232 results about "Acoustic source localization" patented technology

The invention relates to a microphone array sound source positioning method and system based on cross-correlation-beam forming closed-loop optimization, and belongs to the technical field of sound source positioning. The method comprises the following steps: collecting multichannel sound signals through a microphone array and preprocessing the multichannel sound signals to extract time-frequency features and suppress noise interference; time delay information among the microphones is estimated by adopting a generalized cross-correlation phase transformation algorithm, and an optimization strategy is introduced to improve estimation stability and anti-interference performance; enhancing the target sound source signal in combination with a minimum variance undistorted response beam forming algorithm and an adaptive Kalman filtering mechanism; constructing a closed-loop feedback optimization mechanism based on the beam output signal to realize feedback adjustment; and adopting a hybrid network architecture, taking the beam output signal amplitude spectrum as input, and outputting the frequency spectrum or mask of the obtained target sound source signal. The method has the advantages of high calculation efficiency, high positioning precision and strong anti-interference capability, and is suitable for real-time acoustic signal processing in a complex environment.

The invention discloses an all-weather sound source positioning system and method based on multi-sensor fusion, and the method comprises the steps: firstly, obtaining acoustic sensing data, millimeter wave radar data and infrared thermal imaging data, and enabling each data to have a collection timestamp; then, cross-modal time alignment processing is carried out on the multi-source data to map the multi-source data to a unified time reference, and multi-modal fusion features under a unified time axis are obtained; for each time point in the time-aligned multi-modal fusion features, according to the confidence coefficient of the data of each sensor, adaptive weighted fusion is carried out on the data of different modals, and fused common feature representation is generated; and finally, time sequence modeling and joint reasoning are carried out based on the common feature representations of a plurality of continuous time points, and continuous position information of the sound source in the three-dimensional space is regressed. According to the method, high-precision positioning of three-dimensional positions of a plurality of sound sources is realized, so that the accuracy and the stability of sound source positioning are improved in a complex environment and an all-weather condition.

The invention relates to the technical field of environmental noise monitoring, and discloses a road traffic noise intelligent monitoring and three-dimensional sound field reconstruction system, which comprises an acoustic sensor module, a data collection and storage module, a data analysis and evaluation module, a three-dimensional sound field construction and display module and a traffic flow feature library. According to the system, a differential geometry principle is adopted, a sound field is regarded as a Riemannian manifold with a local microstructure, and accurate description of an irregular sound field is realized through a curvature self-adaptive sound field manifold construction technology; introducing a covariant derivative in Riemannian geometry, and constructing a sound propagation model adapted to a complex road environment; and realizing hierarchical decomposition and reconstruction of the sound field by using a multi-scale analysis theory. According to the method, the sound source positioning precision and the calculation efficiency are improved, seamless analysis from microcosmic to macroscopic is realized, an innovative solution is provided for traffic and noise collaborative management, and intelligent traffic and environmental noise management are effectively supported.

The invention discloses a sound source localization method based on multi-frequency separation and Newton optimization deconvolution, relates to the technical field of array acoustic signal processing, and is used for solving the problem that weak sound sources and multiple sound sources are difficult to identify. According to the method, dominant frequency is extracted through multichannel frequency domain analysis, a cross-spectrum matrix is constructed in combination with a near-field propagation model and a guide vector, delay summation beam forming is executed to obtain sound source preliminary distribution, then the distribution is regarded as a convolution result, a maximum likelihood model is introduced, and a two-stage deconvolution strategy of coarse estimation and Newton method fine optimization is adopted to obtain a high-resolution sound source. According to the multi-sound-source positioning method, subgrid-level analysis of sound source positions and amplitudes is achieved, finally, all frequency results are fused, continuous sound source images are smoothly output through a two-dimensional Gaussian kernel, the resolution and real-time performance of multi-sound-source positioning are remarkably improved, and the multi-sound-source positioning method is suitable for high-precision acoustic imaging in a complex sound field.

The invention discloses a formation decoupling sound source localization method and device based on deep learning and a readable medium, and the method comprises the steps: constructing and training a sound source localization model, and obtaining a trained sound source localization model; acquiring a first sound source signal received by a first microphone and a second sound source signal received by a second microphone in the microphone array; calculating generalized cross-correlation frequency domain representation between the first sound source signal and the second sound source signal based on the first sound source signal and the second sound source signal; obtaining a frequency domain feature based on the guide vector between the first microphone and the second microphone and the generalized cross-correlation frequency domain representation; determining an input feature based on the frequency domain feature, and inputting the input feature into a trained sound source localization model to obtain a candidate sound source angle and a confidence coefficient corresponding to the candidate sound source angle; and post-processing the candidate sound source angle to obtain a sound source positioning result. The invention solves the problems that the existing sound source localization method based on deep learning is large in calculated parameter quantity and cannot be applied to embedded equipment and the like.

The invention belongs to the field of sound source localization, and particularly relates to an FPGA accelerated sound source localization algorithm implementation method and system. The method comprises the following steps: acquiring an audio signal through a microphone array with more than two channels to obtain a time domain digital signal corresponding to the audio signal; after the digital signal is converted into a frequency domain signal through the FPGA, the frequency domain signal is stored in a memory outside the FPGA in parallel through a parallel interface of a protocol with parallel transmission capability; the capacity of the memory is greater than that of storage resources in the FPGA chip; acquiring the stored frequency domain signal from the memory by using the protocol through the FPGA, calculating the cross-power spectral density of each microphone pair in the microphone array in parallel according to the frequency domain signal, and storing the calculated cross-power spectral density in the memory in parallel through a parallel interface of the protocol; through the FPGA, the cross-power spectral density is obtained from the memory, and in combination with the obtained pre-stored TDOA value, the response intensity of each microphone pair in the microphone array is calculated in parallel for realizing sound source localization.

The invention relates to the technical field of intelligent vehicle-mounted systems, and discloses a vehicle-mounted voice interaction method and system and a readable storage medium, and the method comprises the steps: synchronously collecting initial voice and video data in a vehicle-mounted environment; performing wake-up word detection through a local acoustic model, and based on the detection confidence, extracting a mouth shape visual feature sequence by using a mouth shape recognition model to perform mouth shape verification so as to obtain a wake-up state and sound source positioning information; activating an interaction module at a corresponding position, and performing semantic recognition on the collected interaction voice and video data through a local model and a cloud model respectively; and finally, carrying out fusion cross validation on the local semantic recognition result and the cloud semantic recognition result to generate a final semantic recognition instruction, and executing corresponding operation by the vehicle-mounted system. According to the method, the recognition accuracy, the response speed and the robustness of vehicle-mounted voice interaction in a complex environment are improved, the false wake-up rate is effectively reduced, and the user experience is optimized.

The invention discloses a sound source localization and distance measurement method, device and equipment based on a microphone array and a storage medium, and relates to the technical field of acoustics, and the method comprises the steps: collecting a frequency sweeping signal played by a sound source through the microphone array, obtaining the audio data of each channel, obtaining a cross-correlation sequence through generalized cross-correlation and phase transformation weighting processing, and obtaining a distance measurement result; constructing a current to-be-processed set, processing other sequences again by taking the first sequence as a reference to obtain a new sequence group, discarding the reference sequence, taking the new group as a to-be-processed set, repeating the iteration step until only one to-be-processed sequence is left in the set, and performing up-sampling on the sequences to obtain a to-be-processed set; extracting a first main peak position to obtain a time delay estimation value of a decimal sampling level so as to determine an incident angle of a sound source relative to an array normal, performing time delay alignment on each cross-correlation sequence, constructing a single-channel beam forming signal based on an alignment result, and determining a second main peak position so as to determine a sound source distance, the efficiency of sound source localization and distance measurement is improved.

The invention provides a far-field sound source localization method applied to a transformer substation, and relates to the field of sound source localization, and the method comprises the steps: obtaining a target frequency sound in the transformer substation, and converting the target frequency sound into a spherical harmonic domain multi-channel observation vector; inputting the spherical harmonic domain multichannel observation vector into a preset network model to obtain a spherical harmonic domain mask matrix, and obtaining an in-band smooth covariance based on the spherical harmonic domain mask matrix; the preset network model is used for filtering the spherical harmonic domain multi-channel observation vector; according to the in-band smooth covariance, a spatial spectrum matrix is obtained, and the spatial spectrum matrix comprises spatial spectrum values in different candidate sound source directions; performing multi-sound source distinguishing on the spatial spectrum matrix to obtain a multi-source direction set, and realizing far-field sound source positioning of the transformer substation; the multi-source direction set comprises polar angle and azimuth angle coordinates of each target sound source direction and is used for reflecting the sound source direction in the transformer substation. The method solves the problems that the noise interference of the transformer substation is large and multiple sound sources are difficult to distinguish, and realizes the accurate positioning of the sound sources in the transformer substation.

The invention relates to the field of audio data analysis, in particular to an audio data processing method and device, equipment and a storage medium. The method comprises the following steps: carrying out multi-angle audio continuous acquisition and scene noise suppression on a conference room through a surrounding microphone array, and generating a filtering optimization audio signal; performing three-dimensional time difference positioning calculation according to the filtered and optimized audio signal to obtain accurate sound source positioning information; personalized voiceprint analysis is carried out on the filtered and optimized audio signals, audio distribution modeling is carried out based on accurate sound source positioning information, and a multi-person conference audio field is constructed; performing parallel audio stream separation according to the multi-person conference audio field to generate an intelligent spliced audio segment; and performing deep semantic analysis and semantic logic correction on the intelligent spliced audio segment to generate an audio analysis result. According to the invention, rapid and accurate speaker audio recognition of a multi-person parallel conference is improved.

The invention provides a voice recognition method and device and a vehicle, and the method can achieve the fusion of endowing a higher proportion to an early-arriving audio signal and endowing a lower proportion to a late-arriving audio signal through the arrival time of each audio signal, thereby preferentially maintaining voice components from a near-end or a direct path under the condition of not needing sound source positioning, and improving the voice recognition efficiency. Audio signals caused by reflection, reverberation or far-end interference are suppressed, the definition and the signal-to-noise ratio of the first recognition audio are effectively improved, and the reliability of voice wake-up and recognition is enhanced.

The invention discloses a directional radio receiving method, a directional radio receiving device and electronic equipment. The method comprises the following steps: acquiring multi-channel audio data acquired by a microphone array; sound source localization is carried out on the multi-channel audio data, a target localization result is obtained, and the target localization result at least comprises a sound source orientation determined based on a historical localization result; audio signals in multiple target directions corresponding to the multi-channel audio data are determined, and the audio signals are determined based on the target positioning result; and determining an output signal corresponding to the multi-channel audio data according to the audio signal. According to the invention, the technical problem of beam forming direction deviation caused by insufficient precision and poor stability of sound source positioning in noise and reverberation environments in the prior art is solved.

The invention is applicable to the technical field of grouting of mining engineering and geotechnical engineering, and provides a grouting diffusion space real-time sensing and tracking method based on slurry dynamic sound source localization, which comprises the following steps: active enhancement and characteristic modulation of a slurry dynamic sound source; building a stereo wave sensing network and acquiring high-fidelity sound wave data; carrying out AI-driven slurry sound source identification and three-dimensional space positioning; and the grouting diffusion process is subjected to three-dimensional real-time visualization and intelligent early warning. The invention creatively provides a comprehensive solution integrating sound source active enhancement modulation, stereophonic sensing network, artificial intelligence deep noise reduction and three-dimensional space inversion positioning, and the core of the comprehensive solution lies in that slurry flow sound signals which are originally weak and difficult to use are actively enhanced and are endowed with features; a traceable slurry power sound source signal with a high signal-to-noise ratio is constructed, and three-dimensional space real-time sensing and visual tracking of the slurry diffusion frontal surface position and the overall diffusion range are realized in combination with an advanced signal processing and positioning algorithm.

The invention provides gas phase pipeline leakage detection rapid positioning equipment and a rapid positioning method. The system comprises a sound source acquisition module, a visible light imaging module, a thermal infrared imaging module, an information processing module and a display module. The visible light imaging module is used for collecting visual image signals. The thermal infrared imaging module collects a temperature electric signal. The information processing module is used for processing the sound source electric signals, the visual image signals and the temperature electric signals and completing image algorithm superposition calculation of all the signals. Dual judgment of sound source positioning and temperature verification is adopted, rapid space positioning of a leakage sound source is achieved in combination with leakage sound source detection of the sound source collection module, and meanwhile misjudgment caused by interference factors such as environmental noise and equipment vibration is effectively eliminated through temperature anomaly verification of the thermal infrared imaging module. The positioning precision is high, and the position of a leakage point is quickly and accurately locked.

This invention provides a sound source localization method, apparatus, and product based on incremental learning and imbalance correction. The method includes: acquiring a dataset for a new task, a parameter-frozen feature extractor, and regularization parameters; enhancing the tail category of the new task dataset; initializing an autocorrelation matrix, a cross-correlation matrix, and a category count based on the number of categories in the new task dataset; iterating through each sample in the new task dataset, updating the autocorrelation matrix, cross-correlation matrix, and category count; calculating the category weight for each category and calculating the Gini coefficient describing the category distribution; adaptively adjusting the regularization parameters based on the Gini coefficient; and solving for and outputting the optimal weight matrix based on the adjusted regularization parameters and aggregating the global autocorrelation matrix and global cross-correlation matrix of different categories. The optimal weight matrix is ​​used to update the sound source localization model. This invention addresses the problem of intra-task and inter-task imbalance in sound source localization, improving the sound source localization effect.

The invention discloses an agricultural product wholesale order automatic generation method and device, equipment and a medium, and relates to the technical field of order management. The method comprises the following steps: firstly, receiving a plurality of field sound signals collected by a sound pickup array on an agricultural product selling communication field in real time, and carrying out voice signal extraction processing by applying an empirical mode decomposition technology, a sound source positioning technology and a voiceprint matching technology in real time so as to obtain speech signals of a seller and a buyer; then, the extraction result is integrated into an agricultural product selling communication dialogue text data stream in real time, the text data stream is imported into a large language model in real time to carry out semantic understanding and entity information extraction processing, then a matched agricultural product wholesale order template is retrieved according to the extraction result, and content filling and dynamic updating are carried out on the template; and generating a new agricultural product wholesale order, and finally pushing the order to the seller terminal equipment in real time and outputting and displaying the order in real time, so that the order generation efficiency can be improved, the order information is ensured to be generated correctly, and the seller experience is improved.

The invention relates to the technical field of noise detection, in particular to a high-precision equipment noise detection method, which comprises a mobile vehicle, a protective frame is fixedly connected to the surface of the mobile vehicle, a master control box is fixedly mounted on the surface of the mobile vehicle, a damping mechanism is arranged on the surface of the mobile vehicle, and the damping mechanism comprises a damping spring. The damping springs are fixedly connected to the surface of the moving vehicle, a telescopic damper is fixedly connected to the surface of the moving vehicle, a damping plate is fixedly connected to a telescopic rod of the telescopic damper, and an adjusting mechanism is arranged on the surface of the damping plate. According to the invention, the sound source is positioned through the sound source positioning instrument, after the noise source is determined, the adjusting mechanism is started to detect the detection equipment, and then the height and angle of the noise detector are changed, so that the detection precision is improved, and the noise can be better evaluated.

The application relates to the field of real-time translation, and provides a real-time duplex translation method based on multi-channel parallel processing and a corresponding product.The method comprises the following steps: collecting multi-channel voice signals of at least two user groups in real time through a group of audio acquisition modules respectively; dynamically adjusting beam forming parameters of each audio acquisition module in the group of audio acquisition modules based on a sound source positioning result and feeding back to the corresponding audio acquisition module; monitoring voice activity of each audio channel corresponding to each audio acquisition module; when the voice activity of any audio channel reaches a predetermined condition, automatically activating a translation processing procedure of the audio channel and keeping the remaining audio channels in a monitoring state; using a parallel processing mechanism for the voice signals of the activated audio channel, simultaneously performing real-time translation of the currently activated audio channel and voice activity monitoring of the remaining audio channels; and transmitting the translation result of the current speaker to other users participating in the conversation in the user group, so that multi-channel data is synchronously coordinated.

The invention relates to the technical field of sound source localization, and particularly discloses a far-field dual-dynamic unmanned aerial vehicle sound source localization truth value construction method and system, and the method comprises the steps: building a unified space-time reference, synchronously collecting dual-dynamic original space data, and carrying out the real-time localization. Performing time alignment on the first absolute space coordinate sequence and the second absolute space coordinate sequence to generate a time-aligned coordinate pair sequence; according to the coordinate pair sequence, calculating a theoretical direction of arrival truth value and a theoretical distance truth value of the target sound source unmanned aerial vehicle relative to the acquisition system; packaging the theoretical direction-of-arrival truth value and the theoretical distance truth value into a truth value data unit, storing the truth value data unit, and constructing a truth value database. The direction-of-arrival and distance truth values required by an evaluation positioning algorithm are directly derived through time alignment and geometric solution; and finally, carrying out standardized association packaging on the truth value and the original multi-modal data to construct a structured database.

The present application relates to a kind of based on GS-RBF and time delay estimation multi-element microphone array sound source positioning method, belong to sound source positioning technical field, comprising the following steps: S1: the near-field model of fault sound source is established, and microphone array design is carried out;S2: based on generalized cross-correlation phase transformation algorithm GCC_PHAT calculation time difference of arrival TDOA matrix;S3: construct the sound source positioning model based on RBF neural network;S4: based on grid search algorithm GS, the sound source positioning model is optimized, and the model after optimization is used to locate sound source.The present application guarantees the rapidity and accuracy of fault sound source positioning when unmanned aerial vehicle inspection, and the anti-noise performance and spatial resolution capability in complex environment are considered, and an efficient, reliable technical solution is provided for the detection of transmission line hidden fault.

The application provides a recording processing method and related devices. The method can include: an electronic device can perform sound source positioning based on the sound collected by a microphone, obtain the position of a target sound source and the number of sound sources in the recording environment, and then perform sound source separation on the sound collected by the microphone according to the position of the target sound source and the number of sound sources in the recording environment to obtain the sound corresponding to the target sound source, i.e., a target audio signal. The electronic device can also determine the signal-to-noise ratio and display the current sound pickup quality to the user. This method can monitor and display the sound pickup quality to the user in real time, so that the user can adjust in time when the sound pickup quality is poor, thereby obtaining high-quality audio and improving the user experience.

The invention relates to an acoustic signal processing and biological recognition technology, in particular to a method for fusing sound source localization and voiceprint recognition, a chip and electronic equipment. The method comprises the following specific steps: synchronously acquiring an environment audio signal through an audio acquisition module array and processing the environment audio signal to obtain audio data; and analyzing the audio data to obtain deep fusion voiceprint features. And judging whether a registered user feature library contains the deep fusion voiceprint feature or not, if so, outputting a sounder identity, and if not, marking and endowing an identification number of the sounder identity. And positioning the sound source based on the audio data to obtain a space coordinate thereof. And establishing an identity-position association model, binding identity information output by voiceprint recognition with space coordinates obtained by sound source positioning, and generating identity-position association data. According to the method, the technology is fused and innovated, accurate linkage of identity-position is realized, high real-time performance and strong anti-interference capability are realized, and the method is adaptive to complex scenes.

The invention relates to the field of robot system control, and particularly discloses a robot tracking and positioning method and system based on audio-visual fusion, and the method comprises the following steps: synchronous collection of audio-visual data, multi-modal fusion judgment, setting of a visual data continuity judgment threshold value, and setting of a visual data continuity judgment threshold value. When the visual detection equipment continuously detects a face target with confidence greater than a set value in multiple frames, the system preferentially enters a visual tracking mode; when the interruption time of the visual data exceeds the set time, the system is automatically switched to a sound source positioning mode; and driving element coordination control: according to the obtained image offset or the obtained target sound source angle change, the driving element performs angle coordination adjustment on the head of the robot at the horizontal and longitudinal angles. According to the method, the limitation of a single mode is overcome, smooth mode switching is realized, the positioning precision is improved, and motion control is optimized.

Example signal processing methods and example electronic devices are disclosed. One example method is applied to an electronic device, where the electronic device includes a microphone array and a camera. The example method includes performing sound source localization on a first audio signal obtained by using the microphone array, to obtain sound source direction information. A first video obtained by using the camera is processed to obtain user direction information. A target sound source direction is determined based on the sound source direction information and the user direction information. A user lip video is obtained in the target sound source direction by using the camera. A second audio signal is obtained by using the microphone array. A third audio signal is obtained based on the second audio signal and the user lip video by using a voice quality enhancement model.

The invention discloses a low-cost handheld portable photoacoustic electromagnetic wave multi-mode insulator detection device, and relates to the technical field of electrical equipment detection, an optical detection module is used for comprehensively collecting appearance images, thermal radiation data and deep ultraviolet signals of an insulator, and the defects of appearance damage, heating, corona discharge and the like of the insulator are accurately recognized; the acoustic detection module collects abnormal sound wave signals of the insulator and achieves accurate sound source positioning by means of an 8-array-element far-field microphone array and a sound source positioning signal processing system. The electromagnetic wave detection module collects a radio frequency signal of the insulator through a radio frequency antenna and a radio frequency spectrum analysis system, carries out deep spectrum analysis and rapidly judges a discharge fault of the insulator, and the central controller is responsible for carrying out synchronous collection, preprocessing and fusion analysis on multi-modal data, generates fault early warning information and carries out on-site processing prompt, and carries out real-time monitoring on the discharge fault of the insulator. Problems existing in existing insulator detection equipment can be effectively solved, and a powerful guarantee is provided for safe and stable operation of a power system.

The application relates to the technical field of unmanned aerial vehicle search and rescue, and discloses a sound source positioning system based on an unmanned aerial vehicle platform, which comprises a flying vehicle, a closed bin, a main image acquisition module, a noise reduction module, an anti-inertia damping release module, a sound acquisition module, a collection control module and a micro image acquisition module. The anti-inertia damping release module is arranged in the inside of the closed bin and is used for adjusting the distance between the main image acquisition module and the sound acquisition module, so that the sound acquisition precision in a dense forest area is improved. The sound acquisition module and the main image acquisition module are separated and arranged in parallel by using the anti-inertia damping release module, time delay positioning is combined, a positioning mode of high-position sound acquisition and low-position sound acquisition is realized, the influence of the dense forest on sound energy reduction is reduced by using the crown-adhering flight of the sound acquisition mechanism, and the sound acquisition quality is poor when the sound acquisition is applied to mountainous areas and dense forest search and rescue, the low-position flight image acquisition area is too small, and the search and rescue precision and the search and rescue quality are improved.

The invention relates to a uniform spherical array time domain sound source localization method based on a neural network, and belongs to the technical field of sound source localization. The method comprises the following steps: constructing a uniform spherical microphone array, and arranging microphones on a spherical surface by adopting a golden angle distribution strategy; collecting a multi-channel time domain audio signal; a time domain signal is input to a deep regression neural network, the network comprises a time domain encoder, a Transform encoder and a full connection layer, and continuous coordinates of a sound source in a three-dimensional space are directly output. According to the method, frequency domain conversion and space grid division are avoided, the positioning precision, the calculation efficiency and the three-dimensional full-space applicability are improved, and the method is suitable for real-time sound source positioning in a complex acoustic environment.

The application discloses an anti-reverberation sound source positioning method and device, equipment and medium, and relates to the technical field of sound source positioning. The method comprises the following steps: synchronously recording original sweep frequency signals by a preset multi-channel microphone array to obtain a plurality of recording signals corresponding to the number of channels, and performing cross-correlation operation on the recording signals and the original sweep frequency signals respectively to obtain cross-correlation sequences; performing peak value detection on the cross-correlation sequences to obtain peak value detection results, and determining target stable peak value positions based on the peak value detection results by using a preset iteration screening strategy to obtain each target delay position corresponding to the number of channels; performing straight line fitting based on the target delay positions to calculate fitting errors according to the fitting results, and judging whether the current sound source positioning is successful by the fitting errors; if successful, determining a target angle and a target distance based on the fitting results and the target delay positions to position the target sound source.

The invention provides an audio processing method and device, a vehicle machine and a medium, and relates to the technical field of vehicle-mounted audios, and the method comprises the steps: carrying out the wake-up word detection of at least two paths of audio signals; the at least two paths of audio signals comprise an external audio signal and an internal audio signal; detecting a wake-up word, and performing interference elimination on the audio signal to obtain a first target audio signal of the audio signal outside the vehicle and a second target audio signal of the audio signal inside the vehicle; determining a target sound source area through a cross attention mechanism according to the first target audio signal and the second target audio signal; and executing a vehicle control instruction according to the audio signal corresponding to the target sound source area. According to the method, wake-up word detection and interference elimination are cooperatively performed on multiple paths of audio signals inside and outside the vehicle, and the target sound source area is accurately positioned in combination with a cross attention mechanism, so that the anti-interference performance of wake-up word detection and the accuracy of sound source positioning in a complex vehicle-mounted acoustic environment are improved, and the vehicle is ensured to only respond to a wake-up instruction of the target sound source area.

The invention discloses a pair of hearing-impaired auxiliary intelligent glasses supporting bidirectional multi-modal interaction and a communication method, and relates to the technical field of voice recognition and processing, and the method comprises the following steps: collecting the position of a sound source and the horizontal coordinate of face detection, and calculating the sound source arrival angle of the sound source based on the position of the sound source and the horizontal coordinate of face detection; the short-time voiceprint feature of each sound source is collected, and a feature identity tag is obtained based on the short-time voiceprint feature and the sound source arrival angle of each sound source; according to the invention, through fusion of visual and auditory multi-modal data, accurate positioning of a sound source space position is realized; the horizontal coordinate obtained by face detection and the arrival angle of the audio signal are cooperatively calculated, so that the problem of direction confusion of a traditional single microphone array in a plurality of adjacent sound source scenes is effectively solved, and the accuracy and reliability of sound source positioning are remarkably improved.