119 results about "Directional microphone" patented technology

A real-time system for the multimodal generation of virtual reality scenes based on artificial intelligence for the creation of immersive three-dimensional environments from natural language narratives, consisting of: a speech capture module configured to continuously record a user's spoken narrative via one or more directional microphones, preprocesses the captured signal by noise reduction and temporal alignment, and outputs a digital speech stream; A speech-to-text processing unit that is operationally coupled to the speech capture module and configured for real-time speech recognition using a continuous neural transformer model. The unit is trained to transcribe natural language utterances into structured text data while maintaining contextual continuity throughout the evolving narrative. a semantic interpretation processing unit that is communicatively linked to the speech recognition unit and configured to perform natural language understanding techniques to extract contextual entities, spatial references, temporal relationships, and object attributes from the transcribed narrative; the engine includes a large language model that is fine-tuned for spatial reasoning tasks; a scene graph generation module configured to transform the interpreted semantic data into a structured, hierarchical representation that defines nodes for identified entities and edges for corresponding relationships, with each node associated with metadata describing geometry, position, orientation, texture, and linking attributes between objects; a multimodal image-language model processor coupled with the scene graph generation module, wherein the processor is configured to retrieve, adapt, or synthesize appropriate three-dimensional elements from a pre-trained visual-lexical embedding space and align these elements with their semantic and spatial definitions derived from the scene graph; a scene assembly and rendering controller configured to create a cohesive virtual scene from the aligned assets, perform real-time rendering using a GPU-accelerated ray tracing pipeline, and produce a stereoscopic visual output that corresponds to the evolving narrative; A head-mounted virtual reality visualization device connected to the rendering engine and configured to display the generated immersive environment to the user in real time. The device features motion sensors and inside-out tracking cameras to detect head and body movements, dynamically updating viewing angles and perspective within the rendered scene; and a bidirectional feedback module integrated into the head-mounted device and connected to the semantic interpretation processing unit; the module is configured to interpret corrective commands, gestures, or supplementary comments from the user to refine or modify specific scene elements without interrupting the real-time visualization; The system continuously updates the virtual scene as the narrative develops, ensuring temporal synchronization between speech input and rendered output below a defined latency threshold, thus enabling a natural, dialogic construction of complex three-dimensional virtual environments.

A system for real-time analysis of emotional feedback during motivational speeches, consisting of: a series of multimodal sensors, including at least one visual sensor configured to capture facial expressions of spectators, at least one directional microphone configured to capture the audio responses of the audience, and optionally one or more physiological sensors configured to capture biometric signals from spectators; an edge-based processing unit that is communicatively coupled to the arrangement of multimodal sensors, wherein the edge-based processing unit comprises the following: (a) a feature extraction module configured to extract visual features from captured facial images, acoustic features from voice responses, and physiological features from biometric signals; (b) an emotion inference machine configured to process the features using a deep learning-based emotion recognition model comprising a convolutional neural network (CNN) for classifying facial expressions, a recurrent neural network (RNN) for classifying voice emotions, and a multimodal late fusion layer configured to compute a composite emotion state vector representing the aggregated emotions of the audience; (c) a timestamp and speech alignment module configured to correlate the calculated composite emotion state vector with segmented portions of a live motivational speech based on real-time speech-to-text transcription and semantic analysis; and (d) a session-based storage unit configured to log time-indexed emotional state vectors and corresponding speech segments for post-event analysis; A speaker feedback interface comprising a portable display or a podium-mounted visualization panel, wherein the interface is configured to display visual indicators of emotional feedback in real time, the indicators being derived from the emotional state vector and including at least emotional trend graphs, threshold alerts, or engagement indices.

The invention relates to the technical field of data processing, in particular to an automatic conference recording and abstract generating method for an intelligent conference, which comprises the following steps of: capturing voice streams of multiple speakers in real time through a directional microphone array, generating an original text stream with a time sequence mark based on real-time voiceprint clustering, and synchronously extracting intention intensity parameters in the voice streams; performing intention-driven dynamic segmentation processing on the original text stream; performing agenda-perceived abstract block generation on the segmented text, wherein decision expressions meeting a semantic density threshold value are extracted as abstract core blocks; and assembling the abstract core blocks into a structured abstract document according to the hierarchical structure of the agenda template. Compared with a traditional linear transcription mode based on voice recognition, the method has the advantages that the mapping precision between the speech content and the identity of the participant is remarkably improved, and a solid foundation is provided for semantic segmentation and decision tracing.

A concentric circular microphone array (CCMA) may include a number of omnidirectional microphones and an equal number of directional microphones, wherein the omnidirectional microphones and the directional microphones form a plurality of concentric rings on a substantially planar platform. Each of the plurality of concentric rings includes a subset of the omnidirectional microphones and a subset of the directional microphones (e.g., arranged in mixed pairs of microphones). Responsive to a sound source, the omnidirectional microphones and the directional microphones may respectively generate first and second electronic signals. A target beampattern of Nth order may be specified for the CCMA. An Nth order beamformer for the CCMA, that is steerable in a three-dimensional space including the sound source, may be determined based on the specified target beampattern. The beamformer may be executed to calculate an estimate of the sound source based on the first electronic signals and the second electronic signals.

The invention relates to the technical field of sound reception, and discloses a system for multi-directional sound reception and noise reduction of a smart glasses microphone array, and the system comprises at least five microphones which are disposed at different positions of a smart glasses frame, and the first microphone is a unidirectional microphone, is located below the glasses frame, and is used for picking up the voice of a wearer; the second microphone and the third microphone are bi-directional microphones, are located right in front of the glasses frame and are used for picking up voice signals right in front of the wearer. The directivity weight of the microphone array is adjusted in real time through the dynamic optimization module, and the pickup effect of the target voice signal is enhanced preferentially. The voice enhancement module is matched to extract a target voice signal by using a Bayesian inference method, so that noise components in a mixed signal can be effectively separated, voice distortion is reduced, the definition and fidelity of the voice signal are remarkably improved, and the voice has better intelligibility in various complex noise environments.

The invention discloses a conference sound reinforcement system based on an AI intelligent algorithm and 360-degree omni-directional noise reduction, and relates to the technical field of audio signal processing, the conference sound reinforcement system comprises a conference management center, the conference management center is in communication connection with the following modules: a multi-sound-source sensing module used for capturing 360-degree omni-directional sound field information in a conference environment and constructing a sound source space distribution model; according to the invention, the omnidirectional microphone array unit covers all directions of a conference space, synchronously collects audio data streams, eliminates the limitation of a conventional unidirectional microphone, combines the sound source positioning space mapping unit, constructs a sound source space distribution model based on the time difference of arrival and the phase difference through a deep learning model, and improves the sound source positioning accuracy. The azimuth angle, pitch angle and distance parameters of the sound source are accurately analyzed, the position of the sound source is mapped to a virtual space coordinate system, a dynamically updated 3D sound source distribution diagram is generated, the position change and intensity distribution of the sound source are reflected in real time, and the positioning precision in a complex acoustic environment is remarkably improved.

The invention provides a factory boundary noise online monitoring traceability system and method, and belongs to the technical field of noise monitoring. The system comprises a data acquisition module; the data processing module is used for judging whether factory boundary noise processing is carried out or not; when the boundary noise processing is determined to be carried out, determining a direction corresponding to the directional microphone with the maximum sound level as a noise source direction; based on the noise source direction, a data acquisition instruction is generated and sent to the data acquisition module, and audio frequency information and video information, collected in real time and fed back by the data acquisition module, in the noise source direction are recycled; and taking the audio frequency information and the video information in the noise source direction as input parameters, and based on a pre-established abnormal sound source positioning model and a pre-established sound feature recognition model, obtaining a noise source visual sound field cloud picture and a noise classification recognition result. The purposes of noise comprehensive response, noise orientation definition and noise accurate traceability are achieved, evidence is provided for factory boundary noise source definition, and disputes are effectively avoided.

The invention provides an AI glasses automatic shooting method and system based on voice control, and relates to the technical field of intelligent wearable equipment, accurate interaction is realized through a multi-channel directional microphone array and an end-cloud collaborative voice recognition engine, the microphone array optimizes the pickup angle based on the wearing position characteristics of a user, and the user experience is improved. In combination with real-time voice activity detection, environmental noise is filtered out, it is ensured that a clear voice instruction can still be captured in a noisy environment, an end-cloud cooperation mode operates a lightweight model locally to guarantee the off-line response speed, a cloud large model is called when a network is available to improve the complex instruction analysis capability, and a composite statement containing parameter adjustment can be recognized; and an operation intention and parameters are automatically bound through a natural language processing engine, so that one-step execution of the instruction is realized.

A concentric circular microphone array (CCMA) may include a number of omnidirectional microphones and an equal number of directional microphones, wherein the omnidirectional microphones and the directional microphones form a plurality of concentric rings on a substantially planar platform. Each of the plurality of concentric rings includes a subset of the omnidirectional microphones and a subset of the directional microphones (e.g., arranged in mixed pairs of microphones). Responsive to a sound source, the omnidirectional microphones and the directional microphones may respectively generate first and second electronic signals. A target beampattern of Nth order may be specified for the CCMA. An Nth order beamformer for the CCMA, that is steerable in a three-dimensional space including the sound source, may be determined based on the specified target beampattern. The beamformer may be executed to calculate an estimate of the sound source based on the first electronic signals and the second electronic signals.

The invention discloses a machine tool state real-time intelligent monitoring and abnormity early warning method. The method comprises the following steps that a directional microphone array is used for collecting cutting voiceprint signals in the machining process of a machine tool; focusing sound in a cutting area through a sound beam forming technology, and inhibiting noise in a non-target area; performing frequency separation, de-noising enhancement and signal segmentation preprocessing on the collected voiceprint signals; extracting a time domain feature, a frequency domain feature and a time frequency feature of the voiceprint; based on process steps in a machine tool CNC program, dynamically matching the real-time voiceprint features with the reference voiceprint model; recognizing the voiceprint feature deviation condition through an anomaly detection algorithm, and triggering a multi-stage early warning signal according to the deviation degree; through the unique signal acquisition and processing technology, a dynamic matching mechanism and intelligent early warning classification, extremely high creativity is shown, the limitation of the prior art is exceeded, and the accuracy, adaptability and intelligent level of the machine tool state monitoring system are remarkably improved.

The invention discloses a wind turbine blade internal damage diagnosis method based on a sound field graph neural network, and belongs to the technical field of wind turbine blade state detection, and the method comprises the steps: deploying a microphone array in a cabin, and collecting an acoustic signal when a blade rotates; constructing a space sound field graph structure by taking the microphone as a node and the sound wave propagation path as an edge; extracting nonlinear acoustic features by using a physical constraint graph neural network PC-GNN; generating a damage embedding vector based on self-supervised training contrast learning; and outputting a damage probability thermodynamic diagram and positioning information. According to the method, a directional microphone array is deployed in a cabin, and a sound wave propagation space diagram structure is constructed; designing a physical constraint graph neural network PC-GNN, and embedding an acoustic wave equation as a regularization item; the problem of scarcity of damaged samples is solved by adopting self-supervised contrast learning; and finally outputting a positioning thermodynamic diagram of the internal damage of the blade.

The invention relates to the technical field of multi-modal data processing, and discloses an elevator advertisement audience behavior analysis system and method based on multi-modal data acquisition, and the system comprises a data acquisition layer which is used for collecting multi-modal sensing information in an elevator space, the data acquisition layer comprises a millimeter wave radar module, a low-resolution thermal imaging module, an ambient light and distance sensing module, a directional microphone array module and an elevator state interface module; and the data processing and fusion layer is connected with the data acquisition layer and used for preprocessing and dynamically fusing the multi-modal sensing information, and the data processing and fusion layer comprises a space-time alignment unit, a dynamic weight adjustment unit and a conflict processing unit. According to the method, a complete audience behavior analysis scheme for the elevator closed space is formed through non-intrusive multi-modal data acquisition, a dynamic fusion strategy adaptive to the elevator scene and fine-grained behavior modeling.

The invention relates to a method and an apparatus for improving the intelligibility of a group of persons engaged in conversation, each person in the group being able at times to be a speaker and at times to be a listener. The persons are situated at two or more positions P. For this purpose, a system (1) is used comprising: two or more directional microphones M which are directed towards the positions P; two or more directional loudspeakers L which are likewise directed towards the positions P; and a digital signal processor DSP which is capable of processing and forwarding acoustic signals in real time. Each microphone M continuously receives acoustic signals Ai and forwards them to the DSP. The DSP identifies, in real time, a speech signal Si1 from each acoustic signal Ai and generates therefrom a processed speech signal aSi1. The DSP transmits the processed speech signal aSi1 to one or more other loudspeakers L. The DSP detects later-arriving speech signals Si2, Si3 having the same characteristics as reflections of the speech signal Si1, and generates therefrom, in real time, processed reflected signals aSi2, aSi3, which the DSP transmits to another loudspeaker L within a predetermined time window (4) having a duration of at most 40 ms.

The application relates to the technical field of data processing, in particular to an audio acquisition method, an electronic device and a storage medium. The audio acquisition method of the application uses an acoustic vector sensor (AVS) array with directivity as a pickup device, which is better than an omnidirectional microphone array, to acquire voice signals in a space, wherein each AVS in the AVS array comprises an omnidirectional microphone and a directional microphone, then the weights of the voice signals acquired by the omnidirectional microphone and the directional microphone in each AVS are adjusted according to a target direction, the voice signals of each AVS after enhancement in the target direction are obtained, then a designed super-directivity beamformer is applied to the enhanced signals acquired by each AVS for further enhancement processing, and the voice signals of the entire AVS array after enhancement in the target direction are obtained.

This invention relates to the field of acoustic signal analysis technology, specifically to a sound collector and diagnostic method for SCR system fault diagnosis. It includes multiple directional microphone arrays arranged in a circular pattern on the outer ring of an elastic clamp. The elastic clamp is fitted onto the tube at the end of the jet valve. The directional microphone arrays are inserted into a mounting base, which is slidably connected to the outer ring of the elastic clamp. The mounting base has slots on its surface, and clamping blocks are inserted into the side walls of the slots. The advantages are: the use of an elastic clamp fitted onto the tube at the end of the jet valve allows the clamp to adapt to tubes of different sizes, simplifying the installation process. Furthermore, the rubber gaskets clamping the clamp between the clamp and the tube prevent loosening of the clamp and effectively reduce vibrations transmitted through the tube, improving the stability and accuracy of sound collection.

This utility model discloses an adjustable pickup angle microphone, relating to the field of microphone technology. It includes a directional microphone body and an adjustment component. The bottom of the directional microphone body has a support frame, and the adjustment component is located on one side of the support frame. The adjustment component includes a side plate, a side cover, a first motor, a worm gear, a worm wheel, and an adjustment shaft. The side cover is mounted on one side of the side plate, and the first motor is fixed to the bottom of the side cover. Through the adjustment component, when the pickup angle of the directional microphone body needs to be adjusted, the first motor drives the worm gear to rotate, which in turn drives the adjustment shaft to rotate, adjusting the pitch angle of the directional microphone body. In conjunction with a second motor, a gear and a gear ring drive the base plate to rotate, changing the orientation of the directional microphone body. This allows for precise pickup at specific angles, further expanding the device's application range.

The present application relates to the technical field of sound recognition, and particularly relates to a sound recognition system for sheep feeding behavior. The technical scheme comprises a sound collection module, a voice enhancement module, a voiceprint feature extraction module, a multi-modal classification module, a data fusion unit, the sound collection module is arranged on a wearable device on the neck of a sheep, is provided with an anti-wind-noise directional microphone array, and is used for collecting environmental sound signals in real time; the voice enhancement module is connected with the sound collection module. The present application realizes accurate recognition and analysis of sheep feeding behavior, effectively solves the problems of sound signal processing in a complex environment, individual and group monitoring, privacy protection and energy supply, not only improves the intelligent level and efficiency of pasture management, reduces labor costs, but also can timely find health problems of sheep, protect the health of the sheep, protect the privacy of sheep data, and improve the energy utilization efficiency of equipment.

The invention provides an electronic device and a pickup method, and belongs to the technical field of terminals.The electronic device comprises a directional microphone array, and the directional microphone array at least comprises a first directional microphone and a second directional microphone. The pickup enhancement direction of the first directional microphone is perpendicular to the pickup enhancement direction of the second directional microphone. The direction with the maximum sound signal gain of the directional microphone array is a target direction, and the target direction is jointly determined by the first directional microphone and the second directional microphone. According to the scheme provided by the invention, the array structure of the first directional microphone and the second directional microphone is combined with the back-end algorithm, so that pickup enhancement in a specific direction can be realized.

The embodiment of the utility model discloses audio and video recording equipment, the audio and video recording equipment comprises a base assembly and a holder assembly, the base assembly comprises a base body, a panoramic camera and an omnidirectional microphone, and the panoramic camera and the omnidirectional microphone are arranged on the base body at an interval; the holder assembly comprises a shell, a zoom camera and a plurality of directional microphones; the zoom camera and the directional microphones are both connected with the shell, the shell is provided with a daylighting opening and a plurality of pickup holes, the pickup holes are annularly formed in the peripheral side of the daylighting opening, the directional microphones correspond to the pickup holes one to one, the zoom camera can shoot through the daylighting opening, the directional microphones correspond to the pickup holes, the shell is connected with the base body, and the base body is connected with the shell. The zoom camera and the directional microphone are arranged on the base body and can move relative to the base body, so that the zoom camera and the directional microphone are aligned with a spokesman to carry out audio and video recording, the pickup effect of the directional microphone can be further optimized, and the pickup definition of the spokesman is improved.

The application discloses a kind of sports dance training rhythm synchronism quantitative analysis and feedback method, it is related to sports dance training data analysis technical field, comprising the following steps: training data multimodal acquisition, 12 camera 16 key joint data are collected, 48kHz directional microphone audio is collected, wrist heart rate instrument is collected heart rate;Preprocessing, action data filter noise completion is removed exception, audio is extracted feature and is removed noise, heart rate is smoothed, unified time sequence;Rhythm characteristic bidirectional extraction, audio is extracted beat cycle and stress, action is extracted force and action cycle;Synchronism quantization, calculate time difference, sequence consistency, identify synchronism decline section;Analysis result, grade, mark weak link, calculate progress amplitude;Generation feedback, AR and voice prompt, adjust plan and push APP.This method solves the problem of traditional training synchronism subjective fuzzy;Real-time multi-sensory feedback helps immediate adjustment, individualized report and plan fit demand.

The present application discloses a hearing aid including a directional microphone system. The hearing aid includes: a forward path, which includes at least two input transducers, a beamformer filter, a signal processor, and an output transducer; a feedback estimation system for estimating a current feedback from the output transducer to each of the at least two input transducers and providing a corresponding feedback metric indicating the feedback; a controller configured to receive the feedback metric from the feedback estimation system; wherein the controller is configured to switch between two operating modes of the hearing aid, namely a single input transducer operating mode and a multi-input transducer operating mode, according to the feedback metric.

A conference terminal, an echo cancellation method and apparatus, and a sound pickup device are provided. The conference terminal comprises a loudspeaker and at least one omni-directional microphone group. The omni-directional microphone group comprises at least two omni-directional microphones. According to the conference terminal, a weight vector of a beam former enabling the at least two omni-directional microphones to form a dipole beam mode is determined, so that an echo signal in the direction of the loudspeaker is suppressed, and a sound signal in a target direction is enhanced. The sound signal is collected by means of the omni-directional microphones. For the at least two omnidirectional microphones, the weighted sum of at least two sound signals is determined according to the weight vector as an echo cancellation signal.

The application discloses a sports action intelligent teaching system and method based on a multimodal large model, and relates to the technical field of intelligent teaching.The method is characterized in that a directional microphone and a high-speed camera are arranged in a badminton training ground, and the method combines band-pass filtering, frame processing and short-time energy calculation of acoustic signals, and target detection and optical flow analysis of visual images, so that the method can obtain a hitting window energy Ehit and a background noise energy Ebase, and further construct an acoustic energy ratio index Rste.The method guarantees high signal-to-noise ratio extraction of the hitting acoustic signal in a complex environment, and can accurately locate a visual contact time Tvis in a visual mode.Compared with the existing teaching method which only relies on visual detection, the method still has high robustness under the conditions of illumination change and field noise interference, and significantly improves the accuracy of hitting action data acquisition and analysis.

To provide a method and an apparatus for detecting the position of a wasp with high accuracy.SOLUTION: A method for detecting a wasp includes collecting an environmental sound in front of a directional microphone by using the directional microphone, performing a frequency analysis on the environmental sound to identify, from the environmental sound, a magnitude of a sound of a fundamental frequency derived from a movement of a wing of a wasp, magnitudes of harmonics of the fundamental frequency, and a high-frequency noise, and determining whether each of the sound of the fundamental frequency, the sounds of the harmonics, and the high-frequency noise is greater than a predetermined threshold. The hornet detection device includes a sound collection means, a determination means, and an output means.SELECTED DRAWING: Figure 11

The present invention relates to the field of data processing technology, and specifically to a method for generating automated meeting records and summaries for intelligent conferences, comprising the following steps: real-time capture of multi-speaker voice streams through a directional microphone array, generation of a time-stamped original text stream based on real-time voiceprint clustering, and simultaneous extraction of intent intensity parameters from the voice stream; performing intent-driven dynamic segmentation processing on the original text stream; performing agenda-aware summary block generation on the segmented text, including extracting decision statements that meet a semantic density threshold as summary core blocks; and assembling the summary core blocks into a structured summary document based on the hierarchical structure of the agenda template. Compared with traditional linear transcription methods based on speech recognition, the present invention significantly improves the mapping accuracy between speech content and participant identity, providing a solid foundation for semantic segmentation and decision tracing.

A conference speech presentation system, a sound localization method and apparatus, a conference system and a pickup device. The method includes the following steps: collecting (S101) a multi-channel voice signal through a directional microphone array; determining (S103) a steering vector including phase information and amplitude information according to array shape information and microphone pointing direction information; determining (S105) sound direction information according to the steering vector and the voice signal. By adopting this processing mode, both the phase information and the amplitude information are considered when determining the steering vector, which can effectively improve the accuracy of sound localization.

The utility model discloses a motor vehicle vibration abnormal sound source positioning device, which belongs to the technical field of abnormal sound source positioning and comprises a base and a positioning device body, a plurality of sound sensors in an index spiral array are arranged on the positioning device body, the sound sensors adopt directional microphones, and the directional microphones are arranged on the base. The positioning device body is provided with four picture capturers which are distributed in a rectangular array mode, the top of the base is rotationally connected with a rotating ring, the rotating ring is provided with a first arc-shaped plate through a supporting plate, a second arc-shaped plate is fixed to the bottom of the positioning device body, and the second arc-shaped plate is located above the first arc-shaped plate; sleeves are fixed to the two sides of the bottom of the second arc-shaped plate correspondingly, connecting rods are arranged at the positions, located in the sleeves, of the bottom of the second arc-shaped plate correspondingly, and the bottom ends of the two connecting rods are connected with second anti-skid pads correspondingly. When the motor vehicle vibration abnormal sound source positioning device is used, the position of abnormal sound can be relatively accurately measured.

The utility model relates to the technical field of electronic products, and provides a recording pen which comprises a frame assembly, a lifting assembly and a microphone assembly. The frame assembly is provided with an accommodating cavity. The lifting assembly is located in the containing cavity and comprises a supporting frame and a lifting module, the supporting frame is matched with the frame assembly in an inserted mode, and the lifting module is arranged on the frame assembly, connected with the supporting frame and capable of driving the supporting frame to do lifting motion; the microphone assembly comprises a directional microphone and a first omnidirectional microphone, the directional microphone is rotationally connected with the supporting frame body, and the first omnidirectional microphone is arranged on the frame body assembly and located at the matching position of the supporting frame body and the frame body assembly; the supporting frame is driven by the lifting module to hide or expose the pickup channels of the directional microphone and the first omnidirectional microphone. According to the utility model, the microphone assembly does not extrude the screen space, the screen ratio can be increased, and the high requirement of a user on the screen ratio can be met; and dust can be prevented from falling on the microphones, so that the pickup effect of the microphones can be prevented from being influenced.

The application provides a directional microphone, which comprises a device substrate, a MEMS microphone sensor assembly, an intermediate plate, a top substrate and a MEMS comb mechanism; the MEMS microphone sensor assembly is fixed in the middle area of the top surface of the device substrate, the intermediate plate is covered on the device substrate and has an inner hole surrounding the MEMS microphone sensor assembly; the top substrate is covered on the intermediate plate and has a horizontal channel on the top surface and a lower end opening on the bottom surface, the horizontal channel is communicated with the inner hole through the lower end opening; the MEMS comb mechanism comprises a movable comb on the top surface of the device substrate and capable of controlled horizontal movement, the movable comb has an opening structure and a shielding structure; during the movement of the movable comb, the opening structure is communicated with the horizontal channel at different positions respectively. The directional microphone has good versatility based on the adjustable directional design of the microphone sensor assembly.

This invention discloses a subject data acquisition system based on multi-mode interaction. The system includes an edge computing gateway, and a flexible piezoelectric thin film array, a millimeter-wave radar module, and a directional microphone array, all connected to the edge computing gateway. The edge computing gateway is configured to control the millimeter-wave radar module to operate at a low duty cycle and extract the compression profile in basic scanning mode. When the piezoelectric signal envelope variance exceeds the limit, it triggers entry into a directional diagnostic mode, calculating the geometric centroid coordinates and extracting the main peak of ventricular ejection. Based on these geometric centroid coordinates, the beamforming weight vector of the directional microphone array is updated to extract the audio signal sequence. The cross-correlation function between the main peak and the Doppler phase signal trough of the chest wall displacement is calculated to obtain the time delay parameter. After sliding window compensation alignment and feature vector extraction, the results are input into the model to output the state confidence assessment result. This application can achieve high signal-to-noise ratio and low false alarm rate for accurate spatiotemporal alignment of multi-source heterogeneous physiological signals and imperceptible monitoring of abnormal states.