Patents
Literature
Patsnap Eureka AI that helps you search prior art, draft patents, and assess FTO risks, powered by patent and scientific literature data.

2238 results about "Spectrogram" patented technology

A spectrogram is a visual representation of the spectrum of frequencies of a signal as it varies with time. When applied to an audio signal, spectrograms are sometimes called sonographs, voiceprints, or voicegrams. When the data is represented in a 3D plot they may be called waterfalls.

Intelligent risk early warning method and system based on multi-dimensional data analysis

The invention relates to the field of enterprise risk early warning analysis, in particular to an intelligent risk early warning method and system based on multi-dimensional data analysis. The method comprises the following steps: acquiring a multi-dimensional enterprise data stream, performing heterogeneous index information analysis and logic hierarchy reconstruction, and constructing an enterprise running state sensing map; performing multi-index local fluctuation amplitude calculation on the enterprise operation state sensing map, and performing dynamic disturbance feature mining to construct a risk disturbance spectrogram; performing deep semantic analysis and node state abrupt change feature analysis on the risk disturbance spectrogram to generate a dynamic transition type of an abrupt change node; and performing full-period time sequence tracing on the risk disturbance spectrogram, performing potential risk node prediction based on the dynamic transition type, and identifying other potential risk propagation ports. Through accurate and efficient enterprise risk perception, the risk intervention decision is made in advance, and the anti-risk capability and the operation stability of the enterprise are improved.
Owner:BEIJING HAOHONGDA XUNJIE TECHNOLOGY DEVELOPMENT CO LTD

Method and system for early diagnosis of parkinson's disease based on multimodal deep learning

A method for early diagnosis of Parkinson's disease based on multimodal deep learning is provided. Audio-visual data of a to-be-diagnosed subject while performing a speech task is acquired. The audio-visual data are preprocessed to extract a plurality of audio segments and a plurality of video segments. A face image sequence is extracted from each of the plurality of video segments. A Mel-spectrogram of each of the plurality of audio segments is calculated. The face image sequence and the Mel-spectrogram are input into a multimodal deep learning model to output a classification result for Parkinson's disease early diagnosis of the to-be-diagnosed subject. A system for early diagnosis of Parkinson's disease based on multimodal deep learning is also provided.
Owner:SHANDONG UNIV

Emphasizing portions of synthesized speech

A neural text-to-speech system may be configured to emphasize words. Applying emphasis where appropriate enables the TTS system to better reproduce prosodic characteristics of human speech. Emphasis may make the resulting synthesized speech more understandable and engaging than synthesized speech lacking emphasis. Emphasis may be manually annotated to, and / or predicted from, a source text (e.g., a book). In some implementations, the system may use a generative model such as a variational autoencoder to generate word acoustic embeddings indicating how emphasis is to be reflected in the synthesized speech. A phoneme encoder of the TTS system may process phonemes to generate phoneme embeddings. A decoder may process the word acoustic embeddings and the phoneme embeddings to generate spectrogram data representing the synthesized speech.
Owner:AMAZON TECH INC

Image semantic segmentation method and device fused with Fourier transform

The embodiment of the invention provides an image semantic segmentation method and device fused with Fourier transform, and the method comprises the steps: constructing a segmentation model based on a U-shaped network, and carrying out the feature extraction through employing a four-layer convolution pooling network as an encoder; constructing a Fourier transform module, performing two-dimensional discrete Fourier transform on the input image to obtain a spectrogram, and performing amplitude spectrum and phase spectrum decomposition on the spectrogram to obtain an initial frequency domain feature; performing preset size scaling and pixel normalization processing on the input image to obtain a pre-processed image, and inputting the pre-processed image into a Fourier transform module to obtain four groups of adjusted frequency domain features; a bilinear interpolation algorithm is adopted to adjust the size of the prediction probability graph to the size of an original input image, pixel points with probability values larger than 0.5 in the prediction probability graph are marked as foreground areas, and a final semantic segmentation result is obtained; according to the method, the semantic segmentation task in a complex scene can be better processed, and the segmentation precision and the model robustness are improved.
Owner:UNIVERSAL UBIQUITOUS TECH CO LTD

Method and system for adaptively adjusting ambient noise of Bluetooth headset

The invention relates to the technical field of audio noise reduction, and discloses a Bluetooth headset environment noise adaptive adjustment method and system, and the method comprises the steps: collecting multi-band sound data of headset environment noise through a preset acoustic sensor array, generating a noise spectrogram, extracting a noise masking threshold through time-frequency joint analysis, and collecting a headset audio signal; next, noise suppression is carried out on the audio signal by using a multi-order digital filter bank to obtain a primary noise reduction signal, the primary noise reduction signal is optimized into an optimized noise reduction signal based on a signal-to-noise ratio, and the frequency response matching degree of the optimized noise reduction signal and the original audio signal is calculated; then, performing phase correction and amplitude-frequency equalization processing according to the frequency response matching degree to obtain an audio equalization parameter; and finally, monitoring residual noise energy based on the parameter, generating a noise regulation and control instruction, driving adaptive sampling, and generating an adaptive noise reduction scheme. According to the invention, the audio noise reduction effect of the Bluetooth earphone can be improved.
Owner:SHENZHEN SHENYU ELECTRONICS TECH CO LTD

Dereverberation and noise reduction

A system configured to improve audio processing by performing dereverberation and noise reduction during a communication session. In some examples, the system may include a deep neural network (DNN) configured to perform speech enhancement, which is located after an Acoustic Echo Cancellation (AEC) component. For example, the DNN may process isolated audio data output by the AEC component to jointly mitigate additive noise and reverberation. In other examples, the system may include a DNN configured to perform acoustic interference cancellation, which may jointly mitigate additive noise, reverberation, and residual echo, removing the need to perform residual echo suppression processing. The DNN is configured to process complex-valued spectrograms corresponding to the isolated audio data and / or estimated echo data generated by the AEC component.
Owner:AMAZON TECH INC

Precise interference avoidance method and device in radio system

The invention discloses a precise interference avoidance method and device in a radio system, and relates to the field of signal processing, and the method comprises the steps: constructing a sensing matrix through sensing node data, and capturing a transient interference signal through aperiodic scanning; performing tensor decomposition on the signal data to extract time domain, frequency domain, space domain and modulation domain features, constructing a dual-mode spectrum analysis model, reconstructing an instantaneous spectrogram by using compressed sensing, and predicting an interference mode through LSTM; after the instantaneous spectrogram and the predicted interference graph are fused, threat assessment is carried out through a multi-stage interference classification model; according to the interference category and the threat level, beam forming is optimized, adaptive null is generated, a power density optimization model is constructed, and the transmitting power is dynamically adjusted; an anti-interference frequency hopping sequence is generated based on a chaotic mapping algorithm, and spectrum camouflage and tracking interference resistance are realized. The method has the advantages that accurate identification and dynamic avoidance of interference are realized through multi-dimensional perception, intelligent prediction and adaptive beam forming, and the interference avoidance capability of a wireless system is improved.
Owner:BEIJING BOHONG KEYUAN INFORMATION TECH CO LTD

Water supply network sound leakage monitoring method and device based on multi-modal data fusion and multi-task learning

The invention discloses a water supply network sound leakage monitoring method based on multi-modal data fusion and multi-task learning, and the method comprises the steps: inputting an original sound signal, and carrying out the labeling of a label; converting the original sound signal to obtain a corresponding logarithmic Mel spectrogram; performing feature extraction on the original sound signal to construct a time-frequency feature set; forming a data set by the original sound signal, the logarithmic Mel spectrogram, the label and the time-frequency feature set; constructing a multistage task prediction network based on a residual convolutional neural network framework; performing multi-task training on the multi-level task prediction network by using the data set to obtain a water supply network sound leakage monitoring model; and inputting the original sound signals of the collection points into the water supply network sound leakage monitoring model to obtain a prediction result. The invention further provides a water supply network sound leakage monitoring device. According to the method provided by the invention, the leakage detection aspect is promoted to more comprehensive sound leakage monitoring application through a data-driven acoustic detection technology.
Owner:INNOVATION CENTER OF YANGTZE RIVER DELTA ZHEJIANG UNIVERSITY +2

Multi-mode power line carrier communication method and carrier communication module

The invention relates to the technical field of carrier communication, in particular to a multimode power line carrier communication method and a carrier communication module, and the method comprises the steps: determining a frequency coincidence degree through analyzing the coincidence degree of a frequency corresponding to a positive number amplitude value on a spectrogram of each modal component in a carrier signal and a signal frequency range of the carrier signal, and determining a difference index by analyzing the difference between each characteristic modal component and all modal components of the noise reference signal in frequency, analyzing the centralized distribution condition of the carrier signal in energy, and determining a step length adjustment value in combination with the signal interference degree. And carrying out noise reduction processing on the carrier signal in the power line carrier communication process at the current moment. The invention aims to eliminate noise interference in carrier signals and improve the reliability of power line carrier communication.
Owner:SHENZHEN LANCHAO TECH CO LTD

Power equipment fault diagnosis method based on time sequence-spectrogram-discrete data multi-modal fusion

The invention relates to a power equipment fault diagnosis method based on time sequence-spectrogram-discrete data multi-modal fusion, and belongs to the field of power equipment fault diagnosis. A multi-modal feature fusion technology based on an attention mechanism is adopted, and a power equipment fault diagnosis model based on multi-modal information fusion is constructed. Compared with a traditional deep learning fault diagnosis method, the problem that a fault single-modal sample is insufficient in diagnosis model accuracy can be solved through multi-modal data fusion, and the universality and diagnosis precision of the model are improved.
Owner:STATE GRID FUJIAN ELECTRIC POWER RES INST +2

Identification method of multi-model fusion signal modulation mode based on time-frequency diagram

The invention discloses a method for identifying a multi-model fusion signal modulation mode based on a time-frequency diagram, and belongs to the field of signal processing. Preprocessing the original signal to obtain a signal sample; constructing a data set by using the normalized signal samples; constructing a fusion model; training a modulation identification module in the fusion model; performing modulation mode identification on an input unknown signal by using the fusion model to obtain a preliminary identification result; and performing comprehensive judgment on the preliminary recognition result of the fusion model by using a comprehensive judgment device to obtain a final recognition result. According to the invention, by combining a plurality of time-frequency analysis methods, the time-frequency domain feature information of the signal is fully extracted; the reliability and robustness of a final decision are improved by adopting a multi-model fusion framework, and the modulation recognition performance of the system under the condition of a low signal-to-noise ratio is improved.
Owner:NORTHWESTERN POLYTECHNICAL UNIV

Method for providing video and electronic device supporting the same

An electronic device is provided. The electronic device includes a memory, and at least one processor electrically connected to the memory, wherein the at least one processor is configured to obtain a video including an image and an audio, obtain information on at least one object included in the image from the image, obtain a visual feature of the at least one object, based on the image and the information on the at least one object, obtain a spectrogram of the audio, obtain an audio feature of the at least one object from the spectrogram of the audio, combine the visual feature and the audio feature, obtain, based on the combined visual feature and audio feature, information on a position of the at least one object the information indicating the position of the at least one object in the image, obtain an audio part corresponding to the at least one object in the audio, based on the combined visual feature and audio feature, and store, in the memory, the information on the position of the at least one object and the audio part corresponding to the at least one object.
Owner:SAMSUNG ELECTRONICS CO LTD

Water turbine fault classification diagnosis method based on multi-modal fusion and meta learning

The invention discloses a water turbine fault classification diagnosis method based on multi-modal fusion and meta-learning. The method comprises the following steps: step 1, respectively extracting time domain features and Mel-language spectrogram features of monitoring noise signals of a water turbine through a feature extraction module; step 2, performing cross-modal attention mechanism fusion on the time domain features and the voiceprint features through a multi-modal fusion module, and adjusting a fusion weight based on a dynamic weight distribution mechanism; and step 3, performing small sample training optimization on the fused features through a meta-learning module, and improving the classification capability of the model for new fault types in combination with a Triplet Loss-KNN algorithm and twin network pre-training. The fault diagnosis method based on the time domain-voiceprint fusion network and meta learning has the advantages of being rapid in diagnosis, accurate in classification, high in generalization ability and the like, and the fault diagnosis precision of the water turbine based on noise signals can be effectively improved.
Owner:CHINA THREE GORGES UNIV

Multi-language cross-culture communication auxiliary method and system based on large model

The invention provides a multi-language cross-culture communication assisting method and system based on a large model. The method comprises the following steps: receiving a source language audio stream during a call, calling a multi-language sound frequency harmonic modulation feature library to extract fundamental frequency harmonic intensity distribution and tone turning features, and generating a cultural acoustic fingerprint vector; based on the vector, controlling a microphone array phase difference, directionally enhancing a fundamental frequency harmonic component of a speaker and suppressing noise, and outputting a high signal-to-noise ratio spectrogram; analyzing the pronunciation rhythm and tone turning characteristics of the spectrogram, capturing the pitch jump and duration of the syllable boundary, and generating an acoustic culture label; associating the spectrogram with a target semantic library, matching harmonic distribution and a cultural context rule based on a large model, and outputting a cultural interpretation prompt containing an ambiguity resolution suggestion; and generating a calibration result according to the acoustic tag and the semantic prompt, and overlapping the dynamic floating subtitles to the face area of the speaker in the video conference picture. According to the invention, cultural tone ambiguity in multi-language communication is eliminated.
Owner:LUSTER LIGHTWAVE CO LTD

Method, system, and device for classifying feeding intensity of fish school

Provided are a method, system and device for classifying feeding intensity of fish school, relating to the field of aquaculture. The method includes: extracting features of an audio clip to be detected to determine a Mel spectrum-based fish school feeding depth speech spectrum feature vector, a CQT-based fish school feeding depth speech spectrum feature vector, and an STFT-based fish school feeding depth speech spectrum feature vector; fusing the Mel spectrum-based fish school feeding depth speech spectrum feature vector, the CQT-based fish school feeding depth speech spectrum feature vector and the STFT-based fish school feeding depth speech spectrum feature vector to generate a fused feature spectrogram; and inputting the fused feature spectrogram into a deep convolutional neural network model constructed by historical audio clips corresponding to different types of feeding intensities to determine a feeding intensity type corresponding to the audio clip to be detected.
Owner:CHINA AGRI UNIV

Systems and methods for multimodal indexing of video using machine learning

Systems, methods, and computer-readable media are disclosed for systems and methods multimodal indexing of video using machine learning. An example method may include deceiving, by a video encoder of an audio-video transformer neural network comprising one or more computer processors coupled to memory, a first frame and a second frame associated with a first segment of a video. The example method may also include receiving, by an audio encoder of the audio-video transformer neural network, an audio spectrogram comprising first audio data associated with the first segment of the video. generating, by the video encoder, a first video embedding. The example method may also include generating, by the audio encoder, a first audio embedding. The example method may also include determining a fusion of the first video embedding and the first audio embedding using a multimodal bottleneck token. The example method may also include determining an output including the first video embedding and the first audio embedding. The example method may also include determining a classification of the first portion of the video based on the output.
Owner:AMAZON TECH INC

Multi-modal forged video detection method based on multi-head addition cross attention mechanism

The invention discloses a multi-mode counterfeit video detection method based on a multi-head addition cross attention mechanism, and belongs to the technical field of video counterfeit detection. The method comprises the following steps: preprocessing a video stream, decomposing a single-frame positioning face, and extracting an audio to generate a Mel spectrogram slice; the 3D convolutional network extracts video spatio-temporal features and motion differences, and the filter bank extracts audio features in combination with the residual network; audio features are mapped to a video alignment space through asymmetric projection, the video features are subjected to bidirectional interaction with an audio input multi-head addition cross attention module after being subjected to time sequence coding, and audio dominant and video dominant features are generated and are cascaded and fused with original features; and constructing cross-modal similarity loss constraint feature distribution, and fusing feature dynamic weighting and time sequence compression to output four classification probabilities of audio-visual double true, audio-visual double pseudo, video pseudo-audio true and video pseudo-audio pseudo. The multi-mode counterfeiting recognition precision is improved, and texture abnormity and audio and video mismatch features are captured.
Owner:ARTIFICIAL INTELLIGENCE INNOVATION RES INST OF ZHEJIANG UNIV OF TECH BINJIANG DISTRICT HANGZHOU

Centralized station terminal with data real-time acquisition and analysis function

The invention relates to the technical field of ground fault detection, in particular to a centralized substation terminal with a data real-time acquisition and analysis function. Performing difference on the current effective value sequence to obtain a target moment; obtaining a local number according to the historically connected differential value of the target moment; acquiring a local current segment and a corresponding spectrogram according to the target moment and the local number; obtaining a harmonic ratio coefficient according to the number of peak points in the spectrograms of the target moment and the historical adjacent target moments and the amplitudes of the peak points; obtaining a target coefficient according to a difference value between the target moment and a historical adjacent target moment and a harmonic ratio coefficient, and carrying out clustering; and obtaining a detection reference value according to the discrete characteristics of the target coefficient in the cluster. The abnormal characteristic value is obtained according to the target coefficient and the detection reference value at the current moment; and high-resistance grounding fault monitoring is carried out according to the abnormal characteristic value, so that the fault detection accuracy is improved.
Owner:SHANDONG DEYUAN POWER TECHNOLOGY CORP LTD

Hierarchical emotional speech generation method and device, equipment and medium

The invention relates to the technical field of speech synthesis, can be applied to business scenes of medical health, financial science and technology and the like, and discloses a hierarchical emotional speech generation method, which comprises the following steps: acquiring an input text and an emotional speech sample, extracting a text embedding feature from the input text, extracting a Mel spectrum feature from the emotional speech sample, and obtaining a text embedding feature; the method comprises the following steps: extracting phoneme-level, word-level and statement-level sentiment distribution characteristics through a hierarchical sentiment distribution extraction module, carrying out time dimension alignment, generating a multi-level sentiment guidance matrix, and inputting the multi-level sentiment guidance matrix and text embedding characteristics into a sentiment synthesis module to generate a target Mel spectrum; and finally, converting the target Mel spectrum into target voice through a vocoder. The emotion control is expanded from the statement level to the fine-grained level of phonemes, words and the like, and the multi-level emotion guidance matrix is combined for generation, so that the generated speech is finer and more abundant in emotion expression and conforms to the context, and the naturalness of speech generation and the accuracy of emotion transmission are improved.
Owner:PING AN TECH (SHENZHEN) CO LTD

Computer implemented method and computing system for monitoring the blades of a wind turbine

The present invention relates to a computer implemented method for monitoring the blade (210) of a wind turbine (200), the method comprising fixing a sound acquisition device (110) to an outer surface of a tower (220) of the wind turbine (200), wherein the sound acquisition device (110) comprises a microphone, a sound encoder and a communication unit, acquiring a sound sample of sound emitted by the blade (210) during operation of the wind turbine (200), encoding the sound sample into a raw audio file, sending the raw audio file to a remote computing device (120), preprocessing the raw audio file, generating a spectrogram of the pre-processed audio file, detecting an anomaly by applying an image analysis machine learning model to the spectrogram, classifying the anomaly to a damage type by applying a damage type feature analysis machine learning model to frequency domain features of the audio file at the time and frequency of the anomaly, determining a damage location by applying a damage location feature analysis machine learning model to frequency domain features of the audio file at the time and frequency of the, and tracking of a damage severity by extracting audio features of the audio file at the time and frequency of the anomaly indicative of the damage severity depending on the damage type.
Owner:WEREOVER GMBH

Speech synthesis method and device based on hierarchical emotion distribution, equipment and medium

ActiveCN119207372AImprove naturalnessImprove emotional expressionSpeech synthesisSpeech rateTime domain
The invention relates to the technical field of artificial intelligence, and discloses a speech synthesis method based on hierarchical emotional distribution, which comprises the following steps: acquiring audio data and text transcription, segmenting audio into phoneme, word and sentence-level speech units, extracting acoustic characteristics such as pitch, speed and rhythm, generating an emotional distribution vector by using a pre-training model; the method comprises the following steps: processing text transcription to generate text embedded information, combining an emotion distribution vector, the text embedded information and acoustic features, generating rhythm features through a variable converter, generating a Mel-frequency spectrogram by a decoder based on the rhythm features, and converting the Mel-frequency spectrogram into a time domain audio signal by a vocoder and outputting the time domain audio signal. According to the method, the audio data are hierarchically segmented into phonemes, words and sentences, the acoustic features, the emotion distribution vectors and the text embedding information are combined, and the rhythm features are generated through the variable converter, so that fine control over emotion expression is achieved, and the naturalness and emotion expressive force of speech synthesis are remarkably improved.
Owner:PING AN TECH (SHENZHEN) CO LTD

Pipeline defect identification method and device, storage medium and equipment

The invention discloses a pipeline defect identification method and device, a storage medium and equipment, and belongs to the technical field of pipeline defect detection. The method comprises the following steps: S1, acquiring a time sequence sound signal acquired by an acoustic sensor arranged on the inner wall and / or the outer wall of a pipeline; s2, performing spectrogram extraction on the time sequence sound signal to obtain a Mel spectrogram and a LOFAR spectrogram; and S3, performing feature extraction on the Mel spectrogram, the LOFAR spectrogram and the time sequence sound signal through a deep learning model of multi-modal signal fusion to obtain a feature vector containing spectrogram features and time sequence features, and performing classification according to the feature vector to obtain a classification result of pipeline defects. According to the method, the acoustic signals are converted into different types of spectrograms, the original time sequence signals are combined, feature extraction, fusion and classification are carried out through a deep learning model, complementary information of the spectrograms and the time sequence signals is fully utilized, the limitation of a single recognition mode is overcome, and the accuracy and robustness of pipeline defect detection are improved.
Owner:CHINA URBAN CONSTR DESIGN & RES INST CO LTD

Analysis method and system for improving spectrum quality of mass spectrum

The invention relates to the technical field of mass spectrum analysis, in particular to an analysis method and system for improving the spectrum quality of a mass spectrum. The method comprises the following steps: scanning a sample by using a mass spectrometer, collecting original mass spectrogram data, and preprocessing the original mass spectrogram data; based on the preprocessed original mass spectrum data, identifying potential peak positions through a multi-scale Gaussian fitting technology, and screening out candidate peaks by calculating the signal-to-noise ratio of each peak; fitting each screened candidate peak by using an asymmetric Gaussian function model, and describing the shape of the candidate peak; and constructing a mass offset correction model to correct the mass spectrum data, and performing super-resolution processing on the corrected mass spectrum data by using a convolutional neural network model. According to the design, peak fitting is carried out by adopting a multi-scale Gaussian fitting technology, a signal-to-noise ratio screening mechanism and an asymmetric Gaussian function model.
Owner:TIANJIN ZHIPU INSTR CO LTD

Mobile operation device with audio and video decomposition function

The invention relates to the technical field of data denoising, in particular to a mobile operation device with an audio and video decomposition function. Obtaining a noise probability value according to the data distribution in the sliding window range in the peak point sequence of the audio signal and the audio signal length corresponding to the sliding window range; obtaining a signal segment according to the noise probability value; clustering is carried out according to peak features in the spectrogram of the signal segments, and an audio feature cluster and a noise feature cluster are obtained; according to the time interval between the signal segment and other signal segments and the peak value of the audio feature cluster, obtaining a signal confidence coefficient; obtaining a reference segment according to the signal confidence and the noise feature cluster; and obtaining a noise degree characterization value according to the noise feature clusters of the signal segment and the reference segment and the signal confidence of the signal segment. According to the noise degree characterization value, the self-adaptive step length factor is obtained, the signal segments are de-noised, the de-noising efficiency and accuracy are improved, and the working performance of the mobile operation device is better.
Owner:BEIJING GUOWANG SHENGYUAN INTELLIGENT TERMINAL SCI & TECH CO LTD

Radio frequency fingerprint identification method and system based on convolution-attention mechanism and multi-packet reasoning

The invention relates to a radio frequency fingerprint identification method and system based on a convolution-attention mechanism and multi-packet reasoning, and belongs to the technical field of communication networks and artificial intelligence. Comprising the following steps: step 1, capturing an equipment transmission signal, and preprocessing the transmission signal to obtain a spectrogram; step 2, constructing a radio frequency fingerprint identification model, and inputting the obtained spectrogram into the radio frequency fingerprint identification model for training to obtain a trained radio frequency fingerprint identification model; and step 3, performing prediction by using the trained radio frequency fingerprint identification model to obtain a final prediction result. Aiming at the problems of high calculation complexity and redundant module interaction existing in a traditional model fusion mechanism, a module separation strategy is adopted, so that a convolution module and a Transform module are independently optimized, the calculation complexity is reduced, and the stability of the model is improved.
Owner:SHANDONG NORMAL UNIV

Full-automatic cable vibration high-precision measurement method based on machine vision technology

The invention discloses a full-automatic cable vibration high-precision measurement method based on a machine vision technology, and relates to the technical field of structural vibration measurement. The method comprises the following steps: installing and calibrating equipment, and ensuring that a cable plane coincides with an imaging plane; high-frame-rate video acquisition is carried out, and comparison verification is carried out in combination with an accelerometer; screening effective pixels based on the pixel intensity time sequence variance, and constructing a pixel intensity space-time matrix by using the time sequence of the pixels; sVD decomposition is adopted to extract an accumulated energy ratio gt; 90% of the main modes are reconstructed, and real signals are revealed; a plurality of inhaul cables are automatically positioned and coded through gradient features and Hough transform; sub-pixel-level vertical displacement is calculated by applying an optical flow method, and median filtering is combined to resist noise; fourier transform is carried out to obtain a spectrogram, and a fundamental frequency is identified; and calculating cable force based on a string pulling theorem. According to the method, remote, full-field and multi-target synchronous monitoring is realized, the displacement measurement precision reaches a sub-pixel level, and a high-precision and full-automatic solution is provided for health monitoring of structures such as bridges.
Owner:NINGBO ORIENTAL UNIV OF TECH (TEMPORARY NAME)

Multi-source federal cross-domain and source-domain enhanced millimeter wave action recognition method and system

The invention discloses a multi-source federal cross-domain and source-domain enhanced millimeter wave action recognition method and system, and the method comprises the steps: generating a micro-Doppler spectrogram through a millimeter wave radar, and extracting the motion features of human body actions through a signal processing module; in the federal multi-source domain adaptation module, dynamically evaluating and fusing knowledge of a plurality of source domains by adopting a voting-based pseudo-tag method and a weighted knowledge aggregation mechanism, and optimizing the generalization ability of a target model; and through a generalization gap optimization method, the performance of the source domain model is improved, and the robustness of the system in different environments is ensured. Through combination of a federated learning framework and a multi-source domain adaptation technology, unsupervised learning under the condition that a target domain has no annotated data is realized, only a single set of millimeter wave equipment is needed, a millimeter wave communication protocol is compatible, and the method has the characteristics of privacy protection, unsupervised learning, multi-source knowledge fusion and strong generalization ability. The method is suitable for application scenes of smart home, health monitoring, man-machine interaction and the like, and has wide practical application value and research prospect.
Owner:XI AN JIAOTONG UNIV

Bluetooth communication intelligent speech translation method and system based on multi-mode enhancement

The invention relates to the technical field of artificial intelligence, and discloses a Bluetooth communication intelligent speech translation method and system based on multi-mode enhancement, and the method comprises the steps: collecting a multi-channel audio signal through a built-in multi-microphone array of a Bluetooth device, carrying out the dynamic direction self-adaptive beam forming of the multi-channel audio signal, and carrying out the self-adaptive beam forming of the multi-channel audio signal; extracting a Mel spectrogram feature of the direction enhancement signal, identifying lip regions of a plurality of candidate speakers in each frame of real-time speaking video captured by a camera, performing time sequence convolution on the lip regions to obtain a lip movement time sequence embedded vector, calculating a correlation score with the Mel spectrogram feature, separating the direction enhancement signal, and obtaining a lip movement time sequence embedded vector; and performing text transcription and conversion on the high-confidence separation voice to obtain a translation language text, and sending the synthesized target translation voice to a preset mobile terminal through the Bluetooth device to obtain a target translation result. According to the method, the real-time performance and accuracy of speech translation are improved in a multi-person scene, far-field speech, noise interference and accent difference.
Owner:SHENZHEN DIE MICRO SEMICON CO LTD

Underwater sound radiation noise identification method

The invention relates to the related field of underwater acoustic signal processing and noise recognition, in particular to an underwater acoustic radiation noise recognition method, and provides an underwater acoustic radiation noise real-time recognition system based on combination of a convolutional neural network (CNN) and a long short-term memory network (LSTM). The system can perform efficient data preprocessing, feature extraction and optimization processing after collecting underwater acoustic signals in real time, and performs rapid identification of noise types through the trained mixed deep learning model. Specifically, the CNN can extract local features from a spectrogram, the LSTM is good at capturing time sequence information, complex underwater acoustic signals can be better understood and distinguished through combination of the CNN and the LSTM, and the recognition precision is remarkably improved. In addition, the system is further provided with an automatic updating mechanism, the model can be retrained regularly according to newly-collected data, it is ensured that the model is always in the optimal state, and the system adapts to the constantly-changing underwater environment.
Owner:ZHONGCHUAN NO 9 DESIGN & RES INST

Diffusion-based audio purification for defending against adversarial deepfake attacks

Disclosed are systems and methods including software processes executed by a server that detect audio-based synthetic speech (“deepfakes”). Embodiments implement a machine-learning architecture having a diffusion model that generates purified features that are fed to a deepfake detection model. The machine-learning architecture includes input layers that convert an audio signal into a Gaussian or frequency space representation (e.g., log spectrogram) to extract a set of initial features indicative of spoofing or deepfake attacks. The diffusion model identifies adversarial noise on the audio signal in the initial features and generates purified features or clean version of the input audio signal. A deepfake detector includes a neural network architecture and classifier programmed and trained to generate a deepfake detection score and classify the audio signal as genuine or fraudulent using the purified features.
Owner:PINDROP SECURITY INC