20227results about "Speech analysis" patented technology

Herein disclosed is receiving predetermined content, receiving a request to transform the predetermined content into a derivative work, receiving a requested theme for the derivative work, using generative artificial intelligence to create the derivative work generated as a function of the predetermined content and the requested theme, determining if the generated derivative work is approved based on a machine learning model configured to determine a content approval score as a function of content owner preferences, in response to determining the generated derivative work is approved, applying a digital watermark to the approved derivative work, configuring an authorization server to govern use of the approved derivative work based on the digital watermark and providing user access to the authorized derivative work. The requested theme may be determined using a Large Language Model (LLM) and a chatbot interview. The generative artificial intelligence may comprise a diffusion model. The content may comprise music.

The invention relates to a multi-microphone array beam forming signal enhancement method and device, and the method comprises the steps: obtaining a multi-channel time domain signal collected by a multi-microphone array, and carrying out the time-frequency conversion, and obtaining a plurality of time-frequency domain signals; acquiring a spatial distribution vector of the multi-microphone array, and performing deformation compensation processing on the complex number of time-frequency domain signals to obtain sound field consistency signals; performing sound source direction feature extraction on the sound field consistency signal, and performing complex time-space attention beam forming network processing to obtain a beam forming signal; performing beam direction and gain parameter optimization on the beam forming signal through a preset adversarial environment simulator to obtain a dynamic frequency band enhanced signal; and performing inverse time-frequency conversion on the dynamic frequency band enhanced signal, and performing low-delay optimization to obtain a target quality voice output signal. According to the invention, signal deformation compensation can be carried out more accurately.

The invention discloses an adaptive teaching real-time feedback method based on multi-modal fusion, and the method comprises the following steps: synchronously collecting text modal information, voice modal information and image modal information generated by students in a teaching process, forming multi-modal original data information, and extracting historical student interaction behavior data; preprocessing the multi-modal original data information, and respectively generating corresponding text, voice and image sequence features; a visual feature encoder and a sequence feature encoder are adopted to encode each modal sequence feature to obtain a high-dimensional feature; inputting the modal high-dimensional features into a cross-modal fusion network for deep fusion; parameters of the feedback model are optimized through a model-independent element learning feedback regulation and control algorithm, and a personalized feedback strategy is generated; generating comprehensive feature representation according to the fusion features, and outputting personalized teaching feedback; and the interaction information is updated based on the feedback behavior data to realize closed-loop optimization.

The invention provides a digital human interaction control method and device fusing emotion semantics and logical reasoning and a storage medium. The method comprises the following steps: analyzing multi-modal input data of a user, constructing emotion-semantics joint representation, and generating a logic decision path; and through a cognitive fusion module, emotion-semantic representation and a logic decision path are fused, and an interaction response adapting to emotion and logic consistency is generated. The system optimizes an emotion semantic model and a logical reasoning rule on line according to user feedback and interaction history, and real-time interaction of emotion dynamic and logical rules is achieved. The system can dynamically adjust the logic decision path based on the multi-mode emotional state of the user, and improves the naturalness and situation adaptability of interaction. A dynamic time warping algorithm and a factorization machine are introduced to process a multi-modal feature fusion problem, and the accuracy and robustness of emotional state recognition are improved. The online optimization mechanism enables the model and the rule to be evolved continuously, and reasoning errors are corrected automatically through user feedback, so that error circulation is avoided.

The invention discloses a video feature extraction and multi-dimensional matching-based movie advertisement real-time pushing system, and belongs to the technical field of digital advertisements. According to the system, multi-modal analysis is carried out on visual, audio, text and semantic features of a movie through a video feature extraction module, a dynamic interest tag constructed by a user portrait analysis module is combined, and the weighted matching degree of a video scene, user preference and an advertisement tag is calculated by utilizing a multi-dimensional matching engine; and millisecond-level advertisement putting is completed through the real-time pushing decision module. According to the method, context awareness and streaming computing technologies are fused, the relevance between advertisements and content scenes is remarkably improved, cross-platform deployment is supported, the method is suitable for short videos, live broadcast and other real-time scenes, the advertisement click rate is increased by 55%-72% through tests, the response delay is lower than 200 ms, and the method has the technical advantages of being efficient, accurate and low in delay.

The invention relates to the technical field of data processing, in particular to a multi-modal data alignment method and system, and the method comprises the steps: collecting multi-modal data, and carrying out the preprocessing of the multi-modal data; the preprocessed multi-modal data are sent to an event anchor point detection module, and multi-modal emergency detection, anchor point marking and preliminary global alignment are carried out; performing time resampling and numerical value standardization on the multi-modal data after coarse alignment; mapping the resampled and standardized multi-modal data to a unified time axis; performing local fine alignment and global adjustment on the multi-modal feature sequence after unified time axis mapping; and performing confidence evaluation and anomaly correction on the fine alignment result, and outputting a final alignment sequence. According to the invention, a high-precision and multi-modal data real-time alignment method which is light in weight and has adaptive correction capability can be realized on end side equipment with limited resources.

A method for early diagnosis of Parkinson's disease based on multimodal deep learning is provided. Audio-visual data of a to-be-diagnosed subject while performing a speech task is acquired. The audio-visual data are preprocessed to extract a plurality of audio segments and a plurality of video segments. A face image sequence is extracted from each of the plurality of video segments. A Mel-spectrogram of each of the plurality of audio segments is calculated. The face image sequence and the Mel-spectrogram are input into a multimodal deep learning model to output a classification result for Parkinson's disease early diagnosis of the to-be-diagnosed subject. A system for early diagnosis of Parkinson's disease based on multimodal deep learning is also provided.

The invention relates to the technical field of man-machine interaction, in particular to a robot behavior mode dynamic adjustment method based on multi-mode perception, which comprises the following steps: S1, collecting a visual image, a voice signal and an environment parameter of a target scene in real time; s2, extracting each modal feature; s3, dynamically allocating weights, and generating a fusion feature vector; s4, identifying a current scene type and a user attribute; s5, matching a corresponding interaction mode from a preset strategy library based on the scene type and the user attribute identified in the step S4; and S6, executing the matched interaction mode in the S5, and updating the weight distribution rule according to the feedback data. According to the method, vision, voice and environment characteristics are fused through a multi-mode perception technology, the behavior mode of the robot is dynamically adjusted based on a self-adaptive weighted fusion algorithm, and the interaction strategy is optimized in combination with user feedback, so that the interaction accuracy and the intelligent level of the robot under different scenes and user attributes are improved.

The invention discloses a multi-modal data real-time identification and cooperative processing system based on edge computing and federated learning. The multi-modal data real-time identification and cooperative processing system comprises a cloud center coordination node, a plurality of edge computing nodes, a cross-modal encryption engine, a federated learning controller and a model updating verification module. The cloud center coordination node executes federated learning model aggregation and dynamic task allocation, and generates a cross-modal encryption strategy; and the edge computing node is configured with a multi-modal data acquisition module, a local model training unit and a co-processing gateway to realize multi-modal data acquisition and local processing. The system encrypts vision, acoustics and text data by using differentiated algorithms such as spatial confusion, frequency domain permutation and homomorphic encryption; the federated learning controller carries out multi-modal feature fusion, hierarchical encryption and dynamic networking at the edge node; and the model updating verification module performs aggregation updating after ensuring parameter consistency by using secure multi-party calculation. According to the method, real-time processing and privacy protection of multi-modal data are realized, and the data co-processing efficiency is improved.

The invention relates to the technical field of audio and video analysis and data processing, in particular to an AI-based multimedia equipment operation and maintenance management system, which comprises a data acquisition module, an analysis module, an equipment state modeling module, a fault prediction module, a maintenance strategy optimization module, a resource allocation module and a visual platform. The data acquisition module acquires temperature, vibration, audio and video feature data of the equipment through the sensor and the video equipment, and the data are preprocessed and then transmitted to the cloud; the equipment state modeling module constructs a dynamic baseline model based on historical data, abnormal signals are generated through real-time comparison, and the visual platform realizes multi-dimensional state monitoring through 3D topology rendering, AR auxiliary diagnosis and early warning linkage. The system solves the problems of response lag and low resource utilization rate caused by dependence on manual operation and maintenance in the prior art through a closed-loop management mechanism driven by a data stream, and improves the fault diagnosis precision and the operation and maintenance efficiency.

The invention relates to the technical field of artificial intelligence, can be applied to business scenes of financial science and technology, medical health and the like, and discloses a decision-making method, device, equipment and medium based on multi-modal semantic alignment. Executing cross-modal alignment by taking the voice semantic map as a reference to generate associated information, fusing the voice features, the visual features, the action features and the associated information to generate a fusion feature vector, inputting a decision network to generate a decision feature vector and generate a task execution instruction, obtaining execution feedback information of the task execution instruction, and updating the decision network. According to the method, input is dominated by voice instructions, visual features, action features and semantic map depth alignment and fusion are combined, input naturalness and multi-modal data analysis and decision-making efficiency are improved, and interaction adaptability and decision-making accuracy of the model in a complex scene are improved.

The invention discloses a medical whole-course intelligent management system based on a large model, and belongs to the technical field of large models. Comprising a multi-modal data acquisition module, a privacy calculation preprocessing module, a dynamic knowledge enhancement module, a time sequence data analysis module, an intelligent decision engine module, a multidisciplinary collaboration module, a patient interaction platform module, a dynamic intervention feedback module and a system security center module. The cross-mechanism data security sharing is realized, and the compliance of sensitive information processing is also ensured; a two-channel medical knowledge base is constructed, authoritative guidelines can be synchronized, newest clinical research data can be analyzed in real time, the knowledge base is kept in the newest state all the time, and the frontier scientific basis is provided for clinical decisions; dynamic modeling and trend prediction are carried out on long-term monitoring data of a patient by adopting a hybrid neural network model, and potential health risks and development trends can be identified more accurately.

A device capable of using data from multiple sensors to determine an estimated position / direction of a user with respect to the device. The device may use estimated position data, along with confidence data, that originated from a plurality of sensors to fuse the data to determine the user's estimated position and comprehensive confidence of the estimated position. The system may use the location information to perform beamforming / beam steering and / or other downstream operations using the comprehensive estimated position.

The invention relates to the technical field of audio noise reduction, and discloses a Bluetooth headset environment noise adaptive adjustment method and system, and the method comprises the steps: collecting multi-band sound data of headset environment noise through a preset acoustic sensor array, generating a noise spectrogram, extracting a noise masking threshold through time-frequency joint analysis, and collecting a headset audio signal; next, noise suppression is carried out on the audio signal by using a multi-order digital filter bank to obtain a primary noise reduction signal, the primary noise reduction signal is optimized into an optimized noise reduction signal based on a signal-to-noise ratio, and the frequency response matching degree of the optimized noise reduction signal and the original audio signal is calculated; then, performing phase correction and amplitude-frequency equalization processing according to the frequency response matching degree to obtain an audio equalization parameter; and finally, monitoring residual noise energy based on the parameter, generating a noise regulation and control instruction, driving adaptive sampling, and generating an adaptive noise reduction scheme. According to the invention, the audio noise reduction effect of the Bluetooth earphone can be improved.

The invention discloses an auxiliary scheme generation method and system for dysphagia rehabilitation, belongs to the technical field of medical rehabilitation intelligent evaluation, and is used for solving the problem of poor optimization of a swallowing training path. According to the method, myoelectricity, tongue pressure, throat images and acoustic signals in the swallowing process are collected through a multi-channel sensing device, a unified multi-mode swallowing behavior feature vector is constructed, obstacle stage recognition and functional grade judgment are achieved in combination with a pre-training model and historical data, and a structured evaluation result is output; matching training actions with high adaptation degree based on the knowledge graph, fusing muscle group matching degree, edge weight and function level factors to generate a personalized training path, and setting strength, frequency and rhythm parameters; in the training process, the motion and myoelectricity deviation is monitored in real time, a prompt signal is output, and the behavior completion degree is recorded; and the rehabilitation progress map is dynamically updated according to feedback, and if the deviation exceeds the limit, the path is automatically reconstructed and doctor intervention is synchronized, so that the rehabilitation efficiency and safety are improved.

The invention discloses a motor fault detection method and system based on voiceprint recognition. According to the method, an annular microphone array is adopted to collect motor sound signals in a non-contact mode, a three-channel time-frequency data set is constructed through empirical mode decomposition (EMD) and a Mel-frequency cepstral coefficient (MFCC), fault diagnosis is carried out in combination with a CNN + ResNet network, and dynamic time warping (DTW) and CNN fusion matching is supported. The system comprises a preprocessing module, a fault template library and a matching algorithm, integrates wavelet denoising and multi-beam acquisition technologies, covers a frequency band of 50Hz-20kHz, can display a fault type and trend analysis in real time, and triggers secondary verification when the confidence coefficient is insufficient. According to the scheme, the anti-interference capability is improved through array signal processing, model parameters are optimized in combination with transfer learning, non-contact detection is achieved, the real-time performance and accuracy of fault diagnosis are remarkably improved, and the method is suitable for industrial motor health monitoring.

A system configured to improve audio processing by performing dereverberation and noise reduction during a communication session. In some examples, the system may include a deep neural network (DNN) configured to perform speech enhancement, which is located after an Acoustic Echo Cancellation (AEC) component. For example, the DNN may process isolated audio data output by the AEC component to jointly mitigate additive noise and reverberation. In other examples, the system may include a DNN configured to perform acoustic interference cancellation, which may jointly mitigate additive noise, reverberation, and residual echo, removing the need to perform residual echo suppression processing. The DNN is configured to process complex-valued spectrograms corresponding to the isolated audio data and / or estimated echo data generated by the AEC component.

The invention discloses an improved multi-modal information fusion extraction method and system, and relates to the technical field of multi-modal information processing, and the system comprises the following components: S1, data collection and preprocessing, S2, construction of a multi-task collaborative learning framework, S3, multi-modal information fusion and multi-task training, and S4, information extraction and application. According to the method, a multi-task collaborative learning framework and a shared information fusion module are constructed in a multi-modal information fusion and multi-task training stage, so that effective fusion and collaborative training of multi-modal data are realized, and rich and universal feature representation in the data is further mined; the improved method not only improves the accuracy of data processing, but also accelerates the processing speed, and provides powerful support for practical application.

The invention relates to an audio noise reduction method, device and system based on deep learning, and the method comprises the steps: obtaining an input audio signal with noise, and carrying out the multi-scale time-frequency decomposition, and obtaining a mixed time-frequency feature and a noise fingerprint spectrum; performing parameter parallel processing on the noise fingerprint spectrum through a preset dynamic kernel generation network, and performing preliminary noise reduction processing on the mixed time-frequency characteristics to obtain noise-reduced mixed data; performing dual-path processing structure construction on the noise reduction mixed data to obtain amplitude optimization data and phase optimization data; performing dynamic time-frequency domain cross fusion on the amplitude optimization data and the phase optimization data to obtain fused audio data; and carrying out differentiable acoustic equation constraint adversarial training on the fused audio data, and carrying out inverse time-frequency transformation processing to obtain a target noise-reduced audio signal. According to the invention, the overall efficiency and effect of audio signal processing can be effectively improved.

The invention relates to an intelligent microphone pickup and speech enhancement method, system and device, and the method comprises the steps: obtaining an audio signal collected by a dual-silicon microphone array, and carrying out the cross-correlation analysis of the audio signal, and obtaining a corresponding human voice signal correlation feature; performing sound source positioning analysis on the audio signal based on the human voice signal correlation feature to obtain a corresponding target human voice signal; performing frequency characteristic analysis on the target human voice signal, and performing segmented dynamic gain processing on the signal according to a preset frequency band range to obtain a corresponding voice enhancement signal; performing noise component adaptive filtering processing on the target human voice signal to obtain a corresponding noise suppression parameter; and inputting the speech enhancement signal and the noise suppression parameter into a preset echo cancellation model for joint optimization to obtain a corresponding output human voice signal. According to the invention, the coupling problem of multiple acoustic interferences can be effectively solved.

The invention relates to an intelligent monitoring alarm method and system for a motor controller, and the method comprises the steps: obtaining an internal circuit signal and a body vibration signal of the motor controller, carrying out the combined noise analysis, and generating a dynamic noise baseline; performing component fault identification based on the dynamic noise baseline and the internal circuit signal to obtain a fault precursor current feature; performing component impact response separation based on the dynamic noise baseline and the body vibration signal to obtain hidden fault vibration characteristics; performing component association mapping on the fault precursor current feature and the hidden fault vibration feature to obtain an internal fault component identifier; and performing alarm analysis based on the fault component identifier, the fault precursor current feature and the hidden fault vibration feature to obtain a graded alarm decision. According to the invention, early fault features can be captured more sensitively, and false alarms and missing alarms are reduced.

The invention relates to the technical field of artificial intelligence, and discloses an English teaching training system and method fusing semantic matching and cognitive assessment, and the method comprises the steps: synchronously capturing a text response, a voice intonation, an eye movement track, a facial micro-expression and a touch rhythm generated in a learning process; semantic deviation deconstruction and cognitive intention quantization processing are carried out on the learning interaction original sequence, and a bidirectional deep semantic matching network is adopted to carry out context alignment on student answers and target corpora; based on the word meaning divergence point set and the cognitive load multi-scale vector, extracting a nonlinear diffusion trajectory of a learning state by using a time gating multi-layer recursive trajectory evolution algorithm; the knowledge point nodes, the deviation type nodes and the emotion triggering nodes associated with the emotion instability candidate segments are fused to construct a local learning map; and forming an emotion cognition feedback result driven by learning interest based on the local learning map and the self-adaptive error correction intervention sequence. The method has the advantage of improving the learning interest of students.

The invention relates to the field of natural language processing. The invention discloses a cross-modal knowledge optimization system for improving localization adaptability of a large language model, and the system comprises a modal data collection module which collects cross-modal localization data of characters, audios and videos; the data preprocessing module is connected with the data acquisition module and preprocesses acquired data; the cross-modal knowledge fusion module is connected with the preprocessing module, fuses data and large language model general knowledge, and performs mining association by means of a cross-modal learning algorithm to form a localized cross-modal knowledge graph; the model fine-tuning module is connected with the fusion module and uses the map to finely tune the large language model; the evaluation feedback module is connected with the fine tuning module and evaluates the localization performance of the model. Through multi-modal data integration, dynamic preprocessing, deep knowledge fusion, efficient fine adjustment and intelligent feedback, various problems in the large language model localization process are systematically solved, and the expression of the model in dialect understanding, cultural questions and answers and cross-modal tasks is remarkably improved.

The noise online monitoring system comprises a multi-source sensing module, a positioning analysis module, a baseline generation module, a state association module and an execution control module. The multi-source sensing module synchronously collects target area parameter signals and generates noise identification signals through coherence analysis; the positioning analysis module receives the noise identification signal and generates a positioning signal containing a noise source position and a motion trail; the baseline generation module calls historical noise spectrum data of a corresponding scene according to a noise source position in the positioning signal to generate an adaptive noise baseline threshold signal; the state association module carries out space-time alignment on the baseline threshold signal and the equipment working condition data; the execution control module receives the positioning signal and the correlation degree signal. The noise on-line monitoring system can solve the problems that a traditional noise monitoring system is inaccurate in positioning, poor in base line adaptability and disjunction between noise and the equipment state.

The invention provides an AI-based virtual digital human interaction system, and relates to the technical field of data processing, and the system is used for obtaining a user feature data set by performing feature extraction on an interaction data set, calculating an audio change rate and speech speed stability, obtaining speech emotion data, calculating syntactic complexity and counting the number of emotion words. Calculating a facial expression symmetry degree; obtaining facial emotion data; calculating body stability and identifying a visual staying area; obtaining attention state data; adjusting tone and speed of the virtual digital person according to the virtual digital person text, optimizing sentence pattern complexity and sentiment words in the virtual digital person text, adjusting changes of eyes, eyebrows and mouth corners of the virtual digital person, and adjusting information display sequence and visual focus guidance. According to the method, the interaction efficiency of the virtual digital human is improved by analyzing the facial expressions and the body actions of the user.

The invention provides a dynamic calibration method and system for a vehicle-mounted emotion recognition system, and the method comprises the steps: S1, obtaining multi-source data which comprises a facial image, a voice signal and a physiological signal; the obtained multi-source data are preprocessed, and preprocessed multi-source data are obtained; s2, performing feature extraction based on the preprocessed facial image, the voice signal and the physiological signal to obtain a facial expression feature vector, an audio feature vector and a physiological state feature vector; s3, evaluating the current environment credibility based on an environment credibility evaluation function; s4, dynamically distributing the weight of the multi-source data according to the credibility of the current environment and the real-time scene; and S5, constructing a multi-modal fusion vector based on the dynamically distributed weight of the multi-source data, the facial expression feature vector, the audio feature vector and the physiological state feature vector, and performing emotion recognition by using the constructed emotion recognition model based on the multi-modal fusion vector.

The invention relates to the technical field of voice processing, can be applied to business scenes of financial science and technology, medical health and the like, and discloses an audio coding and decoding method, device, equipment and medium, and the method comprises the steps: carrying out the sliding window segmentation processing of an input audio signal, and generating signal segments; processing the signal segments through an encoder containing a multi-layer self-attention mechanism to generate continuous potential representations; performing decomposition vector quantization processing on the continuous potential representation to generate a discrete code; the discrete codes are processed through a decoder comprising a multi-layer self-attention mechanism, and reconstructed signal segments are generated; and splicing the reconstructed signal segments to generate a complete audio signal. According to the method, a traditional convolution structure is replaced by a multi-layer self-attention mechanism, global time sequence dependence modeling is carried out on the audio signals after sliding window segmentation, effective compression of potential representation is realized in combination with decomposition vector quantization, and continuity and fidelity of audio reconstruction are improved on the premise that calculation complexity is not increased.

The invention relates to the technical field of artificial intelligence and emotion calculation, in particular to an audio and video dual-mode emotion recognition method and system based on adapter fusion. The method comprises the following steps: acquiring a video frame sequence and an audio signal, and preprocessing the video frame sequence and the audio signal; constructing an emotion recognition model; based on a bimodal feature extraction module, a space adapter and a global adapter are embedded in sequence, and corresponding modal enhanced space features and global features are obtained in sequence; generating intermediate representations of the corresponding modes based on the global features, and performing feature fusion according to the intermediate representations to obtain fusion features of the corresponding modes; the fusion features are spliced, time sequence features are extracted, and final features are obtained; inputting the final features into a classifier to obtain a predicted emotion category, training an emotion recognition model by adopting a loss function, and determining an optimal emotion recognition model; and inputting a to-be-recognized video frame sequence and an audio signal into the emotion recognition model, and outputting a recognition result.

The invention relates to the technical field of voice processing, can be applied to business scenes of financial science and technology, medical health and the like, and discloses a voice enhancement method, device and equipment based on noise perception and a medium. Environment feature information is extracted and input into an audio enhancement model to generate an enhanced audio signal; obtaining a reference audio sample, extracting a personalized feature vector, and carrying out personalized processing on the enhanced audio signal; and collecting playing feedback data, determining a playing time domain adjustment parameter and a playing frequency domain adjustment parameter, adjusting the personalized enhanced audio signal, and generating an optimized audio signal. According to the method, dynamic adjustment is realized in combination with the feedback parameters in the playing process by fusing the environmental perception information and the personalized speaker characteristics, clear and natural optimized audio output with personalized styles can be generated in a complex environment, and the voice interaction quality and adaptability are improved.

The invention discloses a forest ecology monitoring method and system based on sound analysis, and relates to the technical field of sound processing, and the method comprises the steps: obtaining the real-time continuous recording of a forest monitoring area, embedding a timestamp and geographic position information in the recording, and guaranteeing that all recording data have precise space-time identifiers; eliminating a silent part in the recording data by using short-time energy calculation, and dynamically setting an energy threshold value and retaining an effective signal to obtain effective audio data; the effective audio data are converted into a time-frequency matrix through STFT, filter parameters are adjusted in combination with real-time meteorological data, dynamic noise reduction processing is carried out, and a frequency spectrum after noise reduction is obtained; mFCC features are extracted through Mel filter group transformation and DCT, and standardized feature vectors are generated; and performing deep learning classification on the MFCC features by using a pre-trained CNN and LSTM, setting a statistical threshold based on historical data, detecting abnormality, and finally generating output data including a classification result, activity intensity and abnormality early warning information.