28498results about "Speech analysis" patented technology

Systems and methods are provided for machine learning based monitoring. Image data from a camera is received. On the hardware accelerator, a person detection model based on the image data is invoked. The person detection model outputs first classification result. Based on the first classification result, a person is detected. Second image data is received from the camera. In response to detecting the person, a fall detection model is invoked on the hardware accelerator based on the second image data. The fall detection model outputs a second classification result. A potential fall based on the second classification result is detected. An alert is provided in response to detecting the potential fall.

The invention relates to a method and a system for monitoring and analyzing regional noise. The method comprises the following steps: carrying out partition processing on a target area according to an acoustic environment function standard to obtain acoustic environment function area division data and determine a monitoring point position distribution matrix; noise data and environment parameter data are collected, timestamp alignment is carried out, and a multi-dimensional associated data set is obtained; performing spectral analysis and data compensation to obtain a noise feature vector, and performing standard exceeding judgment to generate audio recording data; inputting a voiceprint recognition model for classification to obtain a noise source type identifier, and performing positioning calculation to obtain a noise source space coordinate; carrying out space-time distribution statistics to obtain a functional area noise standard-reaching rate and calculating contribution degrees of various noise sources; and carrying out graded early warning judgment, and generating alarm information and a noise pollution prevention and control instruction. According to the embodiment of the invention, the whole-process automatic management of monitoring and early warning of regional noise is realized.

Herein disclosed is receiving predetermined content, receiving a request to transform the predetermined content into a derivative work, receiving a requested theme for the derivative work, using generative artificial intelligence to create the derivative work generated as a function of the predetermined content and the requested theme, determining if the generated derivative work is approved based on a machine learning model configured to determine a content approval score as a function of content owner preferences, in response to determining the generated derivative work is approved, applying a digital watermark to the approved derivative work, configuring an authorization server to govern use of the approved derivative work based on the digital watermark and providing user access to the authorized derivative work. The requested theme may be determined using a Large Language Model (LLM) and a chatbot interview. The generative artificial intelligence may comprise a diffusion model. The content may comprise music.

A method of wireless communication at a wireless repeater device controlled by a base station in a wireless communication network includes receiving, from the base station, control signaling comprising control information, in association with traffic relayed through the repeater device between a first wireless communication device and a second wireless communication device; and configuring a relay unit of the repeater device to communicate the traffic between the first wireless communication device and the second wireless communication device using at least one of: the time division duplex state, the fronthaul-link TCI state index, the access-link TCI state index, or the time domain resource allocation.

The invention relates to the technical field of environmental sound event detection, in particular to an environmental sound event detection method based on multi-modal data fusion, which utilizes a microphone array, a camera and a sensor to synchronously acquire audio, video and environmental data to form a multi-modal data source. And the collected multi-modal data is preprocessed, so that the data quality and the accuracy of subsequent analysis are improved. And key features are extracted from the processed data, and the features are fused to form multi-modal features. And a deep learning model is constructed, and multi-modal features are used for training, so that a model capable of identifying an environmental sound event can be obtained. And performing optimization such as pruning, quantification and knowledge distillation on the trained model, and deploying the model on edge equipment to realize real-time preliminary feature extraction and event detection. The problems that an environment sound event detection method in the prior art is poor in robustness, low in real-time performance and limited in generalization ability are solved.

The invention relates to a multi-microphone array beam forming signal enhancement method and device, and the method comprises the steps: obtaining a multi-channel time domain signal collected by a multi-microphone array, and carrying out the time-frequency conversion, and obtaining a plurality of time-frequency domain signals; acquiring a spatial distribution vector of the multi-microphone array, and performing deformation compensation processing on the complex number of time-frequency domain signals to obtain sound field consistency signals; performing sound source direction feature extraction on the sound field consistency signal, and performing complex time-space attention beam forming network processing to obtain a beam forming signal; performing beam direction and gain parameter optimization on the beam forming signal through a preset adversarial environment simulator to obtain a dynamic frequency band enhanced signal; and performing inverse time-frequency conversion on the dynamic frequency band enhanced signal, and performing low-delay optimization to obtain a target quality voice output signal. According to the invention, signal deformation compensation can be carried out more accurately.

Disclosed herein is an audio system that can be customized by the user (including the developer) and that may allow the user to control more than just the sound levels and may allow the user to select more than one of a few pre-determined settings. Disclosed herein is a development platform that allows manufacturers (including the developers) to test various different parameters, settings, algorithms, functions, processes, accessories, and the like. The development platform allows the manufacturer to simulate the performances from possible options for the audio system, and then load the selected option from the development platform to the audio system. Additionally, the development platform that enables the end user to have the same level of control and ability to access, upload, and control plugins on the end user's device(s).

The invention belongs to but not limited to the technical field of emotion recognition, and discloses a multi-modal classroom emotion recognition method and system based on modal adaptive learning, and the method comprises the steps: building a multi-modal data collection system, collecting the multi-modal data, such as texts, audios and videos, in a classroom teaching process in real time, and carrying out the real-time collection of the multi-modal data; and preprocessing and feature extraction are carried out on data of different modes. For emotion expression feature differences of different classroom scenes, a multi-modal data fusion method based on adaptive modal scoring (AMS) is provided, and a time sequence modeling and emotion classification model is constructed. The multi-modal fusion and time sequence modeling mechanism simulates the multi-dimensional perception of the teacher on the emotional states of the students in the class, and enhances the interpretability of the emotion recognition system. According to the system, weighted fusion is carried out on data features from texts, audios and videos through fusion of multi-modal data, and complete emotion feature representation is formed.

The invention discloses an adaptive teaching real-time feedback method based on multi-modal fusion, and the method comprises the following steps: synchronously collecting text modal information, voice modal information and image modal information generated by students in a teaching process, forming multi-modal original data information, and extracting historical student interaction behavior data; preprocessing the multi-modal original data information, and respectively generating corresponding text, voice and image sequence features; a visual feature encoder and a sequence feature encoder are adopted to encode each modal sequence feature to obtain a high-dimensional feature; inputting the modal high-dimensional features into a cross-modal fusion network for deep fusion; parameters of the feedback model are optimized through a model-independent element learning feedback regulation and control algorithm, and a personalized feedback strategy is generated; generating comprehensive feature representation according to the fusion features, and outputting personalized teaching feedback; and the interaction information is updated based on the feedback behavior data to realize closed-loop optimization.

The invention provides a digital human interaction control method and device fusing emotion semantics and logical reasoning and a storage medium. The method comprises the following steps: analyzing multi-modal input data of a user, constructing emotion-semantics joint representation, and generating a logic decision path; and through a cognitive fusion module, emotion-semantic representation and a logic decision path are fused, and an interaction response adapting to emotion and logic consistency is generated. The system optimizes an emotion semantic model and a logical reasoning rule on line according to user feedback and interaction history, and real-time interaction of emotion dynamic and logical rules is achieved. The system can dynamically adjust the logic decision path based on the multi-mode emotional state of the user, and improves the naturalness and situation adaptability of interaction. A dynamic time warping algorithm and a factorization machine are introduced to process a multi-modal feature fusion problem, and the accuracy and robustness of emotional state recognition are improved. The online optimization mechanism enables the model and the rule to be evolved continuously, and reasoning errors are corrected automatically through user feedback, so that error circulation is avoided.

The invention relates to a multi-modal digital employee reception system and method based on edge calculation, and relates to the technical field of artificial intelligence, and the system comprises a multi-modal data collection module, an edge calculation module, a data fusion and decision module, a trigger activation module, an emotion recognition module, an interaction response module and a behavior prediction module. The method comprises the following steps: acquiring voice, image and action information of a user through a high-resolution camera, a microphone array and an infrared sensor; locally processing data by using an edge calculation module, and extracting voice, vision and thermal imaging features; a Transformer model is dynamically optimized through Test Time Training (TTT), multi-modal data are dynamically fused in combination with a multi-head attention mechanism, a comprehensive decision vector is generated, the system can accurately judge the emotional state of a user, user behaviors are predicted based on a fused Token sequence and historical records, personalized services are actively provided, and by dynamically adjusting a trigger threshold value and a weight distribution mechanism, the emotion state of the user can be accurately judged. The false triggering rate and the missed triggering rate are reduced, and the stability and the robustness of the system are enhanced.

The invention discloses a low-delay adaptive wireless audio transmission method, and relates to the technical field of audio data transmission. The invention relates to a low-delay self-adaptive wireless audio transmission method, which comprises the steps of audio signal acquisition and coding, priority scheduling and time slot allocation, wireless channel transmission and interference resistance, congestion control algorithm optimization, decoding and reconstruction of a receiving end, and dynamic buffer management and delay optimization. According to the method, low-delay transmission of audio signals is ensured through a plurality of technologies such as priority scheduling, time slot allocation, dynamic buffer management and adaptive coding, the stability of audio transmission is remarkably improved by combining congestion control, forward error correction (FEC) and adaptive interference elimination technologies, and even in a complex wireless environment, the stability of the audio transmission is greatly improved. And the audio signal can keep high quality, and the transmission stability is ensured by quickly responding to channel change, eliminating interference, correcting errors, reducing packet loss and retransmission.

The invention discloses a video feature extraction and multi-dimensional matching-based movie advertisement real-time pushing system, and belongs to the technical field of digital advertisements. According to the system, multi-modal analysis is carried out on visual, audio, text and semantic features of a movie through a video feature extraction module, a dynamic interest tag constructed by a user portrait analysis module is combined, and the weighted matching degree of a video scene, user preference and an advertisement tag is calculated by utilizing a multi-dimensional matching engine; and millisecond-level advertisement putting is completed through the real-time pushing decision module. According to the method, context awareness and streaming computing technologies are fused, the relevance between advertisements and content scenes is remarkably improved, cross-platform deployment is supported, the method is suitable for short videos, live broadcast and other real-time scenes, the advertisement click rate is increased by 55%-72% through tests, the response delay is lower than 200 ms, and the method has the technical advantages of being efficient, accurate and low in delay.

The invention relates to the technical field of data processing, in particular to a multi-modal data alignment method and system, and the method comprises the steps: collecting multi-modal data, and carrying out the preprocessing of the multi-modal data; the preprocessed multi-modal data are sent to an event anchor point detection module, and multi-modal emergency detection, anchor point marking and preliminary global alignment are carried out; performing time resampling and numerical value standardization on the multi-modal data after coarse alignment; mapping the resampled and standardized multi-modal data to a unified time axis; performing local fine alignment and global adjustment on the multi-modal feature sequence after unified time axis mapping; and performing confidence evaluation and anomaly correction on the fine alignment result, and outputting a final alignment sequence. According to the invention, a high-precision and multi-modal data real-time alignment method which is light in weight and has adaptive correction capability can be realized on end side equipment with limited resources.

A method for early diagnosis of Parkinson's disease based on multimodal deep learning is provided. Audio-visual data of a to-be-diagnosed subject while performing a speech task is acquired. The audio-visual data are preprocessed to extract a plurality of audio segments and a plurality of video segments. A face image sequence is extracted from each of the plurality of video segments. A Mel-spectrogram of each of the plurality of audio segments is calculated. The face image sequence and the Mel-spectrogram are input into a multimodal deep learning model to output a classification result for Parkinson's disease early diagnosis of the to-be-diagnosed subject. A system for early diagnosis of Parkinson's disease based on multimodal deep learning is also provided.

The invention relates to the technical field of man-machine interaction, in particular to a robot behavior mode dynamic adjustment method based on multi-mode perception, which comprises the following steps: S1, collecting a visual image, a voice signal and an environment parameter of a target scene in real time; s2, extracting each modal feature; s3, dynamically allocating weights, and generating a fusion feature vector; s4, identifying a current scene type and a user attribute; s5, matching a corresponding interaction mode from a preset strategy library based on the scene type and the user attribute identified in the step S4; and S6, executing the matched interaction mode in the S5, and updating the weight distribution rule according to the feedback data. According to the method, vision, voice and environment characteristics are fused through a multi-mode perception technology, the behavior mode of the robot is dynamically adjusted based on a self-adaptive weighted fusion algorithm, and the interaction strategy is optimized in combination with user feedback, so that the interaction accuracy and the intelligent level of the robot under different scenes and user attributes are improved.

The invention discloses a multi-modal data real-time identification and cooperative processing system based on edge computing and federated learning. The multi-modal data real-time identification and cooperative processing system comprises a cloud center coordination node, a plurality of edge computing nodes, a cross-modal encryption engine, a federated learning controller and a model updating verification module. The cloud center coordination node executes federated learning model aggregation and dynamic task allocation, and generates a cross-modal encryption strategy; and the edge computing node is configured with a multi-modal data acquisition module, a local model training unit and a co-processing gateway to realize multi-modal data acquisition and local processing. The system encrypts vision, acoustics and text data by using differentiated algorithms such as spatial confusion, frequency domain permutation and homomorphic encryption; the federated learning controller carries out multi-modal feature fusion, hierarchical encryption and dynamic networking at the edge node; and the model updating verification module performs aggregation updating after ensuring parameter consistency by using secure multi-party calculation. According to the method, real-time processing and privacy protection of multi-modal data are realized, and the data co-processing efficiency is improved.

The invention relates to the technical field of intelligent robots, in particular to a service robot intelligent emotion interaction method, which comprises the following steps: acquiring emotion related data of a user through a multi-modal sensor on a service robot, analyzing the acquired data based on a deep learning model, and extracting emotion features of the user; future emotion changes are predicted through a time sequence analysis algorithm, a service robot generates a personalized interaction strategy through an emotion decision-making system according to the current and predicted emotion states, and the system adopts a self-adaptive algorithm based on reinforcement learning, so that the interaction strategy can be continuously optimized along with the emotion changes of a user; the robot performs modeling and optimization on long-term interaction data by utilizing a self-learning mechanism in combination with emotional feedback of the user, so that the precision and interaction effect of emotional response are gradually improved; according to the method, the problems of inaccurate emotion recognition, inflexible interaction strategy and lack of adaptive optimization in the prior art are effectively solved, and the interaction experience of the service robot and the user is remarkably improved.

A computer implemented system for authenticating an audio conversation occurring between a human target and an unknown speaker. The system adjusts messages sent to the speaker to include characteristics that evoke different responses in bots controlling vocoders to generate deepfakes than are evoked in humans. By analyzing these responses, harms from deepfake threats can be avoided by turning the tools used by deepfake generating bots against those very bots.

The invention relates to the technical field of audio and video analysis and data processing, in particular to an AI-based multimedia equipment operation and maintenance management system, which comprises a data acquisition module, an analysis module, an equipment state modeling module, a fault prediction module, a maintenance strategy optimization module, a resource allocation module and a visual platform. The data acquisition module acquires temperature, vibration, audio and video feature data of the equipment through the sensor and the video equipment, and the data are preprocessed and then transmitted to the cloud; the equipment state modeling module constructs a dynamic baseline model based on historical data, abnormal signals are generated through real-time comparison, and the visual platform realizes multi-dimensional state monitoring through 3D topology rendering, AR auxiliary diagnosis and early warning linkage. The system solves the problems of response lag and low resource utilization rate caused by dependence on manual operation and maintenance in the prior art through a closed-loop management mechanism driven by a data stream, and improves the fault diagnosis precision and the operation and maintenance efficiency.

The invention discloses a self-adaptive education method based on artificial intelligence, and relates to the technical field of education, and the method comprises the following components: intelligent emotion monitoring, learning performance evaluation, self-adaptive teaching strategy adjustment, personalized learning path planning and a feedback and optimization mechanism. According to the method, by introducing a learning performance evaluation and prediction module and a prediction and adaptive teaching strategy adjustment module, individual differences, including learning styles, interest preferences and ability levels, of students are fully considered, so that teaching strategies are intelligently recommended and adjusted, and personalized learning paths are dynamically generated; the personalized teaching mode effectively overcomes the defects of a traditional'one-cut 'teaching mode, the teaching effect is remarkably improved, each student can obtain the most suitable learning experience, and the learning enthusiasm and the learning effect of the students are greatly improved.

The invention relates to the technical field of artificial intelligence, can be applied to business scenes of financial science and technology, medical health and the like, and discloses a decision-making method, device, equipment and medium based on multi-modal semantic alignment. Executing cross-modal alignment by taking the voice semantic map as a reference to generate associated information, fusing the voice features, the visual features, the action features and the associated information to generate a fusion feature vector, inputting a decision network to generate a decision feature vector and generate a task execution instruction, obtaining execution feedback information of the task execution instruction, and updating the decision network. According to the method, input is dominated by voice instructions, visual features, action features and semantic map depth alignment and fusion are combined, input naturalness and multi-modal data analysis and decision-making efficiency are improved, and interaction adaptability and decision-making accuracy of the model in a complex scene are improved.

The invention discloses a medical whole-course intelligent management system based on a large model, and belongs to the technical field of large models. Comprising a multi-modal data acquisition module, a privacy calculation preprocessing module, a dynamic knowledge enhancement module, a time sequence data analysis module, an intelligent decision engine module, a multidisciplinary collaboration module, a patient interaction platform module, a dynamic intervention feedback module and a system security center module. The cross-mechanism data security sharing is realized, and the compliance of sensitive information processing is also ensured; a two-channel medical knowledge base is constructed, authoritative guidelines can be synchronized, newest clinical research data can be analyzed in real time, the knowledge base is kept in the newest state all the time, and the frontier scientific basis is provided for clinical decisions; dynamic modeling and trend prediction are carried out on long-term monitoring data of a patient by adopting a hybrid neural network model, and potential health risks and development trends can be identified more accurately.

A device capable of using data from multiple sensors to determine an estimated position / direction of a user with respect to the device. The device may use estimated position data, along with confidence data, that originated from a plurality of sensors to fuse the data to determine the user's estimated position and comprehensive confidence of the estimated position. The system may use the location information to perform beamforming / beam steering and / or other downstream operations using the comprehensive estimated position.

This disclosure describes techniques that include assessing trust in a system, and in particular, assessing trust by performing a sentiment analysis for an entity or device within a system. In one example, this disclosure describes a method that includes performing, by a computing system and based on information collected about a network entity in a computer network, a sentiment analysis associated with the network entity; determining, by the computing system and based on the sentiment analysis, a trust score for the network entity; and modifying, by the computing system and based on the trust score for the network entity, network operations within the computer network.

The invention relates to the technical field of audio noise reduction, and discloses a Bluetooth headset environment noise adaptive adjustment method and system, and the method comprises the steps: collecting multi-band sound data of headset environment noise through a preset acoustic sensor array, generating a noise spectrogram, extracting a noise masking threshold through time-frequency joint analysis, and collecting a headset audio signal; next, noise suppression is carried out on the audio signal by using a multi-order digital filter bank to obtain a primary noise reduction signal, the primary noise reduction signal is optimized into an optimized noise reduction signal based on a signal-to-noise ratio, and the frequency response matching degree of the optimized noise reduction signal and the original audio signal is calculated; then, performing phase correction and amplitude-frequency equalization processing according to the frequency response matching degree to obtain an audio equalization parameter; and finally, monitoring residual noise energy based on the parameter, generating a noise regulation and control instruction, driving adaptive sampling, and generating an adaptive noise reduction scheme. According to the invention, the audio noise reduction effect of the Bluetooth earphone can be improved.

The invention discloses an auxiliary scheme generation method and system for dysphagia rehabilitation, belongs to the technical field of medical rehabilitation intelligent evaluation, and is used for solving the problem of poor optimization of a swallowing training path. According to the method, myoelectricity, tongue pressure, throat images and acoustic signals in the swallowing process are collected through a multi-channel sensing device, a unified multi-mode swallowing behavior feature vector is constructed, obstacle stage recognition and functional grade judgment are achieved in combination with a pre-training model and historical data, and a structured evaluation result is output; matching training actions with high adaptation degree based on the knowledge graph, fusing muscle group matching degree, edge weight and function level factors to generate a personalized training path, and setting strength, frequency and rhythm parameters; in the training process, the motion and myoelectricity deviation is monitored in real time, a prompt signal is output, and the behavior completion degree is recorded; and the rehabilitation progress map is dynamically updated according to feedback, and if the deviation exceeds the limit, the path is automatically reconstructed and doctor intervention is synchronized, so that the rehabilitation efficiency and safety are improved.

The invention discloses a motor fault detection method and system based on voiceprint recognition. According to the method, an annular microphone array is adopted to collect motor sound signals in a non-contact mode, a three-channel time-frequency data set is constructed through empirical mode decomposition (EMD) and a Mel-frequency cepstral coefficient (MFCC), fault diagnosis is carried out in combination with a CNN + ResNet network, and dynamic time warping (DTW) and CNN fusion matching is supported. The system comprises a preprocessing module, a fault template library and a matching algorithm, integrates wavelet denoising and multi-beam acquisition technologies, covers a frequency band of 50Hz-20kHz, can display a fault type and trend analysis in real time, and triggers secondary verification when the confidence coefficient is insufficient. According to the scheme, the anti-interference capability is improved through array signal processing, model parameters are optimized in combination with transfer learning, non-contact detection is achieved, the real-time performance and accuracy of fault diagnosis are remarkably improved, and the method is suitable for industrial motor health monitoring.

The present disclosure provides for an audio analytics system. The audio analytics system may comprise at least one audio capture device configured to capture at least one audio source emitted from at least one origin. By executing one or more operations on the audio source, the audio analytics system may identify one or more potential origin characteristics related to the origin of the audio source. The identified potential origin characteristics may be compared to one or more expected origin characteristics of the origin of the audio source to determine one or more origin characteristic results. If at least one discrepancy exists between the identified potential origin characteristics and the expected origin characteristics, the audio analytics system may identify the audio source as being potentially fraudulent.

Method, system, device, and non-transitory computer-readable medium for presenting a conversation. A computer-implemented method may include: obtaining, via a first virtual participant, a first set of audio data associated with the first conversation while the first conversation occurs; transcribing the first set of audio data into a first set of text data while the first conversation occurs; obtaining a set of annotations associated with the set of text data while the first conversation occurs; identifying one or more topic transitions based at least in part upon the set of text data; generating a conversation summary based at least in part upon the one or more topic transitions; obtaining a first set of visual data associated with the conversation; and presenting the set of annotations, the conversation summary, and the first set of visual data embedded in the first set of text data to the first group of actual participants.