31 results about "Time alignment" patented technology

Disclosed in the present invention are an automatic delay calibration method for a smart speaker system, a device, and a storage medium. The method comprises: initializing a smart speaker system and performing environmental testing to preliminarily configure system parameters; exciting a target sound channel and playing a target test signal; locating a first test signal on the basis of a voice activity detection (VAD) algorithm, determining the start and end of the first test signal, and identifying a time difference of the first test signal to obtain a delay of main channels; locating a time of arrival of a second test signal on the basis of a fast peak search algorithm, and identifying a delay of a subwoofer; and performing delay calibration on the subwoofer and the main channels, and dynamically adjusting delay calibration parameters of the channels on the basis of a delay calibration amount, thereby achieving accurate time alignment between the subwoofer and the main channels. The present invention solves the problem of low-frequency signal delays that are difficult to process in conventional technologies, and particularly makes a breakthrough progress in the coordination of a subwoofer and other sound channels.

The invention relates to the field of fan spindle state monitoring, and aims at synchronously acquiring sound and vibration signals through multiple channels, realizing nanosecond time alignment by adopting a precision time protocol, and performing denoising and normalization preprocessing on the signals to improve data integration and signal fidelity. Furthermore, short-time Fourier transform and continuous wavelet transform are combined to extract multi-scale time-frequency features, and a high-dimensional combined feature vector is generated in combination with cross-correlation analysis. And mapping the feature vectors to a low-dimensional manifold space through a local linear embedding algorithm, and constructing a dynamic mode reference template. Indexes such as curvature, track length and direction entropy are monitored in real time, whether the spindle has an abnormal evolution trend or not is judged through a self-adaptive curvature threshold, and abnormity judgment is achieved in combination with track backtracking verification. According to the scheme, the abnormal starting boundary of the spindle state can be caught in a refined mode, and the early warning and operation and maintenance response capacity of fan operation is effectively improved.

The invention provides a multi-modal data real-time analysis and feedback method and system, and the method comprises the steps: collecting a video frame and an audio frame, reading a count value of the same monotonic timer when the collection of the video frame and the audio frame is completed, and generating an audio and video sequence; calculating the behavior popularity of each region in each time slice based on the sequence, and generating a region set for multi-modal event analysis in combination with a preset threshold and a quantity upper limit; scheduling the corresponding video sub-blocks to a visual computing power unit for target positioning and action classification, and executing voice activity detection and keyword category judgment on the time-aligned audio clips at the same time; visual and audio results are fused, and a structured event sequence is generated according to time slice and region compression; and constructing a classroom teaching chain through the sequence, identifying a key event, and finally generating a teaching event description. According to the invention, under the condition of domestic chip combination, cost, time delay, multi-modal consistency and data security controllability are considered, and real-time perception and feedback of teaching behaviors are effectively supported.

This application discloses a distributed audio and video synchronization method based on multi-field coupling, relating to the field of distributed audio and video synchronization technology. It obtains the time progress field, rhythm field, and semantic field through multi-source node state data broadcast by neighboring nodes, and obtains the time progress error, rhythm error, and semantic error based on these fields. The optimal playback speed is determined based on the time progress error, rhythm error, and semantic error, and the synchronization mode is determined based on the semantic error. Finally, an audio and video synchronization control strategy is generated based on the optimal playback speed and synchronization mode. This strategy is executed, and the process enters the next synchronization cycle until the convergence condition is met, completing synchronization. Through multi-field coupling, synchronization from time alignment to perceptual consistency is achieved, balancing playback continuity, rhythm consistency, and semantic consistency, thus improving the audio and video synchronization effect and user experience in a distributed environment.

The invention discloses a multi-sensor monitoring system and method based on time alignment, and belongs to the technical field of multi-sensor monitoring, and the system comprises a sensor collection module which is used for obtaining an electrocardiosignal, a photoelectric volume pulse wave signal, a heart sound signal and a respiratory impedance signal containing a main clock receiving timestamp; the time alignment module is used for performing acquisition time alignment on the electrocardiosignal, the pulse oximeter signal, the heart sound signal and the respiratory impedance signal containing the main clock receiving timestamp to form the electrocardiosignal, the pulse oximeter signal, the heart sound signal and the respiratory impedance signal which are in time alignment; the multi-modal fusion analysis module is used for extracting local morphological features in the electrocardiosignal, the pulse oximeter signal, the heart sound signal and the respiratory impedance signal after time alignment to form a feature map, and extracting dynamic weights between different signals and between different time points / features in the same signal; according to the invention, the interpretability and accuracy of judgment are improved.

The invention discloses an audio automatic detection system and method based on multi-dimensional calibration, and the method comprises the steps: arranging recording equipment through a standardized recording position rule, carrying out the calibration of space parameters through a range finder, an angle ruler and other tools, and guaranteeing the consistency of collected signals; a multi-channel acquisition architecture is constructed to support synchronous recording and detection of multiple devices; by comparing multi-dimensional acoustic feature extraction of standard audio and recorded audio and adopting a two-stage time alignment mechanism, similarity is accurately calculated to identify abnormities such as sound break and sound break, and a detection result is formed. And finally, analyzing and generating a report based on a detection result, and performing visual display and automatic pushing. The accuracy, efficiency and reliability of audio detection are remarkably improved, and the method is particularly suitable for a batch automatic detection scene before consumer electronics leave a factory.

The invention belongs to the technical field of director, and particularly relates to an AI director assistant method and system capable of implementing an intelligent interaction effect. The method comprises the following steps: synchronously accessing multiple paths of audio and video signals and carrying out space-time alignment to generate a synchronized audio and video stream set and space mapping parameters; and carrying out hierarchical analysis and semantic feature extraction on the audio-visual content to form an individual behavior portrait. And identifying a social function role and predicting a behavior intention based on the individual behavior portrait. A scene significance scoring system is constructed according to social function roles and behavior intentions, and a key scene with a propagation value is automatically identified. And finally, candidate lens schemes are generated and sorted based on the key scene, and lens switching and dynamic composition adjustment are executed. According to the method, automatic closed loop from signal perception to behavior understanding to picture output is realized, the event response time is remarkably shortened, the accuracy of wonderful instant capture and the continuity of picture presentation are improved, and the problems of response delay and one-sided judgment existing in manual director are effectively solved.

The application discloses an audio quality automatic scoring method and system combining MFCC and time domain statistical features, comprising the following steps: on the basis of resampling and time alignment of a reference signal and a to-be-detected signal, extracting cepstrum and time domain statistical features according to second segmentation and further 100 ms subsegment extraction, forming 170-dimensional second-level features through mean value+maximum value hierarchical aggregation of subsegment features, splicing the corresponding reference segment into 340-dimensional joint features, inputting the pre-trained support vector machine model to output a second-by-second discrete score of 0 to 5, and realizing high-time-resolution automatic quality evaluation and abnormal positioning under the conditions of light computing power and small samples.

Disclosed are apparatuses, systems, and techniques that use one or more artificial intelligence models for time-aligned automatic speech recognition (ASR) of speech. The techniques include processing, an ASR model, one or more audio frames representative of a speech to generate, for a transcription unit (TU) of the speech a first set of likelihood values and a second set of likelihood values. An individual likelihood value of the first set characterizes a probability that the TU corresponds to a vocabulary token. An individual likelihood value of the second set characterizes a probability that the TU corresponds to a timestamp token. The techniques further include generating, using the first set of likelihood values and the second set of likelihood values, a timed transcription of the speech.

The application discloses a psychological condition recognition method based on multi-modal data, comprising the following steps: S1, using a linear space-time detector to improve the quality of video and audio features; S2, using a cross-modal time aligner to ensure time synchronization between modes; S3, using Transformers to detect key segments, and then obtaining the final prediction result through a linear layer; and inputting the prediction result. The problems that the existing deep learning method has low recognition accuracy for psychological conditions are solved.

The application relates to the technical field of user behavior analysis, and discloses a hearing aid use behavior analysis system based on multi-dimensional time sequence characteristics, which comprises a time alignment module, parameter channel data, environment channel data and behavior channel data are acquired, the data are sorted according to server time alignment to form multi-channel time sequence data, asynchronous acquisition is continuously represented on the same time axis, a feature statistical module is used for calculating short-term features of statistical quantities in a preset time window, long-term features of wearing time and average hearing aid power are statistically calculated according to natural days, the features are combined to form a multi-dimensional time sequence feature sequence, scene and operation changes are stably described, a channel modeling module is driven by the sequence to form a multi-channel time sequence model, user behavior feature representation and hearing loss trend prediction results are output, the short-term fluctuation and the long-term evolution are connected, a behavior clustering module is used for clustering to form a user behavior portrait according to the representation, and individualized test and fitting suggestions are generated in combination with the hearing loss trend prediction results and the parameter channel data.

The invention relates to a key travel control method and system for a mechanical keyboard and electronic equipment, and the method comprises the steps: carrying out the time alignment and feature extraction of collected pressure data, audio data, vibration data, posture data and application data, and generating a multi-dimensional feature vector; inputting the multi-dimensional feature vector into a self-attention converter model, and outputting a scene classification identifier and triggering probability prediction; searching in a hand feeling mapping library according to the scene classification identifier to obtain a target key travel parameter and a target damping parameter, and generating a target control instruction according to the trigger probability prediction, the target key travel parameter and the target damping parameter; and the magnetorheological fluid damper and the linear brake respond to the target control instruction, the magnetorheological fluid damper is controlled to reach target damping, and the linear brake is controlled to stretch out and draw back to the target position. Therefore, the problems that a mechanical keyboard is fixed and single in hand feeling, cannot adapt to dynamic changeable scenes, cannot meet personalized requirements of users and the like are solved, the keyboard input efficiency and the operation accuracy are improved, and the use experience is improved.

The invention discloses a decoupling type recording and broadcasting method and system based on a key frame index and difference additional recording, and the method comprises the steps: firstly continuously collecting multiple paths of audio and video streams, segmenting the audio and video streams into video fragment files, meanwhile, carrying out the parallel real-time detection of a board key event, and generating a key frame index file in which an event type and a millisecond timestamp are recorded; after class, according to the effective starting and ending time of the actual course and the index file, precisely extracting videos of corresponding time periods from the stored fragments; if fragment missing is found, fragments of other machine positions in the same time period are searched based on indexes, patch video fragments are generated through frame-level difference encoding, finally, the patch video fragments and normal video fragments are spliced to form a complete classroom record, and through a decoupling architecture of full-amount recording, event indexing, post-event accurate extraction and additional difference recording, the complete classroom record is obtained. According to the invention, the problem of missed recording and wrong recording caused by equipment delay, class schedule change or recording interruption in the prior art is solved, and the integrity of recorded content, the time alignment precision and the system fault-tolerant capability are remarkably improved.

The embodiment of the invention relates to the technical field of unmanned station equipment fault diagnosis, and provides an unmanned station equipment fault pre-diagnosis method and system based on voiceprint recognition. According to the embodiment of the invention, accurate time alignment is carried out on the original audio stream and the telemetering stream, a parallel network is adopted to extract acoustic and working condition features, deep interaction and time dimension aggregation of the working condition-acoustic features are realized through a Transformer type fusion encoder taking the working condition as a condition, and finally the fault probability is output by a classification head. And the self-adaptive representation and discrimination of the voiceprint under the dynamic working condition are realized. By means of the mode, false alarms caused by normal working condition changes can be remarkably reduced, the recognition rate and diagnosis precision of early-stage tiny fault acoustic symptoms are improved, and meanwhile real-time performance and robustness are considered.

The invention discloses a centralized management and control software platform based on multi-source heterogeneous data fusion, and relates to the technical field of data management and control, first, real-time interconnection of video and audio collectors is ensured by using a heartbeat monitoring mechanism, and position coordinates of the collectors are determined through feedback information; then, a video-audio interconnection mapping network is constructed, and a foundation is laid for subsequent data processing; in a data review stage, accurately positioning abnormal videos and audio segments by analyzing a video frame pixel difference rate and an audio decibel value waveform curve; then, performing time alignment and matching on the abnormal segments to generate a high-value data set; and finally, surveying and mapping a collection area of the collector by means of a plane-coordinate system, and locking target position coordinates of the audio and video source in real time by combining information such as a sound source direction angle and the like, thereby assisting efficient acquisition of key information.

The embodiment of the application provides a multi-task speech enhancement method and device, which utilizes a speech enhancement model to perform feature extraction processing on two signals of a microphone signal and a reference signal, utilizes a dynamic time delay alignment module to perform adaptive time alignment on the two signals, utilizes an adaptive gating module to fuse the features extracted from the two signal branches, extracts multi-scale time features and frequency domain features from the time and frequency dimensions, and improves the accuracy of noise feature extraction, echo feature extraction and speech feature extraction. The application can jointly perform noise suppression and echo cancellation tasks, realize collaborative optimization of multi-task speech enhancement, improve speech call quality, reduce required computing resources, can realize lightweight deployment, and can be suitable for resource-limited application scenarios.

Various aspects of the present disclosure relate to methods and apparatuses for sounding reference signal (SRS) configuration with validity area. According to an embodiment of the present disclosure, a user equipment (UE) can include: at least one memory; and at least one processor coupled with the at least one memory and configured to cause the UE to: receive a first configuration including an area-specific time alignment timer associated with an SRS validity area; and transmit an indication to acquire, activate, or deactivate a second configuration associated with the SRS validity area when the UE is in a non-connected state, wherein the second configuration includes an SRS configuration, or a time alignment configuration, or a timing advance (TA) command.

The invention relates to the technical field of user behavior analysis, and discloses a hearing aid use behavior analysis system based on multi-dimensional time sequence characteristics, which comprises a time alignment module for acquiring parameter channel data, environment channel data and behavior channel data, and sequencing the data into multi-channel time sequence data according to server time alignment; the characteristic statistics module calculates short-term characteristics of statistical magnitude in a preset time window and counts long-term characteristics of wearing duration and average hearing aid power according to natural days, the short-term characteristics are combined into a multi-dimensional time sequence characteristic sequence, scene and operation changes are stably described, and the real-time performance of the hearing aid is improved. The channel modeling module drives a multi-channel time sequence model through the sequence, user behavior feature representation and hearing loss trend prediction results are output, the penetration from short-term fluctuation to long-term evolution is achieved, and the behavior clustering module forms a user behavior portrait according to representation clustering. And personalized test matching suggestions are generated in combination with the hearing loss trend prediction result and the parameter channel data.

The embodiment of the invention provides a conference pickup method, a terminal, a system and a computer storage medium, and the method comprises the steps: a main conference terminal carries out the collection of audio data in a conference process, and obtains main audio data; receiving slave audio data collected for the conference and sent by a slave conference terminal corresponding to the master conference terminal; obtaining time delay information between the master conference terminal and the slave conference terminal, and performing time alignment processing on the master audio data and the slave audio data based on the time delay information to obtain aligned master audio data and aligned slave audio data; and selecting target audio data from the aligned main audio data and the aligned slave audio data to obtain a conference pickup result based on the target audio data. According to the embodiment of the invention, the method can effectively improve the definition of conference pickup, provides high-quality audio experience, and enables participants to be more focused on the conference content.

To provide time alignment of encoded data of an audio encoder with associated metadata, such as spectral band replication (SBR) metadata.SOLUTION: An audio decoder (100, 300) configured to determine reconstructed frames of an audio signal (237) from access units (110) of a received data stream is described. The access unit (110) includes waveform data (111) and metadata (112), where the waveform data (111) and the metadata (112) are associated with a same reconstructed frame of the audio signal (127). An audio decoder (100, 300) comprises a waveform processing path (101, 102, 103, 104, 105) configured to generate a plurality of waveform subband signals (123) from waveform data (111), and a metadata processing path (108, 109) configured to generate decoded metadata (128) from metadata (111).SELECTED DRAWING: Figure 1

The invention discloses a live broadcast multi-language interpretation method, device and system and a storage medium, and relates to the technical field of live broadcast, and the method comprises the steps: separating a video content obtained from a video collection terminal into a picture stream and an original sound audio stream; a translated audio stream of the original audio stream is obtained, the translated audio stream has at least two versions, and different versions correspond to different language types; performing time alignment processing on the original sound audio stream and the translated audio stream; determining a first language type selected by the first playing terminal; and sending the first translated audio stream and the picture stream corresponding to the first language type to the first playing terminal. Compared with the traditional implementation of pushing the stream containing multiple languages to the playing terminal, the method has the advantages that the amount of data needing to be pushed is greatly reduced, and the stream pushing efficiency can be effectively ensured; moreover, according to the scheme, time alignment processing is performed on the translated audio stream and the original audio stream, so that synchronous matching output of the plug stream picture and the audio of the playing terminal can be ensured.

The invention discloses an unmanned mine card operation state voiceprint monitoring system, method and device. The system comprises a three-dimensional space mapping module, a time synchronization module and an abnormity positioning module. Voiceprint and image acquisition devices are symmetrically arranged along the two sides of a mine road, and an accurate space model is constructed by utilizing three-dimensional laser scanning, so that full coverage and no blind area of a monitoring area are ensured. Microsecond-level time synchronization is realized by adopting a precise time protocol supporting a hardware timestamp, and time-space alignment of voiceprint and image data is ensured by combining a linear model to compensate clock drift. Voiceprint spatial-temporal characteristics are extracted based on a time-aware bidirectional multi-scale network, anomaly detection is carried out through a mahalanobis distance, and finally, the real-time position, speed and three-dimensional coordinates of a part are fused, so that accurate traceability from abnormal voiceprint to a specific fault part is realized. According to the method, the defects of single dimension, misalignment of space-time correlation and fuzzy positioning of the existing monitoring technology are overcome, and the timeliness and the maintenance efficiency of fault early warning are remarkably improved.

The present disclosure provides a method and device for processing audio and video, equipment, medium and product, relating to the technical field of audio and video processing, which comprises: obtaining a first timestamp of a server, taking the time corresponding to the first timestamp as a reference time, and the server is connected with at least one audio and video collection terminal; based on the reference time, a corresponding time calibration parameter is established for the audio and video stream of each audio and video collection terminal; according to the time calibration parameter, the local time of each audio and video collection terminal is mapped to the reference time to time-align each audio and video stream. The present disclosure takes the server time as the unified reference and establishes the time calibration parameter, maps the scattered local time to the unified reference time, thereby ensuring the alignment of multiple audio and video streams in the time dimension, and providing a reliable time reference for subsequent cross-validation and positioning cheating behavior.

The invention relates to the technical field of video processing, and provides a full-automatic time sequence video subtitle arrangement method and device based on a multi-mode large model. According to the method, video materials and reference copywriting are matched with preset cue words to be input into a multi-modal large model, and a time-sharing division result of format output is obtained; according to the time-sharing division result, cutting the video material on a time axis, and generating audio sentence-by-sentence with specified timbre for the reference copywriting; and performing audio and video integration on the clipped video material and the generated sentence-by-sentence audio to obtain final output, thereby solving the problems that automatic time-sharing alignment of the video clip and the corresponding copywriting content is difficult to realize, full-automatic subtitle arrangement cannot be realized, and the efficiency and the quality of video production are limited in the prior art.

The invention provides an audio and video synchronous processing method and system for a weak network environment, and relates to the technical field of data processing, and the method comprises the steps: obtaining request data sent by a client through a cloud server, and generating response data based on the request data; dividing the response data into dialogue texts and facial action parameters; the dialogue text and the facial action parameters are transmitted to an anti-jitter buffer area of a local end through a first network and a second network respectively; rendering and generating a video frame based on the facial action parameters, and generating an audio frame based on the dialogue text; and performing time alignment on the video frame and the audio frame to obtain a played video. The data volumes of dialogue texts and facial action parameters are far smaller than those of audio streams and video streams, and delay differences caused by size differences of data packets are reduced. The influence of network jitter on data transmission is eliminated through the anti-jitter buffer area; rendering and time alignment are carried out according to local time without depending on a network arrival time sequence, and the synchronization problem caused by different transmission delays of audio and video streams is solved.

The invention provides a time-space synchronous acquisition method for video sound wave data, which belongs to the technical field of power transmission and transformation engineering, and comprises the following steps: synchronously acquiring video images and sound wave signals, and improving the sound wave measurement precision by adopting cross-correlation time delay estimation and IIR (Infinite Impulse Response) notch filtering; a double-layer game optimization model and a forward clustering method are utilized to realize heterogeneous data stream time alignment, non-uniform motion state parameters of a probe are estimated based on unscented Kalman filtering recursion, and a B-spline curve is adopted to fit a nonlinear mapping function of a depth position and time to correct spatial positioning deviation in a variable speed scene. Characteristic level semantic verification is achieved through a multi-mode space-time alignment mapping model, a synchronous alignment data set containing accurate space-time annotation is output, and the problem that due to non-uniform-speed motion of a probe, the depth position and time are in a nonlinear relation, and consequently video sound wave data space positioning error accumulation is caused is solved.

The present document relates to time-alignment of encoded data of an audio encoder with associated metadata, such as spectral band replication (SBR) metadata. An audio decoder (100, 300) configured to determine a reconstructed frame of an audio signal (237) from an access unit (110) of a received data stream is described. The access unit (110) comprises waveform data (111) and metadata (112), wherein the waveform data (111) and the metadata (112) are associated with the same reconstructed frame of the audio signal (127). The audio decoder (100, 300) comprises a waveform processing path (101, 102, 103, 104, 105) configured to generate a plurality of waveform subband signals (123) from the waveform data (111), and a metadata processing path (108, 109) configured to generate decoded metadata (128) from the metadata (111).

The application relates to an artificial intelligence-based legal mediation data identification recording system and method, and relates to the field of data processing, which comprises the following steps: collecting multi-modal data in a legal mediation process, performing time alignment processing, and obtaining text data, voice data and image data; performing semantic identification on the voice data, obtaining voice text data, performing time sequence verification analysis on the text data, and obtaining a text fuzzy coefficient sequence; performing emotion identification according to the voice data and the image data, obtaining a voice emotion sequence and an image emotion sequence, and fusing and calculating to obtain an emotion parameter sequence; performing time sequence verification analysis according to the voice emotion sequence and the image emotion sequence, obtaining an emotion fuzzy coefficient sequence, labeling the text data in combination with the text fuzzy coefficient sequence, and recording. The application solves the technical problems that legal mediation recording data is not accurate enough, emotions cannot be expressed, and records are not accurate and intuitive in the prior art.

The application discloses a multi-channel synchronous decoding control system based on a remote IPTV signal virtual tunnel, and relates to the technical field of video transmission synchronous control, and specifically comprises the following modules: a global semantic module, a synchronous level module, a tunnel allocation module, a synchronous management module, a synchronous dispatching module and a playback correction module; key frame detection, audio fingerprint extraction and subtitle alignment are performed on a plurality of IPTV signal streams, semantic events are generated and merged, a global time axis and a unified semantic label are established and clustered; a synchronous level record of each IPTV signal stream is generated and updated; a candidate tunnel set is selected for each IPTV signal stream according to tunnel preferences, a time window scheduling table is generated and issued; device node clocks are collected and corrected, limited clocks are marked based on time difference data, and a local compensation strategy is issued; a receiving end decoding buffer is established and a decoding start signal aligned by time is triggered; transmission quality is monitored in real time, tunnel configurations are updated, and logs are recorded.