Patents
Literature
Patsnap Eureka AI that helps you search prior art, draft patents, and assess FTO risks, powered by patent and scientific literature data.

1490 results about "Time stamping" patented technology

A system for the temporal ordering of multi-channel data streams

ActiveDE202025100817U1TransmissionData packData stream
A system for the temporal ordering of multi-channel data streams, comprising the following: a multi-channel data reception module configured to receive real-time data streams from multiple sources, extract temporal metadata, and detect discrepancies in timestamps; a time synchronization module configured to align data packets using timestamp normalization, adaptive buffering, and AI-based predictive modeling; an AI-powered predictive modeling module trained to estimate missing or delayed timestamps and to reorder out-of-order data packets based on learned patterns; an adaptive buffer module configured to dynamically adjust buffer sizes based on real-time data arrival rates and network conditions; an ordering and output processing module configured to reconstruct the correct order of the data packets and transmit them to a processing or analysis system; and a module for error detection and correction that is configured to detect inconsistencies in timestamps, transmission delays and missing data points, thus ensuring increased reliability of the ordered data sequence.
Owner:AGRAWAL SUMIT KUMAR BOTHELL

Dust control method in calcium carbonate manufacturing process

The invention provides a dust control method in a calcium carbonate manufacturing process, which comprises the following steps: preprocessing a data set of dust monitoring data and process parameters, fusing data with different sources, different formats and different timestamps, and normalizing the data into a time sequence analysis form; under preset production efficiency, cost control and quality stability constraint conditions, optimizing and searching equipment decision variable parameters, a product formula and a transportation mode to obtain an equilibrium solution of dust control and production management, and forming an improved process scheme; and the optimized process scheme is issued to a control system of each production device, the temperature, pressure and flow parameters of each process are adjusted in real time, the dust concentration change is dynamically monitored, and closed-loop optimization control of dust is formed.
Owner:LIANZHOU JINYU PLASTIC PACKING CO LTD

Packaging evidence for long term validation

A method for packaging digital evidence for long term validation comprises forming a package of a digital document (10), an electronic signature (12) for the document (10), together with evidence (16) of the authority of the signature in the document and a time stamp (20) indicating when the document was digitally signed. All of the pieces form parts of the packaged evidence.
Owner:GEN DIGITAL INC

Mine safety production data fusion analysis method based on multi-source perception

The invention relates to the technical field of mine safety production analysis, and discloses a mine safety production data fusion analysis method based on multi-source perception, which comprises the following steps: step 1, carrying out real-time data acquisition on physical parameters related to safety production in a mine environment through various types of sensors arranged in the mine environment, the collected data are stored as independent data streams with timestamps, and the data of different sensors have time inconsistency according to the difference set by the sampling frequency and the clock; and step 2, carrying out time delay compensation processing on the multi-source data with the timestamps acquired in the step 1. An optimization model is established in combination with a variational method to perform time delay compensation and time synchronization processing on sensor data, the problem of time inconsistency caused by different sampling frequencies and clocks of sensors is eliminated, fusion processing of multi-source sensor data on a unified time reference is realized, and an accurate data basis with consistent time sequence is obtained.
Owner:åŒ—äŗ¬å®‰åˆä¼—é“å®‰å…ØęŠ€ęœÆęœ‰é™å…¬åø

A method, device, equipment, medium and product for synchronizing clocks between clusters

The present invention discloses a method, device, equipment, medium and product for clock synchronization between clusters. The method is applied to a slave clock synchronization system in each cluster, and the slave clock synchronization system includes: a clock synchronization module and a calculation module; the clock synchronization module obtains the timestamp of the high-precision time synchronization protocol PTP message between the master clock server and the master clock server, and sends the timestamp to the calculation module; wherein the PTP message includes at least: a synchronization message, a follow-up message, a delay request message and a delay response message; the calculation module determines the corresponding master-slave clock deviation and transmission delay according to the received timestamp, and sends the clock synchronization signal determined according to the master-slave clock deviation, transmission delay and the local slave clock to the clock synchronization module; the clock synchronization module generates the corresponding slave clock second pulse according to the clock synchronization signal sent by the calculation module to complete the slave clock correction. This scheme can realize accurate clock synchronization between computing clusters in a multi-computing cluster system.
Owner:SHANGHAI SMARTLOGIC TECHNOLOGY LTD

Time-sensitive network end-to-end time synchronization method, device and storage medium

The present invention relates to the technical field of information processing, and provides a time-sensitive network end-to-end time synchronization method, device and storage medium, including obtaining the timestamp of the source end, performing time calibration on the time synchronization signal according to the timestamp of the source end, collecting the time of the target end after obtaining the calibrated time synchronization signal, obtaining the timestamp of the target end, analyzing the timestamp of the source end and the timestamp of the target end, obtaining the network delay value, combining the local clock information of the source end and the target end, and obtaining the corrected timestamp of the target end, and adjusting the local clock of the target end according to the corrected timestamp of the target end and the timestamp of the source end. By obtaining the timestamp of the source end and performing time calibration and analysis, and correcting the timestamp of the target end, the accuracy and stability of the end-to-end time synchronization are improved, and the problems of insufficient synchronization accuracy and unstable delay in complex network environments are improved.
Owner:SHENZHEN SCODENO TECH CO LTD

Engineering machinery fault prediction and intelligent maintenance method based on deep learning

The invention discloses an engineering machinery fault prediction and intelligent maintenance method based on deep learning, and belongs to the technical field of intelligent operation and maintenance of engineering machinery. The method comprises the following steps: firstly, performing timestamp synchronization and feature enhancement on multi-source sensor data to generate a space-time alignment tensor; fusing the image and time sequence features through a multi-modal feature distillation network, and constructing a cross-modal unified feature vector; thirdly, calculating a fault probability and residual life distribution, and constructing a Markov decision model in combination with a resource state; and finally, dynamically optimizing the maintenance instruction by using deep reinforcement learning, and continuously updating the model through closed-loop feedback. According to the method, early-stage accurate prediction of the fault and dynamic optimization of the maintenance strategy are realized, the problems of inaccurate prediction, decision lag, resource waste and the like in a traditional method are effectively solved, and the availability rate and the maintenance economy of equipment are remarkably improved.
Owner:XIAMEN ZHONGTA RISHENG INFORMATION TECH CO LTD

Centrifugal machine data remote monitoring analysis method and system based on cloud computer

The invention relates to the technical field of data monitoring analysis, and discloses a centrifugal machine data remote monitoring analysis method and system based on a cloud computer, and the method comprises the following steps: collecting centrifugal machine operation index data, carrying out time stamp synchronization, carrying out the recognition and elimination of abnormal values in the data, and carrying out the data integration through data normalization processing, and generating synchronous monitoring data. According to the invention, the data frequency domain analysis of the centrifugal machine is implemented on the cloud platform, the vibration signal frequency spectrum and energy distribution are accurately identified, the efficient marking and trend analysis of abnormal data are realized, the detailed degree and response speed of abnormal detection are enhanced, and the monitoring of the running state of the centrifugal machine is more accurate; by combining real-time trend analysis and historical fault data, the accuracy of fault prediction is improved, the classification of fault modes is more scientific, the situations of false alarm and missing alarm are effectively reduced, the timeliness and pertinence of preventive maintenance measures are enhanced, and the running efficiency and safety of the centrifugal machine are remarkably improved.
Owner:DRAGON LAB INSTR

Data traceability verification method in trusted data space

The invention provides a data traceability verification method in a trusted data space, and relates to the technical field of zero-knowledge certification, the method comprises the following steps: collecting a plurality of operation records of target data in the trusted data space, and arranging the operation records according to a timestamp sequence to generate an operation sequence; converting into a state transition constraint condition; inputting the operation sequence into a pre-trained key node identification model, and outputting position identifiers of one or more key state transition nodes in the operation sequence; constructing a zero-knowledge proof circuit according to the operation sequence, the state transition constraint condition and the key state transition node, and operating the zero-knowledge proof circuit to generate a composite zero-knowledge proof; and providing the composite zero-knowledge proof as a data traceability voucher of the target data to a verification party to complete traceability verification of the whole processing process of the target data. The technical problems of privacy disclosure risk and low verification efficiency of data traceability verification in the prior art are solved.
Owner:LINGSHU TECH CO LTD

Industrial Internet of Things timestamp-level rapid anomaly detection method based on frequency domain spectrum difference

The invention discloses an industrial Internet of Things timestamp-level rapid anomaly detection method based on frequency domain spectrum difference. A non-neural network scheme is adopted to execute fine-grained industrial sensor signal anomaly detection. Comprising the following steps: proposing an evaluation index which only takes frequency spectrum difference in a frequency domain as timestamp-level anomaly detection; point-level and sequence-level spectrum difference calculation modules are designed as two complementary branches to jointly generate the spectrum difference of each timestamp in a frequency domain; designing an exception scoring strategy to generate an exception score for each timestamp in combination with point-level and sequence-level spectrum differences; wherein the point-level spectrum difference calculation module is used for identifying dominant anomalies in the industrial time signals, and the sequence-level spectrum difference calculation module is used for identifying implicit anomalies in the industrial time signals. According to the industrial Internet of Things timestamp-level rapid anomaly detection method based on the frequency domain spectrum difference, high-speed and high-precision anomaly detection can be realized on industrial Internet of Things edge equipment with limited resources.
Owner:HUNAN UNIV OF SCI & TECH

Mimicry encryption method and system for communication data transmission

The invention relates to the technical field of communication data security, and discloses a mimicry encryption method and system for communication data transmission. The method comprises the following steps: synchronizing a high-precision timestamp through an NTP protocol and generating a standardized environment parameter; temperature, voltage and signal strength indexes are extracted based on the environmental parameters, entropy hybrid calculation and dynamic splicing coding are carried out, and a dynamic encryption key is generated; performing multi-layer mimicry permutation and diffusion confusion on plaintext data after the plaintext data is partitioned by using the key to generate a final ciphertext; performing protocol encapsulation and transmission optimization on the ciphertext; and the receiving end realizes decryption through de-encapsulation and inverse mimicry transformation, and dynamically adjusts the key in combination with environmental parameter feedback. According to the method, key dynamics is enhanced through environment randomness, multi-layer encryption and integrity verification are combined, the anti-attack ability and safety of data transmission are remarkably improved, and the method is suitable for communication scenes with high real-time performance requirements.
Owner:CHONGQING DACHENG ZHIXIN TECH DEV CO LTD

AI enhanced real-time occlusion detection system based on intraoral image

The invention relates to the field of oral medical diagnosis, and discloses an intraoral image-based AI enhanced real-time occlusion detection system, which comprises a multi-source intraoral image acquisition module for acquiring dynamic oral image data through a high-definition oral endoscope and recording a three-dimensional structure of a tooth surface by combining a laser scanning device, meanwhile, force distribution of an occlusion contact point is collected through a pressure sensor array; the multi-source intraoral image acquisition module comprises a timestamp synchronization unit and is used for performing multi-source synchronization on an oral image frame sequence and occlusal force distribution data by utilizing a unified timestamp; through the multi-source in-port image acquisition module, synchronous acquisition of dynamic images, three-dimensional structure data and bite force distribution is realized, and time sequence consistency of the data is ensured through timestamp marking. Compared with a traditional single data source collection mode, the multi-source collection can comprehensively record dynamic characteristics in the occlusion process, rich and accurate input is provided for subsequent AI analysis, and data missing or collection deviation is avoided.
Owner:FOURTH MILITARY MEDICAL UNIVERSITY

Multi-camera anti-shake time sequence synchronous control system and method based on FPGA (Field Programmable Gate Array)

The invention belongs to the technical field of signal synchronization, and discloses a multi-camera anti-shake time sequence synchronization control system and method based on an FPGA. The method comprises the following steps: taking an FPGA as a core, generating an FSYNC signal as a camera exposure reference, and obtaining a delay value through a timestamp and differential measurement; acquiring I MU data and obtaining a global motion vector through a gradient descent algorithm; analyzing by combining the two to obtain phase deviation and feeding back the phase deviation to the FPGA, and adjusting the FSYNC signal by combining the global motion vector by the FPGA according to the phase deviation; according to the method, nanosecond timestamp recording is realized through a homologous clock reference and hardware-level signal binding mechanism, and the limitation of dependence on static time difference compensation in the prior art is broken through in combination with global motion vector analysis and dynamic phase offset adjustment.
Owner:SHENZHEN QUNGUANG VISION TECH CO LTD

Circuit breaker action coil current characteristic diagnosis and analysis system and method

The invention provides a circuit breaker action coil current characteristic diagnosis and analysis system and method. Belongs to the technical field of power system automation and substation state monitoring. The method comprises the following steps: carrying out synchronous wave recording on current of a plurality of circuit breaker actuating coils on a direct current feeder screen side to generate multi-channel synchronous wave recording data; meanwhile, SOE message information is collected, a circuit breaker with specific action is automatically matched and positioned by utilizing a timestamp and action identification information in an SOE message, and action circuit breaker positioning data is generated; the current waveform data of the circuit breaker actuating coil are collected in real time, and a high-precision algorithm is used for feature extraction and analysis, so that abnormal fluctuation in current change can be accurately captured, such as overlarge starting current and unstable holding current, the risk of misoperation or rejection of operation of the circuit breaker caused by abnormal current characteristics is reduced, and the service life of the circuit breaker is prolonged. Electric power system accidents caused by equipment faults are avoided, and the operation reliability of the circuit breaker is improved.
Owner:SHENZHEN TIEON ENERGY TECH

Structured information retrieval method based on large language model

The invention relates to the technical field of language information processing, and provides a large language model-based structured information retrieval method, which comprises the following steps of: deploying an adapter, accessing operation and maintenance data, carrying out timestamp synchronization on an original entry and identifying a source, mapping equipment, parts and personnel based on an asset directory and recording mapping confidence, extracting structural elements from the records; injecting three types of metadata into candidate evidences, establishing corresponding nodes and edges in a graph database, analyzing query into a structured retrieval intention, expanding a candidate evidence set along three chains to form candidate sub-graphs, and calculating comprehensive scores for sorting; the candidate evidences should be verified and constrained, meanwhile, LLM is called for inference, feasibility inference confidence is output, actual measurement results, the inference confidence and source confidence are fused according to preset weights, feasibility scores of the candidate evidences are obtained, and for the candidate evidences passing feasibility verification, accurate anchor points are calibrated for each evidence along a source chain; and generating a structured answer.
Owner:LONGYAN UNIV

Trust evaluation data storage tracing method and system based on block chain technology

The invention discloses a trust evaluation data storage tracing method and system based on a block chain technology, and particularly relates to the technical field of data storage management, and the method comprises the following steps: recording trust evaluation data in a block form based on the distributed storage characteristics of a block chain, automatic management of data submission, verification, storage and updating is realized by using an intelligent contract, data authenticity and non-tampering are ensured, time stamps are added for data through a timestamp function, a time-ordered operation chain is formed, a data tracing function is provided, an authorized user is allowed to inquire a data source and historical operation records, and the user experience is improved. Based on data access characteristics, a storage area is dynamically adjusted, and efficient utilization and optimal management of storage resources are achieved; according to the method, the storage position of data can be dynamically adjusted, hot data are stored in a high-performance storage area, cold data are migrated to a low-cost storage area, the access efficiency of high-frequency data is guaranteed, and the storage cost is optimized.
Owner:BEIJING YIJIA LAO XIAO TECH CO LTD

Intelligent factory manufacturing optimization method and system based on energy consumption prediction

The invention provides an intelligent factory manufacturing optimization method and system based on energy consumption prediction, and the method comprises the steps: carrying out the time sequence structural processing of historical energy consumption record data continuously collected in a factory manufacturing process, and generating an energy consumption sequence unit with a continuous time stamp through time window division and data correlation verification; constructing a manufacturing process-energy consumption association knowledge graph based on the energy consumption sequence unit, and establishing a structured knowledge representation containing node attributes and edge relationship weights by identifying a causal association relationship between process execution nodes and energy consumption fluctuation features; calling a pre-trained graph neural network model to carry out time sequence evolution prediction processing on the graph, and generating an energy consumption prediction sequence of a future preset manufacturing cycle; and according to the energy consumption prediction sequence, performing dynamic scheduling optimization on the execution sequence of the current manufacturing process and the resource allocation scheme, and generating an energy consumption optimization oriented manufacturing execution instruction. The energy consumption management level and the resource utilization efficiency in the manufacturing process of the intelligent factory are improved.
Owner:HIMIT (SHENZHEN) TECH CO LTD

Consolidated alarm screen

A method includes receiving, by a computing device, first data from a sensor, the first data including a first timestamp and an event that triggered an alarm; receiving, by the computing device, second data from a remote computing environment, the second data including a second timestamp different from the first time stamp and specifying an action taken in response to the alarm; and rendering a screen on a display of a computing device, the screen including the first and second data represented as a sequence of events ordered by time based on the first and second timestamps.
Owner:SIMPLISAFE INC

Edge devices for anonymized monitoring of moveable objects, method of communication using said edge devices

An edge device and communication method for communicating data captured at an edge device to another device. The edge device is configured to capture data of an area surrounding the edge device, to detect one or more moveable objects of one or more predetermined classes in the area from the captured data, to determine one or more location indicators indicating respective locations of said one or more objects within the area, and to communicate a message to another device at a distance of the edge device, said message including the one or more location indicators together with a time stamp indicating a time when the data was captured.
Owner:SCHREDER ILUMINAƇAO SA

Rolling bearing lightweight fault diagnosis method and system based on multi-source signal fusion

The invention discloses a rolling bearing lightweight fault diagnosis method and system based on multi-source signal fusion. Vibration, temperature and rotating speed signals are synchronously collected, and timestamps are calibrated; respectively carrying out denoising and normalization preprocessing; dividing and aligning windows; differential feature extraction: extracting time-frequency features of the vibration signals by using a one-dimensional residual CNN, and extracting abnormal measurement of the temperature / rotating speed signals by using an LSTM in combination with an isolated forest algorithm; carrying out self-adaptive weighted fusion on the features through an attention mechanism; the lightweight diagnosis model (through knowledge distillation, pruning and quantification) deduces and outputs the fault category, the health index and the confidence coefficient. The system correspondingly comprises an acquisition module, a preprocessing module, a feature extraction module, a fusion module and a diagnosis module. The method improves the early fault sensitivity, enhances the variable working condition robustness, supports the real-time deployment of edge equipment, and is suitable for the intelligent monitoring of industrial bearings.
Owner:XI AN JIAOTONG UNIV

Systems and methods for managing sharing of a video in a collaboration session

A method and a system for managing sharing of a video in a collaboration session are disclosed. A sharing function obtains (B122) a synchronization time stamp of an audio frame of the collaboration session, wherein the audio frame is going to be used for synchronizing the shared video stream with the collaboration session. The sharing function determines, based on a video frame rate difference between the desired frame rate and a source frame rate of the video file, whether more, less or same number of video frames as in the video file are needed in the shared video stream. The sharing function iteratively retrieves (B140) video frames from the video file stored in the storage function (140). The sharing function provides (B160), by the sharing function (112), the shared video stream by sending said one or more generated video frames to the video communication service (110). The video communication service provides (B174), the shared video stream to the two client devices participating in the collaboration session.
Owner:LIVEARENA TECH AB

Smart power grid load prediction and dynamic response coordinated scheduling method

The invention discloses an intelligent power grid load prediction and dynamic response coordinated scheduling method, and relates to the technical field of power system automation, and the method comprises the steps: accessing intelligent ammeters, distributed power controllers and other devices of Modbus, IEC61850 and DL / T645 protocols through a multi-protocol adaptive gateway, and achieving data standardization; time stamps are calibrated by means of Beidou time service and an IEEE1588PTP protocol, and it is ensured that multi-source data synchronization errors are controllable; deploying an edge computing node cluster, distributing high-priority tasks to low-load nodes through an edge coordinator in combination with a load fluctuation level and a greedy algorithm, and ensuring real-time processing efficiency; the edge nodes generate short-term load prediction, and the cloud platform outputs medium and long-term prediction based on a historical data training model; and finally, the coordinated scheduling decision module fuses the two types of prediction results and the real-time parameters of the power grid, and generates a dynamic instruction to control the output of the adjustable load and the distributed power supply.
Owner:HAINAN POWER GRID CO LTD

Dynamic error management and tracing method for flow line production data

The invention relates to the technical field of data management, and discloses a dynamic error management and tracing method for flow line production data, and the method comprises the following steps: collecting multi-source dynamic data in a flow line production process, including equipment sensor data, production parameter data and product quality data; performing real-time dynamic error detection processing based on the multi-source dynamic data, and generating dynamic error characteristic data through a sliding window algorithm and an abnormal value analysis technology; in the dynamic error management and tracing process of flow line production data, equipment sensor data, production parameter data and product quality data are acquired in real time through a multi-source dynamic data acquisition technology, and data integration is performed by using a timestamp alignment algorithm, so that the synchronism and integrity of error detection data are ensured; meanwhile, through real-time dynamic error detection processing, abnormal fluctuation in the production process can be recognized in time, and the real-time performance and reliability of error detection are guaranteed.
Owner:XIANYANG RAINBOW PHOTOVOLTAIC GLASS CO LTD

Coal mine personnel safety behavior analysis method

The invention belongs to the technical field of intelligent safety supervision, and particularly relates to a coal mine personnel safety behavior analysis method, which synchronously associates a management instruction, personnel behavior data, an equipment state and environment parameters according to timestamps through a management platform, obtains the current position of a personnel in real time, dynamically judges whether the personnel moves correctly or not, and improves the safety of the personnel. When a person approaches each maintenance position, automatically checking whether the maintenance position has omission or not, if the omission is found, generating a rechecking instruction in real time and pausing subsequent operation authorization, if the omission is not found, carrying out consistency checking on the operation sequence, the operation video content and the standard regulation, and if the checking does not reach the standard, carrying out rechecking on the operation video content and the standard regulation; if yes, second-level early warning is triggered immediately, a recheck mechanism is started, subsequent operation authorization is paused by dynamically monitoring operation compliance and combining authority control, personnel are prevented from entering a subsequent maintenance position with an illegal state, and the problem of multi-position continuous violation which can only be traced afterwards in an existing method is thoroughly solved.
Owner:COLORFUL GUIZHOU IMPRESSION NETWORK MEDIA CO LTD

Touch screen coordinate data reporting method and device and storage medium

The invention provides a touch screen coordinate data reporting method and device and a storage medium. According to the method, in the sliding operation scene, according to the timestamp of the data transmission signal sent by the target application, multiple pieces of coordinate data collected by the touch screen are screened, and the screened coordinate data are sent to the target application, so that the link that the target application carries out resampling on the multiple pieces of coordinate data is reduced, the performance of the electronic equipment is improved, and the user experience is improved. And the power consumption of the electronic equipment is reduced, so that the user experience is improved.
Owner:HONOR DEVICE CO LTD

Increasing commit times at database shards for implementing consistency and isolation for distributed transactions

Clock times may be stored at nodes of a database system that are used to provide access to database data using Multi-Version Concurrency Control (MVCC) and implement isolation for distributed transactions. When an access request is received at a processing node of a database, the snapshot time of the access request is determined. The snapshot may be a range of time, with an earliest and latest timestamp. If the snapshot time has a latest time stamp greater than a current timestamp in the clock time, then the clock time may be updated. The clock time and snapshot time may be used to implement time-based MVCC to select a version of a tuple stored in the database to use for performing the access request.
Owner:AMAZON TECH INC

Real-time noise reduction method and system supporting Bluetooth audio interaction

The invention discloses a real-time noise reduction method and system supporting Bluetooth audio interaction, and the method comprises the steps: collecting an environment audio signal and a Bluetooth interaction audio signal, and carrying out the timestamp alignment and spatial calibration of the environment audio signal and the Bluetooth interaction audio signal; an improved multivariate variational mode decomposition algorithm is adopted to decompose and extract noise features, the noise features are combined with user historical noise data, a dynamic noise model is constructed through a long-short-term memory network, parameters are updated in real time, Bluetooth interaction audio content features and a user behavior data recognition scene are analyzed, and a noise reduction strategy is determined. And according to the dynamic noise model and the scene result, reinforcing learning to optimize filter parameters, generating a directional anti-noise signal to carry out noise reduction on the Bluetooth interaction audio signal, collecting the noise reduction intensity manually adjusted by a user, calculating voice definition and total harmonic distortion, and carrying out noise reduction on the Bluetooth interaction audio signal. And inputting a feedback and evaluation result into the modeling and noise reduction steps, carrying out dynamic range adjustment, sound channel balance and power amplification on the audio after noise reduction, and outputting the audio through a loudspeaker.
Owner:SHENZHEN GAOWEI COMM TECH CO LTD

Creative thinking auxiliary generation method and system based on AI

The invention discloses an AI-based creative thinking auxiliary generation method and system, relates to the technical field of artificial intelligence, and solves the problem of inaccurate sound and picture matching in a traditional method by performing timestamp alignment and feature extraction on audio data and a visual image frame and calculating the correlation between the audio data and the visual image frame by using a cross-modal attention mechanism. According to the method, user eye movement track data is introduced, real attention points of a user are mapped into a visual sequence, and an optimized weight matrix is generated by constructing attention masks and fusing model attention weights, so that a generation result is more in line with perception key points of the user. And meanwhile, a feedback mechanism is established based on the synchronization error score, and when the sound and the picture are detected to be asynchronous, the visual frame timestamp can be dynamically adjusted, so that the self-adaptive correction of the content is realized. On the whole, the method has remarkable advantages in the aspects of improving modal alignment precision, enhancing user perception consistency and optimizing generation result naturalness.
Owner:ZHEJIANG NORMAL UNIV