Patents
Literature
Patsnap Eureka AI that helps you search prior art, draft patents, and assess FTO risks, powered by patent and scientific literature data.

973results about "Video data indexing" patented technology

Video content intelligent management method and system based on multi-modal analysis

The invention discloses a video content intelligent management method and system based on multi-modal analysis, and belongs to the field of video content management.The method comprises the steps that multi-modal features of video content are extracted; performing fine structure segmentation on the video content based on the multi-modal features, and segmenting the video into fine segments related to semantics; the fragments obtained after fine structure segmentation are subjected to tagging, and metadata is generated for each fragment; constructing a multi-modal semantic index of the video content based on the generated metadata; according to the method, a semantic retrieval function is provided by utilizing multi-modal semantic indexing, fine fragments required by a user are accurately recalled in combination with text label matching, measurement numerical value scoring, cross-modal vector retrieval and filtering based on a shot language and a shooting technology label, and the user queries and positions a specific time point or the fine fragments in a video through a natural language. According to the invention, the accuracy of video content analysis is improved, the user experience is optimized, and the operation complexity is reduced.
Owner:CHENGDU SOBEY DIGITAL TECH CO LTD

Vehicle-mounted video monitoring system and method integrating 360-degree all-round viewing perception and multi-mode intelligent analysis

The invention provides a vehicle-mounted video monitoring system and a vehicle-mounted video monitoring method integrating 360-degree look-around perception and multi-mode intelligent analysis. According to the system, video images around a vehicle are collected through a vehicle-mounted 360-degree all-round camera, high-precision vehicle position information is obtained through a vehicle-mounted multi-sensor fusion positioning technology, videos and position time are stored in an associated mode, and a space-time video database is constructed. The system utilizes a multi-mode perception fusion technology to intelligently analyze a video, extracts semantic information, and supports a user to quickly retrieve a target video clip through natural language interaction. Meanwhile, the system can monitor abnormal driving behaviors and vehicle surrounding environment risks in real time and trigger early warning when necessary. According to the invention, advanced technologies such as 360-degree looking-around perception, multi-sensor fusion positioning, multi-modal intelligent analysis, natural language interaction and the like are innovatively and deeply fused with vehicle-mounted video monitoring, so that the limitation of a traditional vehicle-mounted monitoring system is broken through, the intelligent and humanized level is greatly improved, the development direction of a vehicle-mounted video monitoring technology is guided, and the vehicle-mounted video monitoring system has a wide application prospect. The method is of great significance in guaranteeing driving safety of intelligent automobiles and promoting development of intelligent traffic.
Owner:刘宇

Video and Audio Multimodal Searching System

A multimodal search system using a video query is described. The system can receive video data captured by a camera of a user device. The video data can have a sequence of image frames. Additionally, the system can receive audio data associated with the video data captured by the user device. Moreover, the system can process, using one or more machine-learned models, the sequence of image frames to generate video embeddings related to the sequence of the image frames. The video embeddings can have a plurality of image embeddings associated with the sequence of image frames. Furthermore, the system can determine one or more video results based on the video embeddings and the audio data. Subsequently, the system can transmit, to the user device, the one or more video results.
Owner:GOOGLE LLC

Systems and Methods for Processing, Analyzing, and Visualizing Complex Object Sets

Disclosed are methods, systems, and non-transitory computer readable memory for processing, analyzing, and visualizing complex digital object sets using large language models. For instance, a method may include receiving a set of digital objects; indexing the received set of digital objects to generate indexed data; generating a timeline prompt based on the indexed data; processing the timeline prompt to generate a timeline response; and outputting the timeline response as an interactive timeline of based on the set of digital objects.
Owner:TRANQUILITY AI INC

Digital intelligent teaching recording and broadcasting system and method

The invention discloses a digital intelligent teaching recording and broadcasting system and method, and relates to the technical field of data encryption, and the method comprises the steps: collecting teaching videos and audios of a teacher through a high-definition camera and an audio device, carrying out the preprocessing, obtaining the course content, recognizing the key information of the course content through a natural language processing NLP and computer vision technology, and carrying out the recording and broadcasting. The method comprises the following steps: segmenting a video into single knowledge point fragments, performing classified storage, forming a learning resource library, collecting basic information and learning habit data of students, constructing a user portrait, matching learning resources, extracting the learning habit data in the user portrait, performing feature extraction, and obtaining a comprehensive feature vector; constructing a learning state evaluation model by using a convolutional neural network to comprehensively analyze the comprehensive feature vector to obtain a learning state comprehensive score, formulating a playing strategy according to the learning state comprehensive score and matched learning resources, regularly testing knowledge points of students, evaluating the mastering degree of the students, and updating a user portrait according to a result.
Owner:JIANGSU XUHE EDUCATION TECH CO LTD

Systems and methods for multimodal indexing of video using machine learning

Systems, methods, and computer-readable media are disclosed for systems and methods multimodal indexing of video using machine learning. An example method may include deceiving, by a video encoder of an audio-video transformer neural network comprising one or more computer processors coupled to memory, a first frame and a second frame associated with a first segment of a video. The example method may also include receiving, by an audio encoder of the audio-video transformer neural network, an audio spectrogram comprising first audio data associated with the first segment of the video. generating, by the video encoder, a first video embedding. The example method may also include generating, by the audio encoder, a first audio embedding. The example method may also include determining a fusion of the first video embedding and the first audio embedding using a multimodal bottleneck token. The example method may also include determining an output including the first video embedding and the first audio embedding. The example method may also include determining a classification of the first portion of the video based on the output.
Owner:AMAZON TECH INC

Intelligent database video retrieval method based on big data technology

The invention relates to the technical field of intelligent information retrieval, in particular to an intelligent database video retrieval method based on a big data technology, which comprises the following steps: S1, carrying out space-time slicing processing on an original video stream; s2, extracting a visual feature vector, an audio waveform vector and a text description vector; s3, constructing a cross-modal incidence matrix; s4, constructing a layered mixed index structure which comprises a real-time updating layer and a static storage layer; s5, after a user retrieval request is received, candidate video set screening is carried out based on the layered mixed index structure, and a retrieval result is generated; and S6, updating the cross-modal incidence matrix, and synchronously updating the weight parameter of the layer. According to the method, the retrieval accuracy is improved through cross-modal feature fusion, the data storage and retrieval efficiency is optimized by adopting a hierarchical mixed index structure, and the index weight is dynamically adjusted in combination with user behavior feedback, so that efficient, accurate and intelligent video retrieval is realized.
Owner:HEBEI ZHENGTONG ARCHIVES MANAGEMENT CO LTD

Dual-stream video management

An Internet of Things (IoT) or vehicle dash cam may store both a high-resolution and low-resolution video stream on a device. The video streams are selectively accessible by remote devices. Because of the relatively smaller storage requirements of low-resolution video files, retaining of additional video data on the vehicle device (beyond what would be possible with only high-resolution video) is possible. The user may be provided an option to adjust the amount of low-resolution and high-resolution video to store on the device. A combined media file may be generated by a device to include time-synced high-resolution video, low-resolution video, and / or metadata for a particular time period.
Owner:SAMSARA INC

Video authenticity detection method and system based on large model and retrieval enhancement generation technology

The invention belongs to the technical field of computer vision and natural language processing, and discloses a video authenticity detection method and system based on a large model and a retrieval enhancement generation technology. Jointly constructing a to-be-detected text by combining the video title, the brief introduction, the text obtained by audio conversion and the video description text; completing the construction of an external knowledge base; performing fact verification based on a retrieval enhancement generation technology (RAG); performing common sense reasoning based on a large model: constructing cue words, performing common sense and logic reasoning on the to-be-detected text by utilizing the large model ChatGLM, detecting whether common sense mistakes and logic conflicts exist or not, and judging whether news is true or false according to the common sense mistakes and the logic conflicts; and comprehensively judging the authenticity of the to-be-detected video according to the fact check and common sense reasoning. Through the combination of the large model and the RAG technology, the accuracy and interpretability of video content authenticity detection are improved; the detection efficiency is improved; false news videos can be found and processed in time, and the health of a network environment and public trust can be maintained.
Owner:NORTHEASTERN UNIV CHINA

Method and system for content-based indexing of data streams in streaming storage systems

A method for managing a data stream (DS) includes: receiving a first request from an administrator; obtaining the DS from a streaming storage system (SSS); performing a first analysis of the DS to extract relevant data; making a first determination that the relevant data is worth indexing; generating a set of indexes corresponding at least a portion of the DS; receiving a query about a data item in the DS from a user; sending a second request to a database engine comprising the query; performing a second analysis of the query; making a second determination that a re-indexing of the portion of the DS is not required; identifying an index from the set of indexes that is associated with the query; sending position information of the data item to an API service; retrieving the data item from the SSS; and initiating displaying of the data item to the user.
Owner:DELL PROD LP

National dance digital display system

The invention relates to the technical field of digital display, in particular to a national dance digital display system which comprises a track node extraction module, a rhythm segment optimization module, a trend stability recognition module, an action information acquisition module and a synchronous recombination execution module. According to the method, angle abrupt change points are extracted by constructing spatial vectors, action change recognition is enhanced, the frame sequence structure analysis precision is improved, time length and amplitude dual stability standards are introduced in beat paragraph screening, the accuracy and robustness of rhythm recognition are enhanced, trend stable frames are extracted through slope fluctuation comparison, and the accuracy and robustness of rhythm recognition are improved. The continuity of the rhythm flow direction and the centrality of node selection are ensured, the main limb movement is constructed by the path total distance, the direction angle and the frame time sequence, the clear structure and the consistent direction of the movement expression are ensured, the direction correction and rhythm scaling strategy is introduced in the dynamic recombination process, the smooth movement transition and the rhythm synchronization are ensured, and the reorganization efficiency is improved. And the rhythm coordination and the dynamic coherence of dance actions are obviously improved.
Owner:SICHUAN NORMAL UNIV

Factory real-time digital monitoring method and system based on Internet of Things, and storage medium

The invention relates to the technical field of industrial monitoring, and discloses a factory real-time digital monitoring method and system based on the Internet of Things, and a storage medium. The method comprises the following steps: carrying out encryption access on heterogeneous Internet of Things equipment through dual identity authentication to obtain a security authentication equipment identifier pool; performing multi-dimensional data acquisition on the equipment identification pool based on Hash verification to obtain an encrypted time sequence data chain; performing dynamic identification on the factory visual image by adopting a confidence adaptive threshold to obtain a multi-task fusion feature code; performing heterogeneous fusion on the time sequence data chain and the feature code by using a sliding window weight distribution algorithm to obtain a factory digital state fingerprint; and performing knowledge graph reasoning on the state fingerprint through three-level early warning threshold judgment to obtain a real-time risk level early warning code. The technical problems that heterogeneous Internet of Things equipment cannot be uniformly accessed and authenticated, multi-source data lacks safety fusion processing, and the factory state cannot be intelligently reasoned and pre-warned are solved.
Owner:BEIJING TIANYUAN 3D TECH CO LTD

Systems and Methods for Entity, Relationship, and Timeline Generation from Complex Object Sets

Disclosed are methods, systems, and non-transitory computer readable memory for processing, analyzing, and visualizing complex digital object sets using large language models. For instance, a method may include receiving a set of digital objects; indexing the received set of digital objects to generate indexed data; generating a timeline prompt based on the indexed data; processing the timeline prompt to generate a timeline response; and outputting the timeline response as an interactive timeline of based on the set of digital objects.
Owner:TRANQUILITY AI INC

Al-based video content analysis method and system

The invention relates to the technical field of content recognition, in particular to an Al-based video content analysis method and system, and the method comprises the following steps: carrying out the image segmentation learning based on a video sequence through employing a U-Net convolutional neural network, analyzing scenes and elements in a video frame, recognizing and isolating key visual elements in a video through network learning, and carrying out the recognition of the key visual elements in the video. Comprising objects and figures. According to the method, image segmentation learning is carried out by adopting the U-Net convolutional neural network, key visual elements in the video can be identified and isolated more accurately, a clearer basis is provided for follow-up scene change and key event tracking, video content analysis is carried out by applying the graph neural network, the identification capability of dynamic scenes and events is enhanced, and the identification efficiency is improved. Deeper structured understanding is provided for video content indexing, the video quality is analyzed through a structural similarity index evaluation method, and the reason of visual quality reduction can be accurately recognized and improved.
Owner:CHENDA (GUANGZHOU) NETWORK TECH CO LTD +1

Abnormal behavior detection method, system, device and medium

The present invention relates to a method, system, device and medium for detecting abnormal behavior. The method comprises: obtaining a video to be identified, and extracting a plurality of video frames from the video in sequence; wherein each video frame carries a corresponding time mark; inputting each video frame and a preset behavior description set into a multimodal large language model, performing multimodal fusion on the input video frames and the behavior description set, and identifying abnormal behaviors in each video frame; wherein the behavior description set is used to indicate abnormal behaviors to be detected; arranging the abnormal behaviors in each video frame according to the generation order, and inputting them into a behavior analysis model together with corresponding time marks, and detecting abnormal behaviors in the video; wherein the behavior analysis model is a time series model based on Transformer. The present invention improves the problem that the prior art only uses a single modality analysis method, resulting in a low abnormal behavior recognition rate, and realizes accurate recognition of abnormal behaviors.
Owner:上海锡鼎智能科技有限公司

Generation-Augmented Latent Navigation for Continuous Spatiotemporal Zoom and Rotation in Immersive Environments

A system and method for generation-augmented latent hyperspace navigation in spatiotemporal media using hierarchical and Lorentzian autoencoders. The system compresses media into latent representations while preserving geometric, temporal, and semantic relationships. A latent hyperspace manager organizes compressed data as geodesic trajectories, and a geodesic trajectory mapper computes navigation paths. Symbolic anchors provide persistent reference points, while spatiotemporal routing coordinates decisions across multiple scales. A strategy caching system preserves successful navigation patterns for reuse as procedural memory. A synthetic content generator including latent diffusion models, neural radiance fields, and context-aware refinement produces augmentation for continuous zoom, bidirectional traversal, and rotational reorientation. A user input interface and zoom controller enable interactive exploration and reconstruction, supporting applications in immersive media, visualization, and surveillance.
Owner:ATOMBEAM TECH INC

Multi-modal data pairing method and system based on deep learning

The invention provides a multi-modal data pairing method and system based on deep learning, and relates to the technical field of data processing, and the method comprises the steps: obtaining a video multi-frame sequence and a target text, and respectively extracting an overlapped frame group set and a standardized text sequence; performing spatio-temporal feature extraction and text dependency relationship coding to obtain a video time sequence vector sequence and a text vector sequence; executing cross-modal alignment search, and constructing a monotonic matching path set; calculating a semantic and action entity relationship consistency score of the paired elements on the path to obtain a comprehensive score; and determining an alignment relationship between the video and the text based on the optimal path. According to the method, accurate matching of the video and the text is realized, and the cross-modal retrieval efficiency is improved.
Owner:BEIJING YIZHUANG INTELLIGENT CITY RES INST GRP CO LTD

Multimodal ai-based search for digital assets

Embodiments of the present disclosure relate to multimodal AI-based search for digital assets via an indexing and / or search pipeline. With respect to the indexing pipeline, some embodiments obtain first data and second data associated with a first digital asset. Such data represents different data types or modalities of the same digital asset. After obtaining the first and second data, some embodiments then generate a composite index. After the composite index is built such index can then be used to execute a query via the search pipeline. To execute the query some embodiments compute a relevance score for each digital asset, of multiple digital assets, based at least in part on a measure in which each digital asset satisfies one or more parameters or conditions for two or more data types of the query. Various embodiments then rank each digital asset and present one or more associated indicators.
Owner:NVIDIA CORP

Media information processing method and device, storage medium and electronic equipment

The invention discloses a media information processing method and device, a storage medium and electronic equipment. The method comprises the steps that to-be-retrieved source media information and prompt template information are obtained, the source media information and the prompt template information are input into a target retrieval model, target media information is determined, and the target retrieval model represents a model obtained through joint training according to at least two retrieval task types; a loss function used in the training process of the target retrieval model is a joint loss function, the joint loss function is determined by at least two sub-loss functions in one-to-one correspondence with the at least two retrieval task types, and the target media information represents media information obtained by retrieving the source media information according to the prompt template information. According to the method and the device, the technical problem of relatively low retrieval efficiency of the media information caused by single multi-modal retrieval task is solved.
Owner:TENCENT TECH (BEIJING) CO LTD

Intelligent conference recording and recording method and system

The invention discloses an intelligent conference recording and recording method and system. According to the invention, cloud service equipment receives multi-modal information sent by edge equipment; processing the multi-modal information, and generating conference record information in combination with context sensing error correction, content segmentation, a dynamic segmentation strategy and a preset storage strategy; carrying out encryption processing on the conference record information based on the participant role information, and generating target format conference information; processing the target format conference information to generate initial conference scene mode information including a conference outline, a PPT and a mind map; processing the target format conference information and the initial conference scene mode information to generate conference information data after priority ranking and strategy processing; performing semantic analysis, feature fusion and cross-modal association processing on the conference information data to generate target conference scene mode information; and determining a target application program based on the target conference scene mode information, and sending the target format instruction information to the target application program.
Owner:XIAMEN RGBLINK SCI & TECH CO LTD

Video retrieval generation method and device based on sparse representation and reordering

The invention discloses a video retrieval generation method and device based on sparse representation and reordering, and relates to the technical field of cross-modal video retrieval. The method comprises the following steps: acquiring a query text, a candidate video sequence and a video retrieval generation model; obtaining a query text dense representation and a candidate video sequence dense representation according to the encoder module, the query text and the candidate video sequence; through a sparse representation module, text sparse representation is generated according to the query text dense representation, candidate video sparse representation is generated according to the candidate video sequence dense representation, and reverse index screening is performed on the candidate video sequence to obtain part of candidate videos; determining a global score of each video in the partial candidate videos through a cross attention module, and sorting the partial candidate videos; and through a generation module, generating a target text corresponding to the query text according to the sorted candidate videos and the query text. By adopting the method and the device, the retrieval efficiency is improved in a sparse representation and reordering mode.
Owner:HARBIN INSTITUTE OF TECHNOLOGY (SHENZHEN) (INSTITUTE OF SCIENCE AND TECHNOLOGY INNOVATION HARBIN INSTITUTE OF TECHNOLOGY SHENZHEN)

Latent Geodesic Traversal Across Multi-Axis Hyperspaces for Real-Time Video Reconstruction and Augmentation

A system and method for latent geodesic traversal across multi-axis hyperspaces for real-time video reconstruction and augmentation. Spatiotemporal video data are compressed into navigable latent representations using hierarchical and Lorentzian autoencoders that preserve geometric and temporal structure. A geodesic traversal engine computes paths across spatial, temporal, spectral, and semantic axes, guided by symbolic anchors and spatiotemporal routing protocols. A correlation network restores fine detail, while an augmentation generator synthesizes additional or counterfactual content to enable infinite zoom, continuous multi-scale exploration, and temporally coherent augmentation. A strategy caching system preserves successful traversal patterns for reuse, supporting persistent learning and adaptive real-time performance.
Owner:ATOMBEAM TECH INC

Image / video analysis with activity signatures

Video frames from a video are compressed into a single image or a single data structure that represents a unique visual flowprint or visual signature for a given activity being modeled from the video frames. The flowprint comprises a computed summary of the original pixel values associated with the video frames within the single image and the flowprint is specific to movements occurring within the video frames that are associated with the given activity. In an embodiment, the flowprint is provided as input to a machine-learning algorithm to allow the algorithm to perform object tracking and monitoring from the flowprint rather than from the video frames of the video, which substantially improves processor load and memory utilization on a device that executes the algorithm, and substantially improved responsiveness of the algorithm.
Owner:NCR VOYIX CORP

Multi-dimensional time sequence data compression and rapid retrieval method and system

The invention discloses a multi-dimensional time sequence data compression and rapid retrieval method and system, and relates to the technical field of large data compression retrieval. The multi-dimensional time sequence data compression and rapid retrieval method comprises the following steps: S1, collecting and preprocessing multi-source data in a monitoring abstract video stream, and constructing a standardized time sequence behavior data set; s2, analyzing behavior characteristics of frame segments in the sliding window, and dynamically adjusting anchor point labeling and compression strategies; s3, evaluating the coverage integrity of anchor point information in a compression section, and driving generation of an index path; s4, comprehensively evaluating the path behavior association strength and dynamically adjusting a loading decision; and S5, verifying the matching integrity of the behavior anchor point field and the jump pointer, and guaranteeing the localizability and jump stability of the event in the compression structure. The problems that in an intelligent monitoring scheme, video abstract compression is not bound with behavior semantics, a fine-grained positioning mechanism based on behavior labels is lacked, and key action segments cannot be directly positioned during user playback are solved.
Owner:TIANRUI TECHNOLOGY (TIANJIN) CO LTD

Methods and systems for segmenting video content based on speech data and for retreiving video segments to generate videos

A method includes receiving a series of video segments and providing the series of video segments as input to a first machine learning model to produce text data. The text data is provided as input to a second machine learning model to produce categorized text data that includes a classification indication. The classification indication is added to metadata of the video segment, and the categorized text data is provided as input to a third machine learning model to produce a semantic vector. The method also includes causing the video segment and the metadata that includes the classification indication to be stored at a location of a database based on the semantic vector, the database being configured to be searched based on a search query associated with the semantic vector.
Owner:VIDEOFORCEAI INC

Unsupervised video segment clustering and boundary positioning

Method and apparatus for video processing. A plurality of videos are accessed. An index is generated based on one or more key frames and one or more video features extracted from the plurality of video. A user input is processed to identify user intentions. The index is filtered to identify a plurality of candidate key frames based on the user intentions. A recommended video is generated based on the plurality of candidate key frames, further comprising selecting one or more additional frames based on processing the plurality of candidate key frames using one or more clustering operations, and aggregating the plurality of candidate key frames and the one or more additional frames.
Owner:INTERNATIONAL BUSINESS MACHINE CORPORATION

Multi-modal accident scene library construction method for end-to-end automatic driving test

The invention belongs to the technical field of automatic driving test, and particularly relates to a multi-mode accident scene library construction method for end-to-end automatic driving test. The method specifically comprises the following steps: step 1, extracting text information based on an MCAT-BiLSTM-CRF algorithm; 2, designing the ontology architecture, and storing accident scene information by using a knowledge graph to obtain an accident text knowledge base; step 3, derivative expansion is carried out on the accident scene element combination based on a HyCon-Sg-Net algorithm; step 4, performing enhanced fine tuning on the Open-Sora model to realize modal conversion from the text knowledge base to the accident video database so as to construct a multi-modal accident scene library; the method can be used for testing the performance of an end-to-end automatic driving automobile in an extreme accident scene, and the adaptive capacity of an automatic driving algorithm to the extreme accident scene is remarkably improved.
Owner:JILIN UNIVERSITY