710results about "Video data indexing" patented technology

The invention discloses a video content intelligent management method and system based on multi-modal analysis, and belongs to the field of video content management.The method comprises the steps that multi-modal features of video content are extracted; performing fine structure segmentation on the video content based on the multi-modal features, and segmenting the video into fine segments related to semantics; the fragments obtained after fine structure segmentation are subjected to tagging, and metadata is generated for each fragment; constructing a multi-modal semantic index of the video content based on the generated metadata; according to the method, a semantic retrieval function is provided by utilizing multi-modal semantic indexing, fine fragments required by a user are accurately recalled in combination with text label matching, measurement numerical value scoring, cross-modal vector retrieval and filtering based on a shot language and a shooting technology label, and the user queries and positions a specific time point or the fine fragments in a video through a natural language. According to the invention, the accuracy of video content analysis is improved, the user experience is optimized, and the operation complexity is reduced.

Disclosed are methods, systems, and non-transitory computer readable memory for processing, analyzing, and visualizing complex digital object sets using large language models. For instance, a method may include receiving a set of digital objects; indexing the received set of digital objects to generate indexed data; generating a timeline prompt based on the indexed data; processing the timeline prompt to generate a timeline response; and outputting the timeline response as an interactive timeline of based on the set of digital objects.

Systems, methods, and computer-readable media are disclosed for systems and methods multimodal indexing of video using machine learning. An example method may include deceiving, by a video encoder of an audio-video transformer neural network comprising one or more computer processors coupled to memory, a first frame and a second frame associated with a first segment of a video. The example method may also include receiving, by an audio encoder of the audio-video transformer neural network, an audio spectrogram comprising first audio data associated with the first segment of the video. generating, by the video encoder, a first video embedding. The example method may also include generating, by the audio encoder, a first audio embedding. The example method may also include determining a fusion of the first video embedding and the first audio embedding using a multimodal bottleneck token. The example method may also include determining an output including the first video embedding and the first audio embedding. The example method may also include determining a classification of the first portion of the video based on the output.

The invention relates to the technical field of intelligent information retrieval, in particular to an intelligent database video retrieval method based on a big data technology, which comprises the following steps: S1, carrying out space-time slicing processing on an original video stream; s2, extracting a visual feature vector, an audio waveform vector and a text description vector; s3, constructing a cross-modal incidence matrix; s4, constructing a layered mixed index structure which comprises a real-time updating layer and a static storage layer; s5, after a user retrieval request is received, candidate video set screening is carried out based on the layered mixed index structure, and a retrieval result is generated; and S6, updating the cross-modal incidence matrix, and synchronously updating the weight parameter of the layer. According to the method, the retrieval accuracy is improved through cross-modal feature fusion, the data storage and retrieval efficiency is optimized by adopting a hierarchical mixed index structure, and the index weight is dynamically adjusted in combination with user behavior feedback, so that efficient, accurate and intelligent video retrieval is realized.

An Internet of Things (IoT) or vehicle dash cam may store both a high-resolution and low-resolution video stream on a device. The video streams are selectively accessible by remote devices. Because of the relatively smaller storage requirements of low-resolution video files, retaining of additional video data on the vehicle device (beyond what would be possible with only high-resolution video) is possible. The user may be provided an option to adjust the amount of low-resolution and high-resolution video to store on the device. A combined media file may be generated by a device to include time-synced high-resolution video, low-resolution video, and / or metadata for a particular time period.

A method for managing a data stream (DS) includes: receiving a first request from an administrator; obtaining the DS from a streaming storage system (SSS); performing a first analysis of the DS to extract relevant data; making a first determination that the relevant data is worth indexing; generating a set of indexes corresponding at least a portion of the DS; receiving a query about a data item in the DS from a user; sending a second request to a database engine comprising the query; performing a second analysis of the query; making a second determination that a re-indexing of the portion of the DS is not required; identifying an index from the set of indexes that is associated with the query; sending position information of the data item to an API service; retrieving the data item from the SSS; and initiating displaying of the data item to the user.

The invention relates to the technical field of digital display, in particular to a national dance digital display system which comprises a track node extraction module, a rhythm segment optimization module, a trend stability recognition module, an action information acquisition module and a synchronous recombination execution module. According to the method, angle abrupt change points are extracted by constructing spatial vectors, action change recognition is enhanced, the frame sequence structure analysis precision is improved, time length and amplitude dual stability standards are introduced in beat paragraph screening, the accuracy and robustness of rhythm recognition are enhanced, trend stable frames are extracted through slope fluctuation comparison, and the accuracy and robustness of rhythm recognition are improved. The continuity of the rhythm flow direction and the centrality of node selection are ensured, the main limb movement is constructed by the path total distance, the direction angle and the frame time sequence, the clear structure and the consistent direction of the movement expression are ensured, the direction correction and rhythm scaling strategy is introduced in the dynamic recombination process, the smooth movement transition and the rhythm synchronization are ensured, and the reorganization efficiency is improved. And the rhythm coordination and the dynamic coherence of dance actions are obviously improved.

The invention relates to the technical field of industrial monitoring, and discloses a factory real-time digital monitoring method and system based on the Internet of Things, and a storage medium. The method comprises the following steps: carrying out encryption access on heterogeneous Internet of Things equipment through dual identity authentication to obtain a security authentication equipment identifier pool; performing multi-dimensional data acquisition on the equipment identification pool based on Hash verification to obtain an encrypted time sequence data chain; performing dynamic identification on the factory visual image by adopting a confidence adaptive threshold to obtain a multi-task fusion feature code; performing heterogeneous fusion on the time sequence data chain and the feature code by using a sliding window weight distribution algorithm to obtain a factory digital state fingerprint; and performing knowledge graph reasoning on the state fingerprint through three-level early warning threshold judgment to obtain a real-time risk level early warning code. The technical problems that heterogeneous Internet of Things equipment cannot be uniformly accessed and authenticated, multi-source data lacks safety fusion processing, and the factory state cannot be intelligently reasoned and pre-warned are solved.

Disclosed are methods, systems, and non-transitory computer readable memory for processing, analyzing, and visualizing complex digital object sets using large language models. For instance, a method may include receiving a set of digital objects; indexing the received set of digital objects to generate indexed data; generating a timeline prompt based on the indexed data; processing the timeline prompt to generate a timeline response; and outputting the timeline response as an interactive timeline of based on the set of digital objects.

The invention relates to the technical field of content recognition, in particular to an Al-based video content analysis method and system, and the method comprises the following steps: carrying out the image segmentation learning based on a video sequence through employing a U-Net convolutional neural network, analyzing scenes and elements in a video frame, recognizing and isolating key visual elements in a video through network learning, and carrying out the recognition of the key visual elements in the video. Comprising objects and figures. According to the method, image segmentation learning is carried out by adopting the U-Net convolutional neural network, key visual elements in the video can be identified and isolated more accurately, a clearer basis is provided for follow-up scene change and key event tracking, video content analysis is carried out by applying the graph neural network, the identification capability of dynamic scenes and events is enhanced, and the identification efficiency is improved. Deeper structured understanding is provided for video content indexing, the video quality is analyzed through a structural similarity index evaluation method, and the reason of visual quality reduction can be accurately recognized and improved.

The present invention relates to a method, system, device and medium for detecting abnormal behavior. The method comprises: obtaining a video to be identified, and extracting a plurality of video frames from the video in sequence; wherein each video frame carries a corresponding time mark; inputting each video frame and a preset behavior description set into a multimodal large language model, performing multimodal fusion on the input video frames and the behavior description set, and identifying abnormal behaviors in each video frame; wherein the behavior description set is used to indicate abnormal behaviors to be detected; arranging the abnormal behaviors in each video frame according to the generation order, and inputting them into a behavior analysis model together with corresponding time marks, and detecting abnormal behaviors in the video; wherein the behavior analysis model is a time series model based on Transformer. The present invention improves the problem that the prior art only uses a single modality analysis method, resulting in a low abnormal behavior recognition rate, and realizes accurate recognition of abnormal behaviors.

A system and method for generation-augmented latent hyperspace navigation in spatiotemporal media using hierarchical and Lorentzian autoencoders. The system compresses media into latent representations while preserving geometric, temporal, and semantic relationships. A latent hyperspace manager organizes compressed data as geodesic trajectories, and a geodesic trajectory mapper computes navigation paths. Symbolic anchors provide persistent reference points, while spatiotemporal routing coordinates decisions across multiple scales. A strategy caching system preserves successful navigation patterns for reuse as procedural memory. A synthetic content generator including latent diffusion models, neural radiance fields, and context-aware refinement produces augmentation for continuous zoom, bidirectional traversal, and rotational reorientation. A user input interface and zoom controller enable interactive exploration and reconstruction, supporting applications in immersive media, visualization, and surveillance.

The invention provides a multi-modal data pairing method and system based on deep learning, and relates to the technical field of data processing, and the method comprises the steps: obtaining a video multi-frame sequence and a target text, and respectively extracting an overlapped frame group set and a standardized text sequence; performing spatio-temporal feature extraction and text dependency relationship coding to obtain a video time sequence vector sequence and a text vector sequence; executing cross-modal alignment search, and constructing a monotonic matching path set; calculating a semantic and action entity relationship consistency score of the paired elements on the path to obtain a comprehensive score; and determining an alignment relationship between the video and the text based on the optimal path. According to the method, accurate matching of the video and the text is realized, and the cross-modal retrieval efficiency is improved.

Embodiments of the present disclosure relate to multimodal AI-based search for digital assets via an indexing and / or search pipeline. With respect to the indexing pipeline, some embodiments obtain first data and second data associated with a first digital asset. Such data represents different data types or modalities of the same digital asset. After obtaining the first and second data, some embodiments then generate a composite index. After the composite index is built such index can then be used to execute a query via the search pipeline. To execute the query some embodiments compute a relevance score for each digital asset, of multiple digital assets, based at least in part on a measure in which each digital asset satisfies one or more parameters or conditions for two or more data types of the query. Various embodiments then rank each digital asset and present one or more associated indicators.

The invention discloses a media information processing method and device, a storage medium and electronic equipment. The method comprises the steps that to-be-retrieved source media information and prompt template information are obtained, the source media information and the prompt template information are input into a target retrieval model, target media information is determined, and the target retrieval model represents a model obtained through joint training according to at least two retrieval task types; a loss function used in the training process of the target retrieval model is a joint loss function, the joint loss function is determined by at least two sub-loss functions in one-to-one correspondence with the at least two retrieval task types, and the target media information represents media information obtained by retrieving the source media information according to the prompt template information. According to the method and the device, the technical problem of relatively low retrieval efficiency of the media information caused by single multi-modal retrieval task is solved.

The invention relates to an oil unloading and transportation method and an oil unloading and transportation device. The method comprises the following steps: performing voice recognition on audio information acquired in real time on site to obtain at least one voice instruction; when the voice instruction satisfies a first trigger condition, based on the voice instruction, obtaining traffic information corresponding to the voice instruction, the traffic information including traffic bill identification information and / or field identification information; when the voice instruction and the traffic information meet a second trigger condition, matching the traffic bill identification information, the field identification information and pre-stored information in a database based on a matching algorithm to obtain a matching result; if the matching result is successful, an oil unloading task is executed; and if the matching result is failure, performing voice feedback on the matching result. By adopting the method, the checking flexibility can be improved and the checking cost can be reduced.

The invention discloses an intelligent conference recording and recording method and system. According to the invention, cloud service equipment receives multi-modal information sent by edge equipment; processing the multi-modal information, and generating conference record information in combination with context sensing error correction, content segmentation, a dynamic segmentation strategy and a preset storage strategy; carrying out encryption processing on the conference record information based on the participant role information, and generating target format conference information; processing the target format conference information to generate initial conference scene mode information including a conference outline, a PPT and a mind map; processing the target format conference information and the initial conference scene mode information to generate conference information data after priority ranking and strategy processing; performing semantic analysis, feature fusion and cross-modal association processing on the conference information data to generate target conference scene mode information; and determining a target application program based on the target conference scene mode information, and sending the target format instruction information to the target application program.

The invention discloses a video retrieval generation method and device based on sparse representation and reordering, and relates to the technical field of cross-modal video retrieval. The method comprises the following steps: acquiring a query text, a candidate video sequence and a video retrieval generation model; obtaining a query text dense representation and a candidate video sequence dense representation according to the encoder module, the query text and the candidate video sequence; through a sparse representation module, text sparse representation is generated according to the query text dense representation, candidate video sparse representation is generated according to the candidate video sequence dense representation, and reverse index screening is performed on the candidate video sequence to obtain part of candidate videos; determining a global score of each video in the partial candidate videos through a cross attention module, and sorting the partial candidate videos; and through a generation module, generating a target text corresponding to the query text according to the sorted candidate videos and the query text. By adopting the method and the device, the retrieval efficiency is improved in a sparse representation and reordering mode.

A system and method for latent geodesic traversal across multi-axis hyperspaces for real-time video reconstruction and augmentation. Spatiotemporal video data are compressed into navigable latent representations using hierarchical and Lorentzian autoencoders that preserve geometric and temporal structure. A geodesic traversal engine computes paths across spatial, temporal, spectral, and semantic axes, guided by symbolic anchors and spatiotemporal routing protocols. A correlation network restores fine detail, while an augmentation generator synthesizes additional or counterfactual content to enable infinite zoom, continuous multi-scale exploration, and temporally coherent augmentation. A strategy caching system preserves successful traversal patterns for reuse, supporting persistent learning and adaptive real-time performance.

Video frames from a video are compressed into a single image or a single data structure that represents a unique visual flowprint or visual signature for a given activity being modeled from the video frames. The flowprint comprises a computed summary of the original pixel values associated with the video frames within the single image and the flowprint is specific to movements occurring within the video frames that are associated with the given activity. In an embodiment, the flowprint is provided as input to a machine-learning algorithm to allow the algorithm to perform object tracking and monitoring from the flowprint rather than from the video frames of the video, which substantially improves processor load and memory utilization on a device that executes the algorithm, and substantially improved responsiveness of the algorithm.

The invention discloses a multi-dimensional time sequence data compression and rapid retrieval method and system, and relates to the technical field of large data compression retrieval. The multi-dimensional time sequence data compression and rapid retrieval method comprises the following steps: S1, collecting and preprocessing multi-source data in a monitoring abstract video stream, and constructing a standardized time sequence behavior data set; s2, analyzing behavior characteristics of frame segments in the sliding window, and dynamically adjusting anchor point labeling and compression strategies; s3, evaluating the coverage integrity of anchor point information in a compression section, and driving generation of an index path; s4, comprehensively evaluating the path behavior association strength and dynamically adjusting a loading decision; and S5, verifying the matching integrity of the behavior anchor point field and the jump pointer, and guaranteeing the localizability and jump stability of the event in the compression structure. The problems that in an intelligent monitoring scheme, video abstract compression is not bound with behavior semantics, a fine-grained positioning mechanism based on behavior labels is lacked, and key action segments cannot be directly positioned during user playback are solved.

A method includes receiving a series of video segments and providing the series of video segments as input to a first machine learning model to produce text data. The text data is provided as input to a second machine learning model to produce categorized text data that includes a classification indication. The classification indication is added to metadata of the video segment, and the categorized text data is provided as input to a third machine learning model to produce a semantic vector. The method also includes causing the video segment and the metadata that includes the classification indication to be stored at a location of a database based on the semantic vector, the database being configured to be searched based on a search query associated with the semantic vector.

Method and apparatus for video processing. A plurality of videos are accessed. An index is generated based on one or more key frames and one or more video features extracted from the plurality of video. A user input is processed to identify user intentions. The index is filtered to identify a plurality of candidate key frames based on the user intentions. A recommended video is generated based on the plurality of candidate key frames, further comprising selecting one or more additional frames based on processing the plurality of candidate key frames using one or more clustering operations, and aggregating the plurality of candidate key frames and the one or more additional frames.

The invention belongs to the technical field of automatic driving test, and particularly relates to a multi-mode accident scene library construction method for end-to-end automatic driving test. The method specifically comprises the following steps: step 1, extracting text information based on an MCAT-BiLSTM-CRF algorithm; 2, designing the ontology architecture, and storing accident scene information by using a knowledge graph to obtain an accident text knowledge base; step 3, derivative expansion is carried out on the accident scene element combination based on a HyCon-Sg-Net algorithm; step 4, performing enhanced fine tuning on the Open-Sora model to realize modal conversion from the text knowledge base to the accident video database so as to construct a multi-modal accident scene library; the method can be used for testing the performance of an end-to-end automatic driving automobile in an extreme accident scene, and the adaptive capacity of an automatic driving algorithm to the extreme accident scene is remarkably improved.

The invention relates to the technical field of workflow construction, in particular to an AI workflow construction method and system based on image recognition and electronic equipment. The method comprises the following steps: recording a working process by using a video recording tool, collecting operation data, analyzing video content and extracting key frames and operation steps by adopting image recognition and natural language processing technologies, and performing time sequence division and behavior analysis on operation behaviors in combination with user operation event data so as to determine actual operation steps. The method comprises the following steps: generating a detailed step list with alliance tag information, then combining operation steps with video segments according to an analysis result to form a structured work flow, adding detailed description and annotation information for each step, and finally establishing an index structure, so that a user can quickly retrieve a required business flow video or document through keywords. According to the method, the visualization degree and the understandability of the business process can be improved, the learning cost is reduced, and the working efficiency is improved.

The invention discloses a long video content information acquisition method based on an OCR and voice recognition technology. The method comprises the following steps: S1, carrying out preprocessing n on input long video data to extract an image frame sequence and an audio stream; s2, inputting the image frame sequence into an OCR recognition module, inputting the audio stream into an ASR recognition module, and obtaining a preliminary recognition result; s3, constructing a multi-target fitness function, and optimizing an OCR and ASR parameter combination by using a Kanglizard optimization algorithm; s4, respectively applying the optimal parameter group to an OCR identification module and an ASR identification module to obtain an optimized identification result; s5, constructing a fusion factor graph, executing edge message passing by adopting a belief propagation algorithm, and generating a multi-modal semantic block set; and S6, processing the multi-modal semantic block set to generate a unified multi-modal content information set. According to the invention, through fusion of the horny lizard optimization algorithm and the belief propagation mechanism, high-precision recognition and multi-modal semantic consistency extraction of the image text and the voice information in the long video are realized.

The invention relates to the technical field of multi-mode intelligent video understanding, and provides a long video key frame retrieval method and device based on a knowledge graph. Through the processes of frame-level subtitle generation, frame-level knowledge graph construction, similarity video segmentation and fragment and abstract generation, a long video knowledge graph construction assembly line is constructed, and structured modeling of long video semantic content is realized. By setting a two-stage retrieval mechanism, higher retrieval precision can be obtained while the efficiency is ensured. Vector matching and multi-hop neighbor extension are carried out on a unified knowledge graph, and a node set strongly related to the problem is positioned, so that the semantic gap between the natural language problem and a structured graph is reduced, and the accuracy of key frame selection is improved. By setting an iterative retrieval mechanism, the retrieved key frame can be used as a basis for answering a question text to the maximum extent.

The invention discloses a video processing method and system based on a big data technology, and constructs an intelligent video processing system with semantic driving, man-machine collaboration and continuous evolution by fusing advanced technologies such as big data processing, bimodal AI analysis, knowledge graph, natural language understanding and feedback learning. Compared with a traditional video monitoring system, the scheme has the advantages that the retrieval efficiency, the semantic understanding depth, the event association analysis capability, the user interaction experience, the system self-optimization capability and the like are remarkably improved, and the problems of incomplete seeing, difficulty in finding, inaccuracy in judgment and poor use are effectively solved.

The invention relates to the technical field of traffic management, in particular to a road target detection system, method and device, and the method comprises the steps: collecting a vehicle image of a roadside parking region; analyzing the vehicle image data to detect vehicle information in the parking space; based on the vehicle information, blind area vehicles generated due to shielding of front and rear vehicles are recognized and marked; recording parking information of a vehicle in a parking space in a blind area, wherein the parking information comprises mark information, a position and parking time; after the vehicle in the blind area is driven out of the parking space, whether the marked vehicle is still in a shielding state or not is checked; if the marked vehicle is still in the shielding state, continuously acquiring feature information of the marked vehicle; searching and matching corresponding vehicles in other cameras in a preset range based on the feature information to obtain a target vehicle; and obtaining license plate information based on the target vehicle through a license plate recognition technology, and replacing the mark information in the parking information with the license plate information.