Patents
Literature
Patsnap Eureka AI that helps you search prior art, draft patents, and assess FTO risks, powered by patent and scientific literature data.

1710 results about "Key frame" patented technology

A keyframe in animation and filmmaking is a drawing that defines the starting and ending points of any smooth transition. The drawings are called "frames" because their position in time is measured in frames on a strip of film. A sequence of keyframes defines which movement the viewer will see, whereas the position of the keyframes on the film, video, or animation defines the timing of the movement. Because only two or three keyframes over the span of a second do not create the illusion of movement, the remaining frames are filled with inbetweens.

Long video understanding method based on iterative hierarchical key frame selection

A long video understanding method based on iterative hierarchical key frame selection comprises the following steps: firstly, carrying out hierarchical frame sampling on an input video to obtain hierarchical visual features, and then carrying out a key frame query module on the Nth-layer visual features of a video frame and an input problem text to obtain a key frame query result; video key frame query is carried out in combination with context semantics to generate query vectors, whether the query vectors are enough to generate answers to the input questions is predicted, if not, the query key frames are updated in the (N + 1) th layer of visual features with the higher frame rate, and the input question texts are combined with the context semantics to carry out iterative query of the key frames again; and inputting the inquired key frame and the instruction or the question into a multi-mode large model to generate an understanding result. According to the method, the video frames are queried and the associated frames are screened out based on the user input problem, so that the performance and efficiency of long video understanding are improved, and the method has wide application prospects in scenes such as entertainment, security and education.
Owner:NANJING UNIV +3

Video content semantic understanding and text description generation method based on deep learning

The invention discloses a video content semantic understanding and text description generation method based on deep learning, and relates to the technical field of multimedia information processing.The method comprises the steps that the semantic similarity of a text and a video frame is calculated through a CLIP model, related key frames are selected, and features are aggregated; respectively extracting audio, visual and semantic features; aligning different modal features by using self-attention, unifying dimensions of the LSTM, and then splicing and fusing; attention weights are calculated at a video level, a frame level and a channel level, and key information expression is enhanced; swin Transform encodes fusion features, and LSTM (Long Short Term Memory) decodes step by step to generate natural language description; and a text-video index database is constructed, and rapid retrieval is realized based on semantic similarity. According to the method, the mapping relation between the video features and the natural language is learned end to end through the deep learning model, dependence on a fixed template can be eliminated, and semantic description with various sentence patterns and coherent logic is generated.
Owner:CHINA UNIV OF MINING & TECH YINCHUAN COLLEGE

High-fidelity cloud rendering cluster scheduling method and system

The invention relates to a high-fidelity cloud rendering cluster scheduling method and system, and the method comprises the steps: receiving a rendering task, generating a multi-level priority queue according to the task complexity, timeliness and resource demand classification, and automatically optimizing a scheduling strategy; monitoring the resource state of the heterogeneous computing node in real time; allocating tasks by using preemptive and round-robin scheduling strategies, and adjusting and coping with resource fluctuation in combination with a dynamic code rate; the tasks are decomposed by adopting a spatial blocking and time framing strategy, and an execution sequence is controlled according to a topological sorting algorithm; abnormal nodes are identified through heartbeat detection, and affected subtasks are migrated through incremental task updating. According to the method, efficient resource allocation, scheduling algorithm and idle key frame scheme can be realized, the GPU directly outputs the video stream, the rendering speed is remarkably improved, resource waste and operation cost are reduced through a dynamic resource allocation mechanism, a fault-tolerant mechanism is provided, the task is ensured to be normally completed under the condition of node fault, and the task efficiency is improved. And distributed rendering and synthesis of large-scale high-resolution images are supported.
Owner:SHENZHEN TRAFFIC CONSTR ENG TEST & DETECTION CENT +1

Image target tracking method and device based on deep learning

The invention relates to the technical field of image processing, and provides an image target tracking method and device based on deep learning, and the method comprises the steps: obtaining an original frame sequence of a target video, carrying out the adaptive frame screening of the original frame sequence, and obtaining a plurality of feature frame sequences, key frames and common frames, performing feature extraction and feature mapping on all the feature frame sequences to obtain target trajectory nodes, performing trajectory optimization calculation on the key frames and the common frames by using all the target trajectory nodes to obtain key frame trajectories and common frame trajectories, and inputting the key frame trajectories and the common frame trajectories into a preset deep learning model to perform trajectory reconstruction optimization so as to obtain a target trajectory; and obtaining a tracking target trajectory. By performing adaptive frame screening on the original frame sequence and generating the feature frame sequence, the key frame and the common frame, the data processing efficiency and the information extraction accuracy are improved, the accurate generation of the target motion track is realized, and the problems of reduced tracking accuracy and delayed system response in a complex scene are solved.
Owner:HOHEM TECHNOLOGY CO LTD

Classification method and device based on user video data and medium

The invention relates to the technical field of image processing and mode recognition, in particular to a classification method and device based on user video data and a medium, and the method comprises the steps: collecting video data uploaded by a user, and extracting a plurality of key frames from the video data; using a video feature extraction network and an image-level spatial feature extraction network with time sequence modeling capability to respectively extract time sequence features between the key frames and spatial features of each key frame, and generating video visual features based on the time sequence features and the spatial features; collecting behavior data of the user in the process of watching the video data, and constructing a final user behavior embedding vector based on the behavior data; fusing the final user behavior embedding vector with the video visual features to generate joint feature representation; and inputting the joint feature representation into a pre-trained graph neural network classification model for classification, and outputting one or more semantic category labels to which the video data belong.
Owner:BEIJING CENT BIOLOGY CO LTD

Autonomous lifelong SLAM method and system based on visual language model hidden space representation

The invention relates to an introspection lifelong SLAM method and system based on visual language model hidden space representation, and the method comprises the steps: extracting a semantic tag based on an RGB-D image through a semantic encoder, and generating a scene map and a semantic topological graph based on the RGB-D image and the semantic tag; generating a dynamic mask based on the scene map, obtaining a dynamic mask coverage rate, and screening key frames with high static confidence values based on the coverage rate; calculating camera pose estimation corresponding to the key frame in real time, sampling the key frame to realize layering of the key frame, and performing layering rendering by using a NeRF model to obtain a virtual view; the hidden space difference degree of the virtual view and the corresponding real image is calculated, whether error introspection needs to be carried out or not is judged based on the hidden space difference degree, and the system is used for achieving the method. Compared with the prior art, the method has the advantages that open semantic reasoning of VLM, high-precision reconstruction of NeRF and real-time positioning of SLAM are combined, and positioning and mapping accuracy is improved.
Owner:TONGJI UNIV

Task instruction generation method and device based on cross-modal fusion, equipment and medium

The invention relates to the technical field of artificial intelligence, can be applied to business scenes such as financial science and technology and medical health, and discloses a task instruction generation method, device and equipment based on cross-modal fusion, and a medium, and the method comprises the steps: carrying out the decoding and noise reduction of an input video, generating a frame sequence, and recognizing a plurality of key frames based on the inter-frame similarity; extracting spatial features of the key frames to form a sequence, and generating video spatial-temporal features in combination with time features; performing semantic preprocessing on the input text to obtain text semantic features, and acquiring motion sensor signals to obtain motion features; fusing the video spatio-temporal features, the text semantic features and the action features to generate fused features; and generating a perception vector based on the fusion feature and outputting a task instruction. According to the method, multi-modal fusion is realized through key frame extraction and space-time fusion mechanisms in combination with text semantic features and action features, and the perception expression ability and the task instruction generation accuracy are improved by using time sequence information and multi-source perception input of the video.
Owner:PING AN TECH (SHENZHEN) CO LTD

Converting video semantics into language for real-time query and information retrieval

Implementations utilize a LLM to generate content responsive to a user query directed to a video and cause audio data for the generated content to be rendered as a response to the user query. Implementations extract a subset of frames from all frames of the video as key frame(s) for the video, and utilize a vision-language model in generating a natural language description for the key frame(s) of the video. A prompt can be generated based on a transcription of the user query and based on the natural language description for the key frame(s) of the video. The prompt is processed as input, using the LLM, to generate the content responsive to the user query directed to the video.
Owner:GOOGLE LLC

Video stream-based attitude feature recognition method

The invention discloses a posture feature recognition method based on a video stream, and the method comprises the steps: carrying out the preprocessing of a continuous video stream, obtaining video frame training data, extracting a key frame and an adjacent frame in each frame of image, constructing a feature extraction module for a human body region, and obtaining a global frame, performing local extraction on the human body area by using adjacent frames on the left side and the right side to obtain local frames, and constructing semantic association information for the global frame through time sequence continuity between the local adjacent frames and the current key frame; acquiring enhanced feature representation by adopting a conditional feature aggregation algorithm; obtaining attitude sequence data through the attitude detail features; the method comprises the following steps: establishing three-dimensional coordinates, adaptively extracting posture change data by adopting a human body motion decoupling model, predicting human body posture characteristics through a smooth optimization strategy, and introducing a cross attention mechanism to realize deep fusion of spatio-temporal characteristics, so that the understanding ability of the model to a complex action mode is enhanced; and the attitude expression capability of the model in a sheltered or fuzzy region is obviously improved.
Owner:北京汇畅数宇科技发展有限公司

3D scene reconstruction method and system based on adaptive key frame

The invention provides a 3D scene reconstruction method and system based on adaptive key frames, and relates to the technical field of computer vision, and the method comprises the steps: obtaining continuous multi-frame images and inertial measurement unit data of a current scene; acquiring camera trajectory data and offset data of each inertial measurement assembly based on the inertial measurement unit data; carrying out target feature extraction and target tracking on each frame in the multiple frames of images to obtain feature point data of the to-be-tracked target; determining a key frame based on a scene complexity index and a system resource utilization rate index of the current scene; performing visual inertia joint optimization based on all feature point data, camera trajectory data and offset data, and generating point cloud data containing each feature point; and generating 3D Gaussian distribution of the current scene based on the point cloud data and the key frame, and rendering the distribution to obtain a first 3D rendered image of the current scene. According to the method, real-time 3D scene reconstruction can be carried out in a complex and changeable application scene.
Owner:江淮前沿技术协同创新中心

Long-time pedestrian re-identification method based on dual-path cooperation and key frame guided reconstruction

The invention discloses a long-time pedestrian re-identification method based on dual-path cooperation and key frame guided reconstruction. The method comprises the steps of firstly collecting a pedestrian video to be recognized, and extracting a video feature sequence; space and time position coding is introduced into the video feature sequence; capturing local fine-grained dynamic features through a local dynamic feature capturing path, and modeling long-range time sequence association through a cross-frame global feature modeling path; then, dual-path feature complementation is realized through bidirectional gating interaction; further screening out key frames, and realizing feature reconstruction through a full-frame attention propagation mechanism; and finally fusing the dual-path fusion features, the key frame guide reconstruction features and the refined features to generate pedestrian identity features. And processing pedestrian identity features to obtain standardized feature vectors, performing similarity comparison on the standardized feature vectors and pedestrian features in an image library, and returning a matching list. According to the method, video time sequence information is fully utilized, and the problem of insufficient robustness caused by appearance change in long-time pedestrian re-identification is effectively solved.
Owner:SHIJIAZHUANG TIEDAO UNIV

High-performance video reasoning segmentation method based on time sequence marking

The invention belongs to the technical field of video processing methods, and discloses a high-performance video reasoning segmentation method based on time sequence marking. The method comprises the following specific steps: carrying out time sequence mark coding by using a multi-mode large language model, carrying out time sequence dynamic aggregation on a coded special mark, and carrying out mask decoding and propagation by using an SAM2 based on a key frame selection strategy of the time sequence mark. According to the method, multiple module designs are combined, and the time sequence marks generated by the multi-modal large model are utilized, so that the perception capability of the model on spatial features and time dynamics is effectively improved.
Owner:DALIAN UNIV OF TECH

Panoramic video three-dimensional point cloud sparse reconstruction method and device integrating key frame screening

The invention provides a panoramic video three-dimensional point cloud sparse reconstruction method and device integrated with key frame screening, and the method comprises the steps: obtaining a panoramic video of a target scene, and carrying out the video quality evaluation of the panoramic video, and obtaining a video evaluation result; determining a target resolution of downsampling according to the video evaluation result; carrying out downsampling processing on the panoramic video based on the target resolution to obtain a downsampled second video; for the second video, dynamically adjusting a frame extraction interval based on a preset key frame selection mechanism, and extracting a frame index of the key frame from the second video according to the frame extraction interval to obtain a plurality of frame indexes; and obtaining a plurality of corresponding key frames in the panoramic video based on the plurality of frame indexes, and obtaining the three-dimensional scene sparse point cloud of the target scene based on the plurality of key frames, thereby solving the technical problem of low processing efficiency in the prior art.
Owner:TIANJIN FIRE SCI & TECH RES INST OF MEM

Animation video generation method and system based on video script

The invention discloses a cartoon video generation method and system based on a video script, and relates to the technical field of video synthesis, and the method comprises the steps: based on structured script data, combining a predefined lens rule base and a reinforcement learning model, determining the type, duration and lens operation effect of a lens, and generating a lens splitting sequence through a dynamic lens splitting automatic generation mechanism, based on the split mirror sequence, generating an animation style key frame image through a diffusion model, selecting action data matched with the emotion label through a predefined action library, generating a voice waveform matched with the emotion label through a text-to-voice model, selecting a background music audio matched with the emotion label through a music library, and generating a multi-modal content stream; according to the method, the lens type, the duration and the lens operation effect are dynamically optimized through the reinforcement learning model, the defects of a traditional lens scheduling method based on static rule mapping in time continuity and narrative continuity are overcome, and the narrative fluency and the dynamic adaptability of the lens division sequence are obviously improved.
Owner:CHANGCHUN VOCATIONAL INST OF TECH

Animation real-time capturing and repairing method and system

The invention relates to the technical field of animation processing, in particular to an animation real-time capturing and repairing method and system, and the method comprises the following steps: calculating the pixel change rate between continuous animation frames based on the time sequence data of the animation, determining the change threshold values of key frames and non-key frames, dynamically adjusting image collection parameters according to the change threshold values, and carrying out the real-time capturing and repairing of the animation. And the frame difference before and after adjustment is compared and analyzed. According to the method, through analysis of the animation time sequence data and real-time calculation of the pixel change rate, the continuity between the animation frames is effectively monitored and real-time adjustment is performed, so that the accuracy and efficiency of animation production are improved, visual errors are corrected in real time in the production stage, the high-cost later-stage repair requirement is reduced, and the production efficiency is improved. The frame-by-frame real-time semantic segmentation and visual element identification further enhance the accurate control of the animation content, ensure the image quality of each frame, and ensure the consistency and visual effect of the animation by refining the image quality and synchronously repairing the frames.
Owner:SHENZHEN BENDAO TECH CO LTD

Non-inductive identity recognition and real-time tracking method based on personnel in video

The invention discloses a non-inductive identity recognition and real-time tracking method based on personnel in a video. The method comprises the following steps: S1, acquiring video stream data and extracting a key frame image; s2, key point coordinates are extracted, and a time sequence skeleton sequence is constructed; s3, decomposing gait, action and attitude features to generate standardized features; s4, inputting the standardized features into an improved multi-scale image convolutional neural network, extracting identity representation features, and generating a target identity code; s5, matching identity codes based on similarity measurement, calculating a consistency score, and establishing an identity tracking trajectory; s6, tracking is carried out in combination with identity codes and consistency scores, identity mapping is dynamically updated, and non-inductive recognition is achieved; and S7, updating a track in real time, adjusting a tracking state, and ensuring identity stability. According to the method, the improved graph convolutional network and the consistency measurement technology are combined, the identity recognition precision and tracking stability are improved, and cross-scene non-inductive identity recognition is achieved.
Owner:BEIJING LIYANG ZHIGUANG TECH CO LTD

Video generation method and device based on action coherence, equipment and medium

The invention relates to the technical field of data analysis, can be applied to business system platforms of financial science and technology, medical health and the like, and discloses a video generation method, device, equipment and medium based on action coherence, the method comprises the following steps: obtaining a video action material set, carrying out key frame analysis on the video action material set to obtain a key video frame sequence, extracting an inter-frame residual vector between consecutive frames in the key video frame sequence, adjusting a preset initial diffusion model by using the inter-frame residual vector to obtain an optimized diffusion model, performing cosine scaling on the inter-frame residual vector by using the optimized diffusion model to obtain a scaled residual vector, and obtaining a video adjustment text, and carrying out noise addition and splicing on the key video frame sequence by using the video adjustment text and the zoom residual vector to obtain a noise video, and carrying out noise reduction on the noise video to obtain a target action video. According to the method and the device, the action coherence in the customized generated video can be effectively improved.
Owner:PING AN TECH (SHENZHEN) CO LTD

Generative video coding and decoding method based on multi-modal large model

The invention relates to the technical field of video coding and decoding, and discloses a multi-mode large model-based generative video coding and decoding, which comprises a key frame selection module for determining a key frame by analyzing the semantic and motion characteristics of a video frame; the multi-modal semantic description generation module is used for generating semantic description according to the key frame and the video clip; the key frame compression module is used for realizing efficient compression through latent variable modeling and entropy coding; the key frame reconstruction module reconstructs a key frame by using a conditional diffusion model in combination with the compressed data and the semantic description information; and the video generation module generates a non-key frame by using the semantic description and the key frame, and reconstructs a complete video. Through key frame screening combining semantic and motion information, key frame compression and reconstruction based on a conditional latent variable diffusion model, and frame supplementation and frame insertion generation based on semantic description, efficient compression and high-quality video reconstruction can be realized under a low code rate, and the video storage efficiency and the visual quality are effectively improved.
Owner:上海芯开技术有限公司

Loopback detection method based on active vision and text semantics and related equipment

The invention relates to a loopback detection method based on active vision and text semantics. The loopback detection method comprises the steps that the pose of a mechanical arm is changed to drive a camera to conduct image collection on the surrounding environment to obtain a front view text image; synchronously controlling the laser radar to collect multiple frames of point clouds to screen key frame point clouds; selecting a key text image corresponding to the key frame point cloud from the multiple frames of front view text images, extracting text information in the key text image, and converting the text information into a global coordinate system; calculating the similarity between the text information in the key text image under the global coordinate system and the historical text information obtained at the historical moment, and determining candidate loopback point clouds according to the similarity; and performing geometric consistency calculation on the candidate loopback point clouds to obtain a distance value so as to obtain a loopback detection result. The technical problem of loopback detection errors caused by feature degradation, sparse or similar environments is solved by actively and visually collecting the front view image of the text information and combining the double loopback verification of the point cloud, and the precision and robustness of system positioning and mapping are improved.
Owner:GUANGDONG LAB OF ARTIFICIAL INTELLIGENCE & DIGITAL ECONOMY (SZ)

Game controller animation synchronization method and system

The invention discloses a game controller animation synchronization method and system, and relates to the technical field of data transmission. The method comprises the following steps: step S1, input capture; step S2, data preprocessing; s3, performing a prediction algorithm; step S4, data compression and packaging; step S5, network transmission; step S6, processing by a server; step S7, receiving and analyzing by the client; step S8, generating an animation; step S9, performing synchronous correction; and step S10, a feedback mechanism. According to the method, the input data of the game controller is collected, the next operation of the user is predicted, the skeletal animation is generated according to the predicted result, physical skin rendering is conducted on the animation, the skeletal animation or the key frame difference value is used, the playing speed is adjusted in combination with dynamic time distortion, the animation synchronization delay is reduced, and the animation synchronization effect is improved.
Owner:HANGZHOU ZANRUAN TECHNOLOGY CO LTD

Method for automatically measuring and calculating project progress of transformer substation

The invention relates to the field of construction engineering management, and discloses a substation engineering progress automatic measurement and calculation method, which comprises the following steps: S1) constructing a three-dimensional BIM model containing a geometric model and attribute information by obtaining substation engineering related data, and marking key feature points for subsequent progress comparison; s2) arranging a camera at a construction site, acquiring a construction video stream in real time and transmitting the construction video stream to the central server; s3) performing denoising and key frame extraction on the video image, and extracting image feature points and descriptors by using a deep learning algorithm; s4) matching the image feature points with corresponding points in the BIM model, calculating the deviation between the actual progress and the planned progress, and generating construction progress data; and S5) setting an early warning level according to the construction deviation, generating a progress report, and sending alarm information in multiple modes. According to the method, automatic measurement and calculation of the construction progress are realized by combining the BIM model and video monitoring, and manual intervention is reduced.
Owner:江西博微新技术有限公司

Video clip label generation method and system for scene change detection

The invention discloses a video clip label generation method and system for scene change detection, and the method comprises the steps: carrying out the segmentation of a video clip, and carrying out the preliminary recognition of a scene change point in a video through scene change detection; dividing the video into segments with independent scenes, and extracting representative key frames from each scene segment; using a highlight detection algorithm to identify a highlight area in the video frame; and searching a minimum point of a function of the time and the scene change score, so that the video is divided into segments with non-repeated time, and static information and dynamic information in the segmented video segments are extracted and marked, so that the generation of video segment labels is realized. The invention further discloses a video clip label generation system for scene change detection. According to the method, the video can be accurately segmented into fragments which are not repeated in time, so that the management and retrieval efficiency of the video content is greatly improved, and a user can more quickly find related video fragments.
Owner:SUN YAT SEN UNIV

Key frame extraction method and device based on dynamic reinforcement learning, equipment and medium

The invention relates to the technical field of computer vision, can be applied to the medical field and the financial science and technology field, and discloses a key frame extraction method, device and equipment based on dynamic reinforcement learning and a medium, which are applied to electronic application in a high-frequency transaction abnormal behavior monitoring scene or can be applied to a medical operation key frame extraction scene. The method comprises the steps of obtaining an original video stream and performing preprocessing to generate a standardized video frame; performing feature extraction and feature splicing on the standardized video frame, and performing time sequence modeling on the generated pair frame-level mixed feature vector to generate video-level time sequence representation; generating an enhancement action instruction based on the video-level time sequence representation through the strategy network, and performing enhancement processing on the standardized video frame according to the enhancement action instruction to generate an enhanced video frame; performing optimization processing on the strategy network according to the enhanced video frame to generate an updated strategy network; and performing feature extraction and optimization on the enhanced video frame to generate a target key frame. According to the invention, the key frame extraction precision is improved.
Owner:PING AN TECH (SHENZHEN) CO LTD

Intelligent conference video frame dynamic coding method based on multi-mode semantic understanding

The invention relates to the technical field of computer vision, in particular to an intelligent conference video frame dynamic coding method based on multi-modal semantic understanding, which comprises the following steps: acquiring a video stream sequence and a synchronous audio stream in a conference scene in real time; performing semantic analysis and decoupling on the video stream sequence, and extracting key frames and subsequent frames; extracting a sparse motion field from a subsequent frame, and segmenting a video frame into candidate visual areas including a face, a mouth shape and a background; extracting audio semantic features, executing cross-modal semantic correlation analysis, calculating semantic correlation between the sparse motion field distribution features and the audio semantic features, and positioning a pronunciation area highly related to the voice content; and calculating a quantization offset value of each candidate visual area according to the semantic relevancy, applying the quantization offset values in different areas, and packaging the quantization offset values into a variable-code-rate video code stream. According to the invention, the multi-mode semantic understanding model is constructed to carry out deep semantic analysis on the video frame content so as to realize the dynamic coding of the conference video frame.
Owner:SHENZHEN JIKEYUAN ELECTRONIC TECH CO LTD

Video text cross-modal retrieval method based on spatio-temporal feature fusion

The invention relates to the field of artificial intelligence cross-modal retrieval, and provides a video text cross-modal retrieval method and system based on spatio-temporal feature fusion. The method comprises the following steps: carrying out key frame sampling and time sequence partitioning on an input video, extracting static visual features through a spatial feature network, and extracting motion features through a time dynamic network; a self-adaptive gating fusion module is adopted to dynamically calculate spatial-temporal feature weights and perform weighted fusion; extracting text semantic features by using a pre-training language model; constructing a double-flow projection network to map video fusion features and text features to a unified measurement space, and optimizing a feature distance by adopting a contrast loss function containing difficult negative sample mining and intra-modal constraint; and outputting a retrieval result according to the cosine similarity sequence. The system comprises four units, wherein the gating fusion module is integrated with an FPGA acceleration circuit. According to the method, mAP (at) 10 is equal to 0.78 in a UCF-101 data set, the time sequence action retrieval accuracy rate is 92.8%, and the single video retrieval delay is 23 milliseconds.
Owner:ZHEJIANG UNIV

Short video analysis processing method and system based on AI intelligence, and storage medium

The invention belongs to the technical field of short video abnormity review, and discloses a short video analysis processing method and system based on AI intelligence, and a storage medium, the complexity entropy value and the motion vector change rate of a target short video are analyzed, and then a logic is triggered to execute a dynamic fragmentation operation based on a preset adaptive fragmentation mechanism, so that the short video abnormity review efficiency is improved. The fragmentation strategy can be dynamically adjusted according to the real-time change of the video content, and the universality and adaptability of fragmentation processing are improved. According to the method, the candidate key frames are identified by acquiring the HSV histogram difference degree and the SIFT feature matching degree of each frame of image corresponding to each video clip, and the candidate key frames are screened based on the preset key frame selection redundancy prevention mechanism to obtain each key frame, so that a key frame set is ensured to be simple and efficient, the efficiency and quality of each link of video processing are improved, and the user experience is improved. And the accuracy of subsequent data analysis is ensured.
Owner:HEBEI CANGZHENG INFORMATION TECHNOLOGY CO LTD

Video acceleration processing method under distributed computing framework

The invention discloses a video acceleration processing method under a distributed computing framework, which belongs to the technical field of video processing, and specifically comprises the following steps of: analyzing coding parameters of an input video stream, identifying an original GOP (Group of Picture) structure, reconstructing a GOP by inserting a supplementary key frame, and generating a block unit starting from the key frame and containing a complete GOP; collecting a real-time resource state of each node in the distributed cluster; optimizing a task allocation strategy based on an improved genetic algorithm; calling a corresponding parallel processing mode according to the coding format of the block unit, inputting the blocks into a distributed reasoning cluster, and performing multi-node collaborative reasoning through model segmentation; performing timestamp calibration on a processing result of each node based on a global clock server, and compensating time offset during block splicing through linear interpolation; checking a block processing state according to a set period if a node fails or is overtime; and redistributing the uncompleted blocks to the standby nodes, and continuing processing by inheriting an intermediate calculation result of the original nodes.
Owner:TIANJIN NANKAI HAOYU XINXING INFORMATION TECHNOLOGY CO LTD

Prompt construction method and system of multi-mode large language model, computer equipment and medium

The invention relates to the technical field of multi-modal large language model training, in particular to a prompt construction method and system for a multi-modal large language model, computer equipment and a medium. The method comprises the following steps: extracting a key frame set from an input video stream; executing a motion reconstruction process on the video stream to generate motion track information; and performing visualization processing on the motion track information to generate a track visualization graph. Performing space-time correlation coding on the key frame set and the motion track information to generate an enhanced key frame; a multi-modal prompt is constructed in a mode of integrating visual input and text input, and the multi-modal prompt is input into a preset multi-modal large language model for spatial reasoning. Through the mode, the technical problem that an existing prompting method is difficult to give consideration to the spatial reasoning precision and the calculation efficiency is solved, efficient and accurate spatial reasoning of the multi-modal large language model is achieved, and the calculation efficiency, the reasoning precision and the environmental adaptability of the model are improved.
Owner:HONG KONG UNIV OF SCI & TECH (GUANGZHOU)

Video background music generation method and system, intelligent terminal and storage medium

The invention discloses a video background music generation method and system, an intelligent terminal and a storage medium, and relates to the technical field of computers, and the method comprises the steps: obtaining a to-be-processed video; determining a plurality of candidate key frames from the video to be processed; performing duplicate removal processing on the candidate key frames according to the structural similarity between the candidate key frames to obtain a target key frame; and generating background music matched with the to-be-processed video according to the target key frame. Thus, the candidate key frames with repeated information can be removed based on the duplicate removal processing, redundant calculation is avoided, and the processing efficiency of the background music generation process can be improved.
Owner:SHENZHEN UNIV

Automatic action trajectory labeling system and method based on visual language model

The invention provides an automatic action trajectory labeling system and method based on a visual language model, and relates to the field of visual technical models. The automatic action trajectory labeling system based on the visual language model comprises a key frame extraction module, a VLM semantic labeling module, a historical labeling database, a diffusion model optimization module and an reflection correction module, and the key frame extraction module extracts a key frame sequence from an input video through optical flow analysis and a TSN network scene change detection algorithm; and the VLM semantic annotation module is used for outputting a candidate segmentation point set containing action starting and ending points, the VLM semantic annotation module adopts a multi-modal VLM, takes a key frame image and a context text instruction as input, and generates a preliminary semantic tag and a corresponding vision-language embedding vector. According to the method, multiple bottlenecks of the existing action track labeling technology are solved, the labeling precision is remarkably improved, and the target of time boundary optimization is achieved.
Owner:高杨