1135 results about "Frame sequence" patented technology

The invention relates to the technical field of computer vision, and discloses a real-time video analysis method based on deep learning. The method comprises the following steps: acquiring a real-time video stream through image acquisition equipment, and performing frame segmentation processing to generate a continuous video frame sequence; and extracting features of the video frame sequence by using a pre-trained convolutional neural network to obtain a multi-dimensional feature vector, inputting the multi-dimensional feature vector into the time sequence analysis model to calculate dynamic relevance, and outputting an inter-frame movement track and object behavior features. And constructing a scene understanding map containing a spatial position and a time evolution relationship according to the above-mentioned data, and carrying out abnormal event detection and generating event marking data based on the map. And performing semantic analysis on the event marking data, determining an abnormal event type and a confidence score, triggering a real-time alarm signal according to a result, and updating a historical event database. In the analysis process, the resource occupancy rate of the system is continuously monitored, the calculation precision is dynamically adjusted, a degradation processing mechanism is started when a preset threshold value is exceeded, and key area analysis is preferentially guaranteed.

The invention provides a sign language translation method and system based on a pre-training diffusion large language model, and belongs to the field of sign language video translation. The method comprises the following steps: preprocessing a video containing sign language actions to obtain a sign language video frame sequence, inputting the sign language video frame sequence into a visual feature extraction network to extract features, and fusing to obtain a time sequence visual fusion feature sequence; giving a text cue word of a sign language translation task, constructing an initial mask sequence for a target translation position, taking the text cue word, the time sequence visual fusion feature sequence and the initial mask sequence as guide conditions, injecting the guide conditions into a diffusion language model, iteratively denoising and predicting lexical elements of a masked position in combination with a diffusion mask mechanism, and obtaining the sign language translation task. A natural language translation sequence is obtained, and sign language translation is completed; wherein when the diffusion language model is trained, through an internal feature alignment mechanism, the guiding effect of guiding conditions on text generation is optimized, so that the accuracy, coherence and robustness of long text translation are improved, and the actual requirements of a barrier-free public service scene are better met.

The invention discloses a remote sensing video segmentation method and segmentation system based on text guidance, belongs to the crossing field of remote sensing image processing and computer vision, and relates to a remote sensing video segmentation method and segmentation system. The invention aims to solve the problems that the existing remote sensing video segmentation technology is poor in flexibility, cannot interact with natural languages, is insufficient in generalization ability for new categories or complex targets, and cannot meet the requirement of quickly and accurately extracting semantic information in a dynamic remote sensing scene. The method comprises the following steps: 1, acquiring a video frame sequence, and acquiring a key frame based on the video frame sequence; 2, obtaining an initial segmentation mask; 3, obtaining an optimized mask; 4, calculating a minimum bounding rectangle of the optimized mask, obtaining a bounding box of the minimum bounding rectangle, and obtaining an expanded bounding box; and 5, inputting the expanded bounding box and the video frame sequence obtained in the step 1 into an improved SAM2 video segmentation model, and outputting a frame-by-frame segmentation result of the region of interest by the improved SAM2 video segmentation model.

The invention relates to the technical field of computers, and discloses a multi-mode audio and video synchronous processing method and system based on a distributed architecture. The method comprises the following steps: generating a unique logic sequence identifier consisting of a device code, a media type and a frame number for each frame of audio and video data; distributing the frame carrying the identifier to a distributed node for processing and adding a local timestamp; and the sink node reconstructs an original frame sequence according to the identifier, calculates a synchronous offset by combining high-precision clock calibration, and realizes dynamic reordering and output through a double-buffer structure and a self-adaptive drift compensation algorithm. The system comprises a source end acquisition module, an identifier generation module, a task scheduling module, a distributed processing cluster module, a convergence synchronization module and a synchronous output module. According to the scheme, the system effectively guarantees the audio and video synchronization precision in a high-concurrency heterogeneous environment, the synchronization error is controlled within 20 milliseconds, and the playing quality and the user experience of scenes such as live broadcast and cloud games are remarkably improved.

The invention discloses a multi-mode large model video content understanding reasoning acceleration method and system, and mainly relates to the technical field of artificial intelligence reasoning acceleration. Comprising the following steps: inputting video data and preprocessing the video data to generate a video frame sequence; performing adaptive video Token compression on the generated video frame sequence, and outputting a compressed visual Token set; performing visual feature coding and Key-Value generation on the compressed visual Token set to obtain visual KV data; performing video KV cache partition management on the visual KV data; executing cross-modal reasoning based on the vLLM framework to generate a video content understanding result; and outputting a video content understanding result, and carrying out post-processing and structured mapping. The method has the beneficial effects that the obvious reasoning acceleration and throughput improvement can be realized on the premise of keeping the precision of the original large model.

The invention relates to the technical field of face verification, and discloses a face identity verification data processing method based on dynamic feature extraction, which comprises the following steps: acquiring a continuous face video frame sequence and calculating a full-pixel instantaneous velocity vector to generate an original dense optical flow field, selecting rigid region anchor points to calculate a rigid affine transformation matrix and construct a theoretical rigid motion field, performing differential stripping on the theoretical rigid motion field from the original dense optical flow field, and extracting a non-rigid micro-motion residual field; mapping the non-rigid micro-motion residual field to a facial muscle topological grid to generate a time sequence feature tensor, and calculating a geodesic line distance between a covariance matrix of the time sequence feature tensor and a reference dynamic feature in a Riemannian manifold space; when the geodesic distance is smaller than a threshold value, verification is passed, through a rigid-non-rigid orthogonal decomposition mechanism, the special viscoelastic micro-motion and cooperation law of biological soft tissue is captured by utilizing a residual field, and the high-simulation mask is effectively defended.

The invention discloses a natural resource supervision method based on three-dimensional GIS scene and video fusion, and relates to the technical field of intelligent natural supervision. Comprising the steps of collecting a video stream through a preset observation point, extracting a continuous frame sequence, and calling a space parameter set; performing dynamic region detection on the video frame sequence to generate change feature data; mapping the change characteristic data into a topological structure to be updated based on the projection matrix; calling a pre-constructed three-dimensional GIS scene model, and positioning an update area; according to the type identifier of the change feature data, executing an operation of deleting the associated geometry or constructing and fusing a new geometry; adding a state attribute label to the updated geometry, and storing an update model; the resource change rate is calculated, and when the change rate exceeds a preset industry threshold value, an early warning instruction is generated and pushed to the control terminal. And the space accuracy, the model updating efficiency and the early warning reliability of resource change identification in a complex natural scene are remarkably improved.

The invention relates to the technical field of computer vision, in particular to a space-time consistency data generation method for visual target tracking. The method comprises the following steps: firstly, training a path generator on a target tracking training set, learning a motion law of a target in a time sequence by using optical flow estimation and conditional variation coding technologies, and generating a target motion track conforming to physical constraints; and then, based on the generated target trajectory, introducing a space-time consistency attention mechanism to guide a text-video generation model, and under the condition of keeping basic model parameter freezing, constraining the position, scale and continuity of a target in a generation frame through an attention network, thereby synthesizing a video frame sequence with real motion features. According to the method, target tracking video data with real motion characteristics and high time sequence consistency is generated, and the robustness of the model to complex motion, illumination change and shielding conditions can be improved in different scenes.

The invention relates to the technical field of video coding and image processing, and discloses a code rate control method and system based on video image segmentation, and the method comprises the steps: carrying out the pixel-level semantic segmentation of a to-be-coded video frame sequence; extracting a foreground region of interest; calculating a corresponding segmentation uncertainty parameter; establishing semantic mutation parameters of the foreground region of interest; generating a semantic perception weight; executing region-level target code rate redistribution and quantization parameter mapping; and executing partition coding control. In the prior art, code rate control mainly depends on motion intensity or pixel complexity, and especially when a foreground target suddenly appears or disappears in a monitoring scene, a technical problem that a key target is blurred or a background code rate is wasted is easily caused. Due to the fact that the uncertainty modeling and semantic mutation sensing mechanism of the semantic segmentation result is introduced, priority coding of the foreground interest area is achieved under the frame-level code rate constraint condition, and the video coding quality and the code rate utilization efficiency are improved.

The invention provides a personnel intrusion detection method and system based on a target detection algorithm, and relates to the technical field of safety monitoring, and the method comprises the steps: firstly, defining an intrusion attention region and a judgment rule based on a panoramic layout image of a to-be-monitored scene, then obtaining a real-time monitoring video stream, and extracting a to-be-detected video frame sequence; and calling a pre-trained personnel target detection algorithm to carry out frame-by-frame detection on the video frame sequence to obtain personnel target detection information, carrying out region association behavior characteristic analysis based on an intrusion attention region and the detection information, finally carrying out intrusion judgment according to behavior characteristics and judgment rules, and generating a personnel intrusion detection result. Therefore, the accuracy and reliability of personnel intrusion detection can be improved, and misjudgment and missed judgment are reduced.

The invention relates to a video snapshot compression imaging reconstruction method and system. The method comprises the following steps: inputting a video frame sequence and a time-varying mask set thereof into a measurement model to obtain initial estimation; constructing a reconstruction network which comprises a feature extraction module, a gating residual network module and a video reconstruction module; the feature extraction module comprises two three-dimensional convolution layers, each three-dimensional convolution layer is connected with an activation function, and the feature extraction module extracts initial features from the initial estimation; inputting the initial features into a gating residual network module, and outputting reconstruction information features; and the video reconstruction module fuses the reconstruction information features, and performs up-sampling and detail refining to reconstruct a video sequence. According to the method, on the premise that parameters and computing power are hardly increased, ghosting and flickering are effectively restrained, the stability of long-time reconstruction is improved, and an effective scheme is provided for SCI reconstruction with the high compression ratio, the super-definition resolution ratio and the long sequence.

The invention belongs to the technical field of data analysis and processing, and particularly relates to an equipment state monitoring method based on a time sequence synchronous compression mechanism, and the method comprises the steps: 1, a monitoring terminal collects vibration type, acoustic type and acceleration type signals on a unified time baseline of Beidou time service signals, and registers a second sequence number and a frame sequence number in a unified fragment index table; 2, inputting an original time sequence data packet, constructing a phase grid frame by a phase locking ring according to a reference phase rail, extracting road-level flag event bits frame by frame, and fusing the road-level flag event bits into frame-level flag event bits; and step 3, outputting an equipment state label according to a mutual exclusion priority rule only on the basis of the synchronous compression time sequence packet and the coupling mark, the mode instruction and the abnormal fragment mark in the unified fragment index table in the identification window. The method is obviously superior to the prior art in the aspects of time sequence alignment precision, pattern recognition certainty, data compression efficiency and health state judgment reliability.

Encoding using adaptive transform type sets based on frame level statistics includes obtaining an encoded bitstream by encoding a current block of a current frame of a current sequence of frames of an input video stream using adaptive transform type sets based on frame level statistics and outputting the encoded bitstream. Encoding the current block includes obtaining transform type statistics for previously reconstructed reference frames from the current sequence of frames, the previously reconstructed reference frames including at least one previously reconstructed reference frame, determining, in accordance with the transform type statistics, a current subset of transform types from a set of available transform types, generating encoded block data for the current block using a current transform type from the current subset of transform types, and including the encoded block data in the encoded bitstream.

The present disclosure relates to systems, methods, and non-transitory computer-readable media that leverages a dual-variational autoencoder model. For example, the disclosed systems generate an image embedding from a first frame of a sequence of frames by using a two-dimensional variational autoencoder. Moreover, the disclosed systems generate motion embeddings from motion within a video by using a three-dimensional variational autoencoder. Further, the disclosed systems generate a reconstructed image from the image embedding and a reconstructed video from the motion embeddings and the image embedding. Additionally, the disclosed systems modify parameters of a dual-variational autoencoder model based on a measure of accuracy of the reconstructed image and the reconstructed video.

Systems and methods are disclosed related to a 3D grounded video foundation model. A video generation method and system provide 3D conditioning information to a video diffusion model to improve generated video quality (object and temporal consistency) that is grounded in three dimensions (3D). The video generation method and system also enable precise camera control, cinematic effects, and scene editing. Video output corresponding to a set of camera specifications is generated for a scene from input image(s) including one or more images of a static scene or a sequence of images (video) for a dynamic scene. The input image(s) are used to calculate a 3D cache representing the scene. The 3D cache is rendered according to the set of camera specifications to produce a frame sequence and a mask sequence that identifies missing pixels in each frame. The frame sequence is encoded and masked to generate the output video.

The invention discloses a short video content intelligent generation method based on deep learning, and the method comprises the following steps: obtaining short video data and candidate material frame data, and organizing and generating a candidate material frame sequence; inputting the candidate material frame sequence into an improved Slot-VAE model to generate an object-level potential slot representation set; inputting the short video data into a shot-level semantic coding network, and constructing a double-path latent variable structure; establishing an object-level slot evolution module for time sequence modeling; executing joint training, and updating parameters of the model and the network; inputting the target short video data into a shot-level semantic coding network, and generating a target shot-level semantic slot set and a target object-level slot time sequence potential state sequence; and generating a target short video frame sequence in the object-level decoding network and executing post-processing to generate a short video content intelligent generation result. According to the invention, the lens semantic consistency and the object time sequence continuity are improved.

The invention provides a multi-process decoupling and double-state self-adaptive flow pushing method for a real-time digital human, and relates to the technical field of real-time digital humans. The method comprises the steps that a server initializes a digital human instance and starts two decoupling processes by responding to a client session request; the method comprises the following steps: processing input into an audio frame sequence, forming an audio buffer area through a sliding window, extracting structured features by a first process, and transmitting the structured features with corresponding original audio through cross-process communication; in the second process, a dynamic face area is generated based on the structural features and the visual parameters, and a digital human image frame is synthesized; packaging the image frame, the original audio and the state information into a multimedia unit, and stabilizing frame rate plug flow; based on state and audio input trigger event notification, sessions are monitored, processes are terminated and resources are recycled when the sessions are terminated or abnormal, multi-process decoupling processing and double-state self-adaptive stream pushing of a real-time digital human can be achieved, stream pushing stability is guaranteed, and dynamic monitoring and resource recycling can be achieved.

The invention provides a video content enhancement method for a low-illumination environment, and the method comprises the steps: achieving the data preprocessing based on an original low-illumination video frame sequence through frame synchronization, color space conversion and local brightness analysis, generating a noise sensitivity thermodynamic diagram through multi-feature unsupervised learning, and constructing a noise perception gating mechanism through the combination of affine transformation. Dynamic modulation of the characteristic channel is realized; in the multi-scale network structure, a channel attention module is used for carrying out layer-by-layer self-adaptive adjustment on a noise sensitive area; a basic illumination image and an edge enhancement image are generated through double-branch decoding, and then weighted fusion is carried out in combination with a noise thermodynamic diagram, so that brightness balance and detail enhancement are realized; a noise smoothing regular term is introduced during end-to-end training, so that the network achieves dynamic balance between an enhancement effect and noise control.

The invention discloses a weldment production remote monitoring platform and method based on the Internet of Things, and relates to the technical field of industrial automatic monitoring. The weldment production remote monitoring platform and method based on the Internet of Things comprises the following steps: S1, collecting and preprocessing an image frame sequence, extracting a target position and a stay frequency, and generating structured space trajectory data; s2, constructing a standard task template, positioning a key region, and judging a region hit response state according to a coverage condition; s3, performing coordinate alignment analysis on the identified target spatial position and the standard task template, and identifying a coverage state and an offset feature; and S4, integrating the coverage range, the distribution balance and the key area state, and completing the comprehensive judgment of the task completion condition. The problem that an existing platform cannot judge whether a welding task is completed normatively or not based on operation area distribution is solved.

The invention relates to the field of voiceprint recognition and voice transfer, and discloses a real-time voiceprint recognition method and system for multi-user overlapped voices. The method comprises the following steps: acquiring a multi-source audio stream, and carrying out standardization and framing processing to generate a streaming audio frame sequence; based on the sequence, through circular buffering, noise reduction, endpoint detection and overlap detection model processing, obtaining an overlap interval label; task assembly, speaker separation, track numbering, voiceprint feature extraction and identity judgment are carried out, and a track identity binding structure is generated; and finally, voice transcription, fragment splicing and conflict cutting are executed, and the voiceprint template library is updated. According to the method, real-time separation and identity recognition of overlapped voices of multiple persons are realized, and the accuracy and robustness of voiceprint recognition in a complex scene are effectively improved.

The invention discloses a dynamic human body reconstruction method and system based on double-motion embedding and point cloud fusion, and relates to the field of digital human reconstruction processing. According to the method, the input frame sequence composed of three adjacent frames is intercepted from the monocular video and preprocessed to obtain necessary input parameters, then the dynamic human body reconstruction model is designed to process the input frame sequence and the input parameters to obtain the human body reconstruction image, the overall operation is fast and convenient, and the use effect is good. According to the dynamic human body reconstruction model designed by the invention, on one hand, a DMPF network part is adopted, dual-motion embedding is adopted to extract multi-modal motion features, and efficient fusion of 2D and 3D motion features is realized, so that richer motion information supervision is obtained, and on the other hand, a mixed point cloud encoder is adopted to fuse isolated point and overall point cloud features, so that the dynamic human body reconstruction model is more efficient in motion information supervision. Therefore, the dependency relationship of the local change of the human body surface on the global change is captured, and correct modeling of geometry and texture of the moving human body is further enhanced.

The invention relates to the technical field of image processing, and discloses a mine personnel behavior recognition method and system based on YOLOv5. The method comprises the following steps: acquiring a video frame sequence of a mine site, extracting an environment interference vector from the video frame sequence, and optimizing the video frame sequence based on the environment interference vector to obtain a clear frame sequence; human body key point coordinates and posture vectors are extracted from the clear frame sequence, and time sequence changes of the key point coordinates are analyzed to obtain preliminary action features; calculating a similarity score between the preliminary action feature and a preset standard template, and refining the preliminary action feature based on the similarity score to obtain a refined action component vector; and matching the refined action component vector from a preset action library to obtain an action recognition result, and generating corresponding action response output based on the action recognition result. According to the invention, the accuracy of personnel action recognition in a complex mine environment is improved.

The invention relates to the technical field of endoscope video processing and intelligent edge computing, in particular to an endoscope video enhancement processing intelligent edge computing system. Comprising a data acquisition module which is used for acquiring an original video frame sequence of edge endoscope equipment in real time; the feature extraction module is used for determining an instantaneous feature vector representing the dynamic change of the operation scene; the criticality quantification module is used for quantizing and generating a surgical event criticality score; the tuning logic module is used for generating a discrete and stable calculation normal form switching instruction; the assembly line switching module is used for responding to the calculation normal form switching instruction and executing an asynchronous weight preheating strategy; the utility evaluation module is used for constructing a dynamic utility function to evaluate system performance; and the threshold value correction module is used for performing closed-loop correction on the high-criticality threshold value by adopting a gradient rising strategy. According to the method, the stability of system decision making is enhanced, the smooth transition of the video processing flow among different calculation paradigms is ensured, and the robustness of the system is improved.

The invention discloses a behavior and physiological modal emotion recognition method and system based on orthogonal fusion, and the method comprises the steps: obtaining multi-modal data comprising a video frame sequence, an audio signal and an electroencephalogram signal, and carrying out the preprocessing of the multi-modal data; constructing a preliminary emotion recognition network; training the constructed preliminary emotion recognition network by using the preprocessed multi-modal data to obtain a trained emotion recognition model; and obtaining a to-be-recognized video frame sequence, an audio signal and an electroencephalogram signal, and inputting the to-be-recognized video frame sequence, the audio signal and the electroencephalogram signal into the emotion recognition model to obtain a corresponding emotion recognition result. According to the method, a modularized emotion recognition model is constructed, all functional modules cooperate with one another, redundant information is reduced, complementary information of multiple modes is fused, emotion feature characterization accuracy is improved, and therefore the emotion recognition effect is improved.

The method is mainly applied to the technical field of artificial intelligence processing. The invention discloses a face and license plate privacy protection video processing method, device, equipment and medium, and the method comprises the steps: extracting input video data to obtain a video frame sequence which comprises a plurality of video frames; the video frame sequence is input to a pre-trained face and license plate detection model for target detection, so that a target video frame containing a target image area is determined, position information of the target image area in the target video frame is obtained, and the target image area is a face area or a license plate area; based on the position information of the target image area, performing fuzzy processing on the target image area in each target video frame; and generating a video after fuzzy processing based on the video frame after fuzzy processing. According to the method and the device, the privacy protection and the target detection are collaboratively optimized, so that the efficiency and the accuracy of target detection are improved while the privacy security is guaranteed.

The invention discloses a monitoring video analysis method and system. The monitoring video analysis method comprises the following steps: performing region-of-interest extraction on a video stream at an edge gateway to obtain a first frame sequence; performing inter-frame similarity filtering on the first frame sequence at the edge gateway to obtain a second frame sequence; identifying a preset target in the second frame sequence at the edge gateway by using a lightweight target detection model; the edge gateway sends the third frame sequence of the preset target identified by the lightweight target detection model to the central server; and identifying whether the third frame sequence contains the target alarm information or not by using a multi-modal large model in the central server. According to the monitoring video analysis method and the monitoring video analysis system, the balance among the computing power, the bandwidth cost and the video analysis performance can be realized.

The embodiment of the invention discloses an image target labeling method and device, electronic equipment and a storage medium. According to the embodiment of the invention, a to-be-labeled video frame sequence of a road camera can be acquired; detecting any current frame in the video frame sequence based on a preset target detection model, and generating a detection frame for the target detection object; obtaining a prediction frame in the current frame, wherein the prediction frame is generated by a preset target tracking model according to the motion state of the target tracking object in the previous frame or multiple frames and the position of the detection frame; and matching the detection frame in the current frame with the prediction frame, if matching succeeds, allocating the historical identity identifier of the target tracking object corresponding to the prediction frame to the target detection object corresponding to the detection frame, and if matching fails, allocating a new identity identifier to the target detection object corresponding to the detection frame. Therefore, based on the time-space coherence of the video, the target is tracked and labeled efficiently and accurately.

The invention provides a real-time video monitoring intelligent analysis system based on deep learning, and relates to the technical field of computers, and the system comprises a video collection unit which is used for obtaining a real-time monitoring video stream to be analyzed, and extracting a target video frame sequence from the real-time monitoring video stream; the scene analysis unit is used for processing the target video frame sequence by using a pre-trained scene analysis model to obtain a dynamic object mask and a scene semantic graph corresponding to the real-time monitoring video stream; the acquisition unit is used for acquiring thermal infrared characteristics and dynamic environment variable information of a monitoring area corresponding to the real-time monitoring video stream; and the semantic segmentation unit is used for inputting the dynamic object mask, the scene semantic graph, the thermal infrared features and the dynamic environment variable information into a pre-trained semantic segmentation model to obtain a semantic segmentation result output by the semantic segmentation model. By applying the method provided by the invention, the monitoring video can be accurately analyzed.

The invention discloses a video coding and decoding method, device and equipment and a storage medium, and relates to the technical field of video coding and decoding, and the method comprises the steps: setting a target sequence identifier in a sequence head of a to-be-coded video frame sequence corresponding to a target video, determining a to-be-zoomed video frame, determining a target resolution of the to-be-zoomed video frame, and carrying out the zooming of the to-be-zoomed video frame; scaling each to-be-scaled video frame based on the target resolution, and adding image header data in an image header of the scaled video frame to obtain a target video frame; coding each target video frame and the original video frame in sequence to obtain a code stream; and sending the code stream to a decoding end, so that the decoding end decodes the code stream. By means of adding the image header data containing the image identifier and the target resolution in the image header, the capability of flexibly embedding frames with different resolutions in the same video sequence is realized, and the limitation of fixed resolution coding is broken.