5082 results about "Video based" patented technology

A video localization system localizes actions in videos based on a classification model and an actionness model. The classification model is trained to make predictions of which segments of a video depict an action and to classify the actions in the segments. The actionness model predicts whether any action is occurring in each segment, rather than predicting a particular type of action. This reduces the likelihood that the video localization system over-relies on contextual information in localizing actions in video. Furthermore, the classification model and the actionness model are trained based on weakly-labeled data, thereby reducing the cost and time required to generate training data for the video localization system.

The invention discloses a video analysis-based electric power operation personnel behavior identification and early warning system and method, which realize accurate identification and real-time early warning of electric power operation personnel behaviors by combining a video analysis technology with multi-modal data fusion, and effectively improve the safety management level of an operation site. Compared with a traditional safety supervision mode, the method employs a mode of combining deep learning target detection and time sequence behavior analysis, improves the recognition accuracy of operators and safety equipment, fuses the data of equipment worn by the operators with video data, improves the detection precision, and improves the safety supervision accuracy. And misjudgment caused by illumination change, shielding or complex environment is reduced. Besides, high-risk violation behaviors such as no safety helmet wearing, no safety belt fastening, violation climbing, high-altitude object throwing and the like are accurately recognized through the violation detection module, and different levels of alarm measures are adopted according to the severity of the violation behaviors in combination with an early warning feedback mechanism, so that the pertinence and response efficiency of early warning are improved.

Disclosed is an area monitoring system including a camera module having configured to generate an output representative of a captured image and / or a video of an area including a object. The system includes a state machine including an inference engine configured to: identify and track an object; establish a state and / or a change in state of an object via machine vision inferencing; and generate an output representative of an object identification, an object state, and / or a change in an object's state. The state machine module includes a mapping engine configured to: process the camera module output to assign coordinates to an object contained in an image and / or a video based on a coordinate system of a computer vision map of an area; apply a timestamp in connection with assigning coordinates an object; and generate a coordinate timeseries log for plural coordinate-timestamp pairs.

The invention provides a tunnel construction monitoring analysis method and system based on AI video monitoring, and relates to the technical field of artificial intelligence. A tunnel construction original video stream data set is acquired through a multi-view video acquisition device, and a spatio-temporal feature sequence is generated through spatio-temporal feature fusion processing; the spatiotemporal feature unit comprises a pixel region movement track, a personnel wearing identifier and a construction equipment contour feature, then extracting dynamic features of the spatiotemporal feature sequence to obtain a dynamic spatiotemporal feature set, and executing construction personnel statistical processing based on the dynamic spatiotemporal feature set; generating a hole entering personnel quantity statistical result, a personnel distribution statistical result of each construction part and a personnel identity matching result, performing work efficiency comparative analysis according to the personnel statistical result and a preset construction progress plan, and generating a construction progress deviation index and resource allocation optimization strategy; and when the progress deviation index exceeds the first threshold value, early warning is triggered, and a strategy is output to the management terminal, so that effective monitoring and management of tunnel construction are realized.

The invention discloses an automatic labeling and acquisition standardization method and system based on video content, and the method comprises the steps: decoding a video file into a processable video frame sequence, carrying out the standardization processing, unifying the format and the coding standard, and adjusting the resolution and the frame rate; performing content analysis on the standardized video frame sequence, extracting key frames, identifying objects, scenes and actions, and generating a video analysis result and a preliminary label; according to a video content analysis result, separating an audio stream from the video to carry out noise reduction processing and voice-to-text analysis, and combining the processed text with the video content analysis result to generate multi-modal comprehensive annotation information; based on the comprehensive labeling information, taking feedback of the user in video retrieval, label clicking and watching behaviors as an adjustment basis of label weights, and updating weight distribution of labels by analyzing user behavior data; labeling information and weight distribution are integrated into an index database, and indexes are updated in real time according to user behaviors and content changes.

Systems and methods for generating and providing a virtual walkthrough interface can include generating a virtual walkthrough video based on view synthesis renderings generated by neural radiance field model. The neural radiance field model can be trained based on a plurality of images of an environment and may generate the view synthesis renderings based on processing positions along a determined walkthrough path. The generated virtual walkthrough video can then be scrubbed through to provide the virtual walkthrough interface.

The invention discloses a video feature extraction and multi-dimensional matching-based movie advertisement real-time pushing system, and belongs to the technical field of digital advertisements. According to the system, multi-modal analysis is carried out on visual, audio, text and semantic features of a movie through a video feature extraction module, a dynamic interest tag constructed by a user portrait analysis module is combined, and the weighted matching degree of a video scene, user preference and an advertisement tag is calculated by utilizing a multi-dimensional matching engine; and millisecond-level advertisement putting is completed through the real-time pushing decision module. According to the method, context awareness and streaming computing technologies are fused, the relevance between advertisements and content scenes is remarkably improved, cross-platform deployment is supported, the method is suitable for short videos, live broadcast and other real-time scenes, the advertisement click rate is increased by 55%-72% through tests, the response delay is lower than 200 ms, and the method has the technical advantages of being efficient, accurate and low in delay.

The invention relates to the technical field of monitoring analysis, in particular to a park safety monitoring method and system based on dynamic visual identification. The method comprises the following steps: acquiring a plurality of park monitoring videos based on park monitoring equipment; performing three-dimensional illumination migration rendering on the plurality of park monitoring videos, and performing low-frame smooth completion so as to generate a time sequence monitoring image frame sequence; carrying out spatial position positioning calculation on the park monitoring equipment, carrying out topological association network reconstruction, and constructing an omnibearing monitoring perception topological map; performing dynamic object visual detection on the time sequence monitoring image frame sequence, and performing frame-by-frame optical flow tracking analysis to generate a dynamic target optical flow trajectory; and carrying out multi-time-point sampling on the dynamic target optical flow trajectory, and then carrying out mutation point identification so as to generate an optical flow trajectory mutation point. According to the invention, efficient park safety monitoring analysis and high-precision safety early warning are realized.

The invention discloses an intelligent power plant joint inspection system and method based on video AI and a robot, and the method comprises the steps: dividing an intelligent power plant into a plurality of subareas, evaluating the danger levels of the plurality of subareas according to historical data, and obtaining the danger levels of the plurality of subareas; setting an inspection cycle according to the danger levels of the plurality of partitions, and setting a plurality of robots to acquire operation data according to the inspection cycle to obtain inspection data of the plurality of partitions; acquiring a plurality of monitored videos, and performing video analysis on the acquired monitored videos by using an AI technology to obtain an abnormal position; according to the abnormal position, distributing an inspection robot of a corresponding partition to carry out fixed-point inspection to obtain fixed-point inspection data; and performing fault detection on the periodic inspection data and the fixed-point inspection data by using the fault recognition model, and performing fault early warning according to a fault detection result. The invention relates to the technical field of power plant inspection, and solves the technical problems of unreasonable inspection resource distribution and untimely fault early warning in an existing combined inspection system.

A method implemented by a video coding apparatus. The method includes applying a neural network (NN) filter to an unfiltered sample of a video unit to generate a filtered sample. The NN filter includes an NN filter model generated based on partitioning information of the video unit. Usage of the NN filter is indicated by one or more syntax elements in a bitstream. A conversion is performed between a video media file and a bitstream based on the filtered sample.

A computer-implemented method comprising: receiving, from a user device, video data from a user; training a first machine learning model based on the video data to provide a second machine learning model, the second machine learning model being personalized to the user, wherein the second machine learning model is trained to predict movement of the user based on audio data; receiving further audio data from the user; determining predicted movements of the user based on the further audio data and the second machine learning model; using the predicted movements of the user to generate animation of an avatar of the user.

The invention discloses an underground mine operation state analysis system and method based on video monitoring data, and the system comprises a data collection module which is used for collecting mine video and environment parameter data through a distributed sensor network, and generating a multi-dimensional data fusion set based on a space-time label technology; the edge analysis module is used for extracting feature parameters through a convolutional neural network algorithm based on the multi-dimensional data fusion set and generating a mine operation state recognition result; the fence construction module is used for constructing a three-dimensional digital model and a dynamic safety boundary based on the mine operation state recognition result to form a real-time monitoring reference framework; and the decision execution module is used for performing hierarchical risk assessment on the monitoring data in the security boundary based on the real-time monitoring reference framework, and generating a security early warning and disposal scheme with a tracing identifier. Each piece of early warning and disposal information is attached with a unique tracing identification code, so that follow-up event backtracking analysis is facilitated, and the risk management and control capability is continuously improved.

An action localization method, device, electronic equipment, and computer-readable storage medium are provided. The action localization method includes: identifying at least one target video segment containing a target object in a video; acquiring a first action recognition result of at least one image frame in the at least one target video segment and a second action recognition result of the target video segment; and acquiring an action localization result of the video based on the first action recognition result and the second action recognition result.

The invention belongs to the technical field of artificial intelligence, and particularly relates to a video analysis processing system and method based on a multi-modal large model, and the method comprises the steps: enabling the multi-modal control large model to receive a task which is input by a user and needs to be completed through natural language description, carrying out the semantic analysis of the input of the user, extracting task information, and carrying out the analysis of the task information; a task related to video understanding is decomposed into sub-tasks and the sub-tasks are sent to a video understanding module; the video understanding module receives a video or a video clip required by a task, executes a subtask to analyze the video, and generates embedding and description of video content to be input into a multi-modal control large model; when the multi-modal control large model receives a query request of a user, a query task of the user is analyzed into executable subtasks and mapped to a video understanding task, and a response result based on the query request is generated based on embedding and description of video content generated by the video understanding large model. Therefore, effective fusion and mutual enhancement of cross-modal information are realized, and the video understanding precision is improved.

The invention discloses a long video role scene decoupling generation method and system based on multi-modal diffusion, and relates to the technical field of image processing, and the method comprises the steps: S1, synthesizing the advanced features of a role and a scene through a SigLIP encoder and a DINOv2 encoder; s2, performing cross-modal feature fusion on the advanced features to obtain joint features, and compressing the joint features to obtain compact vectors; s3, generating text features according to the text prompt; s4, potential codes are generated from an input video through a causal 3D convolution encoder, the potential codes pass through a linear projection matrix and then are spliced with a memory state for dimension reduction, and a segmented potential vector sequence is obtained; s5, performing decoupling perception generation on the segmented potential vector sequence through an improved 3D-UNet, and performing deconvolution up-sampling reconstruction after deterministic sampling to obtain an RGB video segmented sequence; according to the method, the key problems of rough dynamic control, limited generation length and over-high resource consumption in long video generation are solved, and the quality and efficiency of the generated video are remarkably improved.

The invention discloses a video analysis-based multi-scene operator violation behavior identification method and system, and belongs to the technical field of intelligent operation safety monitoring and artificial intelligence identification, and the method comprises the steps: collecting a real-time video stream of a multi-scene operation site; recognizing a continuous action time sequence in the real-time video stream by using an action recognition depth model; constructing the continuous action time sequence into an action behavior sequence; the action behavior sequence is constructed into a directed behavior graph with time, space and action labels, the directed behavior graph is compared with a directed behavior graph corresponding to the standard action behavior sequence, and illegal behaviors are recognized; and carrying out multi-mode early warning on the identified illegal behaviors. According to the method, the bottleneck that the traditional image recognition technology is weak in action sequence semantic understanding and poor in environmental adaptability is broken through, and accurate recognition and real-time early warning of illegal behaviors in multi-scene operation are achieved.

The invention provides a real-time video translation and audio and picture synchronization method and system based on a multi-modal large model, and relates to the technical field of video translations, and the method comprises the steps: obtaining a source video; extracting the source video based on the multi-modal large model to obtain a multi-modal feature; fusing the multi-modal features through a cross-modal attention mechanism to generate a context semantic vector; translating into a target language text in real time based on the context semantic vector, and processing the translated language text based on the multi-modal features to obtain a translated language sound source; and performing mouth shape adjustment on the source video based on the translation language sound source, and merging the translation language sound source and the mouth shape animation video to obtain a real-time translation video with synchronous sound and picture. According to the method, the limitation of traditional single-modal translation is broken through, and the semantic accuracy of translation is remarkably improved by dynamically aligning the context information through the multi-modal features in combination with a cross-modal attention mechanism.

Systems, methods, and computer-readable media are disclosed for systems and methods multimodal indexing of video using machine learning. An example method may include deceiving, by a video encoder of an audio-video transformer neural network comprising one or more computer processors coupled to memory, a first frame and a second frame associated with a first segment of a video. The example method may also include receiving, by an audio encoder of the audio-video transformer neural network, an audio spectrogram comprising first audio data associated with the first segment of the video. generating, by the video encoder, a first video embedding. The example method may also include generating, by the audio encoder, a first audio embedding. The example method may also include determining a fusion of the first video embedding and the first audio embedding using a multimodal bottleneck token. The example method may also include determining an output including the first video embedding and the first audio embedding. The example method may also include determining a classification of the first portion of the video based on the output.

The invention discloses a gesture action video generation method and device based on fine-grained semantic description, and the method comprises the steps: carrying out the preprocessing and data enhancement of an original video, and constructing sample data composed of a video frame and a fine-grained semantic description text used for describing a gesture action in the video frame; constructing a training architecture consisting of a video coding module, a text coding module, a noise reduction generation module and a video decoding module; a loss function is constructed based on the difference between added noise in the video coding module and de-noised noise in the de-noising generation module, only parameters of the de-noising generation module are optimized by using the loss function, and a video generation model is formed by the text coding module, the de-noising generation module and the video decoding module after the parameters are optimized; and generating a gesture action video based on the input random vector and the fine-grained semantic description text for describing the gesture action by utilizing a video generation model. In this way, a high-quality general digital human video containing gesture actions is generated based on the fine-grained semantic description text.

Methods are provided for generating distilled multimedia data sets tailored to user's persona and / or task(s) to be performed associated with an enterprise network and enable interactive contextual learning using a multi-modal knowledge graph. Methods involve obtaining multimedia data from one or more data sources related to operation or configuration of an enterprise network and determining context for generating a distilled multimedia data set based on at least one of user input and user persona. The methods further involve generating, based on the context, the distilled multimedia data set that includes a set of multimedia slices generated from the multimedia data using a multi-modal knowledge graph. The multi-modal knowledge graph is generated using a graph neural network and indicates relationships among a plurality of slices of the multimedia data. The methods further involve providing the distilled multimedia data set for performing one or more actions associated with the enterprise network.

The invention discloses an abnormal behavior security early warning system based on multi-video image analysis, and relates to the field of security early warning, and the system comprises a path construction module which constructs a security path based on the installation position of video collection equipment, and constructs an abnormal behavior comparison library; the feature comparison module is used for constructing a security comparison model by using an algorithm, identifying abnormal behaviors of personnel, performing quantitative analysis to obtain a comprehensive risk index, and determining abnormal security personnel; the personnel judgment module is used for grading the security paths and determining security risk personnel; the personnel analysis module is used for judging the risk judgment value change condition of security risk personnel and determining a priority troubleshooting path; the regional analysis module is used for determining a dangerous region and security and protection high-risk personnel through clustering analysis; and the model optimization module is used for checking high-risk suspicious personnel to obtain false marks and missing personnel, and optimizing a security comparison model. The security early warning capability can be improved, and the method is suitable for various security scenes.

The invention discloses a virtual fitting method and device, a storage medium and electronic equipment, and is applied to the technical field of computer image processing, and the method comprises the steps: processing a target video, and obtaining the human body posture data of each video frame in the target video; constructing a 3D human body model of each video frame by using the human body posture data of each video frame; for the 3D human body model of each video frame, acquiring clothing rendering information of the 3D human body model, and rendering virtual clothing on the 3D human body model based on a dynamic fitting algorithm and the clothing rendering information to obtain a virtual rendering video frame of the video frame; and generating an AR fitting video based on each virtual rendering video frame, and displaying the AR fitting video to the user. Therefore, the selected costume can be tried on in the AR form, the fitting effect of the costume can be displayed for the user, the problem that the size is improper due to the fact that the costume purchased online cannot be tried on is solved, the probability of refunding and changing goods is reduced, and good shopping experience is provided for the user.

The invention discloses a short video network public opinion information identification method based on an image processing technology, and relates to the technical field of artificial intelligence and image processing, and the method comprises the following steps: S001, through obtaining image frames, audio tracks and time sequence information of a short video, constructing a multi-modal fusion model, extracting continuous image frames with suspicious identity features, and carrying out the recognition of the short video network public opinion information; generating a forgery risk area distribution map; and S002, performing semantic consistency verification according to the counterfeit risk region distribution map, and extracting space and time anomaly features existing among facial micro-expressions, pronunciation actions and background semantics in the image frame. According to the method, a multi-modal model is constructed by fusing image, audio and time information, fine abnormal features, traceability forgery starting points and propagation paths in a deep forgery video are identified, and an identification strategy and a public opinion response mechanism are dynamically adjusted, so that accurate identification, adaptive processing and closed-loop control of short video public opinion risks are realized; and the identification accuracy and the treatment efficiency are improved.

Methods, systems, and computer programs are presented to perform semantic analysis of video data for event detection and validation. One method includes an operation for training a machine-learning model with text-image pairs to create a semantic model. The semantic model generates text embeddings based on text describing events and image embeddings from video frames. The system calculates similarity values between text and image embeddings to determine event occurrences when the similarity exceeds a predetermined threshold. The system enables precise event detection by analyzing object relationships and attributes within video frames, reducing false alarms and enhancing monitoring efficiency. Further, these techniques can be used to detect any event describable by human text or by some image examples.

Systems and methods of compressing video content as encoded data and selectively reconstructing portions of the content are disclosed. The proposed systems provide a computer-implemented process configured to classify a person's behavior(s) during a video and encode the behaviors as a representation of the video. When playback of the video is requested, a video navigation assistant will allow the end-user to select specific segments of the video based on topics discussed in the video and the codes that were generated to represent the video. The user is then able to move through segments of the video in a sequence that aligns with their viewing preferences.

The invention discloses a lecturer video automatic generation method based on AI speech synthesis and animation driving. The method comprises the following steps: performing structured analysis on a PPT or a text script through an improved interior point method and an incremental shortest path algorithm; performing semantic grouping by applying a full-dynamic parallel single-link clustering algorithm and generating an enhanced script with an expressive mark; a CosyVoice technology is combined with a low-rank approximation method to generate a high-quality voice data stream; establishing a mapping relation between contents and action expressions through semantic analysis, and generating a complete action expression instruction set; and driving the digital human model by using the msueTalk technology, and generating a final lecturer teaching video through a parallel rendering algorithm. According to the invention, the method achieves the efficient and automatic generation of the education video, remarkably improves the content production efficiency, reduces the production cost, and guarantees the specialty and expressive force of the teaching video.

The invention discloses a compression transmission method and device for an electric power safety supervision video and electronic equipment. The method comprises the steps that an electric power safety monitoring video needing to be transmitted is acquired, the electric power safety monitoring video is used for representing the electric power safety monitoring video, the electric power safety monitoring video at least comprises an image frame set and an audio, and the image frame set at least comprises a current image frame; identifying a dynamic area and a static area in the current image frame; code rate distribution is carried out on the current image frame based on the dynamic area and the static area, a code rate distribution result is obtained, and the code rate distributed to the dynamic area in the code rate distribution result is larger than the code rate distributed to the static area; compressing the image frame set based on the code rate distribution result to obtain a compressed image frame set; and constructing a compressed safety supervision video based on the compressed image frame set and the audio, and transmitting the compressed safety supervision video. According to the invention, the technical problem that the compression transmission efficiency of the electric power safety supervision video is low in the prior art is solved.

The invention provides a panoramic video three-dimensional point cloud sparse reconstruction method and device integrated with key frame screening, and the method comprises the steps: obtaining a panoramic video of a target scene, and carrying out the video quality evaluation of the panoramic video, and obtaining a video evaluation result; determining a target resolution of downsampling according to the video evaluation result; carrying out downsampling processing on the panoramic video based on the target resolution to obtain a downsampled second video; for the second video, dynamically adjusting a frame extraction interval based on a preset key frame selection mechanism, and extracting a frame index of the key frame from the second video according to the frame extraction interval to obtain a plurality of frame indexes; and obtaining a plurality of corresponding key frames in the panoramic video based on the plurality of frame indexes, and obtaining the three-dimensional scene sparse point cloud of the target scene based on the plurality of key frames, thereby solving the technical problem of low processing efficiency in the prior art.

The invention discloses a short video editing synthesis method based on fuzzy logic. The method comprises the following steps: S1, carrying out digital video preprocessing on an input original short video; s2, forming a short video feature data set; s3, generating a fuzzy logic decision result; s4, performing multi-target and multi-parameter optimization on the fuzzy logic decision result by adopting a random score programming method to form an optimized editing parameter set; s5, generating a preliminarily synthesized short video; s6, generating a video quality evaluation result; and S7, carrying out iterative optimization on the preliminarily synthesized short video according to a video quality evaluation result, adjusting the editing parameters and re-executing the editing synthesis step until the video quality evaluation result meets a preset standard, and generating a final edited finished product. According to the method, the editing scheme can achieve the best balance in the aspects of visual rhythm, information integrity and emotion expression consistency.

The invention relates to the technical field of computer vision, and discloses a video dynamic target three-dimensional reconstruction method based on artificial intelligence, which comprises the following steps: step 1, image optical flow calculation: obtaining each frame of image in a video, calculating the optical flow value of each pixel point in each frame of image by extracting the spatial gradient of each pixel point in the image and the motion information of a target, and calculating the optical flow value of each pixel point in each frame of image; by analyzing the optical flow value, the displacement and the movement track of the target between the video frames are obtained; and step 2, optical flow weighting processing: on the basis of optical flow calculation, weighting processing is carried out on the optical flow value of each pixel. Weighing processing is introduced into image optical flow calculation, weighting factors are adjusted according to the movement speed and angle of the dynamic target, the precision of depth estimation of the dynamic target under the fast movement and complex background is improved, accurate depth data are obtained, depth information is provided for subsequent three-dimensional reconstruction, and the accuracy of depth estimation of the dynamic target is improved. The common error problem in dynamic target depth estimation is solved.