77 results about "Video content analysis" patented technology

The invention discloses a video content intelligent management method and system based on multi-modal analysis, and belongs to the field of video content management.The method comprises the steps that multi-modal features of video content are extracted; performing fine structure segmentation on the video content based on the multi-modal features, and segmenting the video into fine segments related to semantics; the fragments obtained after fine structure segmentation are subjected to tagging, and metadata is generated for each fragment; constructing a multi-modal semantic index of the video content based on the generated metadata; according to the method, a semantic retrieval function is provided by utilizing multi-modal semantic indexing, fine fragments required by a user are accurately recalled in combination with text label matching, measurement numerical value scoring, cross-modal vector retrieval and filtering based on a shot language and a shooting technology label, and the user queries and positions a specific time point or the fine fragments in a video through a natural language. According to the invention, the accuracy of video content analysis is improved, the user experience is optimized, and the operation complexity is reduced.

The invention belongs to the field of natural language processing crossing, and particularly relates to a visual-text collaborative abstract generation method and system based on multi-modal learning, and the system comprises a multi-modal data receiving module which is used for receiving multi-modal input data in parallel, and the multi-modal input data comprises text data and visual data; the visual semantic understanding module is used for extracting high-level semantic features of the image by adopting a visual semantic understanding model and generating text description; the semantic fusion module is used for carrying out semantic fusion on the original text and the generated image description by adopting a vision-text semantic fusion method based on direction consistency and self-adaptive semantic completion; and the abstract optimization module is used for carrying out semantic reconstruction on the fusion features by adopting a multilayer deep fusion hybrid neural network architecture, and generating an abstract text conforming to context semantics and accurate expression. The method has the advantages that visual and text information is accurately aligned, high-quality abstracts are generated, and the method is particularly suitable for news reports, conference records, video content analysis and other scenes needing cross-modal information fusion.

The invention discloses an automatic labeling and acquisition standardization method and system based on video content, and the method comprises the steps: decoding a video file into a processable video frame sequence, carrying out the standardization processing, unifying the format and the coding standard, and adjusting the resolution and the frame rate; performing content analysis on the standardized video frame sequence, extracting key frames, identifying objects, scenes and actions, and generating a video analysis result and a preliminary label; according to a video content analysis result, separating an audio stream from the video to carry out noise reduction processing and voice-to-text analysis, and combining the processed text with the video content analysis result to generate multi-modal comprehensive annotation information; based on the comprehensive labeling information, taking feedback of the user in video retrieval, label clicking and watching behaviors as an adjustment basis of label weights, and updating weight distribution of labels by analyzing user behavior data; labeling information and weight distribution are integrated into an index database, and indexes are updated in real time according to user behaviors and content changes.

A trained model is retrained for video quality assessment and used to identify sets of adaptive compression parameters for transcoding user generated video content. Using transfer learning, the model, which is initially trained for image object detection, is retrained for technical content assessment and then again retrained for video quality assessment. The model is then deployed into a transcoding pipeline and used for transcoding an input video stream of user generated content. The transcoding pipeline may be structured in one of several ways. In one example, a secondary pathway for video content analysis using the model is introduced into the pipeline, which does not interfere with the ultimate output of the transcoding should there be a network or other issue. In another example, the model is introduced as a library within the existing pipeline, which would maintain a single pathway, but ultimately is not expected to introduce significant latency.

The invention relates to the technical field of content recognition, in particular to an Al-based video content analysis method and system, and the method comprises the following steps: carrying out the image segmentation learning based on a video sequence through employing a U-Net convolutional neural network, analyzing scenes and elements in a video frame, recognizing and isolating key visual elements in a video through network learning, and carrying out the recognition of the key visual elements in the video. Comprising objects and figures. According to the method, image segmentation learning is carried out by adopting the U-Net convolutional neural network, key visual elements in the video can be identified and isolated more accurately, a clearer basis is provided for follow-up scene change and key event tracking, video content analysis is carried out by applying the graph neural network, the identification capability of dynamic scenes and events is enhanced, and the identification efficiency is improved. Deeper structured understanding is provided for video content indexing, the video quality is analyzed through a structural similarity index evaluation method, and the reason of visual quality reduction can be accurately recognized and improved.

The invention relates to the technical field of computer vision, in particular to an AI-based video content analysis system, which comprises a video stream acquisition and detection module, a background interference analysis and filtering module, a target movement track calculation module, a visual angle conversion and track compensation module and an abnormal behavior analysis module. According to the method, the pixel brightness values of continuous frames are extracted, the foreground region recognition accuracy is improved, the brightness change trend is screened to accurately lock a dynamic target, the background change rate is analyzed, similar regions are removed, interference is reduced, and the target edge feature and inter-frame position change are extracted to perform fine-grained trajectory tracking. And track prediction is optimized in combination with the view angle switching time, track changes and speed changes are recorded, the stability and accuracy of the moving target under multiple view angles are improved, abnormal change modes are classified and analyzed, abnormal events are effectively captured, background interference is reduced, and the target detection precision and the abnormal behavior analysis response speed are improved.

The invention relates to the technical field of video coding, in particular to an intelligent video coding optimization system and method based on computer vision, and the system comprises a computer vision video content analysis module; identifying an object, a behavior and a face in the video; the video classification module based on the neural network is in communication connection with the computer video content analysis module; based on the video feature data, classifying videos by using a deep neural network; the video coding parameter optimization module is in communication connection with the video classification module based on the neural network; adaptively optimizing video coding parameters based on the video classification result and the video feature data; the video coding quality evaluation module is in communication connection with the video coding parameter optimization module; evaluating the video quality of coding based on the optimized coding parameters; and the evaluation result is fed back to the video coding parameter optimization module for further optimizing the coding parameter, so that the understanding depth of the video content is greatly improved.

The invention discloses a video generation method, system and device based on a large language model and a medium, and the method comprises the steps: obtaining product information inputted by a user, the product information comprising the name, description and selling point of a product; preprocessing the product information, and carrying out semantic information split processing on the preprocessed product information through a large language model to obtain split description information corresponding to the product information; the method comprises the following steps: acquiring an original video, and performing video clip segmentation on the original video to generate a plurality of video clips and picture description information corresponding to the video clips; semantic matching processing is carried out on the sub-mirror description information and the picture description information, and a video clip with the highest similarity with each piece of sub-mirror description information is obtained through matching; and splicing the video clips according to the sequence of the sub-mirror description information to generate a complete video. Compared with the prior art, the product video can be automatically and efficiently generated by integrating natural language processing, video content analysis and an intelligent matching algorithm.

The invention discloses a video content analysis and post-production optimization method and system based on cloud computing, and relates to the technical field of video processing, and the method comprises the following steps: in a video content analysis stage, obtaining a video through a data interface, dividing the video into frames, analyzing the character action change, forming an action event stream, and storing the action event stream in a database; extracting video content dimension features at the same time; constructing a video content comprehensive feature scoring model, and obtaining a comprehensive feature score through weighted summation in combination with the dimension features of the video content; in the later production and adjustment stage, the performance of the video in each dimension is evaluated based on the comprehensive scoring model, and the video content is adjusted according to the evaluation result and the video clip; according to the method, the frame number meeting the set matching degree threshold condition in the video frame is calculated, the proportion of the frame number higher than the threshold value and the frame number lower than the threshold value is analyzed, when the proportion meets the preset condition, whether the video content needs to be further adjusted in the post-production stage or not is marked, and video content analysis and post-production optimization are achieved.

Aspects of the subject disclosure may include, for example, obtaining current video content of a player repeatedly performing a physical skill, analyzing the current video content based on previous video content, the previous video content comprises other video content of the player repeatedly performing the physical skill, determining biomechanic metrics of the player performing the physical skill based on the analysis, and determining each biomechanic metric of a portion of the biomechanic metrics does not satisfy a respective biomechanic metric success rate. Further embodiments include generating a first image of the player performing the physical skill from the current video content, generating a second image of the player performing the physical skill from the previous video content, and presenting the first image and the second image simultaneously and indicating the portion of the biomechanics that did not satisfy the respective biomechanic metric success rate. Other embodiments are disclosed.

The invention relates to the technical field of video interaction, and discloses an intelligent video content analysis and interaction system, which comprises a video extraction module, an interaction action module, an analysis adjustment module, an intelligent recommendation module and a video evaluation module, the video extraction module is configured to preprocess an input video and extract video features to establish a training data set, the interaction action module is configured to capture gesture actions, voice commands and touch actions for watching real-time video images, and the analysis and adjustment module is configured to determine emotion types according to fusion feature vectors. The intelligent recommendation module is configured to determine a first recommendation video according to an emotion type, determine a second recommendation video according to an interaction strategy, and determine a third recommendation video based on a machine learning algorithm according to historical watching data and historical interaction behaviors, and the video evaluation module is configured to determine historical matching similarity; and evaluating the recommended video quality according to the historical matching similarity. The method has the characteristic of intelligently recommending the video.

The invention relates to the technical field of video content analysis, and discloses a video scene content label determination method based on a knowledge graph. The method comprises the following steps: inputting a multi-modal feature sequence formed by visual, audio and text features extracted from a video stream into a knowledge graph inference engine comprising an entity relationship network and a semantic association rule base; performing node mapping through an engine to generate an initial scene entity set; performing hierarchical reasoning on the set based on a semantic association rule base to obtain a scene semantic topological structure; screening core entities according to entity weight distribution in the topological structure, and generating a candidate tag set; performing time sequence consistency verification on the candidate tag set, and correcting tag time sequence offset in combination with timestamp information; performing cross-modal disambiguation on the corrected label set by using an entity relationship network to eliminate semantic conflicts; and according to a disambiguation result, constructing a scene label knowledge sub-graph containing entity attributes and relation path constraints.

The invention relates to the technical field of big data analysis, and particularly provides an illegal capital clue identification and early warning method and device, and the method comprises the following steps: S1, data collection and preprocessing; s2, text feature extraction and semantic modeling; s3, video content analysis and behavior recognition; s4, performing multi-modal fusion and risk scoring; and S5, early warning output and visual display are carried out. Compared with the prior art, the method has the advantages that potential illegal information can be automatically extracted from massive news reports and short video data, and efficient and accurate risk early warning is realized.

The invention relates to an automatic movie and television video script extraction method based on a multi-modal large model, and belongs to the field of artificial intelligence and video content analysis. Aiming at the problem that an existing automatic speech recognition tool cannot generate a structured script, the method comprises the following steps: firstly, carrying out multi-mode decomposition on an input video, extracting frames by adopting a scene self-adaptive strategy, and establishing sound and picture timestamp alignment mapping; then extracting features through a CLIP-ViT visual feature encoder and a Whisper audio feature encoder, and associating semantics by using a cross-modal attention mechanism; role identity recognition, scene type recognition and action character description are achieved, and finally a script file conforming to the standardization specification is generated and comprises a scene title, a time code mark and a special narrative mark. Compared with a mode of manually dictating marks and an automatic voice recognition tool, the method effectively improves the accuracy and efficiency of movie and television video script extraction.

The invention discloses a language-driven hour-level traffic video analysis method and system, relates to the technical field of video content analysis and retrieval, can generate a video-specific open world detector, can accurately identify and filter fine-grained and composite semantic targets, and improves the accuracy of video analysis. And an hour-level video analysis service driven by open vocabularies and natural languages is really realized. In order to achieve the purpose, the technical scheme of the invention comprises the following steps of: 1) receiving an hour-level traffic video, and positioning a video clip related to natural language query in the video; and 2) automatically constructing an open world target detector in the video clip positioned in the step 1). And step 3) finishing cross-time-period target trajectory generation on the video clip positioned in the step 1). The invention also provides a language-driven hour-level traffic video analysis system for executing the method. The system comprises a video clip positioning module, an open world target detector construction module and a target trajectory extraction module.

The invention discloses a video content automatic auditing method and system based on AI drive, and relates to the technical field of video content analysis, and the method comprises the steps: obtaining video information to be audited, and carrying out the multi-level segmentation of the video information; in the slices, node simulation is carried out according to the content independence of the video information; analyzing associated branches and historical features of each node to generate attribute features of the nodes; content auditing is carried out through an auditing strategy of multiple virtual agents, and illegal space analysis is carried out according to an unstable part in an auditing result; and generating a final video content auditing report according to the auditing result and the analysis result of the illegal space. According to the method, the structured recognition and intelligent auditing efficiency of the multi-modal video content is improved, accurate positioning, label attribution and automatic early warning of complex violation behaviors are realized, and the content security management capability is enhanced.

A method of video encoding is described. The method includes segmenting original video data to obtain an original video segment including multiple video images. Video content analysis is performed on the original video segment to obtain a video image processing parameter corresponding to the original video segment. Image processing is performed on a video image in the multiple video images in the original video segment based on the video image processing parameter to obtain a processed video segment. An encoding parameter of the processed video segment can be determined based on image feature data of the processed video segment. The processed video segment can be encoded based on the encoding parameter to obtain an encoded video segment.

The invention relates to the technical field of video content analysis, in particular to a mine surveillance video key frame extraction method, which comprises the following steps: extracting multi-dimensional features of each frame of a candidate frame set, obtaining a fusion value, and obtaining a candidate key frame set; based on an Euclidean distance method, calculating an Euclidean distance between continuous frames of the candidate key frame set, and determining a total clustering number according to a relationship between the Euclidean distance and a preset clustering threshold value; and clustering the frames in the candidate key frame set by using a preset mixed GWO-FCM clustering algorithm and the total clustering number to obtain a key frame set. According to the method, a hybrid clustering algorithm GWO-FCM combining grey wolf optimization and fuzzy C-means is adopted, global search and local optimization capabilities are considered, and the accuracy and representativeness of key frame clustering are ensured. The high-quality extraction of the key frames is realized, the number of redundant frames is obviously reduced, and the compactness and readability of the video abstract are improved.

The invention belongs to the technical field of video content analysis, and more specifically relates to a video content intelligent analysis and statistical method under the support of a large model. According to the method, rich visual features are extracted through a Swin Transform model, and feature weighting is dynamically adjusted according to an application scene; organizing the extracted features into a time sequence matrix so as to facilitate subsequent time sequence information fusion; through a time sequence modeling network, a time dependency relationship between video frames is captured, a new feature matrix containing dynamic information is generated, then the accuracy and robustness of behavior recognition are improved, finally, a space-time correlation modeling technology is utilized, accurate tracking of a target in a video is achieved, and the behavior recognition accuracy and robustness are improved in combination with a behavior recognition result. Comprehensively analyzing and counting the video content; the accuracy of video analysis is effectively improved, and the problems of dynamic change and insufficient continuous capture of behavior recognition in the prior art are solved.

Disclosed are computerized systems and methods for a decision intelligence (DI)-based framework that automatically and / or dynamically provides an interactive content viewing and content discovery experience to users. The framework includes functionality for real-time video content analysis and interactive entity identification that, inter alia, provides novel capabilities to viewing users related to the extraction, analysis and subsequent interaction with content depicted within video frames during playback of such content. The framework implements artificial intelligence / machine learning (AI / ML) approaches for video processing, user interaction and information delivery through a series of interconnected processes and subsystems. In some implementations, rendered content can be parsed and mined for real-world and / or digital content depicted therein that relate to real-world entities and / or digital resources, whereby interaction with such entities is provided in the form of a provided interface, electronic message and / or recommendations for further information discovery, or some combination thereof.

The invention discloses an intelligent equipment linkage control method and system based on video content analysis, and relates to the field of video content analysis, and the intelligent equipment linkage control method based on video content analysis comprises the following steps: S1, obtaining video parameters, and carrying out the preprocessing; s2, pixel motion vectors are extracted, and a time sequence motion field is formed; s3, extracting a pixel dominant motion information set, calculating an average speed value, and generating a time sequence speed parameter set; s4, performing zero crossing point identification on the time sequence speed parameter set, and generating a time sequence displacement control signal; and S5, sending the time sequence displacement control signal to the external intelligent equipment, and driving the external intelligent equipment to generate physical motion with the video content by adopting the displacement signal. According to the method, by introducing preprocessing means such as multi-scale image analysis, resolution compression and graying conversion, key information is efficiently and accurately extracted from the video, and the precision and efficiency of subsequent analysis are improved.

The application discloses a kind of video intelligent slice method based on multi-modal information fusion and semantic constraint, it is related to video content analysis and intelligent editing technical field, it is characterized in that it proposes semantic priority audiovisual collaborative slice scheme, obtains shot interval by shot boundary detection, constructs voice interval and effective word information by voice recognition, adopts voice interval as semantic mask to remove visual cut point in the process of voice expression, and based on fragment length and effective word, content density determination is carried out to long fragment, and further adopts visual re-cutting, audio energy driving and uniform cutting Secondary slicing strategy of division, the method provided by the application solves the problem of semantic fragmentation and content perception loss in the prior art, can obtain video fragment with complete semantics, high content density and appropriate length, and is suitable for various application scenarios such as automatic editing, short video generation and video content retrieval.

The application provides a video texture mapping and real-time rendering method and system based on a digital twin scene, relates to the technical field of digital twin rendering, and comprises the following steps: acquiring a real scene monitoring video stream and performing real-time image content analysis; dividing the texture details of different regions of the video stream according to the features and numerical values of the image analysis and generating region detail identifiers associated with the video content; identifying the dynamic features of the video picture based on the identifiers and generating scene state codes representing the comprehensive picture conditions; dynamically selecting a target engine and corresponding rendering quality parameters from preset heterogeneous rendering engines in combination with the current load state of a graphics processor; and mapping the video stream image data to the surface of a corresponding model of the digital twin scene as dynamic texture to realize efficient rendering of the picture, so that dynamic texture mapping and adaptive real-time rendering based on video content analysis and hardware load in the digital twin scene can be realized.

The embodiment of the present invention proposes a method and device for parsing audio and video content, which belongs to the field of deep learning. The audio and video to be analyzed are split, feature extracted and feature fused to obtain comprehensive visual features and auditory features. The auditory features and comprehensive visual features are input into a parsing algorithm obtained by optimization training using weakly supervised learning. Through the parsing algorithm, the auditory features and comprehensive visual features are modeled and perceptually predicted with respect to modality to obtain the action events contained in the audio and video to be analyzed and the category and modality to which each action event belongs. The parsing algorithm of the present application proposes a modality perception module and a timing perception module, which can coordinate modality and timing for evidence mining, thereby greatly reducing the model's sensitivity to pseudo-label noise generated during modal classification, improving the robustness of modality dependency judgment and the accuracy of timing annotation, and overcoming the uncertainty problem caused by the lack of timing annotation under weak supervision settings.

Disclosed are computerized systems and methods for a decision intelligence (Dl)-based framework that automatically and / or dynamically provides an interactive content viewing and content discovery experience to users. The framework includes functionality for real-time video content analysis and interactive entity identification that, inter alia, provides novel capabilities to viewing users related to the extraction, analysis and subsequent interaction with content depicted within video frames during playback of such content. The framework implements artificial intelligence / machine learning (AI / ML) approaches for video processing, user interaction and information delivery through a series of interconnected processes and subsystems. In some implementations, rendered content can be parsed and mined for real -world and / or digital content depicted therein that relate to real-world entities and / or digital resources, whereby interaction with such entities is provided in the form of a provided interface, electronic message and / or recommendations for further information discovery, or some combination thereof.

The application provides a video processing cooperation method and device, equipment and a storage medium. The method comprises the following steps: sending a video rendering capability request and a video content analysis capability request to a terminal device; receiving a video rendering capability response and a video analysis capability response of the terminal device, wherein the video rendering capability response comprises a video rendering capability of the terminal device, and the video analysis capability response comprises a video analysis capability of the terminal device; determining an optimal video rendering cooperation configuration according to the video rendering capability of the terminal device, and determining an optimal video analysis cooperation configuration according to the video analysis capability of the terminal device. Therefore, the idle computing resources of the terminal device can be fully utilized under limited cloud server computing resources, so that a user can have a better cloud game quality experience.

The application discloses a video generation method, system, device and medium based on a large language model. The method obtains product information input by a user, the product information including the name, description and selling point of the product. The product information is preprocessed, and the preprocessed product information is subjected to semantic information split-screen processing through a large language model to obtain split-screen description information corresponding to the product information. An original video is obtained, and the original video is subjected to video segment splitting to generate a plurality of video segments and picture description information corresponding to the video segments. The split-screen description information and the picture description information are subjected to semantic matching processing, and the video segments with the highest similarity to each split-screen description information are matched. The video segments are spliced according to the sequence of the split-screen description information to generate a complete video. Compared with the related art, the application can automatically and efficiently generate a product video by integrating natural language processing, video content analysis and intelligent matching algorithms.

The application provides a network public opinion video content analysis method based on multi-modal fusion, and relates to the technical field of network information processing.The method comprises the following steps: by splitting the public opinion video into an audio stream and an image frame sequence, structurally analyzing picture text and speech content respectively, and completing semantic embedding and cross-modal alignment under a unified time axis, introducing a bidirectional cross-modal attention and a gating fusion mechanism to realize deep fusion of multi-modal information, and further combining multi-role consistency, semantic superposition effect and emotion and fact collaborative features for joint modeling at an event level scale, so as to realize overall understanding of network public opinion video content and accurate determination of public opinion risk level.The application can fuse multi-modal information as a whole and realize joint modeling to analyze network public opinion video content.

The invention discloses a video content analysis method, a server, a product, equipment and a storage medium, and relates to the field of video analysis, and the method comprises the steps: calling a preset large language model which cannot be subjected to subjective reasoning, carrying out the information extraction of the video display content of a video file at each moment, and obtaining an information extraction result; and performing risk identification on the extracted structured event data based on a risk identification rule, extracting verification evidence of the risk assessment result of each event entity from the video display content, and outputting a content analysis result based on each risk assessment result and the verification evidence. According to the method and the device, on the basis of the preset large language model which cannot be subjectively reasoned, the disassembly of the complex description logic is realized, the accuracy of each event entity obtained by disassembly is improved, the risk assessment result of each event entity is verified by using the verification evidence, and the content analysis precision of the finally output video file is improved.

The invention discloses a multi-mode-based video advertisement generation method, system and device and a medium, and the method comprises the steps: obtaining a video material of a product, and carrying out the preprocessing of the video material; performing video content analysis and video clip scoring on the preprocessed video material data to obtain a corresponding video content analysis result and a video clip score; generating a mixed cutting video corresponding to the product according to the video clip score; according to the analysis result of the video content, the name of the product and the selling point, generating an advertisement copywriting corresponding to the product; forming audio and subtitle data according to the content of the advertisement copywriting; and integrating the mixed video, the audio and the subtitles to generate a video advertisement of the product. According to the method, technologies such as video content analysis, propaganda copywriting automatic generation and an intelligent matching algorithm are fused, and an efficient automatic process from user input to finished video is constructed, so that the problems that traditional video advertisement generation needs a large amount of manual intervention and the generation efficiency is relatively low are solved.