2316 results about "Video based" patented technology

The invention discloses an underground mine operation state analysis system and method based on video monitoring data, and the system comprises a data collection module which is used for collecting mine video and environment parameter data through a distributed sensor network, and generating a multi-dimensional data fusion set based on a space-time label technology; the edge analysis module is used for extracting feature parameters through a convolutional neural network algorithm based on the multi-dimensional data fusion set and generating a mine operation state recognition result; the fence construction module is used for constructing a three-dimensional digital model and a dynamic safety boundary based on the mine operation state recognition result to form a real-time monitoring reference framework; and the decision execution module is used for performing hierarchical risk assessment on the monitoring data in the security boundary based on the real-time monitoring reference framework, and generating a security early warning and disposal scheme with a tracing identifier. Each piece of early warning and disposal information is attached with a unique tracing identification code, so that follow-up event backtracking analysis is facilitated, and the risk management and control capability is continuously improved.

The invention discloses a short video network public opinion information identification method based on an image processing technology, and relates to the technical field of artificial intelligence and image processing, and the method comprises the following steps: S001, through obtaining image frames, audio tracks and time sequence information of a short video, constructing a multi-modal fusion model, extracting continuous image frames with suspicious identity features, and carrying out the recognition of the short video network public opinion information; generating a forgery risk area distribution map; and S002, performing semantic consistency verification according to the counterfeit risk region distribution map, and extracting space and time anomaly features existing among facial micro-expressions, pronunciation actions and background semantics in the image frame. According to the method, a multi-modal model is constructed by fusing image, audio and time information, fine abnormal features, traceability forgery starting points and propagation paths in a deep forgery video are identified, and an identification strategy and a public opinion response mechanism are dynamically adjusted, so that accurate identification, adaptive processing and closed-loop control of short video public opinion risks are realized; and the identification accuracy and the treatment efficiency are improved.

The invention provides a driving behavior intelligent identification method and system based on video analysis, and the method comprises the steps: obtaining a driver face video stream and a road environment video stream collected by a vehicle-mounted camera, and reading the driving information recorded by a whole vehicle communication network; recognizing an eyelid closing state, a sight line direction and a head posture in the driver face video stream based on a posture recognition model, and performing fatigue distraction analysis to obtain driver state information; performing motion trail analysis on the road environment video stream and the driving information, and performing driving risk assessment in combination with the driver state information to obtain driving assessment information; and performing early warning construction according to the driving evaluation information, generating early warning prompt information, and synchronously writing the early warning prompt information, the driving evaluation information and the driver state information into a safety data protection memory. The fatigue and distraction states of the driver can be recognized more accurately, and the accuracy of state judgment is improved.

The invention provides an interactive automatic explanation method for converting a traditional video into an artificial intelligence digital human, and relates to the technical field of artificial intelligence, and the method comprises the steps: obtaining original video data and an audio track, and carrying out the semantic analysis of the audio track, and obtaining multi-mode deconstruction data; generating an explanation script for each time period of the video based on the explanation text, and performing timestamp labeling on the visual elements to form a time sequence synchronization data structure; in the playing process, a virtual image generator is driven to synthesize digital human dynamic expression output in real time according to the current playing time point; after a user interruption request is received, semantic matching is carried out on a query intention in the explanation script, a target explanation fragment and visual elements are positioned, and complementary explanation content is generated; and driving the virtual image generator to synthesize dynamic output synchronized with the supplementary explanation, and after interaction is completed, recovering playing or skipping to a specified time point according to a user instruction. According to the invention, the conversion from the traditional video to the interactive intelligent explanation video is realized, and the watching experience and learning efficiency of the user are improved.

The invention provides an abnormal blank screen processing method, system and equipment during vehicle-machine interconnection, and the method comprises the steps: predicting the current abnormal blank screen risk probability based on vehicle-machine interconnection parameters, carrying out the abnormal blank screen detection through employing a preset multilayer abnormal blank screen detection algorithm, and when the abnormal blank screen risk probability reaches or exceeds a preset probability threshold value, carrying out the abnormal blank screen detection. And when the abnormal blank screen is detected, controlling the display screen to display a preset picture or video, or when the abnormal blank screen is detected, determining an abnormal blank screen type, and controlling the display screen to display the preset picture or video based on the abnormal blank screen type. According to the method provided by the invention, the technical defects of lack of blank screen risk prediction and active prevention and control capabilities and lack of prevention, control and recovery of various types of blank screens when the existing vehicle-mounted system and the terminal equipment are interconnected are effectively solved.

A video generation framework that is controllable, unsupervised and based on events (CUBE) includes an event camera, which captures changes in light intensity at each pixel of a scene asynchronously and generates event camera data. A text-to-image diffusion model that is conditioned on textual descriptions integrates the event camera data to control video synthesis. Further, an edge extraction module translates event data into a format usable by the text-to-image diffusion model, whereby the diffusion model synthesizes detailed and contextually accurate videos based on textual prompts. Further, an improved system (CUBE Plus) includes a content frame identification module which selectively identifies and uses only the most information-rich event segments of the event camera data to drive cross-frame attention, and an event driven attention mechanism that allows the framework to focus on event-dense moments.

The invention discloses a video gait analysis-based asthenia syndrome evaluation method, medium and equipment, and the method comprises the steps: firstly, synchronously collecting gait videos of different visual angles through a plurality of cameras, and extracting 2D joint point sequences of all visual angles through a stacked hourglass network; then, a high-precision 3D joint point time sequence is reconstructed through a multi-view fusion algorithm, depth features are extracted through a residual connection network, and data of all view angles are dynamically weighted and fused based on the shielding rate and the confidence coefficient; performing sliding window segmentation and downsampling on the 3D sequence, and performing action classification by using an LSTM time sequence model; and finally, key motion parameters are extracted from a classification result to calculate a weakness score, and weakness grade evaluation is realized. According to the method, the problem of single-view shielding is solved through multi-view 3D reconstruction, the evaluation precision is remarkably improved by combining dynamic weighted fusion and time sequence modeling, and compared with a traditional method, the method has the advantages of being non-contact, low in cost and high in objectivity, and is suitable for large-scale screening of community and family environments.

The invention provides an echocardiogram video abstraction method based on a hierarchical space-time attention mechanism, and the method comprises the steps: obtaining an echocardiogram, inputting the echocardiogram into an anatomy and physiological basic model of a key frame recognition task, and obtaining key frame information; inputting the key frame information into a multi-dimensional spatial-temporal feature extraction model to obtain feature information and frequency domain information; fusing the feature information and the frequency domain information; performing key frame enhancement processing on the fusion information by adopting an SE channel attention mechanism to obtain key frame anatomical feature expression; processing the key frame anatomical feature expression by adopting a key frame backtracking strategy and a classification network to obtain a key frame identification classification result; performing left ventricle segmentation on the classified image by adopting an improved U-Net left ventricle region segmentation network; estimating the area and volume of the left ventricle based on a segmentation result; fusing the estimated left ventricular area and volume by adopting a hierarchical attention mechanism and a cross-layer information fusion strategy to obtain a ventricular volume result; according to the method, through abstract extraction of the key information of the ultrasonic cardiac video, the clinical practicability of intelligent screening and auxiliary diagnosis of cardiovascular diseases based on ultrasonic is improved.

The invention relates to the technical field of video analysis, and discloses a linkage alarm method, device and equipment based on video analysis and a storage medium, and the method comprises the steps: carrying out the behavior semantic recognition of a video frame collected by a doorbell, obtaining a visitor behavior mode, coding the video frame and the visitor behavior mode, and obtaining a semantic vector; constructing a door front three-dimensional semantic map according to the video frame and tracking visitors to obtain visitor trajectory data; decoding the semantic vector and the visitor trajectory data to obtain a behavior analysis result; according to the behavior analysis result and the visitor track data, a linkage strategy is selected from a preset action space, an alarm instruction is generated, and the alarm instruction is executed through the doorbell, end-to-end real-time response is achieved, the strict requirement of an actual doorbell application scene for the response speed is met, and the user experience is improved. And the accuracy of doorbell collected video analysis and linkage alarm is improved.

The invention relates to the technical field of image recognition, in particular to a fire behavior recognition method and system based on video monitoring. The technical problem that the early warning reliability of a video monitoring system on fire smoke is insufficient is solved. The method comprises the following steps: acquiring a monitoring video stream of a monitoring area and time sequence data acquired by a sensor in the monitoring area; determining an evaluation value of the to-be-detected area based on the image feature and the morphological change of the to-be-detected area; determining a confidence factor based on the form change trend of the to-be-detected region and the change trend of the time series data; determining a smoke authenticity value based on the evaluation value and the confidence factor; and adjusting an alarm threshold value of the smoke alarm based on the smoke authenticity value. The method is used for fire smoke early warning and monitoring scenes.

The invention discloses a video stream defogging method and system for 5G remote control, and relates to the technical field of image enhancement. The method comprises the following steps: acquiring a foggy video in real time; inputting the single-frame foggy video into a pre-trained monocular depth estimation neural network, and reasoning to obtain a depth map with the same size as the input single-frame foggy video; performing close-shot and long-shot region division on the scene of the current foggy video frame according to the depth map to obtain a region mask with the same size as the depth map; based on the region mask and the foggy video, calculating to obtain an atmospheric light parameter; calculating the transmissivity of each pixel according to the distance estimation result of each pixel in the depth map, and obtaining a transmissivity map with the same size as the depth map; based on the depth map, the atmospheric light parameters and the transmissivity map, adaptive calculation is performed on the foggy video to realize defogging, and a defogged clear video frame is obtained; the method can adapt to different depth-of-field fog effects, realizes different depth-of-field defogging, and outputs clear video frames.

Systems, methods, and apparatuses are described for verifying a user identity in a video conference. A computing device may receive user data and a plurality of security parameters associated with accessing a video conference based on a confidentiality level of the video conference. The computing device may generate a security code that is encoded with user data. The computing device might cause the security code to be displayed on the mobile device for a predetermined time period. The computing device may receive an indication that the first device scanned the security code by using a camera. To verify the identity of a user, the computing device may decode the security code, compare the decoded user data of the decoded security code and expected user data associated with the video conference. The computing device may determine the authenticity of a user video and allow access to the video conference.

According to aspects disclosed herein, a method of using machine learning as part of creating avatars while watching video content is provided. According to an aspect, a method is configured for receiving a request to create an avatar based on the video content and, in response to receiving the request to create the avatar based on the video content, determining one or more processing resources to use to create the avatar based on an output of a machine learning engine, wherein the one or more processing resources include a cloud-based processing resource, an edge-based processing resource, and a local processing resource. The method includes allocating avatar processing operations to the one or more processing resources to create the avatar based on the output from the machine learning engine. The method further includes creating the avatar using the one or more processing resources, and storing the avatar in an avatar database.

A method includes receiving a video comprising one or more frames; executing a first machine learning model using the one or more frames of the video to generate a dynamic mask configured to track a predicted magnitude of attention that individuals will give to different portions of each of the one or more frames of the video during playback of the video, the dynamic mask comprising attention scores for individual portions of each of the one or more frames of the video; generating one or more attention metrics for the video based on an aggregation of attention scores for the individual portions of each of the one or more frames of the video; and generating a record identifying the one or more attention metrics for the video.

The invention provides a construction environment risk early warning method and early warning system based on video monitoring analysis, and the method comprises the steps: obtaining a continuous monitoring video stream of a construction environment, carrying out the construction behavior analysis of the continuous monitoring video stream, and generating a behavior track feature reflecting the dynamic state of an operation main body and a spatial relation feature reflecting the constraint of the operation environment; performing associated risk discrimination processing on the behavior trajectory features and the spatial relationship features through a pre-trained risk assessment model to generate a risk discrimination result; extracting occurrence time sequence information and spatial distribution information of risk events in the continuous monitoring video stream; and generating construction risk early warning information containing the time-space corresponding relation. According to the method, the problem of intervention lag or blindness caused by lack of space-time details in a traditional method is avoided, the comprehensiveness and practicability of construction risk identification are effectively improved, and the accuracy and reliability of early warning are improved.

This disclosure provides an AI filmmaking workflow including AI-assisted storyboarding, AI animation, and post-production processes for creating films. The workflow provides techniques for reconstructing 3D digital environments and characters, and for virtual camera control. The workflow also provides techniques for capturing 2D live-action performances and extracting visual cues. The AI animation process generates synthetic images and video using prompts that are based on virtual camera control in case of 3D digitization and / or visual cues in case of 2D camera capturing. Further, the workflow provides techniques for compositing with AI assistance, to generate a composited video based on the AI-animated video and inputs resulting from 3D digitization and / or 2D video processing. Advanced post-processing techniques are also provided for generating a complete film based on the composited video. This framework is designed to facilitate creative collaborative networks by using a hybrid digitization approach to enhance consistency, directability, and scalability in AI filmmaking.

A system and method for training, assessing, and providing feedback on cardiopulmonary resuscitation (CPR) performance based on at least one video of a CPR training session performed by a trainee on a non-mannequin training object. A preprocessing module is configured to process the at least one video to generate a standardized video. A marking module is configured to use pose estimation to mark points for body movements during the CPR training session based on the standardized video, and a computing module configured to compute body movement parameters for CPR based on the marked points. A classification module implements a machine learning model that classifies CPR compressions on the non-mannequin training object based on the computed body movement parameters, thereby generating compression classifications, wherein the machine learning model is trained to extract CPR-specific features. An editor module maps metrics over the standardized video based on the compression classifications and generates a feedback video based on the mapped metrics. An analysis module identifies deviations from CPR guidelines based on the feedback video and generates analysis results based on the deviations. A feedback module provides performance feedback to the trainee based on the analysis results.

The invention relates to a video understanding method and system based on a multi-modal evidence chain. The method comprises the steps of obtaining a question text, an option set and a target video input by a user; using the question text and the option set to form a complementary analysis angle set; performing frame-by-frame matching on the target video based on a preset angle feature mapping library to obtain key timestamp sets corresponding to different analysis angles; extracting a corresponding key video frame set from the target video; scoring each key video frame in the key video frame set, and constructing an angle-frame mapping table between different analysis angles and corresponding visual evidence frames; obtaining a to-be-reasoned text needing visual evidence, and associating the to-be-reasoned text with the angle-frame mapping table to construct a multi-modal evidence chain; and obtaining a video reasoning result of the problem text based on the multi-modal evidence chain. High-precision and high-interpretability video understanding is realized, and the dependence of video understanding on large-scale annotation data is reduced.

The invention provides a motion video key clip extraction method based on behavior analysis. The method comprises the following steps: firstly, performing decoding and frame standardization on a match video stream, detecting and positioning a coach target, and obtaining and expanding a bounding box; extracting a candidate skeleton key point set in the extended region, selecting the skeleton with the most key points as the skeleton of the coach, and performing cross-frame tracking; analyzing the skeleton in real time, triggering hand fine analysis and extracting hand key points when a scoring trend appears, comprehensively judging based on geometry, kinematics and time sequence rules, and generating a trigger signal when a specific scoring gesture is detected; and the background thread extracts fragments before and after the triggering moment from the video stream as key fragments, and stores the key fragments in association with metadata such as event types, coach identities and competition states.

The invention discloses a physical object three-dimensional positioning method and system based on video data, and relates to the technical field of three-dimensional positioning. The method is used for solving the problems of low object space positioning precision, unstable pose estimation and poor time sequence continuity in a video scene. Firstly, an input video stream is analyzed, an object segmentation mask is generated, feature points are extracted, camera motion parameters are calculated through inter-frame matching, and a scene sparse three-dimensional point cloud is reconstructed; a candidate three-dimensional bounding box is generated according to the segmentation mask and the point cloud, an optimal bounding box is selected by combining geometric matching degree and feature similarity evaluation, and a preliminary three-dimensional positioning result of the object is obtained; a positioning result and historical frame motion data are fused, a space-time constraint optimization model is constructed, and the six-degree-of-freedom pose of the object is solved; and finally, neural radiation field representation is established based on the pose, the pose and neural parameters are optimized through combination of micro rendering and back propagation, continuous and accurate updating of the three-dimensional position of the object is realized, and the positioning stability and robustness in a dynamic scene are improved.

The embodiment of the invention discloses an image target labeling method and device, electronic equipment and a storage medium. According to the embodiment of the invention, a to-be-labeled video frame sequence of a road camera can be acquired; detecting any current frame in the video frame sequence based on a preset target detection model, and generating a detection frame for the target detection object; obtaining a prediction frame in the current frame, wherein the prediction frame is generated by a preset target tracking model according to the motion state of the target tracking object in the previous frame or multiple frames and the position of the detection frame; and matching the detection frame in the current frame with the prediction frame, if matching succeeds, allocating the historical identity identifier of the target tracking object corresponding to the prediction frame to the target detection object corresponding to the detection frame, and if matching fails, allocating a new identity identifier to the target detection object corresponding to the detection frame. Therefore, based on the time-space coherence of the video, the target is tracked and labeled efficiently and accurately.

The invention relates to the technical field of video monitoring, in particular to a livestock farm safety management method and system based on video monitoring, and the method comprises the steps: collecting a monitoring video of a livestock farm, and equally dividing each frame of video image into each CTU block; marking pixel points corresponding to the target object in each frame of video image as salient pixel points, and determining the static saliency of each CTU block; obtaining the motion pixel aggregation degree of each coordinate point in each frame of video image; determining the center-of-mass coordinate and the motion displacement of the target object in each frame of video image, and obtaining the dynamic saliency of each CTU block; and combining the static saliency and the dynamic saliency, determining a saliency weight of each CTU block, distributing a code rate for each CTU block, and carrying out coding compression on the video image for livestock farm safety management. Therefore, the monitoring video quality of the livestock farm is improved, and the safety management effect of the livestock farm is enhanced.

The invention relates to the technical field of industrial visual intelligence, in particular to an intelligent image selection and switching system fusing visual features and quality scores, which comprises the following steps: receiving multiple paths of video coding streams, inter-frame motion vectors and camera parameters; generating a macro block activeness distribution map based on the video coding stream and the inter-frame motion vector, and performing local window positioning and feature reconstruction on the video coding stream to generate an enhanced video vector; calculating a confidence coefficient mean value and a consistency score of the video coding stream according to the enhanced video vector, and fusing the confidence coefficient mean value and the consistency score with a channel transmission signal-to-noise ratio and a quantization noise increment to generate a video quality score; constructing a multi-criterion optimization model, and setting a feature representation vector for the multi-criterion optimization model; mapping the viewpoint weight based on the inner product of the feature representation vector, and calculating with the video quality score to generate a video switching score; and performing priority ranking based on the video switching score, and triggering a mapping switching instruction. And realizing video image scheduling by fusing the visual features and the quality score.

The invention provides a voice recognition method and device, equipment, a storage medium and a program product, and relates to the technical field of image processing. The speech recognition method comprises the following steps: acquiring speech to be recognized, and extracting speech features based on the speech to be recognized; obtaining video content corresponding to the to-be-recognized voice, and extracting video features based on the video content; acquiring a historical voice recognition text of the to-be-recognized voice, and extracting historical text features based on the historical voice recognition text; obtaining a first multi-modal fusion feature based on the voice feature and the historical text feature; obtaining a second multi-modal fusion feature based on the video feature and the historical text feature; and generating a speech recognition text corresponding to the speech to be recognized based on the first multi-modal fusion feature and the second multi-modal fusion feature.

The invention relates to the technical field of video behavior recognition, and discloses a video abnormal behavior recognition method fusing spatio-temporal characteristics. The method comprises the following steps: a video data modeling step: modeling according to a current video frame sequence and preset behavior characteristics to obtain a video behavior model; an experience pool forming step of dividing a plurality of experience layers according to historical identification result differences and record confidence by means of historical video abnormal behavior records to form a multi-layer experience pool; a strategy determination step of determining an intelligent identification strategy of each stage based on a video behavior model target behavior state, and screening a multilayer experience pool according to record confidence and a target matching degree to obtain an experience identification strategy; a strategy adjustment step: dynamically adjusting the two strategies by using a dual-channel mechanism to adapt to a real-time video environment, and determining a target identification strategy; and a behavior state acquisition step of analyzing the video frame sequence according to an identification instruction to acquire an actual behavior state, the identification instruction being generated based on the target identification strategy and the current video frame sequence.

The invention discloses a heartbeat detection method and device based on video and millimeter wave radar fusion, equipment, a medium and a product, and relates to the field of heartbeat detection. The method comprises the following steps: acquiring multi-modal sensor data; performing video modal heartbeat feature extraction according to the face video data stream based on a space attention mechanism to obtain a video heartbeat feature embedded vector; then heartbeat feature extraction is carried out, weighted fusion processing is carried out based on an attention mechanism guided by a video, and a radar heartbeat feature embedded vector is obtained; dimension normalization processing is carried out on the video heartbeat feature embedded vector and the radar heartbeat feature embedded vector, an environment feature vector is obtained through global average pooling, scene division is carried out based on a scene classifier, and a scene classification result is obtained; and after determining a multi-modal fusion weight according to the environment feature vector and a scene classification result, carrying out heartbeat signal reconstruction to obtain a heartbeat signal sequence. The objective of the invention is to improve robustness and detection precision.

The invention relates to a concrete mixing main machine visual monitoring system and a method thereof, and belongs to the technical field of concrete production automation and intelligent quality control, and the method comprises the following steps: S1, collecting a video image sequence of movement of concrete in a mixing main machine in real time; s2, extracting a dynamic visual feature vector and a static feature vector based on the video image sequence; s3, on the basis of the dynamic visual feature vector and a preset dynamic rheological-visual feature coupling model, calculating an internal rheological state index; s4, based on the static feature vector and a preset static correlation model, calculating a predicted value of the foundation slump; s5, determining a final corrected slump through a preset correction function in combination with the intrinsic rheological state index and the predicted value of the foundation slump; s6, according to the final corrected slump, the predicted value of the foundation slump and the intrinsic rheological state index, the comprehensive risk index is calculated, and the intrinsic rheological characteristics which are caused by chemical admixtures and cannot be perceived by a traditional visual method can be deeply captured.

Embodiments of the disclosure provide a solution for video processing. A method for video processing is proposed. The method includes: determining, for a conversion between a video unit of a video and a bitstream of the video, motion information of the video unit based on template matching for intra block copy (IBC) or intra template matching prediction (IntraTMP), wherein the template matching for IBC or IntraTMP is different from template matching for inter prediction; and performing the conversion based on the prediction or reconstruction of the video unit.

The invention discloses a method and a system for automatically generating a video based on multi-agent unstructured knowledge, and relates to the technical field of video automatic generation based on knowledge processing, and through a data source weight evaluation mechanism and a content semantic similarity calculation method, repeated or contradictory knowledge fragments are automatically identified, a disputed knowledge unit list is established, and the video is automatically generated according to the disputed knowledge unit list. And analyzing the matching degree of the dispute content and the preset script by adopting a semantic vector similarity algorithm. And for the recognized dispute knowledge points, a standby knowledge replacement algorithm is applied to retrieve replacement contents with high confidence from a reliable knowledge base, a knowledge unit replacement operation is executed, and high-quality video output with dispute identification is generated, so that the content accuracy and credibility of the multi-source knowledge fusion video are effectively improved. According to the method, a whole-process quality control system from knowledge identification to video generation is established, intelligent matching of knowledge credibility evaluation and a video content presentation mode is realized, and practical deployment of the technology is restricted.