10333 results about "Video streaming" patented technology

The invention relates to the technical field of camera linkage alarm, and discloses a camera linkage alarm method and system for intelligent environment monitoring, and the method comprises the steps: synchronously collecting the video stream data of a plurality of cameras and the multi-parameter monitoring data of an environment sensor; executing multi-path feature analysis and cross-modal feature fusion to obtain environment-vision joint features; environment scene self-adaptive classification is executed, and environment-vision joint features are analyzed through an anomaly judgment model to obtain an abnormal event judgment result; executing multi-camera collaborative verification and vision-environment data association analysis to obtain multi-dimensional abnormal event description information; and selecting a multi-camera linkage response strategy and coordinating the plurality of cameras to carry out three-dimensional monitoring on the abnormal area to obtain a linkage response execution result, so that a false alarm mode can be identified and an abnormal judgment rule can be dynamically adjusted, continuous optimization of the performance of the plurality of cameras is realized, and the false alarm rate and the missing report rate of the plurality of cameras in long-term operation are reduced.

The invention provides an AI-based factory abnormal behavior identification monitoring method and system. The method comprises the steps of obtaining continuous video stream data collected by a plurality of monitoring devices in a target factory in real time; performing time dimension segmentation and space dimension preprocessing on the continuous video stream data to obtain a multi-frame behavior feature set corresponding to the dynamic behavior sequence, and inputting the multi-frame behavior feature set into a pre-trained abnormal behavior recognition model, performing cross-modal association fusion on the skeleton key point features, the environment associated object contour features and the action trajectory features to generate a target behavior feature vector, performing similarity matching with a preset abnormal behavior feature library, determining an abnormal behavior category, and triggering a corresponding early warning signal according to the behavior type when it is determined that the target behavior feature vector belongs to the abnormal behavior category. And real-time alarm display is carried out. According to the invention, the problem that the existing monitoring system is insufficient in accuracy and real-time performance can be solved.

The invention provides an early warning analysis method based on intelligent vision and a server, and the method comprises the steps: obtaining a multi-mode video stream containing a visible light and infrared thermal imaging image sequence through a vision collection device in a monitoring region, and carrying out the spatial-temporal feature extraction of the multi-mode video stream; the method comprises the steps of generating dynamic behavior sequence characteristics including a target object moving track, posture change and thermodynamic distribution characteristics, analyzing the dynamic behavior sequence characteristics through a preset abnormal behavior judgment model in a multi-dimensional matching mode, judging whether the target object triggers abnormal behavior characteristics or not, and if yes, generating a graded early warning signal in combination with an association weight. In addition, according to the spatial distribution and the time length of the abnormal behavior characteristics, the parameters of the acquisition equipment and the coverage range of the monitoring area are dynamically adjusted, and the subsequent video stream anomaly detection precision is optimized, so that intelligent and accurate early warning analysis is realized, and the security efficiency is improved.

The invention relates to the technical field of resource allocation, in particular to a multi-channel video stream cooperative transmission method based on dynamic priority, which comprises the following steps of: acquiring task context characteristic parameters, system resource states, user behavior responses and computing node load data in real time, and constructing a priority allocation model to carry out pattern recognition to generate real-time task priority. And establishing a task scheduling strategy generation model, and performing resource allocation by adopting a priority weighting efficiency evaluation function, a memory-computing unit occupancy rate prediction matrix and a gradient optimization target under an adjustment resource constraint condition to form an initial scheduling strategy. And performing multi-dimensional parameter fusion analysis on the data through an adaptive optimization decision model, generating a resource redistribution correction vector, dynamically adjusting an initial scheduling strategy by adopting an online iterative optimization mechanism, and outputting a final real-time optimization task scheduling strategy. The task priority dynamic evaluation and the closed-loop optimization of the resource allocation are realized, and the efficient cooperative execution of the multi-channel video processing task is ensured.

The invention belongs to the technical field of water conservancy and hydropower engineering, and discloses a water conservancy and hydropower engineering construction safety supervision system based on multi-source data fusion. The system comprises a multi-source sensing acquisition module, a heterogeneous data fusion processing module, a risk identification and early warning module, a safety behavior evaluation and feedback module, and a command scheduling and visualization module. According to the invention, by fusing multi-dimensional data such as image monitoring, environment sensing, personnel positioning, equipment state and the like, a space-air-ground three-dimensional sensing network is constructed, and in a high slope area, the distributed optical fiber strain sensors are linked with thermal imaging data of the unmanned aerial vehicle, so that millimeter-level deformation and temperature field abnormity can be captured in real time; a video stream is analyzed in real time by means of a YOLOv8 algorithm, illegal operation behaviors of personnel can be accurately identified, a cross-modal fusion model of a Transform architecture is combined, the system can dynamically capture potential correlation among data, and millisecond-level response to risks such as side slope landslide, equipment faults and personnel dangerous operation is achieved.

The method is suitable for the technical field of automatic driving, an automatic driving large model framework based on 3D space-time perception and human-like decision reasoning is provided, through deep integration of cross-modal dynamic semantic patch embedding and multi-scale chain reasoning, the three-dimensional space-time semantic reasoning ability of an automatic driving system in a complex scene is effectively improved, and the automatic driving efficiency is improved. The efficient cooperative processing of the full-view scene video stream and the global aerial view is realized; meanwhile, through a multi-level space-time semantic self-adaptive data extraction normal form and semantic action representation generation based on rule driving and semantic action interaction, and in combination with a cross-modal task-oriented multi-level training matrix architecture, the model shows excellent performance in the aspects of scene understanding, decision transparency and end-to-end driving task optimization. The accuracy of the system in dynamic behavior deduction is effectively improved, the decision interpretability is improved, and powerful technical support is provided for global path planning and human-like decision making of automatic driving.

The invention relates to the technical field of industrial intelligent monitoring, in particular to a remote online monitoring method and system based on machine vision and artificial intelligence, and the method comprises the following steps: synchronously collecting a visible light video stream, infrared thermal imaging data and three-dimensional vibration spectrum data through an edge calculation node, and forming a multi-modal sensing data set; performing cross-domain feature alignment processing on the data set to generate a space-time synchronous composite feature matrix; inputting the composite feature matrix into a cascaded deep learning model, and outputting a three-dimensional diagnosis vector which comprises an equipment health state, an abnormal region and a fault probability; and matching the three-dimensional diagnosis vector with a historical reference vector through a DTW and cosine similarity combination algorithm to generate a deviation index, and outputting a self-adaptive alarm threshold and a maintenance suggestion scheme according to the deviation index. The method has the advantages of high multi-modal fusion degree, light diagnosis model and intelligent and adjustable alarm mechanism, and is suitable for real-time remote operation and maintenance in a high-risk industrial scene.

A video streaming system including a video cache system having a cache database and a video storage device, and including a video server configured to communicate with a gateway system configured to store multiple video streams, communicate with a cloud backup system configured to store video in a file system, and communicate with a cloud server to receive a stream request, the stream request indicating requested video that comprises multiple video segments. The video server is further configured to receive the stream request and determine a playlist that indicates where on the video cache system, the cloud backup system, and the gateway system the video segments that are needed to fulfill the stream request are stored, and to communicate needed video segments to the video cache system in preparation for providing streaming video to fulfill the stream request.

The invention provides a tunnel construction monitoring analysis method and system based on AI video monitoring, and relates to the technical field of artificial intelligence. A tunnel construction original video stream data set is acquired through a multi-view video acquisition device, and a spatio-temporal feature sequence is generated through spatio-temporal feature fusion processing; the spatiotemporal feature unit comprises a pixel region movement track, a personnel wearing identifier and a construction equipment contour feature, then extracting dynamic features of the spatiotemporal feature sequence to obtain a dynamic spatiotemporal feature set, and executing construction personnel statistical processing based on the dynamic spatiotemporal feature set; generating a hole entering personnel quantity statistical result, a personnel distribution statistical result of each construction part and a personnel identity matching result, performing work efficiency comparative analysis according to the personnel statistical result and a preset construction progress plan, and generating a construction progress deviation index and resource allocation optimization strategy; and when the progress deviation index exceeds the first threshold value, early warning is triggered, and a strategy is output to the management terminal, so that effective monitoring and management of tunnel construction are realized.

The invention relates to the field of personnel action monitoring and recognition, and discloses an intelligent human shape trajectory prediction and alarm system and method based on multi-modal video analysis, and the system comprises a video collection module which is used for collecting video stream data of a monitoring scene; the data processing module is used for extracting motion state information of the target human shape; the trajectory modeling module is used for constructing a trajectory model of the target person; the risk assessment module is used for calculating a behavior risk value; the alarm module is used for triggering alarm; and the optimization module is used for optimizing the trajectory model. Target motion state information is extracted through the data processing module, high-precision input is provided for the trajectory modeling module, and the trajectory modeling module generates a prediction trajectory in combination with a dynamic optimization algorithm. Compared with a scheme depending on coarse-grained detection or simple position tracking in the prior art, the method has the advantages that fine modeling and dynamic prediction of the motion state are realized, the problem that complex behaviors cannot be accurately reflected is solved, and meanwhile, the track prediction precision and the system response capability are improved.

The invention provides an edge computing traffic light intelligent decision-making method integrated with AI video analysis, and the method comprises the steps: collecting a real-time traffic video stream of a target intersection through an edge computing device, and extracting a dynamic traffic feature set in the real-time traffic video stream, calling a pre-trained space-time analysis model to carry out multi-modal fusion processing on the dynamic traffic characteristic set, generating a traffic state vector of the target intersection, matching a candidate control strategy in a preset decision rule base based on the traffic state vector, and carrying out parameter adjustment on the candidate control strategy through a strategy optimization model to obtain a target traffic state vector; generating a target signal lamp control parameter; and issuing the target signal lamp control parameters to a traffic signal control terminal of the target intersection, and monitoring traffic state change data of the target intersection in real time to update weight parameters of the space-time analysis model. According to the invention, the traffic efficiency, pedestrian safety and traffic management intelligence degree of urban intersections can be improved.

The invention relates to the technical field of traffic management, in particular to a video monitoring traffic intelligent optimization system which comprises a time sequence fragment evaluation module, a path priority adjustment module, a traffic scheduling optimization module and a buffer strategy adaptation module. According to the invention, the transmission efficiency and quality of the video monitoring data are improved by analyzing the video stream in real time and adjusting the frame rate and the compression rate, the importance of each time sequence segment in the video stream is evaluated, the frame rate is adjusted according to the importance, more resources are ensured to be obtained by important events, unnecessary data transmission is reduced, network congestion is effectively reduced, and the transmission efficiency of the video monitoring data is improved. The network transmission path is monitored in real time, and the data transmission proportion is adjusted, so that the video stream passes through the network path with higher quality more preferentially, the stability and speed of video transmission are enhanced, the priority and bandwidth allocation are dynamically adjusted, and the video stream with high priority is protected to ensure that the video quality of a key monitoring area is not influenced by network fluctuation.

Systems, devices, and methods are provided for determining factually consistent generative narrations. A narrative may be generated by performing steps to determine one or more metadata messages for a first portion of a video stream, determine transcribed commentary for a second portion of the video stream, wherein the second portion includes the first portion, and determine a prompt based at least in part on the one or more metadata messages and the transcribed commentary. The prompt may be provided to a generative model that produces an output text. Techniques for performing a factual consistency evaluation may be used to determine a consistency score for the output text that indicates whether the output text is factually consistent with the one or more metadata messages and the transcribed commentary. A narrated highlight video may be generated using the consistent narrative.

The invention provides a data center inspection robot monitoring analysis method and system based on machine vision, and relates to the technical field of inspection robots, and the method comprises the steps: obtaining a video stream and environment parameter data collected by an inspection robot, and carrying out the detection and recognition of an abnormal state through a deep learning target; and constructing a multi-modal data fusion analysis model to form a knowledge graph to generate a root cause analysis result, and planning an inspection path based on priority scores. According to the invention, intelligentization and precision of data center monitoring are realized, and inspection efficiency and fault diagnosis accuracy are improved.

The invention discloses a personnel positioning method and system based on a 3D Gaussian splash model and video fusion. The method comprises the following steps: firstly, acquiring input data of at least two visual angles through a multi-visual angle video stream acquisition module, and constructing an initial three-dimensional Gaussian model by utilizing a sparse point cloud initialization module; a time sequence dynamic tracking module is combined with an optical flow algorithm to realize cross-frame parameter updating of a Gaussian ellipsoid, and a time sequence consistency optimization module is adopted to suppress parameter drift; identifying a constructor bounding box by using a target detection module, and establishing a corresponding relation between pixels and three-dimensional coordinates through a three-dimensional-two-dimensional space matching module; three-dimensional coordinates are calculated through a multi-view fusion positioning module, and the positioning precision is improved through a multi-sensor fusion optimization module in combination with IMU data. A 3D Gaussian splash model is combined with multi-view geometry and time sequence optimization, high-precision personnel dynamic positioning without marking in a construction scene is realized, and the problems of tracking drift and shielding in a complex environment in a traditional method are effectively solved.

The invention discloses an intelligent port crane operation monitoring method and system, and relates to the technical field of crane systems.The system comprises a data acquisition unit, an edge computing node, a cloud processor and an early warning monitoring unit, through deep fusion and efficient treatment of multi-source data, a dynamic twinborn data pool architecture is established, and the dynamic twinborn data pool architecture is established; cross-data-type space-time alignment is realized through multi-modal coding, and the feature fusion accuracy is improved; the dynamic twin data pool is intelligently analyzed, and the crane operation path is optimized in real time, so that the track of the crane operation path is dynamically adjusted, and the working efficiency is improved; through a digital twin model and integration of a multi-modal physical engine, the operation physical state deviation of the crane is evaluated, an operation plan is corrected in real time, and the operation precision is improved; video streams are deeply analyzed, virtual-real comparison is carried out, and model prediction video deviation is monitored, so that the analogue simulation precision is improved; by establishing a closed-loop early warning mechanism, full-process digital management and control are realized, and intelligent upgrading of a port is promoted.

The invention discloses a multi-modal image automatic labeling system and method, and relates to the technical field of image data processing. According to the multi-modal image automatic labeling system and method, time sequence alignment is carried out on video streams and laser radar point cloud data through an asymmetric dynamic time warping algorithm, semantic and geometric features are extracted, and the elastic coefficient of the algorithm is dynamically adjusted. And combining a modal perception attention mechanism, dynamically allocating fusion weights of the video stream and the laser radar according to the features, generating a cross-modal joint feature vector, and outputting a preliminary labeling result. And generating an annotation robustness index by calculating the prediction entropy and the three-dimensional intersection-to-union ratio confidence of the target detection frame, and iteratively optimizing the annotation result. And mapping the cross-modal features and the labeling result into a space-time correlation map, and displaying the three-dimensional positioning, motion trail and modal contribution degree thermodynamic diagram of the target in real time. The problems of time alignment, feature fusion and labeling robustness are effectively solved, and a high-precision and interpretable automatic labeling solution is provided.

A system for dynamic latent space adaptation using spatiotemporal kernel context for multiscale rendering with hierarchical and Lorentzian autoencoders. The Spatiotemporal Kernel Estimator (SKE) analyzes media through motion field, temporal recurrence, frequency band, and scene semantics analyzers to generate adaptive kernel parameters encoding content-specific importance distributions. The system dynamically adapts latent manifold geometry by modifying metric tensor properties according to kernel context, enabling content-aware compression that allocates representational capacity based on visual significance. A multiscale cache implements kernel-adaptive retention policies prioritizing important regions. An adaptive renderer provides intelligent level-of-detail selection based on zoom level and kernel-estimated importance, optimizing processing allocation. The self-optimizing architecture continuously refines kernel context and geometric adaptation based on user interaction and performance feedback, achieving superior compression ratios and perceptual quality. Applications include bandwidth-efficient video streaming, virtual reality, scientific visualization, and cognitive video analytics requiring intelligent context-aware visual processing.

The invention provides a large-scale scene multi-level-of-detail cloud rendering processing method and device based on 3DGS, and relates to the technical field of three-dimensional modeling, and the method comprises the steps: dividing a target modeling scene into a plurality of sub-blocks; performing particle redundancy reconstruction and overlapping region marking on boundary regions between adjacent sub-blocks of each sub-block to obtain processed sub-blocks; performing multi-detail level division on each processing sub-block to generate a particle level set; determining a current visual area according to the user motion data, and scheduling a target hierarchy of a particle hierarchy set in the current visual area; the rendering tasks of all the processing sub-blocks of the target hierarchy are distributed to a plurality of rendering nodes to execute real-time rendering operation; and performing video stream coding on pictures rendered by each rendering node, decoding and displaying received video stream data, and performing particle level updating and re-rendering operation according to an interaction instruction. According to the invention, high-quality detail rendering can be realized for a model of a large scene.

The invention relates to the technical field of safety monitoring, in particular to a smart park safety management method and system based on AI visual identification, and the method comprises the following steps: obtaining a video stream captured by a monitoring camera of a smart park, extracting the movement track data of a person through AI, and carrying out the statistics of the stay duration and access times of a target object in each grid; and constructing retention heat distribution of the grid region to obtain a personnel trajectory retention heat value. According to the invention, through the data processing of the monitoring video stream and the combination of the gridding analysis of the personnel motion trail, the accurate capture of the abnormal cruise behavior is enhanced, and through the calculation of the change rate of the access times between the adjacent grids, the statistics of the round-trip condition of the target among the plurality of areas and the combination of the passing authority data, the abnormal screening is carried out. The method improves the recognition capability of abnormal movement modes, achieves the dynamic adjustment of regional safety risk levels, improves the response capability of park security, enables the recognition capability of abnormal behaviors to be enhanced, and reduces the false alarm rate.

The invention relates to the technical field of intelligent inspection, and discloses a regional abnormal condition real-time early warning method based on high-point panoramic intelligent inspection, which comprises the following steps: collecting visible light and infrared thermal imaging video streams of a target region to form a panoramic video sequence, and carrying out intelligent analysis, feature extraction and analysis, construction of a spatio-temporal topological graph and detection of an abnormal behavior mode. Predicting environmental risks, constructing a spatio-temporal evolution model, and further generating graded early warning information of regional abnormal conditions; the method effectively solves the problems of large panoramic inspection data volume, exception complexity, easy environmental influence on target detection and the like in a large-scale scene, significantly improves the accuracy and real-time performance of exception early warning, and guarantees the regional safety.

The invention belongs to the technical field of intelligent perception and mode recognition, and particularly relates to a human body behavior recognition method based on millimeter wave radar and multi-task learning. The method comprises the steps that millimeter wave radar point cloud data and RGB video streams are synchronously collected, and a five-dimensional point cloud scene is generated through three-dimensional analysis and dynamic target extraction; a hierarchical Point Transform network is constructed to extract radar space-time features, and human body key point features are generated by using visual auxiliary attitude estimation; radar and key point features are dynamically fused through a cross-modal attention mechanism, a multi-task joint optimization strategy is combined, behavior classification serves as a main task, attitude estimation serves as an auxiliary task, and a behavior recognition result is output. According to the method, through multi-task cooperation and cross-modal feature interaction, on the premise of ensuring privacy security, the accuracy and robustness of human behavior recognition in a complex scene are remarkably improved, and efficient and reliable technical support is provided for the fields of intelligent monitoring, human-computer interaction and the like.

Embodiments include methods for generating advertisement videos for insertion into a video stream to promote a product, service, or brand in a manner that is consistent with the context and storyline of the video stream before and at the time of ad insertion. Methods may include capturing an image from the video stream and generating caption text using an image-to-text description model. A product, service, or brand that is consistent with the context and storyline of the captured image is selected and ad video sequence description text is generated that includes descriptions and a storyline blending descriptions of the selected product, service, or brand with the context and storyline of the primary video stream. The ad video sequence description text is used to prompt a text-to-video generation model that generates a new advertisement video clip, which is inserted into the primary video stream before distribution to video content rendering devices.

One example method includes receiving, during a virtual conference from a client device, a request to generate a real-time summary of the virtual conference, a plurality of client devices participating in the virtual conference and exchanging audio and video streams; receiving, during the virtual conference, a plurality of utterances generated by automatic speech recognition (“ASR”) of the audio streams; generating a group of consecutive utterances from the plurality of utterances; determining a segment based on the group of consecutive utterances; generating, using a large language model (“LLM”), a segment summary based on the segment; and providing the segment summary to the client device.

The invention discloses a video stream dynamic fragmentation encryption and block chain evidence storage method, and relates to the technical field of video content security, and the method comprises the steps: calculating a color histogram difference value and an optical flow vector change rate between adjacent frames of an input video, marking the difference value as a scene switching point when the difference value exceeds a preset threshold value, and storing the scene switching point; the method comprises the following steps: preliminarily dividing a video into a plurality of scene segments according to scene switching points, performing content complexity evaluation on the scene segments, calculating gray level co-occurrence matrix characteristics of each frame of image through texture density analysis, calculating edge complexity to extract the number and distribution of Canny edges, and performing motion vector statistics to analyze the size and direction of inter-frame object displacement. The change rate between adjacent pixels in the color space is measured according to the color change gradient; the video stream dynamic fragment encryption and block chain evidence storage method is suitable for video contents of different types and complexities, and has relatively high detection accuracy and robustness.

The invention provides an intelligent event identification method and system based on a high-speed camera, and the method comprises the steps: setting the collection parameters of the high-speed camera, and triggering the camera to collect a target scene video stream. And hardware acceleration decoding processing is carried out on the collected original video data stream, real-time environment illumination information of the environment illumination sensor is obtained, and dynamic brightness equalization processing is executed. And performing motion adaptive denoising processing on the video sequence. Geometric distortion correction is carried out on the image sequence through camera calibration parameters, sub-pixel-level displacement vectors and dense optical flow field data of a moving object are extracted, and multi-scale morphological features are extracted. And the features are fused to generate motion feature data, the data are processed through a spatio-temporal joint event classification model, an event identification result is output, the result is bound with a high-precision timestamp, and event identification information is output to an industrial control system display device in real time. According to the invention, the accuracy and real-time performance of event identification can be improved.

The invention discloses a human abnormal behavior monitoring method based on a large-model multi-agent, which is executed by a modular multi-agent system deployed on a back-end server, obtains information through a monitoring camera, and comprises the following steps: obtaining a video stream from the monitoring camera by a sensing agent and extracting human body posture features; analyzing the key frame by a scene understanding agent by using a visual large model, and constructing a time sequence dynamic scene graph; the core reasoning agent evaluates the scene semantic conformity based on the pre-trained large model and performs abnormal preliminary judgment; performing fine-grained classification, interpretation generation and risk assessment on the abnormal behaviors; and the report and action agent generates an alarm and records event data. According to the invention, through multi-agent cooperative work and a large model technology, efficient and accurate monitoring of human abnormal behaviors is realized, and the intelligent level of the monitoring system and the abnormal behavior identification accuracy are improved.

The invention relates to a high-fidelity cloud rendering cluster scheduling method and system, and the method comprises the steps: receiving a rendering task, generating a multi-level priority queue according to the task complexity, timeliness and resource demand classification, and automatically optimizing a scheduling strategy; monitoring the resource state of the heterogeneous computing node in real time; allocating tasks by using preemptive and round-robin scheduling strategies, and adjusting and coping with resource fluctuation in combination with a dynamic code rate; the tasks are decomposed by adopting a spatial blocking and time framing strategy, and an execution sequence is controlled according to a topological sorting algorithm; abnormal nodes are identified through heartbeat detection, and affected subtasks are migrated through incremental task updating. According to the method, efficient resource allocation, scheduling algorithm and idle key frame scheme can be realized, the GPU directly outputs the video stream, the rendering speed is remarkably improved, resource waste and operation cost are reduced through a dynamic resource allocation mechanism, a fault-tolerant mechanism is provided, the task is ensured to be normally completed under the condition of node fault, and the task efficiency is improved. And distributed rendering and synthesis of large-scale high-resolution images are supported.

The invention relates to the technical field of video behavior recognition, and discloses a behavior recognition and early warning system for a real-time video stream. The system comprises a spatio-temporal feature modeling module, a behavior fragment extraction module, an anomaly propagation modeling module, a risk area positioning module and an early warning strategy generation module. The spatial-temporal feature modeling module builds a dynamic model based on historical data, captures a skeleton key point three-dimensional coordinate sequence, a motion optical flow vector field and a micro-expression intensity spectrum, and outputs a theoretical behavior mode vector; the behavior fragment extraction module generates a multi-modal difference feature tensor through cross-modal difference analysis; the exception propagation modeling module generates an exception propagation path risk probability distribution cloud picture in combination with spatial constraint and trajectory information; the risk area positioning module identifies a high-risk area and marks a boundary; and the early warning strategy generation module dynamically configures monitoring parameters, starts high-frame-rate micro-expression capture for a high-risk area, and performs a track disturbance test on an adjacent area.

The invention provides a multi-mode cooperative sensing power station high-risk operation inspection method and system, and relates to the technical field of video recognition, and the method comprises the steps: activating an unmanned plane and a quadruped robot after a power station operation task is started; starting a video acquisition unit, and establishing a synchronous video stream; carrying out fusion alignment with the global reference coordinate system through an external synchronization signal; inputting the video sequence of the fused view angle into a multi-view angle action behavior recognition network, and establishing a dangerous behavior grade score; auditory data and olfactory data of the quadruped robot are obtained, and linkage abnormity is established; and polling abnormity is reported according to linkage abnormity and dangerous behavior grade scores. Through the method and the device, the technical problem of low inspection efficiency caused by difficulty in comprehensively identifying potential risks in a dynamic environment due to limitation of a single sensing mode is solved, and the inspection efficiency of a power station is improved by fusing multi-mode data, timely finding and processing the potential risks and improving the accuracy of video and audio identification.