254 results about "Temporal modeling" patented technology

The present application relates to the technical field of load prediction, and provides an electrical load prediction method and device based on spatial-temporal correlation. The method comprises: performing spatial-temporal modeling on the basis of historical electrical load data and historical environment data to obtain a multi-layer spatial-temporal model, wherein each layer model is a directed graph and represents spatial causality in the fixed time, and a directed edge between the layer models represents temporal causality; on the basis of an interpretable spatial-temporal attention transformer, performing feature extraction and priori knowledge fusion on the multi-layer spatial-temporal model to obtain spatial-temporal characteristics of an electrical load, wherein the interpretable spatial-temporal attention transformer comprises a spatial causal attention network, a temporal attention network, and a spatial dependence comparison module; and on the basis of the spatial-temporal characteristics of the electrical load, and the real-time electrical load data and the real-time environment data of a target electrical node, predicting by using a trained STEF-DHNet model to obtain the predicted electrical load data of the target electrical node. The present application can improve the prediction accuracy and the model applicability.

An AI-enhanced distributed system for neural network-based data compression leverages reinforcement learning optimization and privacy-preserving computation across edge and central computing devices to autonomously optimize efficiency and quality. The system includes a lightweight compression subsystem at edge devices that applies privacy-preserving preprocessing and partially compresses input data before securely transmitting it to central computing devices. A reinforcement learning agent continuously monitors system performance and automatically optimizes compression parameters, model selection, and task allocation based on multi-objective rewards. The central compression subsystem processes data using AI-optimized parameters and temporal modeling components. The system incorporates hardware detection capabilities that automatically select optimal compression models based on available processing resources and implements homomorphic encryption for computation on encrypted data while coordinating federated learning across distributed devices. This AI-enhanced distributed approach improves bandwidth efficiency, energy consumption, and adaptability while ensuring data privacy and security.

The invention discloses a gas pipe network fault positioning method based on a space-time diagram neural network, and the method employs an edge-cloud collaborative architecture, and achieves the precise detection of leakage points through simulation data generation, multi-modal perception, dynamic space-time modeling and hierarchical positioning. Firstly, simulation modeling is conducted on a pipe network, a multi-working-condition leakage data set is generated, and monitoring point layout is optimized through fuzzy clustering; a multi-modal sensing unit and a lightweight anomaly detection module are deployed at an edge end to realize coarse-grained anomaly detection and data hierarchical transmission; the cloud constructs a dynamic space-time diagram neural network, integrates a pipe network topological structure, multi-source time sequence data and physical constraints, and realizes high-precision positioning of leakage points through a hierarchical positioning strategy; and finally, realizing online evolution of the model through elastic weight solidification and hierarchical parameter updating. According to the method, the space-time diagram neural network and the physical characteristics of the pipe network are deeply fused, and the problems that space-time coupling features are difficult to extract and physical constraints are missing in complex pipe network fault positioning are solved.

The invention relates to the technical field of image processing and mode recognition, in particular to a classification method and device based on user video data and a medium, and the method comprises the steps: collecting video data uploaded by a user, and extracting a plurality of key frames from the video data; using a video feature extraction network and an image-level spatial feature extraction network with time sequence modeling capability to respectively extract time sequence features between the key frames and spatial features of each key frame, and generating video visual features based on the time sequence features and the spatial features; collecting behavior data of the user in the process of watching the video data, and constructing a final user behavior embedding vector based on the behavior data; fusing the final user behavior embedding vector with the video visual features to generate joint feature representation; and inputting the joint feature representation into a pre-trained graph neural network classification model for classification, and outputting one or more semantic category labels to which the video data belong.

The invention provides a spatial-temporal feature fusion-based dynamic graph convolution electroencephalogram depression detection method, which comprises the following steps of: firstly, segmenting a sample into fragments with the length of 1 second, and calculating power spectral density (PSD) as an input feature by adopting a Welch method; the time sequence and spatial characteristics of the EEG signals are synchronously extracted through a double-branch architecture, wherein one branch captures the long-term time sequence dependence of the EEG signals by using a GRU; and the other branch adopts an improved TSCN (separable convolution is introduced), multi-scale spatial features from fine to rough are extracted through causal convolution and expansion convolution of residual layered stacking, after double-branch features are adaptively fused based on an attention mechanism, a dynamic graph structure is constructed, functional connection evolution of brain intervals is modeled by using a graph convolution network, and a dynamic graph structure is constructed. The topological structure of the network is optimized through a back propagation process, and finally depression identification is realized through a Softmax classifier. According to the method, the time sequence modeling capability of the GRU and the multi-scale spatial analysis capability of the TSCN are fused, the representation limitation of a single model is broken through, the dynamic change of a brain function network is adaptively captured through dynamic graph convolution, the physiological interpretability is enhanced, deep complementary fusion of EEG spatial and temporal characteristics is realized, the depression recognition accuracy is remarkably improved, and the method is suitable for popularization and application. And an efficient tool is provided for auxiliary diagnosis of mental diseases.

The invention provides a traffic flow spatio-temporal feature adaptive extraction method based on a dynamic Kolmogorov-Arnold network, and aims to solve the defects of a traditional model in the aspects of dynamic spatio-temporal modeling, structural adaptability and feature expression efficiency. According to the method, a network structure is dynamically adjusted through differential topology search and a parameterized primary function library, a double-flow coupling architecture is designed to extract space and time dependent features respectively, and adaptive fusion of spatial and temporal features is realized by using a gating mechanism. The method comprises the following specific steps: performing space-time normalization and graph structure coding to generate a node feature matrix; extracting multi-scale spatial-temporal features in parallel by the dynamic graph convolution KANs and the time sequence convolution KANs; feature alignment and cooperative enhancement are realized based on a local correlation matrix and a bidirectional interactive attention mechanism; and the lightweight KAN decoder is combined with the dynamic basis function library to output a prediction result. Experiments show that the MAE is reduced to 1.78 vehicles per minute and the reasoning speed is improved by 1.8 times in a traffic accident emergency scene on a PeMS08 data set, and the method is suitable for an intelligent traffic management and vehicle-road cooperation system.

The invention relates to the technical field of network security, and particularly discloses an edge device network threat detection method and system based on an electric power large model, and the method comprises the steps: capturing a network message sequence in real time, extracting a time sequence randomness feature and a semantic deviation feature from a time dimension and a protocol dimension, and carrying out the fusion to form a comprehensive threat feature vector; performing multi-dimensional feature analysis and time sequence modeling by adopting a lightweight electric power large model to realize millisecond-level threat assessment; establishing a multi-level response mechanism, dynamically triggering a differential protection strategy according to the threat level, and ensuring the reliability and consistency of response actions through digital signature and collaborative verification; according to the method, the complex network attack in the power edge equipment can be effectively identified, the threat detection accuracy and the system defense capability are improved, and the strict requirements of a power system on real-time performance and reliability are met.

The invention relates to the technical field of computer vision, can be applied to the medical field and the financial science and technology field, and discloses a key frame extraction method, device and equipment based on dynamic reinforcement learning and a medium, which are applied to electronic application in a high-frequency transaction abnormal behavior monitoring scene or can be applied to a medical operation key frame extraction scene. The method comprises the steps of obtaining an original video stream and performing preprocessing to generate a standardized video frame; performing feature extraction and feature splicing on the standardized video frame, and performing time sequence modeling on the generated pair frame-level mixed feature vector to generate video-level time sequence representation; generating an enhancement action instruction based on the video-level time sequence representation through the strategy network, and performing enhancement processing on the standardized video frame according to the enhancement action instruction to generate an enhanced video frame; performing optimization processing on the strategy network according to the enhanced video frame to generate an updated strategy network; and performing feature extraction and optimization on the enhanced video frame to generate a target key frame. According to the invention, the key frame extraction precision is improved.

The invention provides a pumping method based on multi-agent reinforcement learning control, and belongs to the technical field of deep reinforcement learning. S2, feature extraction and fusion; s3, time sequence modeling; s4, generating an action; s5, carrying out environment interaction; s6, virtual experience generation; s7, experience pool management; s8, performing network training; and S9, performing closed-loop control. Discrete and continuous actions are controlled through the DQN and the DDPG respectively, collaborative optimization of start-stop and frequency adjustment of the water pump is achieved by combining the Critic network, and the problem that a traditional single agent cannot process a mixed action space is solved.

The invention discloses an intelligent temperature control system and method for a non-oriented silicon steel continuous annealing process, and relates to the technical field of non-oriented silicon steel temperature control, the intelligent temperature control system comprises a multi-source sensing fusion acquisition module, data are acquired through multi-modal sensing and edge calculation, a depth space-time modeling module is used for modeling prediction through LSTM and GNN, and the temperature of the non-oriented silicon steel continuous annealing process is obtained. The intelligent cooperative regulation and control module combines reinforcement learning with MPC temperature control and further comprises a monitoring feedback module, a multi-dimensional early warning module and a data insight optimization module, and all the modules work cooperatively. According to the invention, the temperature control precision is improved, the defective rate is reduced, the production efficiency is improved, the energy consumption is reduced, the quality is scientifically monitored, the parameters are adaptively adjusted, the remote operation is realized, and the continuous annealing temperature control of the non-oriented silicon steel is comprehensively optimized.

The invention discloses a sleep disorder prediction method and a sleep disorder prediction system based on deep learning and signal decomposition, the system adaptively determines parameters of variational mode decomposition (VMD) through an RIME optimization algorithm, and decomposes s-EEG signals into intrinsic mode functions (IMFs); the IMFs are dynamically weighted by using a multi-head self-attention mechanism (MHSA), and key features are extracted, and then the IMFs are subjected to dynamic weighting by using the multi-head self-attention mechanism (MHSA), and the IMFs are subjected to dynamic weighting by using the multi-head self-attention mechanism (MHSA); and finally, time sequence modeling is carried out through a long and short time memory network (LSTM), and high-precision prediction is realized. According to the method, the accuracy and stability of sleep disorder prediction are remarkably improved, technical support is provided for early diagnosis and intervention, and clinical application value is achieved.

The invention discloses a continuous time dynamics prediction method, system, terminal and medium fusing a diffusion model and a graph frequent differential equation, and relates to the technical field of dynamics prediction.The method comprises the steps that a multi-node time sequence is obtained, network structure inference is conducted on the multi-node time sequence through the diffusion model, and a potential graph structure between nodes is obtained; carrying out continuous time dynamic modeling by adopting a Scheng ordinary differential equation, and predicting a node state at any time point; diffusion reconstruction loss, dynamic prediction errors and structure sparsity constraints are constructed, total loss is established, network structure inference based on a diffusion model and dynamic modeling based on a Shenzheng differential equation are coupled based on the total loss, and collaborative training optimization is achieved. According to the method, the potential graph structure of the system can be stably recovered in a complex noise environment, high-precision and continuous prediction can be carried out on dynamic evolution of the potential graph structure, and the limitation that structure inference and continuous time modeling cannot be considered in the prior art is overcome.

The invention discloses a fake video detection method and system based on potential space learning, and belongs to the technical field of multimedia forensics, and the method comprises the steps: reinforcing the capturing capability of a model for the potential distribution difference of true and false videos through triple contrast learning, combining GRU network modeling time sequence dynamic features and 3D CNN extraction time-space content features, and obtaining a fake video detection result. The joint representation of video time-space domain features is realized, potential spatial features and content features are dynamically fused through an attention mechanism to focus and forge contradictory information, and a global self-attention mechanism of a time encoder is utilized to mine a long-range time sequence dependency relationship. The method overcomes the problems of insufficient generalization ability, weak time sequence modeling and single feature expression of an existing deep counterfeiting detection method, can effectively distinguish a high-quality counterfeiting video from a real video, improves the adaptability to an unknown counterfeiting technology, enhances the detection robustness of a time sequence anomaly and a complex counterfeiting scene, and improves the detection efficiency. And a solution is provided for deep counterfeiting identification.

The invention relates to a rolling bearing residual life prediction method based on space-time degradation characteristic decoupling, and the method comprises the steps: obtaining a vibration signal in a full life cycle operation process of a rolling bearing, and obtaining a sample sequence after preprocessing; performing complete ensemble empirical mode decomposition on the sample sequence, and extracting a plurality of IMF signals; on the basis of statistical threshold criterion in combination with energy mutation and self-correlation structure mutation analysis, identifying a degradation starting moment, adding a label to a sample sequence, and dividing a training set and a test set; training the space-time degeneration decoupling network by using the training set to obtain an RUL prediction model; testing the RUL prediction model by using the test set to obtain a prediction result; the space-time degeneration decoupling network combines a degeneration guide feature deconstruction module and a collaborative modeling strategy of a time modeling branch and a space modeling branch, captures time dynamic characteristics and a space hierarchical structure, and improves the accuracy, stability and reliability of RUL prediction.

The invention discloses a global dynamic perception space-time traffic flow prediction method based on graph packet representation learning, and belongs to the technical field of traffic flow prediction, and the method comprises the following steps: S1, traffic data input, S2, traffic graph packet construction, S3, graph packet initial feature extraction, S4, time sequence feature extraction, S5, spatial feature extraction, and S6, traffic flow prediction and output. Through a space-time modeling technology of graph packet representation learning and global dynamic perception, space-time characteristic elements of a traffic road network can be comprehensively covered, traditional traffic indexes such as flow and speed are concerned, elements such as road network topological association and cross-regional multi-hop association are also included, a dynamic dependency relationship between a time sequence and a spatial dimension is deeply mined, and a real-time dynamic perception effect is achieved. Therefore, the prediction result can reflect the real evolution law of the traffic flow more accurately, and a more scientific basis is provided for traffic management and decision making.

The invention relates to the technical field of integrated circuit reliability, and discloses an MOS device service life prediction method and system and electronic equipment, and the method comprises the steps: constructing a continuous time dynamic model, and expressing the degradation process of an MOS device; constructing a device relation graph structure, and capturing spatial correlation between devices; executing multi-time scale decomposition, and analyzing time sequence characteristics of device degradation; a self-adaptive numerical solution algorithm is applied to realize degradation track prediction; physical constraint regularization is implemented, and the physical rationality of the model is improved. By fusing the graph neural differential equation and the multi-time scale decomposition technology, the problems of insufficient discrete time modeling, neglect of a synergistic effect between devices, difficulty in multi-scale characteristic processing and the like in the prior art are solved, and high-precision prediction of the service life of the MOS device is realized.

The invention discloses an abnormal traffic detection method and system based on industrial process space-time modeling, and relates to the technical field of abnormal traffic detection. According to the method, more accurate spatial features and time sequence features can be extracted, a model which better conforms to an actual scene is constructed, and the anomaly detection effect is improved. Through the GRU and Informer combined time sequence feature extraction method, the short-term time sequence feature and the long-term time sequence feature of the industrial process data are extracted, the characteristics of the industrial process data are met, and the abnormal flow detection accuracy of the industrial process is improved. A joint optimization strategy combining a prediction model and a reconstruction model is provided, so that the model can identify global distribution characteristics of a time sequence, and can also pay attention to individual characteristics of variables, thereby enhancing the accuracy and robustness of the model in an anomaly detection task.

The invention provides a traffic flow prediction method combining a dynamic graph neural network and a Mama mechanism. The future short-term traffic flow is predicted by using historical traffic data. According to the method, firstly, normalization preprocessing is carried out on traffic state data collected by multiple sensors, a training sample is generated by adopting a sliding window, and a traffic flow value in the next one hour is predicted according to data in the past 24 hours. On the basis of the model structure, a time modeling module composed of multiple layers of MambaBlocks is constructed and used for capturing historical time sequence dependence; constructing a spatial modeling module of dynamic graph convolution, and combining a static adjacency matrix of the road network with a learnable adaptive adjacency structure to extract spatial association; and finally, the outputs of the modules are fused, and a prediction result is obtained through a prediction output module. In the training process, a Huber loss function is used as an optimization target, and evaluation indexes such as a mean absolute error (MAE), a root mean square error (RMSE) and a mean absolute percentage error (MAPE) are used for evaluating the performance of the model. According to the method, the traffic space-time dynamic characteristics are effectively mined, and the long-range dependence modeling capability and the prediction precision are improved.

The invention provides a multi-modal semantic fusion image and text relevance dynamic analysis method and system, and belongs to the technical field of multi-modal data processing. The method comprises the steps of image and text data preprocessing, feature extraction, fusion semantic vector generation through a bidirectional cross attention mechanism, dynamic relevance score calculation through a time sequence attention long-short-term memory network and combined loss function end-to-end training. According to the method, accurate alignment of image and text features can be realized through a bidirectional cross attention mechanism, the cross-modal matching accuracy is improved, the dynamic correlation analysis capability is enhanced by using time sequence modeling, and the problems of insufficient feature coding suitability and limited time sequence modeling capability in the prior art are solved.

The invention discloses a deep learning model for picking up a seismic phase from a seismic signal with a low signal-to-noise ratio. The deep learning model comprises a feature extraction trunk, a bidirectional time sequence-channel attention module (BTCA) and a multi-scale dilated convolutional layer module (DSCN). According to the feature extraction trunk, a cascaded LiteMobileBlock module is used, shallow high-resolution features are extracted from an original waveform step by step, and an initial feature map is generated; the bidirectional time sequence-channel attention module integrates the time sequence modeling capability of the bidirectional LSTM and the spectrum sensing capability (emphasizing importance of different sensors / directions) of a channel attention mechanism, and outputs fusion features; and the multi-scale cavity convolution layer module utilizes parallel cavity convolution modules with different expansion coefficients to synchronously capture a local mutation and global oscillation mode of a waveform to generate multi-scale enhancement features, further enhance the capture capability of the model on long-range time dependence in seismic signals through LSTM, and output time sequence enhancement features. According to the model, a multi-scale cavity convolution module, a bidirectional time-frequency attention mechanism module and an LSTM enhanced sequence modeling module are fused, so that the feature extraction and time sequence modeling capability of a seismic signal with a low signal-to-noise ratio is improved, and robust pickup of a seismic phase is realized.

The invention discloses an oil pipeline monitoring method and system based on spatio-temporal modeling and multi-dimensional analysis, and belongs to the technical field of oil and gas pipeline safety monitoring, data processing and intelligent prediction.The method comprises the steps that multi-source spatio-temporal data is collected to construct a probability generation model; performing inversion on the model by using the observation value to reconstruct four-dimensional posterior probability distribution; performing causal inference on the posterior probability distribution to generate a dynamic causal information flow map; and analyzing topological evolution of the atlas to generate a stability monitoring report. According to the method, a technical path of combining probability modeling based on an information field theory and causal dynamics inversion is adopted, and accurate prediction of a systematic risk critical transition precursor can be realized by reconstructing a pipeline holographic state field and analyzing topological evolution of a causal network of the pipeline holographic state field; and the operation safety and the intelligent monitoring level of the long-distance oil pipeline are obviously improved.

The invention provides a 3D small sample segmentation method and system based on registration alignment and interlayer consistency. The method comprises the following steps: S1, supporting memory extraction; and S2, carrying out sequential segmentation. According to the invention, a 3D medical image is regarded as a series of 2D slice sequences, and sequential processing is carried out by fully using the time sequence modeling capability of SAM2. In this way, the FSMIS-SAM2 can maintain good space consistency between the slices, and the segmentation precision is remarkably improved. The core technology of the FSMI-SAM2 framework is to process a 3D medical image sequence by using a memory enhancement converter of the SAM2. In addition, the invention further provides an innovative inference initial position estimation method, the most appropriate initial slice can be determined for inference through accurate registration according to changes of the anatomical structure of a patient, it is ensured that a high-confidence-coefficient segmentation result can be obtained in the initial stage of sequence inference, and high-quality initialization is provided for subsequent slice segmentation.

The invention relates to a six-degree-of-freedom visual inertial odometer method based on double attention and a mixed basis function. The method comprises the following steps: collecting visual image data and inertial data and carrying out preprocessing; obtaining a visual feature vector; extracting an inertial feature vector; generating a final fusion feature; and inputting the final fusion features into a long and short term memory network for time sequence modeling, carrying out nonlinear feature enhancement by using a median Colmogorov-Arnold network, and outputting six-degree-of-freedom pose estimation in a target environment through a full-connection regression layer. According to the method, through a self-adaptive regulation and control mechanism of vision-inertia data, redundant information is compressed, and meanwhile key motion features are reserved; a double attention module is innovatively introduced to improve the feature discrimination ability, the mixed basis function modeling ability of the median Colmogolov-Arnold network is combined to realize the accurate representation of the complex nonlinear motion dynamics, and finally stable and high-precision six-degree-of-freedom pose estimation is achieved.

The invention discloses a dynamic scheduling method based on a dynamic priority queue and space-time conflict prediction, and relates to the field of conflict prediction and coordinated scheduling. The method comprises the following steps: acquiring information data of a target airport; based on a multi-service desk queuing model, according to airport flight resource data, determining a time-slot-divided take-off and landing queue sequence; based on the conflict prediction model, potential conflict thermodynamic diagram information is predicted according to radar and airport topological data; the conflict prediction model is obtained by training a space-time diagram convolutional network obtained by time sequence modeling based on combination of a diagram convolutional network and a gating cycle unit according to historical radar and airport topological data; determining a scheduling scheme by adopting a mixed integer programming model according to the take-off and landing queue sequence and the potential conflict thermodynamic diagram information; the scheduling scheme is used for scheduling flights in an airport. According to the method and the device, the conflict and collision risk between flights can be effectively reduced while the operation efficiency is improved, and scheduling optimization is realized.

The invention provides a monocular vision and sparse IMU rehabilitation action whole body posture estimation method and system, and the method comprises the steps: synchronously collecting video data and inertial data of human body rehabilitation actions through a monocular RGB camera and a plurality of IMUs, and cutting and zooming an image to a preset resolution; extracting a key point thermodynamic diagram from continuous N frames of images by using a sliding window and a residual neural network, and calculating 2D key point pixel coordinates of each frame; splicing the N frames of 2D key point pixel coordinates, the rotation matrix of the IMU and the acceleration signal into an input sequence; cross-modal time sequence modeling is carried out on an input sequence through time Transform, and after high-dimensional features are extracted, weighted average is carried out through a convolutional layer, and 3D relative key point coordinates of the last frame are output through a regression head. According to the method, by fusing monocular vision and sparse IMU cross-modal data, the problem of visual information loss caused by limb self-shielding is effectively solved, and the defect that a traditional pure vision method is insufficient in precision in rehabilitation actions is overcome.

The invention relates to the technical field of video analysis, in particular to a time sequence action positioning method and device based on bidirectional interaction and dynamic feature enhancement. The method comprises the following steps: constructing a self-adaptive feature enhancement strategy, and carrying out layered network architecture integrated dynamic modeling and local and global time sequence interactive modeling through the self-adaptive feature enhancement strategy; the adaptive feature enhancement method comprises a multi-scale dynamic time sequence modeling module and a global and local adaptive bidirectional interaction module. Introducing a self-adaptive feature enhancement strategy into the encoder; and performing feature enhancement on the initial feature through an encoder to generate an enhanced time sequence feature. The invention provides an effective self-adaptive two-way interactive dynamic time sequence enhancement framework, which uses learnable local and global affine matrixes to carry out time sequence modeling in parallel. The parallel structure supports iterative bidirectional information transmission between characteristics and adaptive balance of short-term and long-term dependencies.

The invention discloses a millimeter wave electrocardiogram reconstruction system and method based on adaptive MODWT and CNN-BiLSTM-CA, and the method comprises the steps: collecting a tiny phase displacement signal of a target thoracic cavity region, and carrying out the multi-band decomposition and key component adaptive screening of an original signal through combining with a multi-scale stationary wavelet decomposition algorithm; and introducing a channel attention mechanism to enhance key information representation, inputting a reconstruction signal into a deep learning model combining a convolutional neural network and a bidirectional long-short term memory network, completing time sequence modeling and nonlinear mapping, and outputting a reconstruction waveform highly consistent with a standard electrocardiogram. According to the method, the signal reduction capacity under the complex interference condition is remarkably improved, high-precision and privacy-friendly remote physiological signal monitoring can be achieved under the condition of not depending on a lead electrode, and the method is superior to a traditional baseline model in the aspects of waveform reduction precision, signal time sequence consistency, model generalization capacity and the like; the method is suitable for various application scenes such as intelligent medical treatment.

The invention discloses a weak supervision video anomaly detection method based on multiple prompts, and the method comprises the steps: 1) obtaining a video with a video-level label, dividing the video into non-overlapping segments, and extracting the initial visual features of the segments through a freezing CLIP image encoder; 2) constructing a Mama-based global-local dependence modeling module, fusing self-attention and a Mama state space model, capturing global and local time dependence of visual features, and outputting time enhanced visual features; 3) constructing an attribute category text prompt module, generating attribute-level and category-level text prompts through LLM, and inputting the attribute-level and category-level text prompts into a frozen CLIP text encoder to obtain text features; 4) constructing a cross-modal alignment module, and constructing a loss function to optimize network parameters; and 5) inputting a test video into the trained model, outputting an abnormal score of the fragment, and judging an abnormal event. According to the method, semantic information is supplemented through multiple text prompts, global and local time modeling and cross-modal alignment are combined, and the accuracy and robustness of weak supervision video anomaly detection are remarkably improved.

The invention discloses a time-space multi-mode video understanding method based on Video LLeMA2. The time-space multi-mode video understanding method comprises the following steps: S1, extracting an image frame sequence and an audio stream sequence of video data; s2, carrying out pretreatment; s3, inputting the image frame sequence into a visual encoder to generate a visual initial feature; s4, inputting the visual initial features into a space-time convolution connector, and generating visual modal features by using three-dimensional convolution and a RegStage module; s5, inputting the audio stream sequence into an audio encoder to obtain audio modal features; s6, aligning the audio modal features with the visual modal features; s7, performing feature fusion; and S8, obtaining a video content understanding result through a language decoder. The video content semantic understanding method is based on the Video LLeMA2 model, integrates multi-modal space-time modeling and language generation technologies, realizes video content semantic understanding, and has the advantages of natural expression and high precision.

The invention discloses a near-shore wave breaking type classification method based on video image deep learning. The method comprises the steps that video image data of a near-shore area are collected based on fixed or temporary video collection equipment, and the visual angle of the video image data covers the high position of a near-shore wave breaking zone; preprocessing the video image data, and marking overflow waves, curled waves and collapse waves according to the data; constructing a training set based on the marked data, and training a neural network model based on the training set; and based on the trained neural network model, carrying out feature extraction and time sequence modeling on the coastal wave image sequence, and predicting a wave breaking type corresponding to the coastal wave image sequence. According to the method, the time sequence video data is collected, and the deep learning model is constructed for training and reasoning, so that wave breaking type identification is automatically completed. The method is efficient and reliable, can adapt to various sea conditions, and has high practical value in practical application.