2471 results about "Frame sequence" patented technology

The invention discloses a video super-resolution reconstruction method and system, and relates to the technical field of video restoration processing, and the method comprises the steps: collecting a to-be-reconstructed video frame sequence and a reference frame sequence; extracting a dense optical flow field between the reference frame and the target frame; performing geometric alignment on the target frame to the reference frame according to the dense optical flow field to obtain a motion consistency feature; performing feature extraction on the video frame sequence through a parameterized residual scaling module; inputting the extracted features into a bidirectional propagation module to process forward and backward image sequences respectively, extracting time sequence forward features and time sequence reverse features, and fusing the features to obtain detail features; carrying out adaptive fusion on the motion consistency features and the detail features, and sending the fused features into a reconstruction network to obtain deep visual features; pixel-level adaptive reconstruction is realized according to the deep visual features, and a super-resolution video sequence is obtained; according to the method, the feature space consistency is kept, and meanwhile, the feature matching precision in a complex motion scene is remarkably improved.

The invention discloses a real-time video quality optimization and enhancement method based on deep learning. The method comprises the following steps: S1, collecting a video stream and extracting continuous frames; s2, extracting motion area features by using an optical flow estimation algorithm, and generating a multi-modal feature matrix; s3, carrying out normalization processing; s4, outputting a de-noising optimization frame by using an adaptive multi-dimensional residual network and a frequency domain analysis technology; s5, generating a detail enhancement frame by using a progressive generative adversarial network; s6, constructing an inter-frame relation graph through locality sensitive hashing, correcting the frame by using a double-flow time sensing network, optimizing the frame sequence based on a dynamic time warping method, and outputting the corrected frame sequence; s7, inputting the corrected frame sequence into a dynamic feedback optimization module to generate an optimized frame sequence; and S8, recombining the optimized frame sequence into a continuous video, and transmitting the continuous video to a target platform. According to the method, the video definition and the time consistency are comprehensively optimized through methods of multi-modal feature fusion, dynamic optimization, progressive network generation and the like.

The invention relates to an intelligent MV generation method, system and device based on AIGC and a medium, and belongs to the technical field of artificial intelligence content generation. The method comprises the following steps: receiving an original music file input by a user and a user instruction; performing audio preprocessing on the original music file to generate a standardized audio signal, and mapping a user instruction into a semantic vector; rhythm features and emotion features are extracted, music feature vectors are generated and subjected to feature alignment with semantic vectors, and an initial key frame sequence is generated based on a diffusion model; performing particle effect density dynamic adjustment and shot switching point generation on the initial key frame sequence, outputting an enhanced key frame sequence, performing optical flow frame insertion and long time sequence prediction, and generating an optimized video frame sequence; and performing block parallel rendering on the optimized video frame sequence according to the target platform parameters to generate an MV file and metadata. According to the invention, deep fusion of audio and visual contents can be realized, and high-quality video contents can be generated according to individual demands of users.

In various examples, systems and methods are disclosed relating to generation of four-dimensional (4D) content models, such as 4D content models to render realistic sequences of frames of 3D data. The systems can initialize a 3D component of the 4D content model, such as a 3D Gaussian splatting representation, based at least on a prompt for the 4D content. The system can configure motion and / or dynamics for the sequence of frames by evaluating frames rendered from the 4D content model using one or more latent diffusion models (LDMs), including a video LDM. The system can perform operations such as autoregressive generation of frames to create long sequences of content, motion amplification to facilitate realistic, dynamic motion generation, and regularization to facilitate generation of complex dynamics.

The invention discloses an automatic labeling and acquisition standardization method and system based on video content, and the method comprises the steps: decoding a video file into a processable video frame sequence, carrying out the standardization processing, unifying the format and the coding standard, and adjusting the resolution and the frame rate; performing content analysis on the standardized video frame sequence, extracting key frames, identifying objects, scenes and actions, and generating a video analysis result and a preliminary label; according to a video content analysis result, separating an audio stream from the video to carry out noise reduction processing and voice-to-text analysis, and combining the processed text with the video content analysis result to generate multi-modal comprehensive annotation information; based on the comprehensive labeling information, taking feedback of the user in video retrieval, label clicking and watching behaviors as an adjustment basis of label weights, and updating weight distribution of labels by analyzing user behavior data; labeling information and weight distribution are integrated into an index database, and indexes are updated in real time according to user behaviors and content changes.

The invention relates to the technical field of monitoring analysis, in particular to a park safety monitoring method and system based on dynamic visual identification. The method comprises the following steps: acquiring a plurality of park monitoring videos based on park monitoring equipment; performing three-dimensional illumination migration rendering on the plurality of park monitoring videos, and performing low-frame smooth completion so as to generate a time sequence monitoring image frame sequence; carrying out spatial position positioning calculation on the park monitoring equipment, carrying out topological association network reconstruction, and constructing an omnibearing monitoring perception topological map; performing dynamic object visual detection on the time sequence monitoring image frame sequence, and performing frame-by-frame optical flow tracking analysis to generate a dynamic target optical flow trajectory; and carrying out multi-time-point sampling on the dynamic target optical flow trajectory, and then carrying out mutation point identification so as to generate an optical flow trajectory mutation point. According to the invention, efficient park safety monitoring analysis and high-precision safety early warning are realized.

A method, apparatus, non-transitory computer readable medium, and system for generating synthetic videos includes obtaining an input prompt describing a video scene. The embodiments then generate a plurality of frame-wise token embeddings corresponding to a sequence of video frames, respectively, based on the input prompt. Subsequently, embodiments generate, using a video generation model, a synthesized video depicting the video scene. The synthesized includes a plurality of images corresponding to the sequence of video frames.

The invention provides a smart campus safety early warning method and system based on edge computing and big data. The method comprises the following steps: collecting multiple paths of video data streams in real time through an edge computing node, performing frame sequence segmentation and spatial-temporal feature extraction, generating an initial behavior feature set, performing multi-dimensional correlation analysis on the initial behavior feature set based on a preset behavior semantic tag, extracting a spatial-temporal behavior feature vector corresponding to a target monitoring scene, and obtaining a spatial-temporal behavior feature vector; and transmitting the time-space behavior feature vector to a central server, inputting the time-space behavior feature vector into a target behavior recognition model, generating a behavior semantic description sequence corresponding to the video data stream, determining a real-time behavior monitoring result according to a matching result of the behavior semantic description sequence and a preset abnormal behavior rule base, receiving the real-time behavior monitoring result through an edge computing node, and sending the real-time behavior monitoring result to the central server. And adaptive adjustment is carried out on acquisition parameters of the video data stream based on a dynamic priority strategy. According to the invention, the real-time performance, the accuracy and the system sustainability of campus behavior monitoring can be improved.

The invention discloses a network edge monitoring and early warning method based on video image AI analysis, and the method comprises the following steps: S1, obtaining video data, processing the video data, and generating an image frame sequence; s2, analyzing an image frame sequence, and extracting space and time features; s3, taking the target as a hypergraph node, constructing a hyperedge based on space and time features, and dynamically adjusting hyperedge connection by using a genetic algorithm; s4, constructing a multi-layer hypergraph Transform network, extracting multi-scale spatial-temporal characteristics, and calculating a semantic relationship between nodes; s5, setting a butterfly optimization algorithm initial population, and dynamically optimizing model parameters through global and local search; s6, constructing an anomaly detection model, identifying an abnormal behavior, and feeding back a result to optimize model parameters and hyperedge selection; and S7, deploying the model at an edge node, triggering early warning when an abnormal behavior is detected, and pushing information to a management platform. According to the invention, through video image AI analysis, accurate detection and real-time early warning of abnormal behaviors in a network edge scene are realized.

The invention relates to the technical field of artificial intelligence and emotion calculation, in particular to an audio and video dual-mode emotion recognition method and system based on adapter fusion. The method comprises the following steps: acquiring a video frame sequence and an audio signal, and preprocessing the video frame sequence and the audio signal; constructing an emotion recognition model; based on a bimodal feature extraction module, a space adapter and a global adapter are embedded in sequence, and corresponding modal enhanced space features and global features are obtained in sequence; generating intermediate representations of the corresponding modes based on the global features, and performing feature fusion according to the intermediate representations to obtain fusion features of the corresponding modes; the fusion features are spliced, time sequence features are extracted, and final features are obtained; inputting the final features into a classifier to obtain a predicted emotion category, training an emotion recognition model by adopting a loss function, and determining an optimal emotion recognition model; and inputting a to-be-recognized video frame sequence and an audio signal into the emotion recognition model, and outputting a recognition result.

The invention relates to the technical field of computer vision, and discloses a real-time video analysis method based on deep learning. The method comprises the following steps: acquiring a real-time video stream through image acquisition equipment, and performing frame segmentation processing to generate a continuous video frame sequence; and extracting features of the video frame sequence by using a pre-trained convolutional neural network to obtain a multi-dimensional feature vector, inputting the multi-dimensional feature vector into the time sequence analysis model to calculate dynamic relevance, and outputting an inter-frame movement track and object behavior features. And constructing a scene understanding map containing a spatial position and a time evolution relationship according to the above-mentioned data, and carrying out abnormal event detection and generating event marking data based on the map. And performing semantic analysis on the event marking data, determining an abnormal event type and a confidence score, triggering a real-time alarm signal according to a result, and updating a historical event database. In the analysis process, the resource occupancy rate of the system is continuously monitored, the calculation precision is dynamically adjusted, a degradation processing mechanism is started when a preset threshold value is exceeded, and key area analysis is preferentially guaranteed.

The invention relates to the technical field of image processing, and provides an image target tracking method and device based on deep learning, and the method comprises the steps: obtaining an original frame sequence of a target video, carrying out the adaptive frame screening of the original frame sequence, and obtaining a plurality of feature frame sequences, key frames and common frames, performing feature extraction and feature mapping on all the feature frame sequences to obtain target trajectory nodes, performing trajectory optimization calculation on the key frames and the common frames by using all the target trajectory nodes to obtain key frame trajectories and common frame trajectories, and inputting the key frame trajectories and the common frame trajectories into a preset deep learning model to perform trajectory reconstruction optimization so as to obtain a target trajectory; and obtaining a tracking target trajectory. By performing adaptive frame screening on the original frame sequence and generating the feature frame sequence, the key frame and the common frame, the data processing efficiency and the information extraction accuracy are improved, the accurate generation of the target motion track is realized, and the problems of reduced tracking accuracy and delayed system response in a complex scene are solved.

The invention discloses a video semantic segmentation method based on a time sequence cross attention mechanism, and belongs to the field of computer vision and the field of material detection.The video semantic segmentation method comprises the steps that firstly, a video used for training is preprocessed, a frame sequence is extracted, and then a coding-decoding network for multi-level feature extraction and fusion is constructed; according to the method, feature extraction is enhanced through a time sequence cross attention module, network parameters are optimized through weighted IoU loss and binary cross entropy BCE loss, then frame-by-frame prediction segmentation is carried out on a target video by using a trained model, and a multi-classification segmentation result is exported. According to the method, a time sequence cross attention mechanism is integrated into the SAMUNet network, the segmentation precision is effectively improved for image data with time sequences, the time cost and the labor cost of material video processing are greatly reduced, the method can be widely applied to the field of industrial detection, and the product quality and the production efficiency are improved.

The invention discloses a safety monitoring video optimization transmission method and a video monitoring terminal, and relates to the technical field of video monitoring, and the method comprises the following steps: extracting a current frame and a previous frame from a continuous monitoring video frame sequence, extracting two adjacent frames of images from a video stream, and providing basic data for subsequent dynamic change analysis; preprocessing the extracted two adjacent frames of images, calculating the difference between the two adjacent frames of images by using an image difference technology, and generating a difference image through pixel-level comparison; according to the method, characteristics such as motion intensity and change density are extracted by extracting preprocessing and difference calculation of two adjacent frames of images, dynamic changes are analyzed by inputting a deep learning model, and a scene is divided into rapid and normal dynamic changes. An initial compression algorithm is used for a normal scene, the bit rate is dynamically adjusted for a rapid dynamic scene, details are captured, video quality and compression efficiency are balanced, and the performance and practicability of a monitoring system are improved.

The invention provides a game scene optimization method and system based on dynamic rendering, and belongs to the technical field of game development.The game scene optimization method comprises the steps that firstly, a real-time rendering state data set of a current game scene, rendering resource occupation information containing scene elements and picture frame rendering time consumption parameters are obtained; then, a pre-trained rendering demand analysis model is called for analysis, a rendering demand feature set of the scene elements is obtained, the rendering demand feature set comprises element visual importance features and rendering resource sensitivity features, and a scene element dynamic optimization strategy is generated based on the rendering demand feature set; the scene element dynamic optimization strategy comprises an element detail level adjustment rule and a rendering resource allocation priority parameter, executing a rendering parameter adjustment operation on a target scene element according to the scene element dynamic optimization strategy, generating adjusted scene rendering configuration data, and finally inputting the scene rendering configuration data into a game rendering pipeline for real-time rendering processing. And outputting the optimized game scene picture frame sequence. Therefore, the game picture quality and the operation performance can be improved.

The invention relates to the technical field of artificial intelligence, can be applied to business scenes of medical health, financial science and technology and the like, and discloses a video stream processing method which comprises the steps of collecting current environment parameters, generating a mode switching instruction and determining a target processing mode; obtaining multi-dimensional context awareness data, and selecting a target detection model; key area coordinate parameters in the video frame sequence are extracted, and grading resolution parameters are determined; and based on the target processing mode, the target detection model and the grading resolution parameter, constructing a video processing strategy matrix, executing the video processing strategy matrix to perform coding processing on the video stream, and generating a target coding video stream. According to the invention, through intelligent mode switching based on the current environment parameters, dynamic adjustment of the video processing mode is realized, and the adaptive capacity of the system in a complex environment is improved; through target detection model selection in combination with multi-dimensional context awareness data, the detection precision is optimized, and the reliability of visual analysis is improved.

The invention discloses a micro-expression recognition method and system based on a state space model and double-flow fusion, and relates to the technical field of expression recognition, and the method comprises the steps: extracting a starting frame image and a top frame image of a micro-expression continuous video frame sequence; inputting the initial frame image and the top frame image into a micro-expression recognition model based on a double-flow fusion network, extracting an optical flow image, and performing face key point detection and clipping operation on the top frame image and the optical flow image respectively; carrying out image block embedding operation to obtain texture features of the top frame image after cutting and motion features of the optical flow image; the two features learn the global dependency relationship and the local correlation between the image blocks from the positive direction and the negative direction through a parallel multi-layer bidirectional Mama fusion module, and deep image features and motion features are extracted after feature fusion; and the deep features of the two modes are interacted through a double-flow cross attention classification module, comprehensive features are learned, and a plurality of classifiers are combined to identify and output the micro-expression type more accurately.

The invention relates to the technical field of video detection, in particular to a monitoring video detection and identification method and system based on deep learning, and the method comprises the steps: obtaining a target monitoring video of a monitoring region based on monitoring camera equipment, obtaining a video frame sequence of the target monitoring video, and carrying out the target screening of the video frame sequence, so as to obtain a video frame set; and performing target detection on each video frame in the video frame set based on a trained target detection model to obtain a target detection result set of each video frame in the video frame set. According to the method, through combination of keyword extraction and a large language model, complex monitoring data can be converted into structured event description, the textualized information not only facilitates management personnel to rapidly know the field condition, but also can provide reference for subsequent event processing, and the event description generated in real time is combined, so that the management personnel can conveniently and rapidly know the field condition. And a manager can make a coping strategy more efficiently.

The invention relates to the technical field of artificial intelligence, can be applied to business scenes such as financial science and technology and medical health, and discloses a task instruction generation method, device and equipment based on cross-modal fusion, and a medium, and the method comprises the steps: carrying out the decoding and noise reduction of an input video, generating a frame sequence, and recognizing a plurality of key frames based on the inter-frame similarity; extracting spatial features of the key frames to form a sequence, and generating video spatial-temporal features in combination with time features; performing semantic preprocessing on the input text to obtain text semantic features, and acquiring motion sensor signals to obtain motion features; fusing the video spatio-temporal features, the text semantic features and the action features to generate fused features; and generating a perception vector based on the fusion feature and outputting a task instruction. According to the method, multi-modal fusion is realized through key frame extraction and space-time fusion mechanisms in combination with text semantic features and action features, and the perception expression ability and the task instruction generation accuracy are improved by using time sequence information and multi-source perception input of the video.

The invention relates to the field of data processing, and provides a 5G-based video call information processing method and system, and the method comprises the steps: continuously obtaining a real-time video frame sequence and 5G network environment perception data in a video call scene, carrying out the multi-dimensional state mapping processing of the 5G network environment perception data, constructing a network transmission adaption model, and carrying out the real-time video frame sequence and 5G network environment perception data. Generating a video coding control instruction based on the network transmission adaptation model, performing content-aware coding conversion on the real-time video frame sequence, and outputting a coding optimization stream; in the transmission process of the coding optimization stream, link state fluctuation information is obtained through a 5G network feedback channel, transmission strategy dynamic calibration is performed on the coding optimization stream according to the link state fluctuation information, and a calibration transmission stream is obtained; and carrying out decoding time sequence alignment processing on the calibration transport stream, generating a visual call output sequence which is synchronous with the time of the original video stream unit, and pushing the visual call output sequence to a receiving end presentation device.

The invention discloses a non-contact heart rate detection method, system and device based on visual Transform and multi-scale feature aggregation and a medium. The method comprises the steps that a face visible light video is obtained, the unified video frame rate is re-sampled through the frame rate, face key point positioning and region division are carried out on each frame of image, and a face visible light video sequence is obtained; performing time migration operation on the video sequence to obtain a difference frame sequence, performing channel fusion on the video sequence and a difference frame, and performing down-sampling on a spatial dimension to obtain a low-resolution feature tensor; constructing a non-contact heart rate extraction model, inputting the low-resolution feature tensor into the non-contact heart rate extraction model, and outputting a heart rate value of the target; the non-contact heart rate extraction model comprises a feature enhancement module, a multi-scale mask feature aggregation module, a Transform time sequence modeling module and an rPPG predictor, the system, the device and the medium are used for achieving the non-contact heart rate detection method based on visual Transform and multi-scale feature aggregation, and the precision of rPPG signal extraction is improved.

The invention provides a multi-mode depth forgery detection method and device, and the method comprises the steps: obtaining a video frame sequence and an audio sequence of a target source video, carrying out the feature extraction of the video frame sequence based on a preset visual feature extraction model, and obtaining a visual feature, performing feature extraction on the audio sequence based on a pre-trained audio spectrogram coding model to obtain audio features, fusing the visual features and the audio features to obtain corresponding fusion features, obtaining a preset target multi-modal deep counterfeiting detection model, performing counterfeiting identification on the fusion features based on the target multi-modal deep counterfeiting detection model, and obtaining a target multi-modal deep counterfeiting detection model. And determining an identification result of the target source video.

The invention discloses a video monitoring abnormal behavior identification method and system based on edge AI, and relates to the technical field of intelligent video analysis, and the method comprises the steps: carrying out the frame segmentation processing of a video monitoring data stream, obtaining a video frame sequence, building a target trajectory prediction model, predicting the position region of a current frame, and generating target position prediction data; performing multi-scale feature extraction on the video frame sequence, performing fusion matching on a target feature vector and target position prediction data, and constructing an enhanced feature matrix; carrying out weight distribution on the key behavior characteristics by adopting an attention mechanism, setting a dynamic threshold adjustment mechanism, and dynamically adjusting an abnormal behavior judgment threshold according to the personnel density; and inputting the adjusted feature data into an abnormal behavior classifier for identification and judgment, outputting an abnormal behavior identification result, and generating an abnormal event report. According to the method, the abnormal behavior detection accuracy in a complex monitoring scene is improved, the false report and missing report rate is reduced, and the millisecond-level real-time response capability is realized.

The invention relates to the technical field of artificial intelligence and the field of medical health, and discloses a video generation method based on multi-modal information fusion, which comprises the following steps: inputting image information and text information, and generating image feature representation and text feature representation; injecting noise into the image feature representation to generate a noisy potential representation; inputting the noisy potential representation into a denoising model to generate an optimized potential representation; inputting the text feature representation into a denoising model, and carrying out semantic fusion on the text feature representation and the optimized potential representation to generate a semantic potential representation; and inputting the semantic potential representation into a decoder, generating a video frame sequence and outputting the video frame sequence. According to the invention, through the multi-modal fusion of the image feature representation and the text feature representation, the generated video is ensured to be consistent with the input content semantically; through noise injection and denoising optimization, the diversity and detail quality of video generation are enhanced; and a video frame sequence is generated through the decoder, so that the visual coherence and natural transition of the video are ensured.

The invention relates to a television box intelligent interaction control unit image processing method and device. The method comprises the following steps: carrying out feature analysis on an input video stream of the television box to obtain an enhanced video stream; the method comprises the following steps: collecting visual and voice interaction information of a user based on an intelligent interaction control unit, and inputting the visual and voice interaction information into a double-branch Transform model for multi-modal information fusion to obtain a fusion feature vector; performing conjoint analysis to obtain a user interaction intention and an interaction response strategy; performing local contrast enhancement and motion compensation to obtain a high-quality video frame sequence; and performing scene adaptive color space mapping and dynamic color temperature adjustment on the high-quality video frame sequence according to the environment light information acquired in real time and the pre-stored color characteristics of the display device to obtain an output video stream. According to the implementation of the method, limited computing resources can be fully utilized, efficient video processing and natural man-machine interaction are achieved, and meanwhile the display effect is dynamically optimized according to environmental changes.

The invention provides a video stream analysis method and system based on unmanned aerial vehicle routing inspection, and relates to the technical field of unmanned aerial vehicles. Firstly, original video stream data of a target area is collected by using a camera device carried by an unmanned aerial vehicle, and then feature extraction is performed on the original video stream data; the method comprises the steps of obtaining a space correlation feature set and a time dynamic feature set of a video frame sequence, then carrying out anomaly detection on the two feature sets based on a preset anomaly detection model, generating an anomaly distribution feature set of a target area, generating a dynamic path optimization instruction according to the anomaly distribution feature set and real-time flight parameters of an unmanned aerial vehicle, and optimizing the dynamic path according to the dynamic path optimization instruction. The method is used for adjusting the inspection path of the unmanned aerial vehicle so that the unmanned aerial vehicle can focus on covering the abnormal area, and calling the incremental learning module to update the parameters of the anomaly detection model based on the anomaly distribution feature set so as to ensure that the anomaly detection model can adapt to the change of the target area and improve the anomaly detection accuracy. Therefore, more efficient, comprehensive and accurate inspection of the target area is realized.

The invention discloses a video image color correction method and system based on artificial intelligence, and relates to the technical field of color correction, and the method comprises the steps: collecting video data, carrying out the image size normalization and denoising processing, and obtaining the preprocessed video frame sequence data; and extracting reflectivity data and illumination data by using a Retinex-Net deep learning model, calculating an illumination adjustment coefficient, carrying out pixel correction, and carrying out color temperature correction on the pixel data of the video frame subjected to pixel correction according to the illumination adjustment coefficient. According to the method, the preliminary color mapping matrix is generated through the color histogram matching method, the color distribution of the image can be optimized, the image is enabled to better conform to the color features of the target reference frame, the color temperature adjustment is performed on the image according to the illumination adjustment coefficient and the illumination threshold, the cold and warm tones of the video are enabled to be accurately adjusted, and the image quality is improved. Through successively carrying out pixel correction, color correction and local area correction, a stable color adjustment frame is provided, so that the video styles are consistent.

The invention provides a network camera monitoring identification method and system based on artificial intelligence, and the method comprises the steps: firstly obtaining a monitoring data stream which is outputted by a network camera and comprises a video frame sequence and a corresponding time sequence metadata sequence, and then carrying out the spatial-temporal context coding processing of the monitoring data stream; the method comprises the following steps: generating a context feature cube containing spatial position information and time evolution information, then executing a normal behavior mode learning operation based on the context feature cube, and generating a reference feature library containing typical scene feature templates and feature evolution rule description; and dynamically matching and comparing the context feature cube of the current time period with the reference feature library, calculating a feature matching deviation value and generating an abnormal confidence score, and finally generating a monitoring early warning instruction containing abnormal occurrence time, a space coordinate range and a confidence level identifier according to the abnormal confidence score and corresponding space-time position information. And the accuracy and the early warning effect of monitoring and identification of the network camera are effectively improved.

The invention relates to a knowledge-intensive visual question and answer automatic data generation method and device, and the method comprises the steps: constructing an original visual data set containing the professional knowledge of a target domain according to a static image, a video stream and multimedia content; extracting a representative frame sequence, converting the audio information into text information, and extracting character information in the static image to construct a structured visual instance database; according to the prompt text meeting the preset professional depth condition, establishing a three-level prompt system containing domain knowledge, an evaluation standard and a generation specification; generating a corresponding visual question and answer pair data set according to the dynamic cooperation of the main agent and the domain expert agent; generating a multi-agent quality evaluation system according to the quality evaluation result; and designing a difficulty grading mechanism according to the negative example sample. According to the method, the professionality, the accuracy and the diversity of the visual question and answer data are remarkably improved, and reliable data support is provided for training and evaluation of a multi-modal large model.

Systems and methods for summarizing an infrastructure surface inspection video, which can involve processing frames of the infrastructure surface inspection video through a spatial in-frame process configured to conduct feature and text extraction on the frames on detected observations, the extracted features and text used to generate spatial metadata; processing the frames of the infrastructure surface inspection video through a temporal cross-frame process configured to detect changes across sequences of the frames including the spatial metadata, the temporal cross-frame process configured to generate temporal metadata encapsulating the detected changes; and processing the frames of the infrastructure surface inspection video through an activity decision process configured to intake the spatial metadata and temporal metadata to detect and track structural features or defects of interest as activity in the infrastructure surface inspection video and generate summary metadata from the spatial metadata, temporal metadata, and detected segments of interest.