232 results about "Frame based" patented technology

The invention provides an unsupervised video monitoring anomaly detection method and system based on semantic modeling, and belongs to the technical field of computer vision, artificial intelligence and video monitoring. Comprising the following steps: carrying out key frame identification on a monitoring video by adopting an image-text joint embedding model, and carrying out target cross-frame tracking based on a depth target detection algorithm to extract behavior semantic information so as to construct a semantic behavior map; sending the key frame sequence of the graph structure information into a frame prediction model, and predicting a next frame image or a target state; and carrying out abnormal scoring on the obtained prediction result, and carrying out threshold judgment by outputting a comprehensive abnormal score value so as to determine whether the current frame is an abnormal event or not. According to the method, the intelligent level and the overall efficiency of video anomaly detection can be effectively improved on the premise of ensuring the real-time performance and the stability, and support is provided for video monitoring anomaly detection in actual scenes such as smart cities, rail transit, industrial parks and commercial security.

The invention discloses a digital production plan scheduling method and system, and belongs to the technical field of optimal scheduling, and the method comprises the steps: constructing a distributed storage architecture based on edge computing nodes; a central coordinator is adopted to realize cross-node data synchronization through an improved Raft consensus algorithm, multi-version concurrency control is realized based on a vector clock, and a global consistent data view is established; a visual scheduling platform is built based on a Vue3 framework, and man-machine interaction is realized by adopting a Canvas and WebGL collaborative rendering framework; establishing a dynamic coordinate conversion model based on bilinear interpolation, designing a space mapping function containing distortion compensation, establishing a multi-thread coordinate service based on WebWorker, and realizing submillimeter-level bidirectional mapping of pixel coordinates and physical coordinates; constructing a three-dimensional space-time analysis model fused with the multi-dimensional features; and all the units are subjected to feature fusion through residual connection, and finally a scheduling scheme with a confidence coefficient weight is output. The method and the device have the effect of meeting various scheduling requirements.

The invention belongs to the technical field of artificial intelligence and data science, and discloses a causal discovery method and related device based on a large model and structural constraint, comprising the following steps: according to meta-information of each causal variable, adopting a large language model and combining multiple rounds of dialogues, multi-role play and auxiliary cue word modes, generating an initial potential causal graph of each causal variable; repeating the following steps until a preset condition is met, and outputting a candidate causal graph with the highest value score: generating the candidate causal graph by adopting a policy network based on an encoder-decoder architecture, and carrying out iterative optimization on the candidate causal graph by adopting a reinforcement learning algorithm based on an act-critic framework; and calculating a value score of the candidate causal graph after iterative optimization according to the value network, and updating parameters of the policy network based on the encoder-decoder architecture according to the value score. The efficiency and accuracy of causal relationship recognition are remarkably improved, and a causal relationship graph with high confidence can be generated in a complex data environment.

A device for processing frame data may be configured to identify one or more semantic characteristics for the frame data, wherein the frame data comprises a plurality of frames, each frame of the plurality of frames being acquired for a same scene and by a different frame source of a plurality of frame sources; extract features from each respective frame of the plurality of frames; determine a non-uniform sampling pattern of the plurality of frames based on the one or more semantic characteristics; and project, using the non-uniform sampling pattern, a portion of the extracted features into a bird's-eye-view (BEV) space having a grid structure to generate a fused set of BEV features.

The invention provides a high-altitude operation risk early warning method and system based on camera image recognition, and relates to the technical field of computer vision, and the method comprises the steps: firstly collecting a video image sequence of a high-altitude operation scene, and generating a fusion feature map containing environment and operation main body features through multi-level feature extraction; performing spatial dimension segmentation and regional feature comparative analysis on the fusion feature map to obtain a spatial risk distribution map containing risk region identification information, processing the spatial risk distribution map of continuous frames based on a time sequence feature fusion rule to generate a dynamic risk evolution map, and calling a risk decision model to perform mode recognition to obtain a dynamic risk evolution map; and generating a risk level classification result and a risk position coordinate set according to the risk level classification result and the risk position coordinate set, and finally generating a risk early warning signal and sending the risk early warning signal to the monitoring terminal, thereby comprehensively, accurately and dynamically monitoring the high-altitude operation risk, and improving the accuracy and timeliness of risk early warning.

The invention discloses a multi-mode large model video content understanding reasoning acceleration method and system, and mainly relates to the technical field of artificial intelligence reasoning acceleration. Comprising the following steps: inputting video data and preprocessing the video data to generate a video frame sequence; performing adaptive video Token compression on the generated video frame sequence, and outputting a compressed visual Token set; performing visual feature coding and Key-Value generation on the compressed visual Token set to obtain visual KV data; performing video KV cache partition management on the visual KV data; executing cross-modal reasoning based on the vLLM framework to generate a video content understanding result; and outputting a video content understanding result, and carrying out post-processing and structured mapping. The method has the beneficial effects that the obvious reasoning acceleration and throughput improvement can be realized on the premise of keeping the precision of the original large model.

The invention relates to the technical field of integrated learning deformation prediction, in particular to a double-weight stacking deformation prediction integrated model modeling method based on complex sample guidance, and the method comprises the steps: building five heterogeneous base models: CLAnet, RBF, MLP, CNN and XGBoost; establishing a support vector regression machine as a meta-model; constructing a secondary integration model on the basis of a stacking framework; a complex sample-oriented five-fold cross validation strategy is adopted to optimize training set distribution, and learning of complex samples is dynamically enhanced; by calculating an error index of each base learner, an initial weight is manually allocated to the base model before the meta-model automatically and implicitly allocates the weight, and metadata set distribution is optimized; and a whale optimization algorithm is introduced to adjust hyper-parameters of each model. According to the integrated prediction model, the problems that a single model is insufficient in generalization ability to complex samples and limited in modeling ability of a multivariable coupling relation can be solved by integrating different learning modes of each base model to features and by means of the quadratic fitting ability of the meta-model.

The invention relates to the technical field of building risk monitoring, in particular to an InSAR building scatterer positioning method, device and equipment, a storage medium and a computer program product. According to the method, urban point cloud data is segmented to obtain an urban building monomer instance point cloud set; based on the point cloud set, adopting a surface reconstruction algorithm to construct a building grid model; based on the grid model and the SAR satellite imaging parameters, adopting an SAR simulation technology to identify a radar illuminable vector frame of the building; converting the vector frame from a geographic coordinate system into an SAR image coordinate system, performing bilinear interpolation based on the geographic coordinates of the angular points of the vector frame, and determining the geographic coordinates of the scatterer in the vector frame; and if the scatterer is located in the overlapping region of the plurality of vector frames, performing overlay identification on the pixel points in the vector frames based on the geographic coordinates, determining the unique geographic coordinates, and determining the geographic positioning of the building scatterer in combination with the deformation information of the scatterer, thereby improving the positioning precision of the building scatterer.

An approach is provided for computer-vision-based object motion detection. The approach involves receiving a video sequence captured from a perspective of a vehicle / device traveling at street level. The approach also involves processing frames of the videos sequence using computer vision to determine semantic localization feature(s) associated with object(s) detected in the frames. The approach further involves clustering the object(s) into object cluster(s) across the frames based on the semantic localization feature(s). The approach further involves creating an object vector connecting the object(s) in a given cluster of the object cluster(s) across the frames. The approach further involves computing an angle between the object vector and a reference vector associated with a movement of the vehicle / device across the frames. The approach further involves classifying a motion of the object(s) relative to the vehicle / device based on the angle.

The invention discloses a robust online vector map construction method for space-time simplified query, and relates to the technical field of automatic driving perception, and the method comprises the steps: fusing multi-view features into a unified bird's-eye view for representation; matching instance features of the current frame and the historical frame based on time association; updating query features and optimizing query representations; a high-quality vector map construction model is used as a teacher model, a map construction model focused on taking an instance as a center is used as a student model, and features of a teacher network are adaptively distilled to a student network; and designing a loss function to evaluate the student model, performing training optimization, and if the output requirement is not met, further optimizing the student model through a middle distance perception mechanism. According to the method, the robustness, the precision and the efficiency of online vector map construction are remarkably improved, especially the performance in a dynamic environment and a complex scene, and by means of the mode, multi-view, space-time correlation and complex 3D target detection tasks can be better processed.

Various examples, systems, and methods are disclosed relating to frame selection via activity-based ranking and optimization. A first computing system can receive a plurality of frames and metadata from a capture device capturing a video stream. The first computing system can generate, using a ranking model, a plurality of rankings for the plurality of frames based on a plurality of video parameters of the plurality of frames and the metadata, wherein the plurality of rankings correspond to a summarization of the video stream. The first computing system can determine at least one of the plurality of frames to provide to at least one buffer based on the plurality of rankings, wherein the at least one buffer stores a subset of frames of the plurality of frames. The first computing system can provide, from the at least one buffer, the subset of frames as input to a machine-learning model.

The embodiment of the invention provides a training method and device of a visual text pre-training model, equipment and a medium, and belongs to the technical field of artificial intelligence. The method comprises the steps of inputting an obtained sample video and an obtained sample text into an initial multi-modal processing model, performing segmentation processing on the sample video to obtain a plurality of initial video frames, and extracting text features from the sample text; performing space-time importance evaluation on each initial pixel block in each initial video frame to obtain space-time information; determining a masked video frame based on the spatio-temporal information, and performing feature reconstruction processing on a part in a masked state in the masked video frame based on the text features to obtain a sample reconstruction result; and calculating a loss value according to a sample reconstruction result, and adjusting model parameters of the initial multi-modal processing model according to the loss value to obtain a trained target multi-modal processing model. According to the invention, the ability of the multi-modal processing model obtained by training to understand the multi-modal information can be improved.

The invention discloses an automatic verification excitation generation method based on a backtracking thinking tree, and the method comprises the steps: constructing a backtracking thinking tree frame based on a self-feedback mechanism based on a thinking tree for a to-be-verified processor function; under the backtracking thinking tree framework, decoupling a to-be-verified function from top to bottom through a large language model, and recursively decomposing the whole to-be-verified function into function points capable of being independently verified layer by layer through a tree structure; according to each function point, determining a normal function and a boundary of each function point, and for each function point, generating verification excitation layer by layer in a tree structure; after the verification excitation is generated, self-verification and self-backtracking are carried out layer by layer through a tree structure, and finally, the generated verification excitation is executed on the processor design to verify the correctness of the processor function. According to the method and the system, the function verification efficiency and the verification coverage rate of the processor are remarkably improved.

The invention relates to a cross-view online motion detection method based on probability time sequence mask attention, and the method specifically comprises the steps: S1, obtaining training data, and obtaining video streams of a plurality of views in a target scene; s2, data preprocessing: extracting RGB and optical flow features from a video stream based on a pre-training model; s3, a double-branch network structure is constructed, probability branches are used for compressing the view-angle-level features into potential view-angle specific codes, and classification branches are used for conducting autoregressive classification through the codes; s4, a GRU-TMA module is constructed, and interference of remote historical information on current action detection is reduced; s5, performing model training and optimization, optimizing potential representation of probability branches by minimizing reconstruction loss and KL divergence loss, and optimizing action classification capability of classification branches through cross entropy loss; and S6, performing online action detection, performing action classification of sequence frames based on output of the double-branch network, and performing performance evaluation according to actual action labels. According to the invention, the accuracy and robustness of action detection are improved.

The invention provides a big language model user query rewriting method and system based on historical context, and belongs to the technical field of artificial intelligence. The method comprises the following steps: firstly, collecting a data set containing user query and historical dialogues, redesigning a new instruction, taking a DeepSeek model as a query revisor, and further optimizing the preliminarily rewritten query to obtain a final query version; the method comprises the following steps: calling a DeepSeek model through an API (Application Program Interface), rewriting a query by utilizing the DeepSeek model under the guidance assistance of a designed instruction, optimizing an original query, generating a plurality of improved candidate queries, storing the improved candidate queries in a query pool, and then scoring all queries in the query pool by adopting a candidate query-context alignment scoring mechanism, and selecting the candidate query with the highest score as a preliminary optimization result. According to the method, the query rewriting frame based on the open source large model is constructed, so that the rewriting quality of user query is effectively improved, and the performance of information retrieval is improved.

The invention provides a real-time translation method and interaction system based on streaming voice segmentation and semantic verification. The method comprises the following steps: receiving a conference audio stream, identifying effective audio frames based on dual-channel voice activity detection, and pressing the effective audio frames into a dynamic buffer area; according to a preset dynamic segmentation strategy, outputting an initial voice segment from the dynamic buffer area for voice recognition, and obtaining a corresponding initial text segment; performing multi-stage reliability verification on the initial text fragment, and dynamically correcting or complementing the initial text fragment according to a verification result to obtain a reliable recognition result; and after the reliable identification result is obtained, triggering an asynchronous parallel translation task. According to the method, a semantic correction mechanism cooperating with the adaptive truncation depth is designed, so that the semantic fragmentation problem of the long-sequence audio during streaming truncation is solved, and the translation accuracy of a complex word order language is greatly improved while low delay is ensured.

A video decoding method employing a UNet neural network and a Transform neural network, comprising: parsing a first syntax element from a bitstream, the first syntax element indicating whether to enable neural network inter-frame prediction; in response to the first syntax element indicating that neural network inter prediction is enabled, a second syntax element is parsed from the bitstream, the second syntax element indicating N frames after an intra-coded frame (Intra frame), where: inter prediction is performed on the N frames using the UNet DM (Diffusion Model), performing inter-frame prediction on a frame after the N frames by using a combination of UNet DM (Data Management) and Transform DM (Data Management); and in response to the first syntax element indicating that neural network inter prediction is enabled, parsing a third syntax element from the bitstream, the third syntax element indicating one of: (a) inter prediction is performed on frames following the N frames using UNet DM and Transform DM, respectively, to generate two frames, respectively, and selecting an optimal prediction frame based on a quality comparison between the two frames; or (b) respectively carrying out inter-frame prediction on the frames after the N frames by using UNet DM and Transform DM so as to respectively generate two frames, and carrying out frame fusion on the two frames so as to obtain a final prediction frame.

The present disclosure relates to methods, apparatus, systems, and non-transitory computer-readable storage media for training and using a multi-scale machine learning model for the enhancement of compressed video. According to some examples, a computer-implemented method includes receiving a video at a content delivery service; performing an encode on a frame of the video by the content delivery service that converts the frame from a pixel domain to a transform domain and back to the pixel domain to generate first pixel values and a first residual for a block of the frame at a first resolution; generating a first set of features, by a machine learning model of the content delivery service, for an input, at a first resolution, of the first pixel values and the first residual of the block; generating a second set of features, by the machine learning model of the content delivery service, for an input, at a second lower resolution, of second pixel values and a second residual of the block; upsampling the second set of features to the first resolution to generate an upsampled second set of features; generating a modified version of the frame based on the first set of features and the upsampled second set of features; and transmitting the modified version of the frame to a frame buffer or from the content delivery service to a viewer device.

Systems and methods are provided for encoding visual content using a hybrid framework based on convolutional and neural radiance networks. An encoder accesses video data having a sequence of frames. The encoder generates a first frame based on averaging pixel attributes of the sequence of frames. The encoder determines a sequence level representation based on the first frame. The encoder trains a neural network model based on the sequence of frames to determine a cross-resolution representation corresponding to the sequence of frames. Training the neural network model comprises generating a plurality of model parameters for reconstructing the sequence of frames based on the sequence level representation and the cross-resolution representation. The plurality of model parameters comprises neural radiance network parameters. The encoder transmits bitstreams of the plurality of model parameters, the sequence level representation, and the cross-resolution representation.

An example operation may include one or more of storing a safety specification for an industrial equipment and a video of an operation that is performed with the industrial equipment, identifying a plurality of video frames within the video that are associated with the operation that is performed with the industrial equipment, generating a description of the plurality of video frames based on execution of a multi-modal artificial intelligence (AI) model on the plurality of video frames, determining a safety issue with respect to the operation that is performed based on execution of a language machine learning model on the description of the plurality of video frames and text content from the safety specification, and displaying an identifier of the safety issue on a display screen associated with the industrial equipment.

Systems, methods and media for vision-based prediction of joint actions and gesture motion. A computer-implemented method includes: extracting frames of a video sequence; estimating the current pose of an object in each frame and determining the joint positions of joints associated with the object in each frame; calculating optical flow between each pair of consecutive video frames; extracting motion features from each video frame of the video sequence based on the optical flow; encoding state information based on the current pose and the motion features; determining a current action label for each motion feature in the state information for a first frame; predicting a future action label for each motion feature in a second frame after the first frame based on the current pose, the current action label and the state information; predicting a future pose for each motion feature in the second frame based on the current pose and the state information; and refining the current action label, the future action label and the future pose.

The invention discloses a pipeline fault intelligent detection method based on multi-model fusion, and the method is characterized in that the method comprises the following steps: determining defect feature types based on obtained pipeline internal images, collecting the pipeline internal images with the same defect feature types, and constructing a plurality of types of training data sets; inputting each type of training data set into a corresponding type of detection model for training to obtain a plurality of expert detection models; pipeline internal images obtained in real time are input into the expert detection models for prediction, prediction results of all the expert detection models are spliced in the column direction and aligned line by line, a new tensor is formed, and a pipeline detection preliminary result is obtained; and removing the redundant prediction frame based on a non-maximum suppression algorithm to obtain a pipeline detection result. According to the method, a multi-model strategy is adopted, the low prediction precision of a single model can be made up, the error detection rate is reduced, and the stability and reliability of the whole prediction model are enhanced.

Disclosed is interactive labeling of a 4D dynamic object based on time series data, which aims at time series-related point cloud dynamic object data. Multi-frame local point clouds in the same time series are transformed into the same global coordinate system with corresponding poses to obtain global point clouds in the same time series are obtained, which clearly shows the moving trajectory of the dynamic object. Taggers can label key 3D boxes based on the moving trajectory of the dynamic object, and automatically generate 3D prediction boxes of other frames based on these key 3D boxes, which significantly reduces the number of frames that need manual operation and solves the problem that 3D prediction boxes generated based on deep learning model are inaccurate and efficiency can hardly be improved.

The invention provides a quadrilateral surface re-topology method based on frame field prediction and electronic equipment. The quadrilateral surface re-topology method based on frame field prediction comprises the following steps: inputting a triangular mesh model into a frame field neural network model, and carrying out direction regression and amplitude diffusion taking the direction as the condition to obtain a corresponding frame field; performing deformation based on the frame field to obtain a deformed grid model; performing isotropic quadrilateral re-mesh division on the deformed mesh model to obtain an initial quadrilateral mesh model; and performing inverse deformation on the initial quadrilateral mesh model to obtain a quadrilateral mesh model corresponding to the triangular mesh model. Through the steps of frame field inference, deformation, isotropic quadrilateral grid re-division and inverse deformation, the quality of the finally generated quadrilateral grid is ensured, so that the purpose of improving the quality of the quadrilateral grid is achieved.

Techniques and systems are provided for shape estimation. For instance, a process can include determining a first adjacency overlap (AdO) score of a first detected object in a first frame based on overlap between the first detected object and a second detected object in the first frame; and extracting features of the first detected object based on a comparison of the AdO score and a threshold AdO score.

The invention relates to a two-dimensional code image recognition method based on boundary completion and uncertainty modeling, and belongs to the field of computer vision. The method comprises the following steps: firstly, acquiring an input two-dimensional code image, detecting a cutting area through gray level conversion and edge gradient analysis, complementing the cutting area by utilizing a generative network, and generating enhanced features by adopting deformable convolution; and then extracting rotation irrelevant features through rotation isovariant convolution, fusing spatial information by applying a global attention mechanism, and correcting the angle of a detection frame based on a positioning mark. Secondly, extracting semantic features of candidate areas, calculating similarity with a standard template to generate semantic scores, detecting positioning mark angle distribution to generate geometric constraint scores, and adjusting candidate box confidence; and finally, establishing a confidence coefficient normal distribution model through the multi-layer perception mechanism, calculating a high confidence interval probability, constructing combined loss including classification, positioning, confidence coefficient and uncertainty loss, optimizing the model according to specified parameters, and storing the optimal model to obtain a two-dimensional code recognition result.

Systems and techniques are provided for vision perception processing. An example method can include determining an attention demand score or characteristic per region of a frame from a sequence of frames; generating attention votes per region of the frame based on the attention demand score or characteristic per region, the attention votes per region providing attention demands and / or attention requests; determining an attention score or characteristic per region of the frame based on a number of attention votes from one or more computer vision functions; based on the attention score or characteristic per region of the frame, selecting one or more regions of the frame for processing using a neural network; and detecting or tracking one or more objects in the one or more regions of the frame based on processing of the one or more regions using the neural network.

The invention discloses a software full-life-cycle intelligent collaborative optimization method and system based on a large model, belongs to the technical field of artificial intelligence and DevOps crossing, and aims to solve the technical problem of how to eliminate LLM model performance degradation caused by development, test and production environment data splitting. According to the technical scheme, the method comprises the steps that a large language model is called through a gRPC interface to generate a program code, and a requirement specification with a version mark is obtained; calling a large language model to generate a boundary value test case based on an extension plug-in of a JUnit framework, and obtaining a defect report with a severity label; an anomaly detector integrated with a Prometheus monitoring tool calls a large language model to predict a container fault, and obtains a resource utilization rate time sequence report; and constructing a structured data lake: storing development, test and operation and maintenance data by using Elasticsearch, and establishing space-time association among developers, test cases and container instances through a Neo4j graph database.

The invention discloses a video coding method and device. The video coding method comprises the following steps: performing semantic segmentation processing on a current frame in a video to obtain a semantic tag graph of the current frame; determining prediction information of a coding unit in the current frame based on the semantic tag graph; and coding the current frame based on the prediction information. The video coding efficiency and the compression performance can be improved.

The invention provides a driver behavior recognition method and device, equipment and a storage medium, and belongs to the technical field of computer image recognition. The driver behavior recognition method comprises the following steps: acquiring a video frame sequence and a heat map frame sequence containing driver behaviors; generating a first feature map and a second feature map based on the video frame sequence and the heat map frame sequence by using a multi-modal interactive neural network and time weight adaptive convolution; the time weight adaptive convolution is used for endowing each frame with a corresponding time weight based on the position of each frame in the respective time sequence; the first feature map is a skeleton-guided appearance feature map based on time weighting, and the second feature map is a skeleton-guided appearance feature map based on time weighting; and performing driver behavior recognition based on the first feature map and the second feature map to obtain a recognition result. According to the invention, the multi-modal interaction and time weight adaptive mechanism is introduced, and the method focuses on the fine-grained behavior of the driver, so that the accuracy of the recognition result is effectively improved.