2824 results about "Image frame" patented technology

The present disclosure relates to the field of computer vision and discloses a three-dimensional dynamic scene reconstruction method and apparatus, and a storage medium. The three-dimensional dynamic scene reconstruction method comprises: acquiring synchronized videos of a plurality of viewpoints of a dynamic scene; computing matching points between video images of different viewpoints, and estimating intrinsic and extrinsic parameters of each camera; obtaining a Gaussian splatting point set {p0} on the basis of a sparse point cloud constructed according to the depth of each matching point; for the first image frame of each video, using {p0} to perform static training thereon, to obtain a Gaussian splatting point set {p}; for the remaining image frames, dividing {p} into a static point set {S} and a dynamic point set {D}, performing dynamic training on {D}, and constructing a dynamic Gaussian splatting point set {P} from {p}, {S}, and the final {D}; and, in view of the intrinsic and extrinsic parameters of each camera, rendering {P} using a Gaussian splatting rendering pipeline, to obtain rendered images at different moments from new viewpoints.

Disclosed in the present invention are an edge-deployed semi-supervised anomaly detection method and system for a railway track foreign object. The method comprises the following steps: an edge device encoding and decoding a video stream captured by a camera to obtain an image frame sequence, and performing frame extraction; and using a semantic segmentation model to perform image segmentation on a certain image frame obtained by means of frame extraction, to obtain a railway track region segmentation image. The use of a single image as input may generate an expert model result having a high weight value; however, the determination based on a single image is not stable, multiple consecutive images of the task scene need to be inputted, the frequency of each expert model obtaining the highest weight is computed, and the expert model corresponding to the highest frequency is the final solution. The present invention supports scene-adaptive foreign object detection algorithm automatic selection, and a user can perform selection on the basis of prior knowledge, or selection may be performed by a scene-adaptive automatic algorithm selection method; the user only needs to provide a batch of image data of the current scene, and the optimal algorithm selection can be evaluated.

The present invention discloses an artificial intelligence-based (AI-based) system and method for generating optimised operation planning and scheduling output. The AI-based system obtains at least one of: one or more data explanation videos, one or more process understanding videos, and unconstrained operational planning data, along with one or more prompts as an input. The AI-based system extracts one or more informative image frames and audio data, to train the one or more AI models and generate a planning standard operating procedure (SOP). The AI-based system processes the planning SOP, the constrained operational planning data, and the one or more prompts to generate the optimised operation planning and scheduling output based on an optimised function with a continuous feedback loop in response to at least one of: the one or more prompts, updated planning SOP, and real-time changes in the constrained operational planning data.

The invention provides a multi-mode remote sensing target tracking and positioning and intention discrimination method and device. The method comprises the following steps: acquiring a plurality of visible light image frames and a plurality of infrared light image frames, and carrying out frame alignment operation on each visible light image frame and each infrared light image frame to obtain a plurality of groups of effective image frame pairs; for each group of effective image frame pairs, determining tracking identification information of each detection object in the effective image frame pairs based on the effective image frame pairs and a pre-trained multi-modal detection tracking model; for each detection object, determining longitude and latitude tracks of the detection object based on the tracking identification information and a back projection mapping function; and determining the behavior intention of each detection object based on a behavior recognition model and the longitude and latitude tracks of each detection object. The accuracy of target tracking and behavior intention recognition in the remote sensing video can be improved.

The invention discloses a power grid intelligent inspection method and system based on an unmanned aerial vehicle, and the method comprises the following steps: collecting image flow data of a target region, constructing a three-dimensional point cloud model, and generating an initial inspection path based on a fast marching tree method in combination with spatial position information; controlling the unmanned aerial vehicle to fly according to a path and collect image frames in real time, and executing an optical flow estimation algorithm through an edge calculation chip to obtain a pixel motion vector; associating the motion vector with a space coordinate corresponding to each inspection point in the inspection path, dividing an optical flow detection area and distributing an initial detection weight; recognizing a dynamic abnormal area according to the motion features, extracting image data and space coordinates, and driving an acousto-optic load assembly carried by the unmanned aerial vehicle to respond; and based on the space coordinates of the abnormal region, re-executing the fast marching tree method to generate a local update path, and adjusting the detection weight of the related region. According to the invention, dynamic sensing and path updating linkage in the unmanned aerial vehicle inspection process can be realized.

Embodiments of this application provide a frame rate switching method and apparatus, applied to the field of terminal technologies. The method includes: drawing and rendering, by an application thread, a first image frame at a frame interval corresponding to a first frame rate in a first period; drawing and rendering, by the application thread, a second image frame at a frame interval corresponding to a second frame rate in a second period, where the second period is preceded by the first period, and the second frame rate is different from the first frame rate.

The invention belongs to the technical field of railway engineering safety monitoring and computer vision, and discloses a railway scene target detection and behavior recognition method based on space-time double-flow features, and the method comprises the steps: obtaining and preprocessing video data into an image frame sequence; a sequence is input to an improved target detection network (Mamba-Yov11), the network integrates local spatial features and global time sequence context information by setting a convolutional neural network path and a state space model path in parallel, and adopts an improved C3k2UIB module to realize dynamic path selection and improve parameter efficiency; the network training adopts a Focaler-IoU loss function to solve the problem of unbalanced training of small samples and difficult samples; and for complex behaviors, the detected target area is sent to the MILA-SF behavior recognition network, and the space-time behaviors are efficiently recognized through fast and slow dual-path design. According to the method, the detection precision of targets such as wearable equipment and operation tools in a railway scene can be remarkably improved.

The invention provides a sewage treatment state monitoring method and system based on image feature analysis. The method comprises the following steps: acquiring a plurality of image frame data and a plurality of sensor time sequence data of a flocculation basin in a first time period; performing data splicing on the image frame data and the plurality of sensor time sequence data to obtain a plurality of multi-modal data at different moments; inputting each piece of multi-modal data into a preset floc detection model, so that the floc detection model performs feature fusion on each piece of data in the multi-modal data through an attention mechanism, and generates a frame-level floc feature vector corresponding to the multi-modal data according to a feature fusion result; and inputting each frame-level floc feature vector and the plurality of sensor time sequence data into a preset sewage treatment state evaluation model, so that the sewage treatment state evaluation model outputs the current sewage treatment state according to the input data, and the accuracy and real-time performance of sewage treatment state monitoring are improved.

A system for multitask detection performs subject tracking by processing image frames from one or more video cameras deployed in a monitored environment. The system uses a neural network to detect human subjects in each frame and extracts feature sets for each subject. These features include a semantic center of the body and directional vectors extending to other body parts, such as the head or face, forming a subject-specific fingerprint. The system compares these fingerprints across frames to identify instances of the same subject over time. By correlating subject positions in image frames with the geolocation data of the capturing cameras, the system computes global coordinates for each subject. Using both the subject-specific fingerprints and spatial coordinates, the system determines trajectories of individuals, including transitions between camera views.

The invention discloses an AI vision-based intelligent inspection system for strong-current transmission line defects, which comprises the following steps of: acquiring a visible light image and an infrared image through an unmanned aerial vehicle, and constructing an image data stream; after preprocessing, introducing a pulse coupling neural network to extract a salient region of the image, and enhancing features of a target region; fusing the saliency mask with the image, inputting a single-time multi-frame detection model to carry out defect identification and positioning, and generating candidate defect bounding boxes and category and confidence information of the candidate defect bounding boxes; and the detection result is optimized through boundary screening and structure rule constraint, and the geographic coordinates of the defect and the number of the tower to which the defect belongs are calculated in combination with image frame positioning information. The method is suitable for intelligent inspection and defect positioning of the power transmission line.

The invention discloses a lightweight high-speed photographing system and method based on an event camera, and the system comprises a hybrid imaging module which is used for synchronously collecting the image data and event data of a scene, and comprises a visible light camera and an event camera; the hardware synchronization module is electrically connected with the visible light camera and the event camera, and is used for receiving a frame synchronization signal of the visible light camera and sending a trigger signal to the event camera, so that the event camera inserts a time stamp corresponding to a visible light image frame exposure moment in an event stream; the data processing and fusion unit is in communication connection with the hybrid imaging module and is used for receiving the image data and the event data and executing the following steps: performing time alignment on an event stream and a visible light image frame stream based on a time stamp; operating a high-frequency frame insertion reconstruction algorithm, and reconstructing a multi-frame intermediate image by using event stream data between two frames; synthesizing the original visible light image frame and the reconstructed intermediate image into a high-frame-rate video stream; and the power supply and interface module is used for supplying power to the system and outputting a high-frame-rate video stream.

To employ low resolution, noisy tone mapping operations for high resolution images, at least one raw image frame is converted to a first image at a higher resolution and a second image at a lower resolution. Tone mapping is applied to the second image to derive a third image at the lower resolution. Histogram matching and regularization are performed to determine a lookup table approximating histogram matching of the second image to the third image. A global gain map is derived based on luma for the first image and the lookup table. A local gain map is derived by up-sampling and denoising residual differences between the third image and the lookup table applied to the second image. Based on the global gain map and the local gain map, a total gain map is determined for tone mapping the first image to produce a fourth image at the first resolution.

According to the target tracking and identification method, the monitoring device and the storage medium disclosed by the invention, the image frames are continuously acquired from the monitoring video source by extracting the appearance characteristics of the specified main target in the current image frame, and the position of each pedestrian target in the image frames is detected in real time; matching the identity label and the motion trail of each pedestrian target in the current image frame; if the identity identifier of the main target is matched in the current image frame, performing similarity judgment on the historical motion trail of the main target and the motion trails of other pedestrian targets in the current image frame, and recording; and if the number of times of similarity in the plurality of continuous frames is greater than a threshold value, querying the final visible position of the main target, screening the main target according to the final visible position, and expanding the detection area step by step until the main target is screened or the whole image is covered when the main target is not screened. Therefore, the target identity abnormity is effectively recognized, the misrecognition risk is avoided, and the recovery efficiency and accuracy are improved.

The invention provides a vehicle throwing object detection and positioning method and system based on multi-channel spatial-temporal feature fusion, and the method comprises the steps: obtaining a continuous time sequence image frame sequence of a road scene, and generating an optical flow image sequence through an optical flow algorithm; outputting a detection frame of each vehicle in each standardized image through a deep neural network target detection model, and generating a region of interest according to the detection frames; a fusion feature vector is generated for each region of interest, a space-time fusion feature sequence is constructed, and a thrown object classification result and a positioning result of each region of interest are generated based on the space-time fusion feature sequence and the multi-branch full-connection network; and based on the classification result and the positioning result of the thrown object, inputting the obtained coordinates of the suspected area of the thrown object into a spherical camera for tracking. According to the method, the thrown object can be efficiently and accurately detected and positioned automatically, the accuracy and real-time performance of thrown object detection are improved, the false alarm rate is reduced, and the precision degree of positioning the position of the thrown object is improved.

The invention relates to the technical field of artificial intelligence, and provides a gastric cancer risk quantitative scoring method, system and equipment based on artificial intelligence, and the method comprises the steps: recognizing the image type of each image frame in alimentary canal endoscope image data; for the electronic dyeing image frame, identifying a part contour region and an intestinal contour region of the feature part in the image frame, determining an intestinal epithelial metaplasia grading category of the feature part according to an area proportion of the intestinal contour region in the part contour region, and performing electronic dyeing intestinal scoring on the image data; for the white light image frame, gastroscope part types included in the image frame and focus area types of all gastroscope parts are recognized; performing atrophy scoring on the image data according to the position distribution of the focus area of the atrophy type; performing table state scoring on the image data according to whether the plica enlargement, nodular gastritis and diffuse redness focus areas exist or not; and realizing gastric cancer risk quantitative scoring according to the electronic staining intestinal scoring, the atrophy scoring and the epistatic state scoring.

The invention provides a method and a device for mapping image coordinates to desktop coordinates, equipment and a storage medium, which are used for improving the positioning precision of a projection touch system in a real environment. The method comprises the steps of performing distortion correction on original gesture video data according to internal parameters of a camera and a lens distortion coefficient to obtain a corrected image frame sequence, and performing interaction space calibration according to the corrected image frame sequence to obtain a distortionless interaction plane, performing perspective change matrix solving on the distortionless interaction plane according to the target vertex coordinate set to obtain orthogonal mapping data, and when gesture key points are detected in the distortionless interaction plane, performing perspective transformation and normalization processing on gesture coordinates of the gesture key points according to the orthogonal mapping data to generate initial desktop control coordinates; and performing display mapping processing on the initial desktop control coordinates according to the physical attribute parameters of the target display to obtain desktop control coordinates.

The invention belongs to the technical field of underground pipeline detection and defect identification, and particularly discloses an underground pipeline defect detection method based on image and point cloud data fusion, and the method comprises the following steps: collecting a video image and laser radar point cloud data of the inner wall of an underground pipeline, and carrying out the synchronous pairing of an image frame and a point cloud frame based on a timestamp; performing defect identification on the synchronized image frames to obtain a defect region ROI; based on a pre-calibrated external parameter matrix of a camera and a laser radar, converting the synchronously acquired point cloud data coordinates into an image coordinate system, calculating through internal parameters of the camera to obtain pixel coordinates, and screening out a point cloud point set falling into a defect region of interest (ROI) as a defect point cloud subset; and sequentially carrying out denoising, registration and clustering segmentation on the defect point cloud subset to obtain a three-dimensional point cloud structure of the defect, and calculating a quantization parameter of the defect based on the three-dimensional point cloud structure. According to the invention, automatic real-time detection and accurate quantification of pipeline defects can be effectively realized.

The invention discloses an unmanned aerial vehicle aerial image imaging optimization method and device fusing a deep learning perception mechanism and physical modeling. The method comprises the following steps: acquiring an original image frame obtained in a flight process of an unmanned aerial vehicle; inputting the image into a MobileViT illumination estimation network, extracting local convolution perception and multi-scale global semantic features, and outputting a scene illumination intensity estimation value; constructing a differentiable imaging parameter reasoning module based on an illumination physical modeling relationship, reversely deducing an optimal exposure parameter combination of a current frame, and constructing a parameter optimization module based on a perceptual error; combining the difference between the reconstructed image and the target image in the semantic perception space to construct a multi-loss function joint training model, and optimizing an exposure combination; deploying an edge computing platform for the trained network model to complete parameter prediction, control feedback and image acquisition link closed loop; according to the method, exposure optimization is realized before imaging, image gamma decoding and target enhancement are realized after imaging, and the image quality in low-light and backlight scenes is improved.

The invention discloses an artificial intelligence-based equipment security and protection monitoring method, equipment and a medium, and relates to the technical field of equipment security and protection monitoring, and the method comprises the steps: constructing a deep learning optical flow estimation model based on image frame sequences, carrying out the optical flow motion vector calculation of adjacent image frames through employing the model, and extracting the motion characteristics of smoke diffusion; textural features are extracted from the image frames, normalization and time alignment are carried out on the textural features and smoke diffusion characterization features, and a space-time joint feature vector sequence is constructed; performing smoke event classification prediction on the sequence through a time sequence classification model containing an attention mechanism to obtain an event type and confidence; and triggering a security response according to the smoke event type and the confidence coefficient. According to the invention, the dynamic characteristics of smoke diffusion are effectively extracted, and the detection precision is improved; the capturing capability of time sequence continuity and space consistency in the smoke diffusion process is enhanced through spatio-temporal joint modeling, and missing report and false report are reduced.

The invention relates to a visual identification and trend prediction method and system for wrinkles of a continuous strip-shaped material, and the method comprises the steps: collecting a continuous image sequence of the surface of the continuous strip-shaped material, obtaining a standardized image frame, and synchronously obtaining a working condition state vector; inputting the standardized image frame into a wrinkle identification model, and outputting wrinkle region information, wrinkle category information and corresponding confidence; constructing a wrinkle index vector at the current moment according to the wrinkle region information, the wrinkle category information and the corresponding confidence coefficient; filtering and time sequence modeling are carried out on the wrinkle index vectors at the current and historical moments, and wrinkle index prediction values, trend labels and prediction uncertainty at multiple moments in the future are output; and constructing a future risk index and security constraint set according to the wrinkle index prediction value, the prediction uncertainty and the working condition state vector, solving an optimal parameter packet, and mapping the optimal parameter packet into an executable control instruction for output.

A method for motion synchronization-based head pose estimation, by a head mounted display (HMD) device and an HMD device for performing the same are provided. The method includes receiving, by the HMD device, motion data from a plurality of motion sensors of the HMD device, receiving, by the HMD device, a plurality of image frames from at least one simultaneous localization and mapping SLAM camera of the HMD device, estimating, by the HMD device, a plurality of motion parameters of head movements of a user from the plurality of image frames received from memory, generating, by the HMD device, a filtered subset of the motion data received from the plurality of motion sensors based on the plurality of motion parameters of the head movements, synchronizing, by the HMD device, the plurality of image frames received from the memory and filtered subset of motion data, and estimating, by the HMD device, the head pose based on the synchronized plurality of image frames and motion data.

The invention relates to a target detection enhanced video code rate control method, and belongs to the technical field of video compression. The method comprises the following steps: preprocessing each frame of image of an input video, dividing a plurality of coding tree units in each frame of image, and calculating gradient features of the image and the coding tree units; the video is coded through different quantization parameters, and the code rate of the coded video and the distortion degree of the video are calculated; fitting a cubic logarithmic model to establish a relationship between the distortion degree and the code rate of the video by combining the gradient characteristics of each frame and the distortion degree and the code rate of the video; marking the ROI region of each image frame of the input video through a target detection network, and dividing the compensation level of the coding tree unit in combination with the ROI region and the gradient feature of the coding tree unit; and in combination with the compensation level of each coding tree unit, through a code rate allocation mechanism, allocating a coding code rate to each coding tree unit in each image frame of the input video. According to the invention, the video compression process can be optimized, and the overall performance of the compressed video is improved.

The invention discloses a rapid robust monocular vision inertial positioning method and system. The method comprises the following steps: configuring IMU and monocular camera sensor parameters; preprocessing the IMU data; iMU attitude initialization is carried out; image features of the monocular camera are extracted, abnormal matching points are removed, and IMU pre-integration is carried out; monocular vision initialization is carried out; odometer attitude optimization: judging whether the feature points are mismatched or not based on an IMU pre-integration result, constructing a feature point re-projection error jacobian matrix, and performing blocking and diagonalization processing on the re-projection error jacobian matrix to respectively optimize inverse depths and image frame attitudes of the feature points; selecting a key frame based on a key frame identification rule after the current frame attitude update is completed; and judging according to the current frame, and removing a certain frame in the sliding window to reserve a sliding window space for adding the latest frame. According to the method, the inverse depth of the feature points and the image frame attitude are optimized respectively based on the counterweight projection error Jacobian matrix partitioning and diagonalization processing, and the calculation complexity is reduced.

The invention discloses a time-space multi-mode video understanding method based on Video LLeMA2. The time-space multi-mode video understanding method comprises the following steps: S1, extracting an image frame sequence and an audio stream sequence of video data; s2, carrying out pretreatment; s3, inputting the image frame sequence into a visual encoder to generate a visual initial feature; s4, inputting the visual initial features into a space-time convolution connector, and generating visual modal features by using three-dimensional convolution and a RegStage module; s5, inputting the audio stream sequence into an audio encoder to obtain audio modal features; s6, aligning the audio modal features with the visual modal features; s7, performing feature fusion; and S8, obtaining a video content understanding result through a language decoder. The video content semantic understanding method is based on the Video LLeMA2 model, integrates multi-modal space-time modeling and language generation technologies, realizes video content semantic understanding, and has the advantages of natural expression and high precision.

The invention discloses a motion capture method based on sports equipment training, and relates to the technical field of intelligent motion capture, and the method comprises the steps: extracting a skeleton key point set of a detected object from a training ground image data stream through employing a human body key point detection algorithm, and extracting a two-dimensional contour region of sports equipment through employing a target segmentation algorithm, meanwhile, recording time index information of the corresponding image frames, performing synchronous comparison by utilizing timestamps of the training event data streams, establishing a time sequence mapping relation, and obtaining time sequence action data; taking the time sequence action data as a space anchor point, carrying out middle frame reconstruction on the high-speed motion stage based on a time sequence mapping relation, recovering an equipment motion track, and carrying out joint matching to generate a time sequence fusion sequence; establishing a multi-view geometric constraint through a time sequence fusion sequence and camera pose parameter information, and recovering a human body three-dimensional skeleton point cloud and an initial six-dimensional pose of the equipment; the space-time consistency and reconstruction precision of motion capture in a high-speed dynamic training scene are effectively improved.

The invention relates to the technical field of point location labeling, in particular to a self-adaptive load balancing point location labeling method and system based on image processing, and the method comprises the following steps: obtaining an image source division region, extracting an edge pixel number texture direction number color channel standard deviation, generating region detail redundant structure distribution information, and monitoring node state data. The method comprises the following steps: constructing a node resource load state mapping set, matching according to a regional detail redundancy ratio and a node state, establishing a task allocation relationship, extracting a boundary gray derivative, screening a boundary stable candidate point location set, calling point location coordinates, sending the point location coordinates to corresponding nodes, executing target verification, writing in an image frame, and generating a labeling result. Quantitative extraction is performed on distribution among image region edge pixels, texture directions and color channels, and task assignment is accurately paired in combination with image structure complexity and node states, so that resource mismatching and processing retardation are avoided, and task distribution accuracy is improved.

The invention provides a graphic display agent device based on a PCIE (Peripheral Component Interface Express) bus, which is characterized in that an instruction analysis module receives a graphic drawing instruction containing an operation type and a corresponding coordinate parameter through a PCIE bus interface, and generates first pixel data based on the instruction; meanwhile, the data conversion module receives original image data containing pixel position information and sensor echo information through a radar video interface, analyzes the original image data and converts the original image data into second pixel data consistent with the first pixel data in format, and the format difference of the two types of data is eliminated to meet the superposition condition; a graph superposition processing module generates a primary and secondary clear composite image according to the logic that the first pixel data covers the second pixel data; then, the image output control module receives the synthesized image by using a first buffer area and a second buffer area which alternately work to ensure that a complete image frame is transmitted to a subsequent module; and finally, the video signal generation module converts the complete image frame into an analog video signal conforming to a preset video time sequence and outputs the analog video signal.

The invention discloses a large video model training method and a related device, and relates to the technical field of video recognition, and the method comprises the steps: collecting a training video data frame to obtain an image frame, inputting a preset prompt word, a user question and the image frame into a large image model, and obtaining a thinking chain and a question answer. Performing cold start on the video large model based on the thinking chain and the question answer to enable the video large model to have thinking chain output capability; and combining training video data and questions to generate space and time disordered data and thinking chain data. And inputting the three types of data into the model to obtain corresponding outputs, calculating the accuracy of each output, obtaining space and time accuracy reward values, and training the model through a group relative strategy optimization algorithm in combination with a thinking chain consistency reward value to obtain an inference video large model. According to the method, the trained video large model can have thinking reasoning capability based on thinking chain implementation.

The invention discloses a method for performing visual inspection on surface forming quality characteristics of a welding seam by utilizing three-dimensional reconstruction, which comprises the following steps of: S1, performing uniform-speed scanning on the welding seam along the direction of a welding bead by using linear structured light emitted by a linear laser, and performing image acquisition on linear structured light stripes formed on the surface of the welding seam by using an industrial camera; s2, converting the acquired RGB image into a gray level image frame by frame, and carrying out image distortion correction and image denoising processing; s3, carrying out ROI (Region of Interest) positioning on the formed line structured light stripe image on the surface of the welding seam by adopting a pixel point gray level distribution calculation method; s4, carrying out image segmentation on the ROI region of the weld line structured light stripe image; and S5, carrying out sub-pixel-level center line extraction on the ROI of the weld line structured light stripe image by adopting a gray extreme value weighted centroid method. Automatic detection of welding seam forming quality characteristics such as laser welding and electric arc welding can be achieved, and the operation efficiency of a production line and the welding seam quality detection precision can be greatly improved.