144 results about "Visual localization" patented technology

Approaches presented herein provide for visual localization by matching features of a query image with features obtained from representation of a three-dimensional (3D) environment. A model such as a neural radiance field (NeRF) can be trained to represent the 3D environment. When a query image is received, query features can be extracted at two different resolutions. A lower resolution set of query features can be compared against NeRF descriptor features for a set of training images, to narrow the search space by finding a set of coarse matches. Higher resolution query features can then be compared against sampled features of these coarse matches, to identify 2D-3D correspondences that can be used to calculate camera pose information for the query image.

The invention discloses a sparse view angle-oriented three-dimensional Gaussian visual positioning method, which belongs to the technical field of positioning, is used for visual positioning, and comprises the following steps of: extracting global image features and local image features of a sparse image, and constructing a feature database; performing nearest neighbor search on the feature database and the RGB image to obtain a sparseness index, calculating a camera pose of a pseudo view angle, rendering a color image and a depth image under the pseudo view angle by using a training main model, extracting global features and local features and storing the global features and the local features into the feature database, constructing a 2D-3D corresponding relation in combination with a target color image, and performing dynamic inner point screening and pose solving. And obtaining a positioning result. According to the method, visual angle deficiency of a key area is intelligently completed, the balance and coverage of feature expression in a three-dimensional scene are effectively improved, and dynamic updating and mismatching elimination of a 2D-3D matching relation are realized, so that the positioning precision and the system stability are remarkably improved, and the method has higher robustness and actual availability under a sparse training condition.

The invention relates to the technical field of artificial intelligence and computer vision, and provides a visual perception method and system based on a multi-modal thinking tree in order to solve the problem that a traditional expansion strategy of purely increasing the parameter scale cannot effectively break through the semantic refinement bottleneck. The visual perception method based on the multi-modal thinking tree comprises the steps of obtaining a to-be-processed original image and a target anaphora text, and constructing the multi-modal thinking tree; defining a reasoning action set for driving node extension; executing a multi-mode Monte Carlo tree search process; iteratively generating a reasoning path until a preset search depth is reached or a termination condition is triggered; all effective leaf nodes generated in the searching process are obtained at the same time; and carrying out aggregation optimization on all effective leaf nodes by adopting a regional feature weighted voting mechanism, and screening out a candidate scheme with the highest comprehensive weight as a final visual perception positioning result. According to the method, the perception performance can be effectively improved on the basis of not changing the original parameter scale of the model, and high-precision visual positioning is realized.

The invention relates to the technical field of machining path planning, in particular to a graphite micropore femtosecond laser machining path planning method based on AI visual localization, which comprises the following steps: S1, acquiring an image of the surface of a graphite workpiece through an industrial camera, and preprocessing the image; s2, processing the image based on an AI visual identification model, and extracting microscopic features of the graphite surface and position coordinates of a preset processing area; s3, based on the extracted feature coordinates and preset micropore design parameters, an initial path of femtosecond laser processing is generated through a path optimization algorithm; and S4, the optimized machining path is converted into a laser galvanometer control instruction, and femtosecond laser is driven to complete micropore machining. According to the method, on the basis of path optimization of the improved grey wolf algorithm, through dynamic weight adjustment, in a multi-taboo area scene, the idle running time of the laser galvanometer is effectively shortened, and the machining efficiency is improved.

The invention discloses a visual positioning method for satellite on-orbit fuel filling docking service, which comprises two stages of off-line training and on-line positioning, and comprises the following steps: firstly, acquiring image and pose information by using a camera, establishing a pre-matching database, carrying out feature matching, calculating the pose of the camera through structural motion, and generating sparse point cloud; carrying out dense reconstruction on the scene by adopting three-dimensional Gaussian sputtering, and generating a three-dimensional Gaussian sputtering model through training; after a camera at the tail end of the refueling satellite mechanical arm captures an original image in real time, a reference image most similar to the current image is recognized based on the pre-matching database, a corresponding initial pose is obtained, and the image is rendered to serve as a starting point of subsequent iterative optimization; and meanwhile, color migration correction is performed on the captured image by taking the rendered image as a reference, and inter-domain differences are eliminated to generate a target image in an optimization stage. According to the invention, a picture captured at an unknown view angle can be well positioned. And the calculation efficiency is improved while the accuracy is ensured as much as possible.

According to one embodiment of the present disclosure, A visual localization method comprises generating at least two or more anchor points from three-dimensional point clouds; generating three-dimensional ray clouds by connecting three-dimensional points included in the three-dimensional point clouds with one of the generated anchor points; extracting feature points of an input image; and clustering a plurality of lines included in the three-dimensional ray clouds based on the at least two or more anchor points, sampling two ray cloud clusters out of the clustered ray cloud clusters, and estimating a pose of a camera that captured the input image based on the sampled ray cloud clusters and the feature points.

The invention discloses a self-adaptive visual positioning method based on multi-scale feature interaction, belongs to the technical field of deep learning and computer vision, and aims to solve the problems of high calculation complexity, low local feature sensitivity and insufficient multi-scale feature fusion of a native Transform in a visual positioning method. The method comprises the following steps: S1, extracting local features of an input image through a CNN-based basic network; s2, extracting multi-scale features of the local features by using a multi-head adaptive adjustment Transform, carrying out interaction and dynamic weighted fusion on the multi-scale features, and outputting global integrated local features; s3, adaptively adjusting the dynamic fusion weight of the local features and the global integrated local features through a dynamic feature adjuster, and outputting a fused feature map; and S4, predicting the three-dimensional scene coordinate of the image and the one-dimensional uncertainty of each pixel based on the regression head, connecting the regression head in series twice to adaptively adjust the Transform and the dynamic feature adjuster, gradually compressing the feature channel, and finally outputting the six-degree-of-freedom pose of the camera.

The system and methods of the various embodiments may enable an aerial vehicle to determine its pose using absolute visual localization of route features and either a keypoint-based pipeline or a template based pipeline. This may result in the aerial vehicle being able to determine its pose when traditional methods of pose detection fail.

The invention discloses an enhanced visual positioning method and system based on cross-domain three-dimensional Gaussian sputtering, and the method comprises the steps: generating a second image in a second image field corresponding to a first image in a first image field through an image editing model, so as to obtain an extended data set of a first image data set in the first image field; performing sputtering model fine tuning on the three-dimensional Gaussian sputtering model adaptive to the feature distribution condition of the second image field according to the corresponding first image and the second image in the extended data set; and generating a positioning image training set for the image positioning model according to the three-dimensional Gaussian sputtering model subjected to fine tuning of the sputtering model, and performing positioning model training on the image positioning model according to the positioning image training set to obtain a trained image positioning model. According to the method, the visual positioning system has higher adaptability and generalization ability, the phenomenon that the positioning precision is reduced is reduced, and the requirements for stability and high precision of cross-domain visual positioning are met.

Examples describe using a two dimensional floor plan, rather than a three dimensional (3D) reconstruction of a scene, to localize a camera. A floorplan of an environment is accessed and an image of the environment captured by a camera in the environment is received. The image is input to a trained neural network to predict an array of rays from the camera to surfaces in the environment indicated in the floorplan. Using the array of rays it is then possible to compute 3D position and orientation of the camera with respect to the floorplan.

The invention discloses an explanatory visual question-answering method and system based on question perception and confidence constraint. The system comprises a selection-enhancement module and a fusion enhancement confidence constraint module, the selection-enhancement module is used for selecting candidate image areas based on question semantics and enhancing salient areas related to questions through a learnable enhancement mechanism to realize accurate visual localization; and the fusion enhancement confidence constraint module is used for fusing the enhanced regional features and the multi-modal features, and ensuring that the confidence of a prediction answer is improved when explanation information is introduced through a double-branch prediction and confidence constraint mechanism, so that the reliability of a prediction result is enhanced. According to the method, the defects of problem insensitive positioning and positioning-reasoning disjunction in the prior art can be effectively overcome, the superiority of the method is verified on a public data set, the answer prediction accuracy and the explanation generation quality are remarkably improved, and the method has wide application prospects.

Provided are systems and methods that invert a trained NeRF model, which stores the structure of a scene or object, to estimate the 6D pose from an image taken with a novel view, 6D pose estimation has a wide range of applications, including visual localization and object pose estimation for robot manipulation.

The invention discloses an unmanned aerial vehicle photovoltaic power station autonomous inspection method, device and equipment based on visual localization, and a medium, and the method comprises the steps: obtaining the field image data of a photovoltaic power station, and selecting a corresponding data processing model according to the preset task content, so as to extract the feature points of a photovoltaic module in the field image data; associating the feature points with data in a preset photovoltaic power station model to obtain a corresponding relation between each photovoltaic module and a position point of the photovoltaic power station model, and determining a current position of the unmanned aerial vehicle in the photovoltaic power station according to the corresponding relation; and according to the current position and a preset next task site, planning a flight path of the unmanned aerial vehicle so as to control the unmanned aerial vehicle to fly to the next task site to execute an inspection task. According to the invention, unmanned aerial vehicle positioning and navigation based on visual positioning are provided, so that the positioning and navigation precision of the unmanned aerial vehicle during photovoltaic power station inspection is improved.

The invention belongs to the technical field of artificial intelligence, and particularly relates to a multi-modal large model illusion detection method based on reverse visual positioning. The method comprises the following steps: constructing a visual instruction fine tuning data set rich in context; training a visual positioning large model with pixel-level positioning and rejection capability based on the data set; performing sentence-by-sentence verification on a response generated by a to-be-detected multi-modal large model by using the trained model, and judging whether illusion exists or not by judging whether text description can be reversely positioned back to image pixels or not; according to the method, illusion rich in details can be effectively detected, pixel-level masks and natural language interpretation are provided, and the accuracy and transparency of evaluation are remarkably improved.

The embodiment of the invention discloses a visual positioning method and device, electronic equipment and a medium. The method comprises the steps of performing image acquisition on a target scene through augmented reality equipment to obtain a positioning image, performing feature point matching on the positioning image and a scene image in a preset image set, and determining a target feature point; determining a two-dimensional coordinate and a three-dimensional coordinate of the target feature point, carrying out positioning calculation, determining a conversion matrix, and carrying out calculation fine adjustment on the conversion matrix based on an optimization algorithm; and determining an evaluation score of the conversion matrix according to at least one of a matching parameter in the feature point matching process, a resolving parameter in the positioning resolving process and a fine tuning parameter in the resolving fine tuning process, and determining an available evaluation result of the conversion matrix according to the evaluation score. According to the scheme, the robustness is high, the positioning precision is not easily influenced by the environment, the dependence on regional textures and map data is reduced, and the positioning success rate and the positioning result precision are improved.

The invention provides an MES collaborative scheduling method and system based on a PLC and visual localization, and the method comprises the steps: collecting a production material image through a dual-mode visual sensor, and generating the visual localization of a material; the PLC system receives the visual positioning, synchronously acquires the current joint state of the mechanical arm, dynamically plans an initial path by applying an optimization A * algorithm, and performs track correction through real-time PID control to obtain track data of the mechanical arm; the mechanical arm track data are input into an MES system, minimization of a busy window of a mechanical arm and high-temperature environment constraint are taken as optimization targets, a task scheduling instruction and path planning of the AGV are optimized, and AGV scheduling output is generated; aGV scheduling output is executed, meanwhile, execution process data are collected, performance deviation is analyzed, thermal risk parameters and control threshold values are periodically updated, and system self-adaptive optimization is achieved. The stability and efficiency of the whole production process are improved.

The invention belongs to the technical field of visual positioning, and relates to a visual positioning method based on feature enhancement and language perception attribute guidance. The core of the method is that two key modules of feature enhancement and language perception attribute guidance are efficiently integrated. In the feature enhancement link, multi-modal features with high distinction degree are refined through multi-round guidance of visual semantic features, so that the feature saliency of the target object is greatly enhanced, and it is ensured that the features of the target object are clear and distinguishable from those of background and other non-target objects. In the aspect of language perception attribute guidance, deep semantic coupling between text embedding and visual representation is deeply mined, core attribute information highly related to a target object is accurately captured, target query is optimized and initialized accordingly, and a solid foundation is laid for subsequent accurate positioning.

A geometric structure aided visual localization method and a system implementing the method are provided. The method includes retrieving a 3D map of initialization location data, the 3D map being constrained by visual structures and geometric structures and modeled by a Gaussian mixture model of a set of Gaussian distributions with mapped landmarks; acquiring a series of real-time image frames by a camera of a mobile system; and for each real-time image frame: extracting local features from the real-time image frame; predicting a camera pose corresponding to the real-time image frame; creating a key frame by tracking the predicted camera pose in the 3D map; identifying temporally visible landmarks with respect to the created key frame; acquiring 2D-3D correspondences between the local features and the temporally visible landmarks; and localizing the mobile system by estimating a state of the real-time image frame based on the 2D-3D correspondences.

This invention relates to a multimodal visual understanding method based on consistency learning and hybrid feature extraction. It includes constructing an end-to-end fine-grained consistency learning framework, introducing a hybrid region extractor, fusing local details and global semantics to generate high-quality hybrid visual cue embeddings, combining self-reconstruction loss and latent spatial consistency loss to force the model to establish explicit alignment between the input visual cue and the output segmentation label, utilizing the geometric boundary constraints of the localization task for description generation, and simultaneously optimizing localization accuracy using the semantic depth of the description task. Furthermore, it constructs a detailed localization index expression and segmentation task to enhance the model's reasoning ability for complex long text instructions. The aim is to address the problems of feature fragmentation and insufficient accuracy in existing large models for fine-grained visual localization and description tasks. Compared with existing technologies, this invention has advantages such as high accuracy and strong generalization ability in pixel-level localization and fine-grained description.

The present invention discloses a three-dimensional Gaussian visual positioning method for sparse perspective, which belongs to the field of positioning technology and is used for visual positioning. The method comprises extracting global image features and local image features of a sparse image and constructing a feature database; performing nearest neighbor search on the feature database and the RGB image to obtain a sparsity index, calculating the camera pose of a pseudo-perspective, rendering a color map and a depth map under the pseudo-perspective using a trained main model, extracting global features and local features and incorporating them into the feature database, constructing a 2D-3D correspondence relationship in combination with the target color map, performing dynamic inlier screening and pose solving, and obtaining a positioning result. The present invention intelligently supplements the perspective loss of key areas, effectively improves the balance and coverage of feature expression in three-dimensional scenes, realizes dynamic updating of 2D-3D matching relationships and elimination of false matches, thereby significantly improving positioning accuracy and system stability, and having stronger robustness and practical usability under sparse training conditions.

The present invention relates to the field of computer vision technology, and discloses a long-term visual positioning method based on an association guidance network. The model training stage includes perceptual network training and concept network training; after the concept network training is completed, the concept network is used as a pre-training model to train the perceptual network, and the concept network is optimized at the same time; the association guidance mechanism performs information interaction; in the image retrieval stage, the trained perceptual network is used to perform image retrieval. The invention uses domain adaptive learning and an association guidance mechanism to use the features in the concept network to guide the feature learning in the perceptual network, so that the features finally obtained by the model training are robust under environmental changes. In addition, the present invention uses a concept database that does not require additional data to perform self-inspirational learning on the concept network, so as to better guide the perceptual network with domain features, thereby improving the final image retrieval performance.

The invention relates to an efficient visual positioning method under conservation of a multi-modal large model dialogue capability, which is characterized in that an adopted positioning network D-LMM comprises a multi-modal large model LMM image encoder CLIP, a multi-modal large model LMM text encoder, a frozen LLM and an instance-level text feature extractor. A dynamic instance feature modulation module DIFM and a fusion segmentation mask head are adopted, and for given images and texts, the process of positioning by using the D-LMM comprises the following steps: inputting the images into an image encoder CLIP to obtain multilayer features; splicing the input visual features and the input text features together; inputting the output text features of the LLM into an instance-perceived text feature extractor ITE to obtain instance-perceived text features; and inputting the average text feature of each instance and the multi-level visual features obtained from the image encoder into a dynamic instance feature modulation module DIFM, and converting the multi-level visual features into a multi-level feature map perceived by the instances.

This application provides a multimodal visual localization method, apparatus, and electronic device, which can be applied to the field of computer vision technology. The method includes: responding to image data acquired by an image acquisition device mounted on a vehicle at a target time, extracting features from the image data to obtain image features; predicting the depth features at the target time based on the expected pose transformation of the image acquisition device and historical depth features to obtain predicted depth features; if the image data also includes a depth image, performing noise reduction processing on the predicted depth features according to the mapping relationship between the depth features and the predicted depth features to obtain optimized depth features; fusing the optimized depth features and visual features to obtain multimodal visual features; and localizing the image acquisition device based on the multimodal visual features to obtain the localization result at the target time.

The present invention discloses a spatial vision positioning method based on the design of a two-color rectangular target. The method includes: designing a two-color rectangular target pattern; performing recognition processing on the two-color rectangular target pattern by the contour corner point ellipse extraction method to obtain the target rectangular corner points in the two-color rectangular target pattern; based on the target rectangular corner points in the two-color rectangular target pattern, performing global numbering positioning processing by the sorting corner point numbering method to obtain the spatial positioning of the two-color rectangular target. The present invention can meet the requirements of spatial positioning and long-distance positioning with the help of the target in a dark environment through a simple target pattern composed of two-color target rectangles. As a spatial vision positioning method based on the design of a two-color rectangular target, the present invention can be widely applied to the field of machine vision technology.

This application discloses a map-free visual localization method, apparatus, device, and medium. The method includes acquiring scene images collected in real time by a robot and extracting an initial feature map from the scene images; enhancing the initial feature map into a voxel feature map using a sparse kernel self-attention mechanism; aggregating the voxel feature map from several dimensions and multiplying the aggregated features element-wise to obtain image feature identifiers; searching for at least one anchor point feature identifier based on the image feature identifiers; and determining a six-degree-of-freedom pose based on the set of anchor point feature identifiers. This application utilizes a sparse kernel self-attention mechanism to enhance the initial feature map using voxels and aggregates the voxel feature map from several dimensions, achieving cross-domain interactive sparse quantization. This overcomes feature interference caused by motion blur or illumination changes, improves the robustness of scene image feature extraction, thereby reducing the probability of matching failure or inaccurate matching results and improving the accuracy of map-free visual localization.

The invention discloses an unmanned aerial vehicle visual positioning method based on a multilevel feature pyramid fusion architecture, and the method comprises the steps: inputting an air-to-ground view angle image shot by an unmanned aerial vehicle and a regional high-resolution satellite image containing the position of the unmanned aerial vehicle; a feature extraction trunk with a self-attention mechanism and a cross attention mechanism is adopted to optimize feature interaction between an unmanned aerial vehicle image and a satellite image. Meanwhile, cross-level feature fusion is carried out by designing a feature fusion module of a symmetrical pyramid structure to generate a feature thermodynamic diagram, and the position of the unmanned aerial vehicle is accurately determined according to heat value distribution in the thermodynamic diagram. According to the method, feature information of different scales is effectively integrated, the calculation amount is reduced, meanwhile, the positioning precision is improved, the method is particularly suitable for the complex environment with GNSS signal loss or interference, high robustness and high efficiency are achieved, meanwhile, the method can be deployed on various small onboard computers, and the method is suitable for popularization and application. The method is widely applicable to autonomous positioning of the unmanned aerial vehicle under denial conditions.

The present invention relates to a computing device for supporting a mobile device in performing visual localization in a target environment. The computing device is configured to obtain and modify one or more 3D models of the target environment to generate a set of 3D models. Then, the computing device is configured to determine a training sample set from the set of 3D models. Each training sample comprises an image derived from the set of 3D models and corresponding pose information. Then, the computing device is configured to train a neural network model from the training sample set such that the trained neural network model is configured to predict pose information for an image captured in the target environment and output a confidence value associated with the predicted pose information.

The invention discloses a visual localization and attitude estimation method and device based on prior search in a rocket recovery section and a medium, and belongs to the technical field of spacecraft guidance, navigation and control, and the method comprises the steps: carrying out the systematic sampling of a rocket recovery tail end operation space containing a three-dimensional position and a three-dimensional camera attitude in advance; constructing a matching data set in which the rocket recovery state is matched with the image; learning mapping from an image to a low-dimensional feature vector by using a deep learning model, and enabling the structure of a feature space to be consistent with the structure of a physical state space through a designed loss function; and in the rocket recovery stage, pre-stored samples closest to real-time image features are retrieved, and state labels of the pre-stored samples are fused, so that pose estimation is realized. The method can adapt to complex three-dimensional attitude changes, is high in robustness, and can meet the real-time requirement.

The present invention provides a visual positioning method based on a point cloud map, including a point cloud map generation module, a visual inertial odometer construction module, and a visual matching and positioning module based on the point cloud map. Among them, the point cloud map module establishes a high-precision point cloud map by fusing laser, IMU, and GPS information; the visual inertial odometer construction module first extracts visual feature points, uses the optical flow method to track the feature points of the front and rear frames, and performs pre-integration on the IMU, and fuses with the visual feature points to construct a visual inertial odometer, outputs the initial pose of each frame, and restores the depth of the feature points; the visual matching and positioning module based on the existing map extracts a sub-map from the point cloud map according to the initial position, projects the feature points restored by vision into the 3D space map coordinate system, and queries the nearest points in the current sub-map. For the nearest points matched by vision and the map, the RANSAC algorithm based on dual quaternion is used to optimize the pose of the current frame.

Provided are a visual positioning method, a non-transitory computer-readable storage medium and an electronic device. Surface normal vectors of a current image frame is obtained. A first transformation parameter between the current image frame and a reference image frame is determined, by projecting the surface normal vectors to a Manhattan coordinate system. A matching operation between feature points of the current image frame and feature points of the reference image frame is performed, and a second transformation parameter between the current image frame and the reference image frame is determined based on a matching result. A target transformation parameter is obtained, based on the first transformation parameter and the second transformation parameter. A visual localization result corresponding to the current image frame is output, based on the target transformation parameter.