167 results about "Visual localization" patented technology

Approaches presented herein provide for visual localization by matching features of a query image with features obtained from representation of a three-dimensional (3D) environment. A model such as a neural radiance field (NeRF) can be trained to represent the 3D environment. When a query image is received, query features can be extracted at two different resolutions. A lower resolution set of query features can be compared against NeRF descriptor features for a set of training images, to narrow the search space by finding a set of coarse matches. Higher resolution query features can then be compared against sampled features of these coarse matches, to identify 2D-3D correspondences that can be used to calculate camera pose information for the query image.

The invention discloses a sparse view angle-oriented three-dimensional Gaussian visual positioning method, which belongs to the technical field of positioning, is used for visual positioning, and comprises the following steps of: extracting global image features and local image features of a sparse image, and constructing a feature database; performing nearest neighbor search on the feature database and the RGB image to obtain a sparseness index, calculating a camera pose of a pseudo view angle, rendering a color image and a depth image under the pseudo view angle by using a training main model, extracting global features and local features and storing the global features and the local features into the feature database, constructing a 2D-3D corresponding relation in combination with a target color image, and performing dynamic inner point screening and pose solving. And obtaining a positioning result. According to the method, visual angle deficiency of a key area is intelligently completed, the balance and coverage of feature expression in a three-dimensional scene are effectively improved, and dynamic updating and mismatching elimination of a 2D-3D matching relation are realized, so that the positioning precision and the system stability are remarkably improved, and the method has higher robustness and actual availability under a sparse training condition.

The invention relates to the technical field of artificial intelligence and computer vision, and provides a visual perception method and system based on a multi-modal thinking tree in order to solve the problem that a traditional expansion strategy of purely increasing the parameter scale cannot effectively break through the semantic refinement bottleneck. The visual perception method based on the multi-modal thinking tree comprises the steps of obtaining a to-be-processed original image and a target anaphora text, and constructing the multi-modal thinking tree; defining a reasoning action set for driving node extension; executing a multi-mode Monte Carlo tree search process; iteratively generating a reasoning path until a preset search depth is reached or a termination condition is triggered; all effective leaf nodes generated in the searching process are obtained at the same time; and carrying out aggregation optimization on all effective leaf nodes by adopting a regional feature weighted voting mechanism, and screening out a candidate scheme with the highest comprehensive weight as a final visual perception positioning result. According to the method, the perception performance can be effectively improved on the basis of not changing the original parameter scale of the model, and high-precision visual positioning is realized.

The invention relates to the technical field of machining path planning, in particular to a graphite micropore femtosecond laser machining path planning method based on AI visual localization, which comprises the following steps: S1, acquiring an image of the surface of a graphite workpiece through an industrial camera, and preprocessing the image; s2, processing the image based on an AI visual identification model, and extracting microscopic features of the graphite surface and position coordinates of a preset processing area; s3, based on the extracted feature coordinates and preset micropore design parameters, an initial path of femtosecond laser processing is generated through a path optimization algorithm; and S4, the optimized machining path is converted into a laser galvanometer control instruction, and femtosecond laser is driven to complete micropore machining. According to the method, on the basis of path optimization of the improved grey wolf algorithm, through dynamic weight adjustment, in a multi-taboo area scene, the idle running time of the laser galvanometer is effectively shortened, and the machining efficiency is improved.

The invention discloses a visual positioning method for satellite on-orbit fuel filling docking service, which comprises two stages of off-line training and on-line positioning, and comprises the following steps: firstly, acquiring image and pose information by using a camera, establishing a pre-matching database, carrying out feature matching, calculating the pose of the camera through structural motion, and generating sparse point cloud; carrying out dense reconstruction on the scene by adopting three-dimensional Gaussian sputtering, and generating a three-dimensional Gaussian sputtering model through training; after a camera at the tail end of the refueling satellite mechanical arm captures an original image in real time, a reference image most similar to the current image is recognized based on the pre-matching database, a corresponding initial pose is obtained, and the image is rendered to serve as a starting point of subsequent iterative optimization; and meanwhile, color migration correction is performed on the captured image by taking the rendered image as a reference, and inter-domain differences are eliminated to generate a target image in an optimization stage. According to the invention, a picture captured at an unknown view angle can be well positioned. And the calculation efficiency is improved while the accuracy is ensured as much as possible.

According to one embodiment of the present disclosure, A visual localization method comprises generating at least two or more anchor points from three-dimensional point clouds; generating three-dimensional ray clouds by connecting three-dimensional points included in the three-dimensional point clouds with one of the generated anchor points; extracting feature points of an input image; and clustering a plurality of lines included in the three-dimensional ray clouds based on the at least two or more anchor points, sampling two ray cloud clusters out of the clustered ray cloud clusters, and estimating a pose of a camera that captured the input image based on the sampled ray cloud clusters and the feature points.

The invention discloses a self-adaptive visual positioning method based on multi-scale feature interaction, belongs to the technical field of deep learning and computer vision, and aims to solve the problems of high calculation complexity, low local feature sensitivity and insufficient multi-scale feature fusion of a native Transform in a visual positioning method. The method comprises the following steps: S1, extracting local features of an input image through a CNN-based basic network; s2, extracting multi-scale features of the local features by using a multi-head adaptive adjustment Transform, carrying out interaction and dynamic weighted fusion on the multi-scale features, and outputting global integrated local features; s3, adaptively adjusting the dynamic fusion weight of the local features and the global integrated local features through a dynamic feature adjuster, and outputting a fused feature map; and S4, predicting the three-dimensional scene coordinate of the image and the one-dimensional uncertainty of each pixel based on the regression head, connecting the regression head in series twice to adaptively adjust the Transform and the dynamic feature adjuster, gradually compressing the feature channel, and finally outputting the six-degree-of-freedom pose of the camera.

The invention discloses a high-flexibility unmanned coffee robot system based on visual localization, and relates to the technical field of mechanical automation, and the high-flexibility unmanned coffee robot system comprises a user interaction unit, a visual localization unit and a mechanical arm unit. The system can accurately capture environment images under various illumination conditions, and data preprocessing is performed through the image acquisition module, so that the image quality is improved. The feature extraction module extracts useful features from the preprocessed image, and the positioning analysis module calculates the positions and postures of the cameras at different time points through an SLAM algorithm by using multi-view image data captured by the plurality of cameras, and constructs a three-dimensional map of the environment; the combination of the multi-modal sensing and advanced image processing technology significantly improves the positioning precision and adaptability of the unmanned coffee robot in a complex environment.

Systems and methods for providing a unified visual localization architecture are described herein. In some implementations, a system includes an image acquisition device mounted to an object, the image acquisition device configured to acquire a query frame of an environment containing the object. The system also includes a memory device configured to store an image database. Further, the system includes at least one processor configured to execute computer-readable instructions that direct the at least one processor to identify a set of data in the image database that potentially matches the query frame; identify a vision localization paradigm in a plurality of vision localization paradigms; and determine a pose for the object using the set of data, the query frame, and lens characteristics for the image acquisition device as inputs to the vision localization paradigm.

The system and methods of the various embodiments may enable an aerial vehicle to determine its pose using absolute visual localization of route features and either a keypoint-based pipeline or a template based pipeline. This may result in the aerial vehicle being able to determine its pose when traditional methods of pose detection fail.

The present invention belongs to the technical field of visual SLAM localization. Disclosed is a robot localization method based on improved point-line feature and IMU fusion. The method in the present invention comprises: 1, extracting feature points from an image, performing tracking and matching, using an improved line segment detection (LSD) method to extract lines from the image, using a line segment merging algorithm to merge short line segments belonging to the same straight line, and then using an LBD to perform matching; 2, performing pre-integration on collected IMU data; 3, using the same initialization policy as VINS-Mono; and 4, using information added to a sliding window, so as optimize the pose of a camera. In the present invention, filtering processing is added, a large number of line segments are no longer extracted in a dense region, a line segment merging algorithm is used to remove some short line segments, and long line segments with more stable characteristics are obtained by means of merging, thereby reducing the matching complexity; and in an environment where there are more texture lines, a trajectory generated in the present invention can also effectively improve the precision of localization.

The invention discloses an enhanced visual positioning method and system based on cross-domain three-dimensional Gaussian sputtering, and the method comprises the steps: generating a second image in a second image field corresponding to a first image in a first image field through an image editing model, so as to obtain an extended data set of a first image data set in the first image field; performing sputtering model fine tuning on the three-dimensional Gaussian sputtering model adaptive to the feature distribution condition of the second image field according to the corresponding first image and the second image in the extended data set; and generating a positioning image training set for the image positioning model according to the three-dimensional Gaussian sputtering model subjected to fine tuning of the sputtering model, and performing positioning model training on the image positioning model according to the positioning image training set to obtain a trained image positioning model. According to the method, the visual positioning system has higher adaptability and generalization ability, the phenomenon that the positioning precision is reduced is reduced, and the requirements for stability and high precision of cross-domain visual positioning are met.

Examples describe using a two dimensional floor plan, rather than a three dimensional (3D) reconstruction of a scene, to localize a camera. A floorplan of an environment is accessed and an image of the environment captured by a camera in the environment is received. The image is input to a trained neural network to predict an array of rays from the camera to surfaces in the environment indicated in the floorplan. Using the array of rays it is then possible to compute 3D position and orientation of the camera with respect to the floorplan.

The invention discloses an explanatory visual question-answering method and system based on question perception and confidence constraint. The system comprises a selection-enhancement module and a fusion enhancement confidence constraint module, the selection-enhancement module is used for selecting candidate image areas based on question semantics and enhancing salient areas related to questions through a learnable enhancement mechanism to realize accurate visual localization; and the fusion enhancement confidence constraint module is used for fusing the enhanced regional features and the multi-modal features, and ensuring that the confidence of a prediction answer is improved when explanation information is introduced through a double-branch prediction and confidence constraint mechanism, so that the reliability of a prediction result is enhanced. According to the method, the defects of problem insensitive positioning and positioning-reasoning disjunction in the prior art can be effectively overcome, the superiority of the method is verified on a public data set, the answer prediction accuracy and the explanation generation quality are remarkably improved, and the method has wide application prospects.

Provided are systems and methods that invert a trained NeRF model, which stores the structure of a scene or object, to estimate the 6D pose from an image taken with a novel view, 6D pose estimation has a wide range of applications, including visual localization and object pose estimation for robot manipulation.

The invention discloses an unmanned aerial vehicle photovoltaic power station autonomous inspection method, device and equipment based on visual localization, and a medium, and the method comprises the steps: obtaining the field image data of a photovoltaic power station, and selecting a corresponding data processing model according to the preset task content, so as to extract the feature points of a photovoltaic module in the field image data; associating the feature points with data in a preset photovoltaic power station model to obtain a corresponding relation between each photovoltaic module and a position point of the photovoltaic power station model, and determining a current position of the unmanned aerial vehicle in the photovoltaic power station according to the corresponding relation; and according to the current position and a preset next task site, planning a flight path of the unmanned aerial vehicle so as to control the unmanned aerial vehicle to fly to the next task site to execute an inspection task. According to the invention, unmanned aerial vehicle positioning and navigation based on visual positioning are provided, so that the positioning and navigation precision of the unmanned aerial vehicle during photovoltaic power station inspection is improved.

The invention belongs to the technical field of artificial intelligence, and particularly relates to a multi-modal large model illusion detection method based on reverse visual positioning. The method comprises the following steps: constructing a visual instruction fine tuning data set rich in context; training a visual positioning large model with pixel-level positioning and rejection capability based on the data set; performing sentence-by-sentence verification on a response generated by a to-be-detected multi-modal large model by using the trained model, and judging whether illusion exists or not by judging whether text description can be reversely positioned back to image pixels or not; according to the method, illusion rich in details can be effectively detected, pixel-level masks and natural language interpretation are provided, and the accuracy and transparency of evaluation are remarkably improved.

The embodiment of the invention discloses a visual positioning method and device, electronic equipment and a medium. The method comprises the steps of performing image acquisition on a target scene through augmented reality equipment to obtain a positioning image, performing feature point matching on the positioning image and a scene image in a preset image set, and determining a target feature point; determining a two-dimensional coordinate and a three-dimensional coordinate of the target feature point, carrying out positioning calculation, determining a conversion matrix, and carrying out calculation fine adjustment on the conversion matrix based on an optimization algorithm; and determining an evaluation score of the conversion matrix according to at least one of a matching parameter in the feature point matching process, a resolving parameter in the positioning resolving process and a fine tuning parameter in the resolving fine tuning process, and determining an available evaluation result of the conversion matrix according to the evaluation score. According to the scheme, the robustness is high, the positioning precision is not easily influenced by the environment, the dependence on regional textures and map data is reduced, and the positioning success rate and the positioning result precision are improved.

The invention provides an MES collaborative scheduling method and system based on a PLC and visual localization, and the method comprises the steps: collecting a production material image through a dual-mode visual sensor, and generating the visual localization of a material; the PLC system receives the visual positioning, synchronously acquires the current joint state of the mechanical arm, dynamically plans an initial path by applying an optimization A * algorithm, and performs track correction through real-time PID control to obtain track data of the mechanical arm; the mechanical arm track data are input into an MES system, minimization of a busy window of a mechanical arm and high-temperature environment constraint are taken as optimization targets, a task scheduling instruction and path planning of the AGV are optimized, and AGV scheduling output is generated; aGV scheduling output is executed, meanwhile, execution process data are collected, performance deviation is analyzed, thermal risk parameters and control threshold values are periodically updated, and system self-adaptive optimization is achieved. The stability and efficiency of the whole production process are improved.

The invention discloses a visual odometer positioning correction method based on an environmental object as a semantic road sign, and belongs to the field of visual localization, and the method comprises the steps: constructing a semantic map through the semantic object in an actual scene, and training a deep learning model for semantic segmentation and classification; semantic segmentation detection and classification detection of a first semantic object are performed on an actual scene image acquired during operation of a positioning system, the first semantic object is used as a semantic road sign to retrieve a matching item from a semantic map, and when the matching item is in a camera view range, pixel coordinates of feature points of the matching item are utilized to perform positioning correction of a visual odometer; according to the method, positioning correction is carried out on the visual odometer based on matching of a semantic segmentation model, a classification model and a map, the application limitation of the visual odometer in an indoor complex environment can be got rid of by using the anti-interference performance of semantic features in the environment, and the robustness and autonomy of a positioning system in the complex environment are improved.

The invention belongs to the technical field of visual positioning, and relates to a visual positioning method based on feature enhancement and language perception attribute guidance. The core of the method is that two key modules of feature enhancement and language perception attribute guidance are efficiently integrated. In the feature enhancement link, multi-modal features with high distinction degree are refined through multi-round guidance of visual semantic features, so that the feature saliency of the target object is greatly enhanced, and it is ensured that the features of the target object are clear and distinguishable from those of background and other non-target objects. In the aspect of language perception attribute guidance, deep semantic coupling between text embedding and visual representation is deeply mined, core attribute information highly related to a target object is accurately captured, target query is optimized and initialized accordingly, and a solid foundation is laid for subsequent accurate positioning.

The invention provides a method for performing side-view camera visual localization by using laser compensation, which comprises the following steps of: 1, acquiring real-time height data of a side-view camera, and adjusting the height of the camera based on the focal length of the side-view camera; step 2, acquiring a Mark point image, and calculating pixel offset of a Mark point based on a visual identification algorithm; and step 3, based on the relative position of the Mark point, calculating the translation offset and the rotation angle of the PCB. Through double detection of laser and vision, the system can accurately calculate translation and rotation offset, and the detection accuracy is remarkably improved. The automatic laser focusing function ensures that images of elements with different heights are clear, the manual focusing time is shortened, and the detection efficiency is improved. Through accurate measurement of the relative position of the Mark point, the rotation error accumulation phenomenon is effectively reduced, and the detection result is more reliable. The compensation system is higher in stability in a complex environment and is not influenced by vibration or illumination fluctuation, and a reliable detection scheme is provided for the modern electronic manufacturing industry.

A geometric structure aided visual localization method and a system implementing the method are provided. The method includes retrieving a 3D map of initialization location data, the 3D map being constrained by visual structures and geometric structures and modeled by a Gaussian mixture model of a set of Gaussian distributions with mapped landmarks; acquiring a series of real-time image frames by a camera of a mobile system; and for each real-time image frame: extracting local features from the real-time image frame; predicting a camera pose corresponding to the real-time image frame; creating a key frame by tracking the predicted camera pose in the 3D map; identifying temporally visible landmarks with respect to the created key frame; acquiring 2D-3D correspondences between the local features and the temporally visible landmarks; and localizing the mobile system by estimating a state of the real-time image frame based on the 2D-3D correspondences.

This invention relates to a multimodal visual understanding method based on consistency learning and hybrid feature extraction. It includes constructing an end-to-end fine-grained consistency learning framework, introducing a hybrid region extractor, fusing local details and global semantics to generate high-quality hybrid visual cue embeddings, combining self-reconstruction loss and latent spatial consistency loss to force the model to establish explicit alignment between the input visual cue and the output segmentation label, utilizing the geometric boundary constraints of the localization task for description generation, and simultaneously optimizing localization accuracy using the semantic depth of the description task. Furthermore, it constructs a detailed localization index expression and segmentation task to enhance the model's reasoning ability for complex long text instructions. The aim is to address the problems of feature fragmentation and insufficient accuracy in existing large models for fine-grained visual localization and description tasks. Compared with existing technologies, this invention has advantages such as high accuracy and strong generalization ability in pixel-level localization and fine-grained description.

The present invention discloses a three-dimensional Gaussian visual positioning method for sparse perspective, which belongs to the field of positioning technology and is used for visual positioning. The method comprises extracting global image features and local image features of a sparse image and constructing a feature database; performing nearest neighbor search on the feature database and the RGB image to obtain a sparsity index, calculating the camera pose of a pseudo-perspective, rendering a color map and a depth map under the pseudo-perspective using a trained main model, extracting global features and local features and incorporating them into the feature database, constructing a 2D-3D correspondence relationship in combination with the target color map, performing dynamic inlier screening and pose solving, and obtaining a positioning result. The present invention intelligently supplements the perspective loss of key areas, effectively improves the balance and coverage of feature expression in three-dimensional scenes, realizes dynamic updating of 2D-3D matching relationships and elimination of false matches, thereby significantly improving positioning accuracy and system stability, and having stronger robustness and practical usability under sparse training conditions.

The present invention relates to the field of computer vision technology, and discloses a long-term visual positioning method based on an association guidance network. The model training stage includes perceptual network training and concept network training; after the concept network training is completed, the concept network is used as a pre-training model to train the perceptual network, and the concept network is optimized at the same time; the association guidance mechanism performs information interaction; in the image retrieval stage, the trained perceptual network is used to perform image retrieval. The invention uses domain adaptive learning and an association guidance mechanism to use the features in the concept network to guide the feature learning in the perceptual network, so that the features finally obtained by the model training are robust under environmental changes. In addition, the present invention uses a concept database that does not require additional data to perform self-inspirational learning on the concept network, so as to better guide the perceptual network with domain features, thereby improving the final image retrieval performance.

The invention relates to an efficient visual positioning method under conservation of a multi-modal large model dialogue capability, which is characterized in that an adopted positioning network D-LMM comprises a multi-modal large model LMM image encoder CLIP, a multi-modal large model LMM text encoder, a frozen LLM and an instance-level text feature extractor. A dynamic instance feature modulation module DIFM and a fusion segmentation mask head are adopted, and for given images and texts, the process of positioning by using the D-LMM comprises the following steps: inputting the images into an image encoder CLIP to obtain multilayer features; splicing the input visual features and the input text features together; inputting the output text features of the LLM into an instance-perceived text feature extractor ITE to obtain instance-perceived text features; and inputting the average text feature of each instance and the multi-level visual features obtained from the image encoder into a dynamic instance feature modulation module DIFM, and converting the multi-level visual features into a multi-level feature map perceived by the instances.

This application provides a multimodal visual localization method, apparatus, and electronic device, which can be applied to the field of computer vision technology. The method includes: responding to image data acquired by an image acquisition device mounted on a vehicle at a target time, extracting features from the image data to obtain image features; predicting the depth features at the target time based on the expected pose transformation of the image acquisition device and historical depth features to obtain predicted depth features; if the image data also includes a depth image, performing noise reduction processing on the predicted depth features according to the mapping relationship between the depth features and the predicted depth features to obtain optimized depth features; fusing the optimized depth features and visual features to obtain multimodal visual features; and localizing the image acquisition device based on the multimodal visual features to obtain the localization result at the target time.

The present invention discloses a spatial vision positioning method based on the design of a two-color rectangular target. The method includes: designing a two-color rectangular target pattern; performing recognition processing on the two-color rectangular target pattern by the contour corner point ellipse extraction method to obtain the target rectangular corner points in the two-color rectangular target pattern; based on the target rectangular corner points in the two-color rectangular target pattern, performing global numbering positioning processing by the sorting corner point numbering method to obtain the spatial positioning of the two-color rectangular target. The present invention can meet the requirements of spatial positioning and long-distance positioning with the help of the target in a dark environment through a simple target pattern composed of two-color target rectangles. As a spatial vision positioning method based on the design of a two-color rectangular target, the present invention can be widely applied to the field of machine vision technology.

This application discloses a map-free visual localization method, apparatus, device, and medium. The method includes acquiring scene images collected in real time by a robot and extracting an initial feature map from the scene images; enhancing the initial feature map into a voxel feature map using a sparse kernel self-attention mechanism; aggregating the voxel feature map from several dimensions and multiplying the aggregated features element-wise to obtain image feature identifiers; searching for at least one anchor point feature identifier based on the image feature identifiers; and determining a six-degree-of-freedom pose based on the set of anchor point feature identifiers. This application utilizes a sparse kernel self-attention mechanism to enhance the initial feature map using voxels and aggregates the voxel feature map from several dimensions, achieving cross-domain interactive sparse quantization. This overcomes feature interference caused by motion blur or illumination changes, improves the robustness of scene image feature extraction, thereby reducing the probability of matching failure or inaccurate matching results and improving the accuracy of map-free visual localization.