97 results about "Visual localization" patented technology

The invention relates to the technical field of artificial intelligence and computer vision, and provides a visual perception method and system based on a multi-modal thinking tree in order to solve the problem that a traditional expansion strategy of purely increasing the parameter scale cannot effectively break through the semantic refinement bottleneck. The visual perception method based on the multi-modal thinking tree comprises the steps of obtaining a to-be-processed original image and a target anaphora text, and constructing the multi-modal thinking tree; defining a reasoning action set for driving node extension; executing a multi-mode Monte Carlo tree search process; iteratively generating a reasoning path until a preset search depth is reached or a termination condition is triggered; all effective leaf nodes generated in the searching process are obtained at the same time; and carrying out aggregation optimization on all effective leaf nodes by adopting a regional feature weighted voting mechanism, and screening out a candidate scheme with the highest comprehensive weight as a final visual perception positioning result. According to the method, the perception performance can be effectively improved on the basis of not changing the original parameter scale of the model, and high-precision visual positioning is realized.

The invention relates to the technical field of machining path planning, in particular to a graphite micropore femtosecond laser machining path planning method based on AI visual localization, which comprises the following steps: S1, acquiring an image of the surface of a graphite workpiece through an industrial camera, and preprocessing the image; s2, processing the image based on an AI visual identification model, and extracting microscopic features of the graphite surface and position coordinates of a preset processing area; s3, based on the extracted feature coordinates and preset micropore design parameters, an initial path of femtosecond laser processing is generated through a path optimization algorithm; and S4, the optimized machining path is converted into a laser galvanometer control instruction, and femtosecond laser is driven to complete micropore machining. According to the method, on the basis of path optimization of the improved grey wolf algorithm, through dynamic weight adjustment, in a multi-taboo area scene, the idle running time of the laser galvanometer is effectively shortened, and the machining efficiency is improved.

According to one embodiment of the present disclosure, A visual localization method comprises generating at least two or more anchor points from three-dimensional point clouds; generating three-dimensional ray clouds by connecting three-dimensional points included in the three-dimensional point clouds with one of the generated anchor points; extracting feature points of an input image; and clustering a plurality of lines included in the three-dimensional ray clouds based on the at least two or more anchor points, sampling two ray cloud clusters out of the clustered ray cloud clusters, and estimating a pose of a camera that captured the input image based on the sampled ray cloud clusters and the feature points.

The invention discloses an explanatory visual question-answering method and system based on question perception and confidence constraint. The system comprises a selection-enhancement module and a fusion enhancement confidence constraint module, the selection-enhancement module is used for selecting candidate image areas based on question semantics and enhancing salient areas related to questions through a learnable enhancement mechanism to realize accurate visual localization; and the fusion enhancement confidence constraint module is used for fusing the enhanced regional features and the multi-modal features, and ensuring that the confidence of a prediction answer is improved when explanation information is introduced through a double-branch prediction and confidence constraint mechanism, so that the reliability of a prediction result is enhanced. According to the method, the defects of problem insensitive positioning and positioning-reasoning disjunction in the prior art can be effectively overcome, the superiority of the method is verified on a public data set, the answer prediction accuracy and the explanation generation quality are remarkably improved, and the method has wide application prospects.

The invention belongs to the technical field of artificial intelligence, and particularly relates to a multi-modal large model illusion detection method based on reverse visual positioning. The method comprises the following steps: constructing a visual instruction fine tuning data set rich in context; training a visual positioning large model with pixel-level positioning and rejection capability based on the data set; performing sentence-by-sentence verification on a response generated by a to-be-detected multi-modal large model by using the trained model, and judging whether illusion exists or not by judging whether text description can be reversely positioned back to image pixels or not; according to the method, illusion rich in details can be effectively detected, pixel-level masks and natural language interpretation are provided, and the accuracy and transparency of evaluation are remarkably improved.

A geometric structure aided visual localization method and a system implementing the method are provided. The method includes retrieving a 3D map of initialization location data, the 3D map being constrained by visual structures and geometric structures and modeled by a Gaussian mixture model of a set of Gaussian distributions with mapped landmarks; acquiring a series of real-time image frames by a camera of a mobile system; and for each real-time image frame: extracting local features from the real-time image frame; predicting a camera pose corresponding to the real-time image frame; creating a key frame by tracking the predicted camera pose in the 3D map; identifying temporally visible landmarks with respect to the created key frame; acquiring 2D-3D correspondences between the local features and the temporally visible landmarks; and localizing the mobile system by estimating a state of the real-time image frame based on the 2D-3D correspondences.

This invention relates to a multimodal visual understanding method based on consistency learning and hybrid feature extraction. It includes constructing an end-to-end fine-grained consistency learning framework, introducing a hybrid region extractor, fusing local details and global semantics to generate high-quality hybrid visual cue embeddings, combining self-reconstruction loss and latent spatial consistency loss to force the model to establish explicit alignment between the input visual cue and the output segmentation label, utilizing the geometric boundary constraints of the localization task for description generation, and simultaneously optimizing localization accuracy using the semantic depth of the description task. Furthermore, it constructs a detailed localization index expression and segmentation task to enhance the model's reasoning ability for complex long text instructions. The aim is to address the problems of feature fragmentation and insufficient accuracy in existing large models for fine-grained visual localization and description tasks. Compared with existing technologies, this invention has advantages such as high accuracy and strong generalization ability in pixel-level localization and fine-grained description.

The present invention relates to the field of computer vision technology, and discloses a long-term visual positioning method based on an association guidance network. The model training stage includes perceptual network training and concept network training; after the concept network training is completed, the concept network is used as a pre-training model to train the perceptual network, and the concept network is optimized at the same time; the association guidance mechanism performs information interaction; in the image retrieval stage, the trained perceptual network is used to perform image retrieval. The invention uses domain adaptive learning and an association guidance mechanism to use the features in the concept network to guide the feature learning in the perceptual network, so that the features finally obtained by the model training are robust under environmental changes. In addition, the present invention uses a concept database that does not require additional data to perform self-inspirational learning on the concept network, so as to better guide the perceptual network with domain features, thereby improving the final image retrieval performance.

The invention relates to an efficient visual positioning method under conservation of a multi-modal large model dialogue capability, which is characterized in that an adopted positioning network D-LMM comprises a multi-modal large model LMM image encoder CLIP, a multi-modal large model LMM text encoder, a frozen LLM and an instance-level text feature extractor. A dynamic instance feature modulation module DIFM and a fusion segmentation mask head are adopted, and for given images and texts, the process of positioning by using the D-LMM comprises the following steps: inputting the images into an image encoder CLIP to obtain multilayer features; splicing the input visual features and the input text features together; inputting the output text features of the LLM into an instance-perceived text feature extractor ITE to obtain instance-perceived text features; and inputting the average text feature of each instance and the multi-level visual features obtained from the image encoder into a dynamic instance feature modulation module DIFM, and converting the multi-level visual features into a multi-level feature map perceived by the instances.

This application provides a multimodal visual localization method, apparatus, and electronic device, which can be applied to the field of computer vision technology. The method includes: responding to image data acquired by an image acquisition device mounted on a vehicle at a target time, extracting features from the image data to obtain image features; predicting the depth features at the target time based on the expected pose transformation of the image acquisition device and historical depth features to obtain predicted depth features; if the image data also includes a depth image, performing noise reduction processing on the predicted depth features according to the mapping relationship between the depth features and the predicted depth features to obtain optimized depth features; fusing the optimized depth features and visual features to obtain multimodal visual features; and localizing the image acquisition device based on the multimodal visual features to obtain the localization result at the target time.

This application discloses a map-free visual localization method, apparatus, device, and medium. The method includes acquiring scene images collected in real time by a robot and extracting an initial feature map from the scene images; enhancing the initial feature map into a voxel feature map using a sparse kernel self-attention mechanism; aggregating the voxel feature map from several dimensions and multiplying the aggregated features element-wise to obtain image feature identifiers; searching for at least one anchor point feature identifier based on the image feature identifiers; and determining a six-degree-of-freedom pose based on the set of anchor point feature identifiers. This application utilizes a sparse kernel self-attention mechanism to enhance the initial feature map using voxels and aggregates the voxel feature map from several dimensions, achieving cross-domain interactive sparse quantization. This overcomes feature interference caused by motion blur or illumination changes, improves the robustness of scene image feature extraction, thereby reducing the probability of matching failure or inaccurate matching results and improving the accuracy of map-free visual localization.

The present invention relates to a computing device for supporting a mobile device in performing visual localization in a target environment. The computing device is configured to obtain and modify one or more 3D models of the target environment to generate a set of 3D models. Then, the computing device is configured to determine a training sample set from the set of 3D models. Each training sample comprises an image derived from the set of 3D models and corresponding pose information. Then, the computing device is configured to train a neural network model from the training sample set such that the trained neural network model is configured to predict pose information for an image captured in the target environment and output a confidence value associated with the predicted pose information.

The invention discloses a visual localization and attitude estimation method and device based on prior search in a rocket recovery section and a medium, and belongs to the technical field of spacecraft guidance, navigation and control, and the method comprises the steps: carrying out the systematic sampling of a rocket recovery tail end operation space containing a three-dimensional position and a three-dimensional camera attitude in advance; constructing a matching data set in which the rocket recovery state is matched with the image; learning mapping from an image to a low-dimensional feature vector by using a deep learning model, and enabling the structure of a feature space to be consistent with the structure of a physical state space through a designed loss function; and in the rocket recovery stage, pre-stored samples closest to real-time image features are retrieved, and state labels of the pre-stored samples are fused, so that pose estimation is realized. The method can adapt to complex three-dimensional attitude changes, is high in robustness, and can meet the real-time requirement.

Provided are a visual positioning method, a non-transitory computer-readable storage medium and an electronic device. Surface normal vectors of a current image frame is obtained. A first transformation parameter between the current image frame and a reference image frame is determined, by projecting the surface normal vectors to a Manhattan coordinate system. A matching operation between feature points of the current image frame and feature points of the reference image frame is performed, and a second transformation parameter between the current image frame and the reference image frame is determined based on a matching result. A target transformation parameter is obtained, based on the first transformation parameter and the second transformation parameter. A visual localization result corresponding to the current image frame is output, based on the target transformation parameter.

The invention discloses a large-scale visual positioning optimization method based on an OCP theory, and relates to the technical field of deep learning. The method comprises the following steps: defining a target function of a visual positioning model; initializing model parameters, momentum vectors and related hyper-parameters; performing iterative optimization training, calculating a small-batch stochastic gradient in each iteration, performing exponential moving average and deviation correction on the gradient by using a diagonal element of an approximate Hessian matrix of element-by-element square of the gradient, performing weight attenuation, and finally calculating a parameter update quantity and updating a model by using an optimization method based on an OCP theory; and outputting the model with the optimal performance on the verification set after iteration is finished. According to the method, the OCP theory and approximate second-order information are combined, a new large-scale visual positioning method is provided, the convergence speed, stability and final test precision of visual positioning model training are effectively improved on the premise that linear complexity is kept, and the method is particularly suitable for large-scale non-convex optimization scenes.

The invention belongs to the technical field of image analysis, and particularly relates to an AGV and turnover equipment docking method and system based on visual localization, and the method comprises the steps: calculating a specular reflection inhibition factor based on the gray value of a pixel point in an image and a neighborhood gray variance, a comprehensive matching cost function is obtained according to a geometric distance between a model projection point and a pixel point of a 3D model of the turnover device, an included angle cosine between an expected 2D normal direction and a gradient unit normal vector and a specular reflection suppression factor, a pixel point with the minimum comprehensive matching cost is searched in a local search neighborhood of the model projection point to serve as a matching point, and the matching point is matched with the model projection point. And updating the pose, and when the number of the effective matching points is smaller than a preset threshold value, switching to a dead reckoning mode based on the last effective pose. According to the method, the problem of unstable positioning caused by environmental interference in a complex industrial environment is solved, and the docking precision and robustness are improved.

The invention relates to the technical field of visual SLAM positioning and mapping, and discloses a visual map data processing method and device, computer equipment and a storage medium. Firstly, panoramic semantic segmentation is carried out on an original image, a panoramic segmented image is generated, the panoramic segmented image comprises semantic information of each pixel, and the semantic information comprises prior dynamic semantic information. And performing optical flow estimation processing on the original image to obtain an optical flow image. Then, based on the panoramic segmented image, the optical flow image and the original image, feature points of a dynamic area are removed in real time, and a static feature image with higher precision is obtained; and finally, under the condition of performing motion tracking based on the static feature image and determining that the static feature image is a key frame, generating a dense point cloud map endowed with semantic information based on the original image, the static feature image and the panoramic segmentation image, and removing a point cloud with prior dynamic semantic information from the dense point cloud map. And a more accurate static environment map is constructed to update the point cloud map.

This application provides a vehicle pure vision localization method based on a scene library, relating to the field of vision localization. The method includes: performing early vision localization of the vehicle using a high-precision map to obtain an optimized image pose; performing quality detection on the image pose based on a deep learning model, and storing qualified image poses into a scene library; recalling matching historical images from the scene library based on the image content and rough position information of the current onboard camera image; and using the recalled historical images to assist or replace the high-precision map for pose estimation to obtain the vision localization result. The technical solution of this application forms a data closed loop in which image data accumulation and localization accuracy mutually promote each other, enabling the sustainable operation of the vision localization process.

The utility model relates to the technical field of laser marking, and provides a laser marking machine with visual localization, which comprises a laser marking machine main body, two connecting plates, a visual localization device and a visual localization device, the center of one side of the outer surface of the top of the laser marking machine main body is fixedly provided with a supporting column, and one side of the supporting column is provided with a marking mechanism; the plurality of sensors are fixedly mounted on the outer surface of the marking mechanism; two electric push rods are started at the same time to drive an arc-shaped plate to move up and down, then a draw-off pump is started, harmful gas generated during marking enters an arc-shaped pipe through a plurality of gas suction hoods, and the arc-shaped pipe conveys the gas into a first telescopic hose and then conveys the gas into a second telescopic hose and a third telescopic hose. And finally, the waste gas enters a waste gas purifier through a connecting pipe to be purified, a first telescopic hose, a second telescopic hose and a third telescopic hose can be shrunk and adjusted, adjustment is conducted according to a marking mechanism and an electric push rod, and gas generated during marking is better extracted and purified.

This invention discloses a method and system for continuous visual localization of unmanned aerial vehicles (UAVs) based on satellite remote sensing imagery. The method comprises: a front-end visual tracking module calculating relative positioning in pixel coordinates based on image matching feature point sets, and then transferring the result to UTM coordinates; when a set number of frames are accumulated or visual image matching fails, a back-end satellite imagery absolute localization module uses UAV-captured images and satellite remote sensing imagery to perform absolute localization and pose correction for the UAV; a back-end forward correction module performs forward correction based on the absolute localization information obtained from the back-end satellite imagery absolute localization module, eliminating the accumulated error of the front-end visual tracking result. The system includes a front-end visual tracking module, a back-end satellite imagery absolute localization module, and a back-end forward correction module. This invention improves the positioning accuracy of UAVs using satellite remote sensing imagery, features a simple algorithm, strong real-time performance, and enables continuous visual localization of UAVs.

This application discloses a fusion localization method, apparatus, and electronic device for autonomous vehicles. The method includes: acquiring localization data from multiple sensors of the autonomous vehicle, including satellite localization data, laser localization data, and visual localization data; determining the confidence level type corresponding to the satellite localization data and the laser localization data using pre-set confidence threshold conditions; determining a fusion localization strategy for the autonomous vehicle based on the confidence level types of the satellite localization data, the laser localization data, and the visual localization data; and performing fusion localization according to the fusion localization strategy to obtain the fusion localization result of the autonomous vehicle. This application performs mutual verification of confidence levels based on localization data from multiple sensors and adopts different fusion localization strategies based on the verification results, ensuring that the fusion localization algorithm has reliable observation input and improving the stability and accuracy of fusion localization.

The invention discloses a large component flatness automatic measurement system based on high-precision real-time visual localization, which relates to the technical field of automatic measurement and comprises a measurer reference building module, a measurement area planning module, an image acquisition and analysis module and a flatness analysis module. The method comprises the following steps: firstly, acquiring spatial state data of a target large component through a measurer reference establishment module, and establishing an actual measurement platform in combination with historical use data of a measurement platform in a database; the measurement area planning module collects component space measurement data, divides a measurement area and generates a platform movement track; the image acquisition and analysis module acquires pictures according to the motion trail, generates a re-detection scheme through analysis, and acquires effective use pictures; and the flatness analysis module calculates the flatness based on the used picture and carries out early warning, so that the automation and precision of the flatness measurement of the large component are improved, and the measurement efficiency and reliability are improved.

This invention discloses a visual localization method based on pose space texture continuity mapping, belonging to the fields of robotics and computer vision. In the mapping stage, image intrinsic parameters, pose, and sparse point cloud are first obtained through sparse reconstruction. Image features are extracted using a pre-trained model, and pose encoding and decoding models are trained to ensure consistency between pose encoding and image features in spatial metrics. Simultaneously, the decoding model outputs a multi-peak distribution to model pose ambiguity. Subsequently, a mapping model from image features to pose encoding is trained and optimized end-to-end to construct a scene map. In the localization stage, features of the image to be localized are extracted, pose encoding is obtained through the mapping model, and the pose distribution is output by the decoding model. A unique location or multiple candidate poses are determined based on the number of peaks in the distribution. This invention effectively improves the robustness and accuracy of robot localization in repetitive texture scenes by maintaining the continuity alignment between the pose space and texture space.

This application provides a method, system, device, and storage medium for visual localization of pathological images, belonging to the field of image recognition technology. The method includes: extracting visual features based on a target pathological image; determining semantic feature vectors and knowledge feature vectors based on a first text description; the target pathological image is the pathological image for which target region localization is to be performed; the knowledge feature vectors are used to represent knowledge information associated with the content of the target pathological image; fusing the semantic feature vectors and knowledge feature vectors to obtain fused text features; performing cross-modal fusion of the fused text features and visual features to obtain fused multimodal features; obtaining a fused representation based on the fused multimodal features; and, based on the fused representation, locating the target region in the target pathological image using a multilayer perceptron to obtain the position information of the bounding box of the target region. This application can improve the ability to accurately and flexibly locate regions at the pathological image level.

A computing device is provided for supporting a mobile device to perform a visual localization in a target environment. The computing device is configured to obtain and modify one or more 3D models of the target environment to generate a set of 3D models. The computing device is further configured to determine a set of training samples based on the set of 3D models. Each training sample comprises an image derived from the set of the 3D models and corresponding pose information. The computing device is further configured to train a neural network model based on the set of training samples, so that the trained neural network model is configured to predict pose information for an image captured in the target environment and to output a confidence value associated with the predicted pose information.

Provided are systems and methods of inverting a trained NeRF model, where the NeRF model stores the structure of a scene or object to estimate 6D poses from images taken at new viewpoints. 6D pose estimation has a wide range of applications, including visual localization and object pose estimation for robotic manipulation.

The application provides a sparse-to-dense visual positioning method and system based on feature Gaussian splash, and the method comprises the following steps: initializing a color-decoupled feature Gaussian field based on a training image set, optimizing the color-decoupled feature Gaussian field based on a query feature map set in combination with feature rendering and feature alignment loss cyclic optimization, and outputting a compact feature Gaussian scene model; screening a Gaussian landmark set in the compact feature Gaussian scene model by using a matching-oriented sampling strategy; training a scene-specific detector; extracting sparse local features of a query landmark heat map corresponding to a query image and performing sparse feature matching with the Gaussian landmark set to obtain an initial pose of a query perspective camera; based on 3D Gaussian splash, rendering a dense feature map and a depth map of the query perspective in the compact feature Gaussian scene model by using the initial pose of the query perspective camera, performing cluster-based proxy matching-based sparse-to-dense accelerated pose optimization, and obtaining accurate positioning of the query perspective camera.

The present application relates to a kind of multimodal visual positioning methods based on loss balance training, comprising the following steps: step S1: the text description and image in the public data set obtained are preprocessed, then input consistency measurement module obtains consistency score and first multimodal feature;Step S2: first multimodal feature is input consistency migration module and is handled to obtain second multimodal feature;Step S3: second multimodal feature is used to associate mask decoder regression and predicts the object in the coordinate bounding box of image in text description reference;Step S4: cross-entropy loss and image-text contrast loss are added in the neural network cascaded in S1-S3, and the loss of consistency score balance model training is balanced, and the coordinate bounding box of inference model after training is completed text description reference object detection and positioning.

The invention discloses a dynamic screen area monitoring and self-adaptive character extraction system based on visual localization, which relates to the technical field of dynamic screen character perception, and comprises the following steps: acquiring an original image frame of an electronic screen, carrying out resolution adaptation, automatically identifying a target area and outputting coordinates; executing character recognition; detecting a target area content change of the target area image; and driving the extraction system to re-execute the positioning-extraction process. According to the method, dynamic screen area monitoring and adaptive character extraction without manual intervention are realized, the manual operation cost and the system maintenance burden are reduced, and high-reliability and high-adaptive intelligent screen sensing capability is provided for application scenes such as software automatic testing, user behavior analysis, content security auditing and barrier-free assistance.