115 results about "Visual odometry" patented technology

The invention discloses a multi-camera radar-inertia-visual odometer method fusing a sonar image, and belongs to the technical field of robot perception and navigation. According to the method, environment perception is realized through cooperative work of multiple sensors; in the air, a multi-camera system and a conventional laser radar are taken as main modes; in water, an active imaging sonar and an underwater laser radar are used as main modes. The system detects the medium of the aircraft according to the resistivity, dynamically switches the main mode and fuses the auxiliary mode information. Through a tight coupling optimization framework, observation data of vision, sonar, laser radar and an inertial measurement unit are uniformly modeled as residual constraints, a nonlinear least square problem is constructed and solved, and efficient fusion of multi-source information is realized. The method has robustness in low-texture, low-illumination and underwater turbid environments, and the positioning precision and stability in a cross-medium scene are remarkably improved.

The invention relates to the technical field of data processing, in particular to a laser and binocular camera-based position positioning system, which comprises an acquisition unit, a processing unit, a fusion unit, an abnormality judgment unit, an adjustment unit and a correction unit. According to the method, multi-source information such as a laser point cloud matching result, binocular visual odometer output and the number of feature points is subjected to joint modeling in an extended Kalman filter, so that the AGV obtains stable pose estimation in different environments, and the pose estimation accuracy is improved by analyzing the change trend of pose covariance in an adjusted monitoring window. A density threshold value, an information entropy threshold value and a dynamic proportion threshold value are automatically corrected, so that the threshold values can adapt to real working conditions for a long time, and performance degradation caused by a fixed threshold value is avoided; the problems that due to the fact that sensor information quality fluctuation cannot be recognized and processed, positioning precision is lowered, accumulative errors are increased, and positioning loss is prone to occurring in a complex environment are effectively solved.

A computer that includes a processor and a memory, the memory including instructions executable by the processor to select first side stereo images of first and second pairs of stereo images acquired at first and second time steps, respectively, and mask first and second first side stereo images, determine point features in masked first and second first side stereo images and determine line features in masked first and second first side stereo images. Matching point features in the masked first and second first side stereo images can be determined using a first attention / graph neural network (attn / GNN) based on keypoints determined based on the line features. Matching line features in the masked first and second first side stereo images can be determined using a second attn / GNN based on keypoints determined based on the line features. Three-dimensional (3D) locations in a scene can be determined by determining stereo disparity based on the matched point features included in the first first side stereo image and point features determined in a first second side stereo image of first and second stereo pairs of images and a 3D stereo camera pose can be determined by determining a perspective-n-point and line algorithm on the 3D locations.

The invention belongs to the technical field of multi-source fusion positioning, and particularly relates to a multi-source fusion positioning method based on LSTM-KF, a program, equipment and a storage medium. According to the method, a long short-term memory (LSTM) neural network is embedded into a Kalman filtering framework, nonlinear error compensation of system state prediction and observation updating is achieved, and then an improved Kalman filter with the dynamic noise adaptive capacity is constructed. By loosely coupling GPS satellite positioning, IMU inertial measurement and VO visual odometer multi-source heterogeneous sensing data, the millimeter-level precision of robot pose estimation in a complex dynamic environment and the anti-interference capability of the system are remarkably improved.

Provided are systems and techniques for automated navigation of vehicles, such as drones. The systems generally include processing unit(s) that, collectively, perform several steps. Such steps include generating metric depth estimates, using a pre-trained model, for each pixel in received image(s) from a monocular camera, or transformed image(s) based on the received image(s). Such steps may also include generating a pose estimate from visual odometry, then generating a truncated signed distance function representation of an environment based on the absolute depth estimates and the pose estimate. The steps may include creating and / or updating a local map based on the truncated signed distance function representation. The steps may include plan a collision-free route towards a goal based on the local map. This may include using motion primitives, which may be generated in a single offline step and stored in a trajectory library.

Eyewear devices that include two SoCs that share processing workload. Instead of using a single SoC located either on the left or right side of the eyewear devices, the two SoCs have different assigned responsibilities to operate different devices and perform different processes to balance workload. In one example, the eyewear device utilizes a first SoC to operate the OS, a first color camera, a second color camera, a first display, and a second display. A second SoC is configured to run computer vision (CV) algorithms, visual odometry (VIO), tracking hand gestures of the user, and providing depth from stereo. This configuration provides organized logistics to efficiently operate various features, and balanced power consumption.

The invention provides an unmanned aerial vehicle path planning method and system based on adaptive dynamic planning. The method comprises the following steps: calculating a solar azimuth angle according to an atmospheric polarization mode, compensating course drift of a visual odometer by using the solar azimuth angle, and generating corrected unmanned aerial vehicle position information; according to the illumination invariant edge information, combining a convolutional neural network, segmenting a key landmark in the texture enhanced image, and generating a semantic navigation landmark point sequence in a constraint space of the key landmark by adopting a bidirectional fast random tree; constructing a model prediction controller according to the semantic navigation road sign point sequence; and through the corrected position information, the flight path of the unmanned aerial vehicle is optimized in a rolling manner through adaptive dynamic planning under the constraint of a dynamic window, and the field angle and exposure parameters of the dual-band imaging unit are adjusted. According to the invention, the autonomous navigation precision and semantic understanding capability of the unmanned aerial vehicle in a complex light environment are improved, and anti-interference and high-precision autonomous path planning and flight control are realized.

The invention relates to the technical field of robots, in particular to a visual perception-based real-time abnormal event recognition method for a quadruped robot, which comprises the following steps of: acquiring body motion data and an environment image sequence of the robot in real time through an inertial measurement unit and a visual sensor; predicting and generating a short-term expected movement track based on the data and the control instruction; meanwhile, estimating an actual motion track of the robot by utilizing a visual odometer technology; fusing the deviation value of the expected trajectory and the actual trajectory with the optical flow statistical features extracted from the dense optical flow field to form a multi-modal fusion feature vector; and calculating a real-time abnormal confidence coefficient score by adopting a pre-trained long-short-term memory network model. Multi-mode sensing information is effectively fused, limitation of a single sensor is overcome, high-precision and real-time recognition and classification of abnormities such as foot slipping, collision and terrain abrupt change are achieved, corresponding emergency control strategies can be triggered, and autonomy and reliability of the quadruped robot in a complex environment are improved.

This invention relates to the field of positioning technology, and more particularly to a positioning method and device. The method includes: determining two-dimensional line segments and visible points in each frame of images acquired by a camera, wherein the visible points include the endpoints of the two-dimensional line segments and interior points between the endpoints; fitting a three-dimensional line based on the two-dimensional line segments and determining the collinearity constraints of the two-dimensional line segments; constructing a cost objective function based on the photometric errors of the visible points in each frame of images, the photometric errors of the two-dimensional line segments in each frame of images, and the collinearity constraints; and minimizing the cost objective function to determine the pose parameters of the camera. This invention proposes a direct visual odometry scheme based on points and lines. This scheme extends the photometric error with respect to lines and adds collinearity constraints on points on the lines, thereby significantly reducing the number of variables to accelerate the optimization process and ensuring that the collinearity constraints are satisfied.

The invention discloses a biased binocular visual odometer method and device based on strawberry cuttlefish bionics. The biased binocular visual odometer method comprises the following steps: synchronously acquiring a bright field image and a dark field image collected by a biased binocular camera and inertial data of an inertial measurement unit; constructing a perceptual attention weight grid according to the bright field image and the dark field image, and performing brightness normalization processing on the bright field image and the dark field image; extracting lateral visual features based on the normalized image and the perceptual attention weight grid; and fusing the biased visual features with the inertial data, and solving through nonlinear optimization to obtain a carrier pose estimation result. According to the invention, the dynamic range of the binocular vision system is expanded, and the imaging and sensing capabilities of the system in an HDR scene are improved. And meanwhile, the positioning precision and robustness of the binocular visual odometer in an HDR scene are improved.

The present application belongs to the technical field of mobile robot navigation, and particularly relates to a SLAM navigation method and system for a mobile intelligent cabinet, which comprises the following steps: acquiring a laser point cloud sequence and a grayscale image of the mobile intelligent cabinet and performing data cleaning; obtaining a geometric feature index based on the spatial jump distribution of the laser point cloud sequence; obtaining a visual texture index based on the local dispersion of the image gradient; calculating laser dynamic weight and visual dynamic weight by using the geometric feature index and the visual texture index, weighting and fusing the pose change quantity calculated by the single-line laser radar odometry and the visual odometry to obtain a fused pose quantity; and updating the global state based on the fused pose quantity and driving autonomous navigation. The present application can adjust the sensor weight in real time according to the environmental characteristics, solves the problem of positioning divergence in the long corridor of a shopping mall or a high-reflectivity environment, and improves the robustness of navigation.

The application discloses a kind of intelligent unmanned vehicle environment perception systems based on multi-sensor information fusion, it is related to unmanned vehicle technical field.The system includes: sensor module, for collecting multi-source heterogeneous data, including GPS / IMU, laser radar, camera and millimeter wave radar;Data acquisition and fusion module, through the Kalman filter to multi-source data space-time synchronous fusion;Environment perception module, based on deep learning identification road boundary, obstacle and traffic sign, constructs dynamic perception model;High-precision map construction module, using SLAM technology and semantic information constructs and updates semantic three-dimensional map;Positioning algorithm module, fusion visual odometry, fusion data and high-precision map, calculates vehicle real-time pose;And control module, according to positioning and perception result carries out path planning and navigation control, the application provides a kind of intelligent unmanned vehicle environment perception systems based on multi-sensor information fusion, can be autonomously navigated and stably driven under complex and severe environment.

This invention discloses a dynamic path planning method for mobile robots suitable for farmland environments, comprising: acquiring access information of the work area and constructing a structured navigation map, performing topological modeling to obtain a navigation map containing navigation nodes and navigation edges; classifying and modeling the navigation edges in the navigation map, and calculating cost weights for each navigation edge based on road conditions, visual odometry confidence, and positioning stability; detecting weed targets and outputting two-dimensional candidate regions for weed targets, mapping each weed target to reachable stopping points in the navigation map, outputting a set of candidate targets, and generating relevant work costs for each weed target; establishing energy feasibility constraints, calculating available energy, minimum return energy consumption, and safety redundancy energy, and, under the premise of satisfying the energy feasibility constraints, performing path planning with a total cost function including navigation edge costs and work costs, and outputting a work path covering the processing order of weed targets.

The invention provides a cabin door positioning method and device based on confidence fusion and related equipment, and relates to the technical field of cabin door positioning. The method comprises the steps that motion information of the boarding bridge and environment information of the environment where the boarding bridge is located are acquired; determining a plurality of confidence coefficients according to the motion information and / or the environment information; fusing the plurality of confidence coefficients to obtain a comprehensive confidence coefficient; according to the comprehensive confidence coefficient and a set decision-making mechanism, cabin door feature point coordinate information is obtained based on first coordinate information and / or second coordinate information, the first coordinate information is cabin door feature point coordinate information obtained through a visual odometer, and the second coordinate information is cabin door feature point coordinate information obtained based on a cabin door recognition model. According to the method, the advantages of accurately predicting cabin door coordinate information by a model and estimating cabin door coordinates in real time by visual tracking can be reserved, more accurate cabin door coordinate information can be quickly obtained, and both accuracy and high efficiency are considered.

The invention discloses a pose estimation method and device based on a visual odometer, equipment and a medium, and the method comprises the steps: obtaining a to-be-processed image collected by a camera, and extracting optical flow data of the to-be-processed image based on an optical flow method; according to the optical flow data, performing feature extraction on the optical flow data to-be-processed image through a multi-head attention mechanism and a feature extraction module to obtain a to-be-optimized motion feature corresponding to the optical flow data to-be-processed image; optimizing the to-be-optimized features of the optical flow data according to the motion consistency constraint to obtain target motion features corresponding to the to-be-processed image of the optical flow data; and performing pose fitting based on the optical flow data target operation features to obtain pose estimation corresponding to the optical flow data camera. According to the invention, high-precision estimation of the pose and motion of the camera is realized.

The application belongs to the field of visual simultaneous localization and mapping (VSLAM), and particularly relates to a VSLAM back-end optimization method based on a maximum cross-correlation entropy of a plurality of convex combinations. The method comprises the following steps: sensor information reading: camera image information reading and preprocessing, so as to obtain pixel information of the image; front-end visual odometry: rough camera motion is estimated according to information of adjacent images, initial values are provided for the back-end, it is judged whether each frame is a key frame, and a preliminary local map is established; back-end optimization: camera poses measured by visual odometry at different times and information of loop detection are accepted, and optimization is performed on the camera poses and the information, so that a globally consistent trajectory and map are obtained; loop detection: it is detected and judged whether the robot has arrived at a previous position, if loop detection is detected, information is provided to the back-end for processing, and a global optimization thread is triggered; and mapping: a map corresponding to a task requirement is established according to an estimated trajectory.

The application discloses a kind of wide-area-oriented end edge cloud collaborative computing visual positioning method and system.For the mobile terminal with binocular camera, the computing framework of end edge cloud is used, and the online map established in advance is used to realize the real-time positioning and mapping based on binocular camera.In the end edge cloud framework, the end side undertakes the work of visual odometry and global map optimization using binocular camera.The end side transmits information to the edge computing unit, while the edge computing unit performs image retrieval and global positioning work.Finally, the positioning result calculated by the end edge collaborative computing is sent to the cloud for real-time visualization.The method improves the calculation efficiency of real-time positioning, and also supports the deployment mode of multiple mobile terminals connected to the edge cloud side, estimates and optimizes the positioning information of each terminal through data exchange, to obtain higher management efficiency.

The present disclosure relates to the technical field of vision, and specifically relates to a constraint construction method and apparatus for visual odometry, and a device and a medium. The method comprises: acquiring a parameter of a support plane; acquiring image feature points of at least two frames of images; establishing an image feature point matching relationship between image feature points of different frames of images; and on the basis of the image feature points, the image feature point matching relationship and the parameter of the support plane, generating a visual odometry constraint term. On the basis of the visual odometry constraint term acquired in the solution, the scale can be effectively constrained during visual odometry, thereby ensuring the stability of the scale, and facilitating an improvement in the accuracy of collecting map data on the basis of a visual odometry result.

The application discloses a positioning method based on ground-air cross visual angle visual odometer, and the positioning precision is judged by judging the mathematical features of the probability graph output by the ground-air visual angle positioning network, which is used as the condition for whether to fuse with the internal odometer, so that the inaccurate ground-air teaching positioning result can be removed, the positioning result with higher precision is fused with the internal odometer, and higher precision can be obtained; the ground panoramic image and the off-line acquired air visual angle image are matched and positioned, and the global positioning information can be provided under the condition that the GNSS signal is interfered; the ground-air cross visual angle positioning and the odometer fusion are used, the cumulative error caused by the long-term operation of the internal odometer is eliminated, and the positioning precision is improved.

Eyewear devices that include two SoCs that share processing workload. Instead of using a single SoC located either on the left or right side of the eyewear devices, the two SoCs have different assigned responsibilities to operate different devices and perform different processes to balance workload. In one example, the eyewear device utilizes a first SoC to operate a first color camera, a second color camera, a first display, and a second display. The first SoC and a second SoC are configured to selectively operate a first and second computer vision (CV) camera algorithms. The first SoC is configured to perform visual odometry (VIO), track hand gestures of the user, and provide depth from stereo images. This configuration provides organized logistics to efficiently operate various features, and balanced power consumption.

The application discloses a monocular visual odometer system and method based on bird's-eye view representation, comprising: a visual BEV encoder for extracting image features from monocular images and predicting the depth distribution probability of the features, then fusing the image features with their depth distribution probability and mapping the image features to BEV to obtain bird's-eye view representation, and then performing camera coordinate system to ontology coordinate system conversion and height direction compression to obtain compressed bird's-eye view representation; a correlation feature extraction module for calculating the correlation between the compressed bird's-eye view representations of adjacent two key frame monocular images according to the compressed bird's-eye view representation and generating a correlation volume; and a pose prediction decoder for performing pose prediction of a target object according to the correlation volume. The system and method not only perform well in reducing scale drift and improving motion estimation accuracy, but also can be widely applied to automatic driving, robot navigation and other fields, and help to improve the navigation ability and reliability of autonomous systems in complex environments.

The present invention discloses a monocular visual odometry method and system based on bird's-eye view representation and a differentiable weighted Procrustes solver. The method and system include: constructing a pose estimation model, which can extract and match key points in a bird's-eye view image and perform relative pose estimation through an interpretable weighted Procrustes solver; constructing a loss function based on pose information, optimizing the pose estimation model parameters using the loss function, and then using the pose estimation model with the optimized parameters to estimate the pose information between two frames of monocular images to realize monocular visual odometry. This method can excel in reducing scale drift and exhibit stable performance under different weather and lighting conditions.

The invention discloses an unmanned aerial vehicle-satellite image matching visual positioning method based on particle filtering, which is used for realizing accurate positioning of an unmanned aerial vehicle in a satellite denial environment. The method comprises the following steps: firstly, realizing relative visual odometer positioning through an unmanned aerial vehicle image on the basis of a Light image feature matching algorithm; then, carrying out scattering point sampling according to a relative positioning result, extracting a candidate sub-graph from a satellite remote sensing image, completing absolute positioning of an unmanned aerial vehicle image and a satellite image by using a GIM-dkm feature matching algorithm, and improving cross-domain matching robustness; on the basis, relative positioning and absolute positioning results are input into a particle filtering framework, fusion is realized based on a weighting function, and a stable and accurate final positioning result is output. Experimental results show that the method of the invention effectively combines relative and absolute positioning advantages, and significantly improves the positioning performance of the unmanned aerial vehicle in a satellite denial environment.

The invention discloses a visual inertia mileage calculation method, device and system and a storage medium, and is based on multi-state constraint Kalman filtering (MSCKF). By fusing inertial measurement unit (IMU) data and stereoscopic vision odometer data and integrating an image enhancement module and a gyroscope zero offset correction mechanism, feature matching of the stereoscopic vision odometer can be effectively assisted. By adopting the technical scheme of the invention, the problem of low positioning precision of the robot in a low-light environment in the prior art is solved.

The invention relates to the technical field of image processing, and discloses a visual odometer online optimization method combining frequency domain and spatial domain constraint spatial-temporal characteristics, which comprises the following steps: acquiring an interval frame and an adjacent frame of an image, and extracting a current frame as a key frame; inputting the adjacent frames into a pose estimation network to generate adjacent frame poses; inputting the current frame and the interval frame into a depth estimation network to generate a depth map of the current frame and the interval frame; performing depth back projection and vertical back projection on the depth maps of the current frame and the interval frames to generate BEV views of the current frame and the interval frames; after fast Fourier transform is carried out on BEV views of the current frame and the interval frame, phase analysis is carried out, and the pose of the interval frame is generated; constructing an overall self-supervision loss, and realizing pose constraint on adjacent frames and interval frames; finally, on-line updating is triggered in a self-adaptive mode based on the similarity and the relative pose of the current frame and the latest key frame, and on-line optimization of the visual odometer is completed. According to the method, the positioning stability and robustness of the visual odometer are remarkably improved.

The invention discloses a large-range collaborative navigation method for an extraterrestrial patroller, and relates to the field of computer vision, visual navigation and map splicing. The method comprises the following steps: performing significance analysis based on spectral residual for a right monocular camera, focusing on sight capture of a typical feature region, establishing a visual odometer model of inertial IMU pre-integration, and performing back-end optimization driven by a factor graph; establishing a plurality of extraterrestrial patroller similar region search models based on an improved bag-of-words model, introducing a similarity relative scoring function based on a norm form, setting a relative threshold value of a similarity suppression function, and obtaining an image overlapping region; according to the method, the calculation capability of an airborne computer is considered, a sparse feature point map and patroller pose splicing model is established, complete splicing of overlapped area maps is completed through splicing of small sparse feature point maps, and the splicing precision and the splicing efficiency are improved.

The invention relates to a method for fusing visual odometry with inertial measurement unit (IMU) data of an ego-vehicle (1) with the following steps: - Capturing an environment of the ego-vehicle (1) by means of at least one optical sensor (2) and generating sensor data, wherein the optical sensor (2) is a camera and the sensor data are consecutively captured camera images; - Applying semantic segmentation on the camera images; - Computing a visual odometry by means of optical flow; - Observing a shape and position of a road (F) and / or road elements (4) and deriving a tilt angle between an optical axis of the optical sensor (z) and the road (F); - Estimating a three axis rotation between two consecutive camera images by means of an essential matrix, which was computed through single value decomposition; - Fusing a delta rotation between the two consecutive camera images with accelerometers of the inertial measurement unit, wherein the fusion is conducted using a filter; - Adjusting rotation parameters of the essential matrix and transforming the detected objects (3) based on the fusion of the delta rotation.

The invention discloses an underwater semantic self-adaptive binocular vision odometer method capable of resisting dynamic interference. The method comprises the following steps: firstly, acquiring a synchronous stereo image pair through a binocular camera; then extracting features by using a SuperPoint model, and performing field adaptive fine tuning on the model through an unsupervised learning method based on descriptor consistency so as to improve the detection quality of the model in an underwater low-contrast and fuzzy environment; secondly, introducing a semantic segmentation network to identify and filter unreliable feature points on a dynamic target and an unstructured medium in real time; then, a SuperGlue model is adopted and epipolar constraint is combined to carry out efficient and robust feature matching; in the pose estimation stage, rapid initialization of a real scale is realized by using binocular geometric characteristics, and continuous tracking is carried out through a PnP algorithm; and finally, constructing a graph optimization model in a sliding window at the rear end, and by taking minimization of a re-projection error as a target, jointly optimizing a camera pose and a three-dimensional map point to obtain a globally consistent motion trail.

This invention relates to a biomimetic brain-like synchronous localization and environmental perception method for underwater robots. The invention addresses the problems of poor visual odometry, insufficient robustness, and low accuracy in loop closure detection in existing underwater robot navigation systems. The robot first acquires environmental feature information through sonar sensors and its own motion information through navigation sensors. A local scene template is obtained by processing the sonar data using an acoustic-visual processing method. The sensor data undergoes pre-integration processing, and the processed data is used as input to a convolutional neural network to output motion displacement, thus forming the robot's perception of its own position. Finally, an empirical map integrates the above information, and loop closure detection updates the empirical map, reducing drift during robot movement and completing the construction of the empirical map. This invention belongs to the fields of bionics and motion navigation technology.

The invention discloses a continuous learning visual mileage estimation method for plateau mountainous regions, which comprises the following steps: acquiring an image sequence, and obtaining continuous multi-frame image data; constructing a batch image database, and screening samples according to a diversity strategy to improve the diversity of the samples to the greatest extent; the online continuous learning module continuously learns attitude transformation between image sequences online by taking an unsupervised monocular depth estimation network as a branch task; the real-time visual mileage reasoning application module is used for continuously carrying out visual mileage calculation according to network output and providing mileage input of downstream tasks, and by adopting the continuous learning visual mileage estimation method for the plateau mountain land, the adaptability of visual mileage estimation to unknown and changeable environments such as the plateau mountain land can be improved.