156 results about "Visual odometry" patented technology

The invention discloses a multi-camera radar-inertia-visual odometer method fusing a sonar image, and belongs to the technical field of robot perception and navigation. According to the method, environment perception is realized through cooperative work of multiple sensors; in the air, a multi-camera system and a conventional laser radar are taken as main modes; in water, an active imaging sonar and an underwater laser radar are used as main modes. The system detects the medium of the aircraft according to the resistivity, dynamically switches the main mode and fuses the auxiliary mode information. Through a tight coupling optimization framework, observation data of vision, sonar, laser radar and an inertial measurement unit are uniformly modeled as residual constraints, a nonlinear least square problem is constructed and solved, and efficient fusion of multi-source information is realized. The method has robustness in low-texture, low-illumination and underwater turbid environments, and the positioning precision and stability in a cross-medium scene are remarkably improved.

The invention relates to the technical field of data processing, in particular to a laser and binocular camera-based position positioning system, which comprises an acquisition unit, a processing unit, a fusion unit, an abnormality judgment unit, an adjustment unit and a correction unit. According to the method, multi-source information such as a laser point cloud matching result, binocular visual odometer output and the number of feature points is subjected to joint modeling in an extended Kalman filter, so that the AGV obtains stable pose estimation in different environments, and the pose estimation accuracy is improved by analyzing the change trend of pose covariance in an adjusted monitoring window. A density threshold value, an information entropy threshold value and a dynamic proportion threshold value are automatically corrected, so that the threshold values can adapt to real working conditions for a long time, and performance degradation caused by a fixed threshold value is avoided; the problems that due to the fact that sensor information quality fluctuation cannot be recognized and processed, positioning precision is lowered, accumulative errors are increased, and positioning loss is prone to occurring in a complex environment are effectively solved.

The invention discloses a visual odometer method fusing a dynamic object detection network and geometric constraints, and relates to the technical field of visual odometers. According to the method, the interference of the dynamic object on the visual odometer is eliminated through effective identification of the dynamic object detection network and combination with a geometric method, so that the problems of feature point matching error and trajectory drift caused by the dynamic object are avoided, and the system can still keep high precision in dynamic scenes such as dense pedestrians and busy vehicles; the dynamic object detection network adopts a lightweight deep learning model and has efficient and real-time detection capability, and meanwhile, the modular design of the system enables the system to have good expandability, and the performance can be further improved in combination with sensors such as a laser radar; after the dynamic object is removed, map construction is based on high-quality static feature points, so that ghosting and false features caused by the dynamic object are avoided, the generated map is more accurate and more consistent, and subsequent positioning and path planning are facilitated.

A non-transitory computer readable medium storing a computer model is described, where the model includes: an encoder module configured to encode first and second images into first and second representations, respectively, the first and second images being from consecutive frames from video; a first decoder module configured to decode the first and second representations and generate first and second depth maps for the images based on the first and second representations, respectively; and a second decoder module configured to determine a six degree of freedom pose translation of a camera that captured the video based on the first and second representations.

Images captured by a camera moving in an environment are received, and for each of a plurality of points in one of the images, outputs are computed using a neural network. The outputs comprise: a trajectory depicting the point in each of the plurality of images, as well as, for each trajectory, a prediction of visibility of the trajectory in each of the images and a prediction of whether the trajectory depicts a static or moving surface in the environment. The neural network receives the images and points as input and computes the outputs, wherein the outputs comprise for each of the trajectories, confidence data. The outputs are sent to a downstream process selected from any of: visual odometry, structure from motion, human body tracking, video editing, vehicle tracking.

Systems and methods for self-supervised learning for visual odometry using camera images, may include: estimating correspondences between keypoints of a target camera image and keypoints of a context camera image; based on the keypoint correspondences, lifting a set of 2D keypoints to 3D, using a neural camera model; and projecting the 3D keypoints into the context camera image using the neural camera model. Some embodiments may use the neural camera model to achieve the lifting and projecting of keypoints without a known or calibrated camera model.

A computer that includes a processor and a memory, the memory including instructions executable by the processor to select first side stereo images of first and second pairs of stereo images acquired at first and second time steps, respectively, and mask first and second first side stereo images, determine point features in masked first and second first side stereo images and determine line features in masked first and second first side stereo images. Matching point features in the masked first and second first side stereo images can be determined using a first attention / graph neural network (attn / GNN) based on keypoints determined based on the line features. Matching line features in the masked first and second first side stereo images can be determined using a second attn / GNN based on keypoints determined based on the line features. Three-dimensional (3D) locations in a scene can be determined by determining stereo disparity based on the matched point features included in the first first side stereo image and point features determined in a first second side stereo image of first and second stereo pairs of images and a 3D stereo camera pose can be determined by determining a perspective-n-point and line algorithm on the 3D locations.

The invention belongs to the technical field of multi-source fusion positioning, and particularly relates to a multi-source fusion positioning method based on LSTM-KF, a program, equipment and a storage medium. According to the method, a long short-term memory (LSTM) neural network is embedded into a Kalman filtering framework, nonlinear error compensation of system state prediction and observation updating is achieved, and then an improved Kalman filter with the dynamic noise adaptive capacity is constructed. By loosely coupling GPS satellite positioning, IMU inertial measurement and VO visual odometer multi-source heterogeneous sensing data, the millimeter-level precision of robot pose estimation in a complex dynamic environment and the anti-interference capability of the system are remarkably improved.

A strip-shaped steel material grabbing control method of an unmanned steel grabbing machine based on machine visual perception comprises the steps that firstly, pose information of the steel grabbing machine and grabbing point position information of a strip-shaped steel material are obtained based on an image obtained by image collecting equipment; then, the grabbing action of an end effector of the steel grabbing machine is controlled according to the two pieces of information. Wherein the pose of the unmanned steel grabbing machine is estimated in real time through a method of combining a depth camera with a visual odometer; a depth camera is used for collecting a steel material scene image, spatial features of a steel material are obtained, and then a grabbing point and a direction vector are calculated; and an initial transformation relation between the sensing equipment and a coordinate system of the steel grabbing machine is established, the pose of the steel grabbing machine is dynamically updated, real-time coordinate transformation of a grabbing point is achieved, the mechanical arm unfolding length and the mechanical claw rotating angle of the end effector are controlled according to the converted grabbing point and the direction vector, and automatic grabbing is completed. According to the unmanned grabbing device, the unmanned grabbing efficiency and precision of the strip-shaped steel materials are remarkably improved.

The invention discloses a visual inertial odometer method and system based on adaptive decision, and relates to the technical field of visual inertial odometers, the specific steps are as follows: obtaining original image data and original IMU data, and constructing a data set containing synchronous image data and IMU data, the original image data being collected based on a visual sensor, and the original IMU data being collected based on a visual sensor; the original IMU data is collected based on an IMU sensor; inputting the image data and the IMU data into a trained visual odometer network model to respectively extract optical flow features and inertial features, performing modal selection by using a dynamic modal selection mechanism, and performing selection or fusion processing on the optical flow features and the inertial features according to the selected modal to generate output features; and decoding the output feature to obtain a relative pose. According to the method, the robustness of the visual inertial odometer system in a complex environment is remarkably improved.

The invention belongs to the technical field of computer vision and robot navigation. The invention provides a monocular vision odometer positioning method based on end-to-end deep learning. According to the embodiment of the invention, an explicit scale correction method is adopted, deep modeling is carried out on a complete time sequence, and fine motion changes between continuous frames can be accurately captured, so that the accuracy of the monocular visual odometer in translation and rotation estimation is remarkably improved, and the problems of error accumulation and scale blur in a traditional method are effectively solved. The fusion module efficiently integrates initial pose estimation, global depth information and time pose information, ensures that the system can maintain stable and reliable positioning performance in various complex scenes such as cities, villages and high-speed driving, and shows excellent robustness and wide applicability.

Provided are systems and techniques for automated navigation of vehicles, such as drones. The systems generally include processing unit(s) that, collectively, perform several steps. Such steps include generating metric depth estimates, using a pre-trained model, for each pixel in received image(s) from a monocular camera, or transformed image(s) based on the received image(s). Such steps may also include generating a pose estimate from visual odometry, then generating a truncated signed distance function representation of an environment based on the absolute depth estimates and the pose estimate. The steps may include creating and / or updating a local map based on the truncated signed distance function representation. The steps may include plan a collision-free route towards a goal based on the local map. This may include using motion primitives, which may be generated in a single offline step and stored in a trajectory library.

Eyewear devices that include two SoCs that share processing workload. Instead of using a single SoC located either on the left or right side of the eyewear devices, the two SoCs have different assigned responsibilities to operate different devices and perform different processes to balance workload. In one example, the eyewear device utilizes a first SoC to operate the OS, a first color camera, a second color camera, a first display, and a second display. A second SoC is configured to run computer vision (CV) algorithms, visual odometry (VIO), tracking hand gestures of the user, and providing depth from stereo. This configuration provides organized logistics to efficiently operate various features, and balanced power consumption.

A robotic work tool (100) comprising one or more satellite navigation sensors (175), a visual sensor (185) and a controller (110), wherein the controller (110) is configured to utilize the one or more satellite navigation sensors (175) to determine a reference for a ground-plane estimation and to utilize the visual sensor (185) to perform visual odometry navigation, and to utilize ground-plane estimation based on the visual sensor (185) and the reference from the satellite navigation sensors (175) to provide a scale (TS) for the visual odometry navigation.

The invention provides an unmanned aerial vehicle path planning method and system based on adaptive dynamic planning. The method comprises the following steps: calculating a solar azimuth angle according to an atmospheric polarization mode, compensating course drift of a visual odometer by using the solar azimuth angle, and generating corrected unmanned aerial vehicle position information; according to the illumination invariant edge information, combining a convolutional neural network, segmenting a key landmark in the texture enhanced image, and generating a semantic navigation landmark point sequence in a constraint space of the key landmark by adopting a bidirectional fast random tree; constructing a model prediction controller according to the semantic navigation road sign point sequence; and through the corrected position information, the flight path of the unmanned aerial vehicle is optimized in a rolling manner through adaptive dynamic planning under the constraint of a dynamic window, and the field angle and exposure parameters of the dual-band imaging unit are adjusted. According to the invention, the autonomous navigation precision and semantic understanding capability of the unmanned aerial vehicle in a complex light environment are improved, and anti-interference and high-precision autonomous path planning and flight control are realized.

The invention relates to the technical field of robots, in particular to a visual perception-based real-time abnormal event recognition method for a quadruped robot, which comprises the following steps of: acquiring body motion data and an environment image sequence of the robot in real time through an inertial measurement unit and a visual sensor; predicting and generating a short-term expected movement track based on the data and the control instruction; meanwhile, estimating an actual motion track of the robot by utilizing a visual odometer technology; fusing the deviation value of the expected trajectory and the actual trajectory with the optical flow statistical features extracted from the dense optical flow field to form a multi-modal fusion feature vector; and calculating a real-time abnormal confidence coefficient score by adopting a pre-trained long-short-term memory network model. Multi-mode sensing information is effectively fused, limitation of a single sensor is overcome, high-precision and real-time recognition and classification of abnormities such as foot slipping, collision and terrain abrupt change are achieved, corresponding emergency control strategies can be triggered, and autonomy and reliability of the quadruped robot in a complex environment are improved.

The present invention discloses a semantic positioning method based on environmental objects, which relates to the field of robotics. It includes semantic mapping, closed-loop detection, and positioning correction. The semantic information of the target object in the image is extracted by a convolutional neural network, and the camera's position information is estimated using a visual odometry. The label information of the target object and the position and size of its 2D bounding box in the image are extracted by a convolutional neural network; a two-way comparison and matching method is used to detect whether the camera has passed through a historical location. If so, geometric verification is performed to confirm whether the camera has reached the closed-loop position; finally, the relative position relationship of the objects is calculated and the position of the camera is corrected through pose graph optimization. The present invention can achieve centimeter-level positioning and map construction in complex environments such as those with changes in lighting and seasons and changes in camera observation angles.

This invention relates to the field of positioning technology, and more particularly to a positioning method and device. The method includes: determining two-dimensional line segments and visible points in each frame of images acquired by a camera, wherein the visible points include the endpoints of the two-dimensional line segments and interior points between the endpoints; fitting a three-dimensional line based on the two-dimensional line segments and determining the collinearity constraints of the two-dimensional line segments; constructing a cost objective function based on the photometric errors of the visible points in each frame of images, the photometric errors of the two-dimensional line segments in each frame of images, and the collinearity constraints; and minimizing the cost objective function to determine the pose parameters of the camera. This invention proposes a direct visual odometry scheme based on points and lines. This scheme extends the photometric error with respect to lines and adds collinearity constraints on points on the lines, thereby significantly reducing the number of variables to accelerate the optimization process and ensuring that the collinearity constraints are satisfied.

The present invention discloses a visual odometry method and system based on image depth prediction and monocular geometry. The method includes inputting two consecutive image frames, detecting and describing local features of the images using the Scale-Invariant Feature Transform (SIFT) algorithm, and then using the Fast Library for Approximate Nearest Neighbors (FLANN) algorithm to match corresponding feature point pairs between the two frames; solving for the essential matrix using epipolar geometry constraints to obtain the relative pose transformation of the camera; constructing and training a monocular depth prediction model to predict dense depth information for each input image frame; if the number of valid depth information pairs formed by two consecutive image frames is greater than a given threshold, using triangulation to estimate the scale factor to obtain the corrected relative pose transformation, otherwise using a combination of the Perspective-n-Point (PnP) projection algorithm, the Random Sample Consensus (RANSAC) algorithm, and local non-linear optimization to solve for the absolute pose transformation. The present invention effectively integrates the advantages of deep learning and traditional geometric methods, can adapt to dynamic environments, and improves the robustness and accuracy of monocular visual odometry.

The invention discloses a biased binocular visual odometer method and device based on strawberry cuttlefish bionics. The biased binocular visual odometer method comprises the following steps: synchronously acquiring a bright field image and a dark field image collected by a biased binocular camera and inertial data of an inertial measurement unit; constructing a perceptual attention weight grid according to the bright field image and the dark field image, and performing brightness normalization processing on the bright field image and the dark field image; extracting lateral visual features based on the normalized image and the perceptual attention weight grid; and fusing the biased visual features with the inertial data, and solving through nonlinear optimization to obtain a carrier pose estimation result. According to the invention, the dynamic range of the binocular vision system is expanded, and the imaging and sensing capabilities of the system in an HDR scene are improved. And meanwhile, the positioning precision and robustness of the binocular visual odometer in an HDR scene are improved.

The present invention relates to the technical field of synchronous positioning and map construction, and in particular to an RGB-D SLAM pose optimization method constrained by dual visual odometry. The method improves on the traditional RGB-D SLAM technology, constructs a first visual odometry based on ORB feature point matching calculation, and constructs a second visual odometry based on a point cloud data registration method. The dual visual odometry is used to add back-end map optimization constraints, thereby effectively correcting the drift generated by system operation without affecting the normal operating speed of RGB-D SLAM. At the same time, local optimization is used to better improve the accuracy of map construction. In addition, since the visual odometry based on ORB feature points in the present invention optimizes all frames in the image, the influence of the superposition of early errors on subsequent frame positioning and mapping can be reduced.

The present application belongs to the technical field of mobile robot navigation, and particularly relates to a SLAM navigation method and system for a mobile intelligent cabinet, which comprises the following steps: acquiring a laser point cloud sequence and a grayscale image of the mobile intelligent cabinet and performing data cleaning; obtaining a geometric feature index based on the spatial jump distribution of the laser point cloud sequence; obtaining a visual texture index based on the local dispersion of the image gradient; calculating laser dynamic weight and visual dynamic weight by using the geometric feature index and the visual texture index, weighting and fusing the pose change quantity calculated by the single-line laser radar odometry and the visual odometry to obtain a fused pose quantity; and updating the global state based on the fused pose quantity and driving autonomous navigation. The present application can adjust the sensor weight in real time according to the environmental characteristics, solves the problem of positioning divergence in the long corridor of a shopping mall or a high-reflectivity environment, and improves the robustness of navigation.

The application discloses a kind of intelligent unmanned vehicle environment perception systems based on multi-sensor information fusion, it is related to unmanned vehicle technical field.The system includes: sensor module, for collecting multi-source heterogeneous data, including GPS / IMU, laser radar, camera and millimeter wave radar;Data acquisition and fusion module, through the Kalman filter to multi-source data space-time synchronous fusion;Environment perception module, based on deep learning identification road boundary, obstacle and traffic sign, constructs dynamic perception model;High-precision map construction module, using SLAM technology and semantic information constructs and updates semantic three-dimensional map;Positioning algorithm module, fusion visual odometry, fusion data and high-precision map, calculates vehicle real-time pose;And control module, according to positioning and perception result carries out path planning and navigation control, the application provides a kind of intelligent unmanned vehicle environment perception systems based on multi-sensor information fusion, can be autonomously navigated and stably driven under complex and severe environment.

This invention discloses a dynamic path planning method for mobile robots suitable for farmland environments, comprising: acquiring access information of the work area and constructing a structured navigation map, performing topological modeling to obtain a navigation map containing navigation nodes and navigation edges; classifying and modeling the navigation edges in the navigation map, and calculating cost weights for each navigation edge based on road conditions, visual odometry confidence, and positioning stability; detecting weed targets and outputting two-dimensional candidate regions for weed targets, mapping each weed target to reachable stopping points in the navigation map, outputting a set of candidate targets, and generating relevant work costs for each weed target; establishing energy feasibility constraints, calculating available energy, minimum return energy consumption, and safety redundancy energy, and, under the premise of satisfying the energy feasibility constraints, performing path planning with a total cost function including navigation edge costs and work costs, and outputting a work path covering the processing order of weed targets.

The invention provides a cabin door positioning method and device based on confidence fusion and related equipment, and relates to the technical field of cabin door positioning. The method comprises the steps that motion information of the boarding bridge and environment information of the environment where the boarding bridge is located are acquired; determining a plurality of confidence coefficients according to the motion information and / or the environment information; fusing the plurality of confidence coefficients to obtain a comprehensive confidence coefficient; according to the comprehensive confidence coefficient and a set decision-making mechanism, cabin door feature point coordinate information is obtained based on first coordinate information and / or second coordinate information, the first coordinate information is cabin door feature point coordinate information obtained through a visual odometer, and the second coordinate information is cabin door feature point coordinate information obtained based on a cabin door recognition model. According to the method, the advantages of accurately predicting cabin door coordinate information by a model and estimating cabin door coordinates in real time by visual tracking can be reserved, more accurate cabin door coordinate information can be quickly obtained, and both accuracy and high efficiency are considered.

The invention discloses a pose estimation method and device based on a visual odometer, equipment and a medium, and the method comprises the steps: obtaining a to-be-processed image collected by a camera, and extracting optical flow data of the to-be-processed image based on an optical flow method; according to the optical flow data, performing feature extraction on the optical flow data to-be-processed image through a multi-head attention mechanism and a feature extraction module to obtain a to-be-optimized motion feature corresponding to the optical flow data to-be-processed image; optimizing the to-be-optimized features of the optical flow data according to the motion consistency constraint to obtain target motion features corresponding to the to-be-processed image of the optical flow data; and performing pose fitting based on the optical flow data target operation features to obtain pose estimation corresponding to the optical flow data camera. According to the invention, high-precision estimation of the pose and motion of the camera is realized.

The application belongs to the field of visual simultaneous localization and mapping (VSLAM), and particularly relates to a VSLAM back-end optimization method based on a maximum cross-correlation entropy of a plurality of convex combinations. The method comprises the following steps: sensor information reading: camera image information reading and preprocessing, so as to obtain pixel information of the image; front-end visual odometry: rough camera motion is estimated according to information of adjacent images, initial values are provided for the back-end, it is judged whether each frame is a key frame, and a preliminary local map is established; back-end optimization: camera poses measured by visual odometry at different times and information of loop detection are accepted, and optimization is performed on the camera poses and the information, so that a globally consistent trajectory and map are obtained; loop detection: it is detected and judged whether the robot has arrived at a previous position, if loop detection is detected, information is provided to the back-end for processing, and a global optimization thread is triggered; and mapping: a map corresponding to a task requirement is established according to an estimated trajectory.

The application discloses a kind of wide-area-oriented end edge cloud collaborative computing visual positioning method and system.For the mobile terminal with binocular camera, the computing framework of end edge cloud is used, and the online map established in advance is used to realize the real-time positioning and mapping based on binocular camera.In the end edge cloud framework, the end side undertakes the work of visual odometry and global map optimization using binocular camera.The end side transmits information to the edge computing unit, while the edge computing unit performs image retrieval and global positioning work.Finally, the positioning result calculated by the end edge collaborative computing is sent to the cloud for real-time visualization.The method improves the calculation efficiency of real-time positioning, and also supports the deployment mode of multiple mobile terminals connected to the edge cloud side, estimates and optimizes the positioning information of each terminal through data exchange, to obtain higher management efficiency.

The present disclosure relates to the technical field of vision, and specifically relates to a constraint construction method and apparatus for visual odometry, and a device and a medium. The method comprises: acquiring a parameter of a support plane; acquiring image feature points of at least two frames of images; establishing an image feature point matching relationship between image feature points of different frames of images; and on the basis of the image feature points, the image feature point matching relationship and the parameter of the support plane, generating a visual odometry constraint term. On the basis of the visual odometry constraint term acquired in the solution, the scale can be effectively constrained during visual odometry, thereby ensuring the stability of the scale, and facilitating an improvement in the accuracy of collecting map data on the basis of a visual odometry result.

An embodiment of the present invention discloses a mapping and navigation method for a legged robot based on binocular visual odometry, which relates to the technical field of multi-legged mobile robots and is capable of autonomous navigation of the legged robot even after leaving the effective signal range of the GPS. The present invention includes: capturing image frames using a binocular camera; extracting feature points of the image frames at each time stamp and establishing a camera coordinate system, and then using the extracted feature points and the camera coordinate system to obtain odometry information; obtaining key frames during the movement of the legged robot; further establishing a sub-map, creating a map based on the sub-map, planning a route in the map, and navigating according to the planned route. The designed image processing method prevents the image effects captured by the legged robot during its fluctuating state from affecting the calculated odometry.