64 results about "Visual odometry" patented technology

The invention relates to the technical field of data processing, in particular to a laser and binocular camera-based position positioning system, which comprises an acquisition unit, a processing unit, a fusion unit, an abnormality judgment unit, an adjustment unit and a correction unit. According to the method, multi-source information such as a laser point cloud matching result, binocular visual odometer output and the number of feature points is subjected to joint modeling in an extended Kalman filter, so that the AGV obtains stable pose estimation in different environments, and the pose estimation accuracy is improved by analyzing the change trend of pose covariance in an adjusted monitoring window. A density threshold value, an information entropy threshold value and a dynamic proportion threshold value are automatically corrected, so that the threshold values can adapt to real working conditions for a long time, and performance degradation caused by a fixed threshold value is avoided; the problems that due to the fact that sensor information quality fluctuation cannot be recognized and processed, positioning precision is lowered, accumulative errors are increased, and positioning loss is prone to occurring in a complex environment are effectively solved.

Provided are systems and techniques for automated navigation of vehicles, such as drones. The systems generally include processing unit(s) that, collectively, perform several steps. Such steps include generating metric depth estimates, using a pre-trained model, for each pixel in received image(s) from a monocular camera, or transformed image(s) based on the received image(s). Such steps may also include generating a pose estimate from visual odometry, then generating a truncated signed distance function representation of an environment based on the absolute depth estimates and the pose estimate. The steps may include creating and / or updating a local map based on the truncated signed distance function representation. The steps may include plan a collision-free route towards a goal based on the local map. This may include using motion primitives, which may be generated in a single offline step and stored in a trajectory library.

The invention relates to the technical field of robots, in particular to a visual perception-based real-time abnormal event recognition method for a quadruped robot, which comprises the following steps of: acquiring body motion data and an environment image sequence of the robot in real time through an inertial measurement unit and a visual sensor; predicting and generating a short-term expected movement track based on the data and the control instruction; meanwhile, estimating an actual motion track of the robot by utilizing a visual odometer technology; fusing the deviation value of the expected trajectory and the actual trajectory with the optical flow statistical features extracted from the dense optical flow field to form a multi-modal fusion feature vector; and calculating a real-time abnormal confidence coefficient score by adopting a pre-trained long-short-term memory network model. Multi-mode sensing information is effectively fused, limitation of a single sensor is overcome, high-precision and real-time recognition and classification of abnormities such as foot slipping, collision and terrain abrupt change are achieved, corresponding emergency control strategies can be triggered, and autonomy and reliability of the quadruped robot in a complex environment are improved.

The invention discloses a biased binocular visual odometer method and device based on strawberry cuttlefish bionics. The biased binocular visual odometer method comprises the following steps: synchronously acquiring a bright field image and a dark field image collected by a biased binocular camera and inertial data of an inertial measurement unit; constructing a perceptual attention weight grid according to the bright field image and the dark field image, and performing brightness normalization processing on the bright field image and the dark field image; extracting lateral visual features based on the normalized image and the perceptual attention weight grid; and fusing the biased visual features with the inertial data, and solving through nonlinear optimization to obtain a carrier pose estimation result. According to the invention, the dynamic range of the binocular vision system is expanded, and the imaging and sensing capabilities of the system in an HDR scene are improved. And meanwhile, the positioning precision and robustness of the binocular visual odometer in an HDR scene are improved.

The present application belongs to the technical field of mobile robot navigation, and particularly relates to a SLAM navigation method and system for a mobile intelligent cabinet, which comprises the following steps: acquiring a laser point cloud sequence and a grayscale image of the mobile intelligent cabinet and performing data cleaning; obtaining a geometric feature index based on the spatial jump distribution of the laser point cloud sequence; obtaining a visual texture index based on the local dispersion of the image gradient; calculating laser dynamic weight and visual dynamic weight by using the geometric feature index and the visual texture index, weighting and fusing the pose change quantity calculated by the single-line laser radar odometry and the visual odometry to obtain a fused pose quantity; and updating the global state based on the fused pose quantity and driving autonomous navigation. The present application can adjust the sensor weight in real time according to the environmental characteristics, solves the problem of positioning divergence in the long corridor of a shopping mall or a high-reflectivity environment, and improves the robustness of navigation.

The application discloses a kind of intelligent unmanned vehicle environment perception systems based on multi-sensor information fusion, it is related to unmanned vehicle technical field.The system includes: sensor module, for collecting multi-source heterogeneous data, including GPS / IMU, laser radar, camera and millimeter wave radar;Data acquisition and fusion module, through the Kalman filter to multi-source data space-time synchronous fusion;Environment perception module, based on deep learning identification road boundary, obstacle and traffic sign, constructs dynamic perception model;High-precision map construction module, using SLAM technology and semantic information constructs and updates semantic three-dimensional map;Positioning algorithm module, fusion visual odometry, fusion data and high-precision map, calculates vehicle real-time pose;And control module, according to positioning and perception result carries out path planning and navigation control, the application provides a kind of intelligent unmanned vehicle environment perception systems based on multi-sensor information fusion, can be autonomously navigated and stably driven under complex and severe environment.

This invention discloses a dynamic path planning method for mobile robots suitable for farmland environments, comprising: acquiring access information of the work area and constructing a structured navigation map, performing topological modeling to obtain a navigation map containing navigation nodes and navigation edges; classifying and modeling the navigation edges in the navigation map, and calculating cost weights for each navigation edge based on road conditions, visual odometry confidence, and positioning stability; detecting weed targets and outputting two-dimensional candidate regions for weed targets, mapping each weed target to reachable stopping points in the navigation map, outputting a set of candidate targets, and generating relevant work costs for each weed target; establishing energy feasibility constraints, calculating available energy, minimum return energy consumption, and safety redundancy energy, and, under the premise of satisfying the energy feasibility constraints, performing path planning with a total cost function including navigation edge costs and work costs, and outputting a work path covering the processing order of weed targets.

The invention provides a cabin door positioning method and device based on confidence fusion and related equipment, and relates to the technical field of cabin door positioning. The method comprises the steps that motion information of the boarding bridge and environment information of the environment where the boarding bridge is located are acquired; determining a plurality of confidence coefficients according to the motion information and / or the environment information; fusing the plurality of confidence coefficients to obtain a comprehensive confidence coefficient; according to the comprehensive confidence coefficient and a set decision-making mechanism, cabin door feature point coordinate information is obtained based on first coordinate information and / or second coordinate information, the first coordinate information is cabin door feature point coordinate information obtained through a visual odometer, and the second coordinate information is cabin door feature point coordinate information obtained based on a cabin door recognition model. According to the method, the advantages of accurately predicting cabin door coordinate information by a model and estimating cabin door coordinates in real time by visual tracking can be reserved, more accurate cabin door coordinate information can be quickly obtained, and both accuracy and high efficiency are considered.

The application discloses a kind of wide-area-oriented end edge cloud collaborative computing visual positioning method and system.For the mobile terminal with binocular camera, the computing framework of end edge cloud is used, and the online map established in advance is used to realize the real-time positioning and mapping based on binocular camera.In the end edge cloud framework, the end side undertakes the work of visual odometry and global map optimization using binocular camera.The end side transmits information to the edge computing unit, while the edge computing unit performs image retrieval and global positioning work.Finally, the positioning result calculated by the end edge collaborative computing is sent to the cloud for real-time visualization.The method improves the calculation efficiency of real-time positioning, and also supports the deployment mode of multiple mobile terminals connected to the edge cloud side, estimates and optimizes the positioning information of each terminal through data exchange, to obtain higher management efficiency.

The application discloses a positioning method based on ground-air cross visual angle visual odometer, and the positioning precision is judged by judging the mathematical features of the probability graph output by the ground-air visual angle positioning network, which is used as the condition for whether to fuse with the internal odometer, so that the inaccurate ground-air teaching positioning result can be removed, the positioning result with higher precision is fused with the internal odometer, and higher precision can be obtained; the ground panoramic image and the off-line acquired air visual angle image are matched and positioned, and the global positioning information can be provided under the condition that the GNSS signal is interfered; the ground-air cross visual angle positioning and the odometer fusion are used, the cumulative error caused by the long-term operation of the internal odometer is eliminated, and the positioning precision is improved.

Eyewear devices that include two SoCs that share processing workload. Instead of using a single SoC located either on the left or right side of the eyewear devices, the two SoCs have different assigned responsibilities to operate different devices and perform different processes to balance workload. In one example, the eyewear device utilizes a first SoC to operate a first color camera, a second color camera, a first display, and a second display. The first SoC and a second SoC are configured to selectively operate a first and second computer vision (CV) camera algorithms. The first SoC is configured to perform visual odometry (VIO), track hand gestures of the user, and provide depth from stereo images. This configuration provides organized logistics to efficiently operate various features, and balanced power consumption.

The invention relates to the technical field of image processing, and discloses a visual odometer online optimization method combining frequency domain and spatial domain constraint spatial-temporal characteristics, which comprises the following steps: acquiring an interval frame and an adjacent frame of an image, and extracting a current frame as a key frame; inputting the adjacent frames into a pose estimation network to generate adjacent frame poses; inputting the current frame and the interval frame into a depth estimation network to generate a depth map of the current frame and the interval frame; performing depth back projection and vertical back projection on the depth maps of the current frame and the interval frames to generate BEV views of the current frame and the interval frames; after fast Fourier transform is carried out on BEV views of the current frame and the interval frame, phase analysis is carried out, and the pose of the interval frame is generated; constructing an overall self-supervision loss, and realizing pose constraint on adjacent frames and interval frames; finally, on-line updating is triggered in a self-adaptive mode based on the similarity and the relative pose of the current frame and the latest key frame, and on-line optimization of the visual odometer is completed. According to the method, the positioning stability and robustness of the visual odometer are remarkably improved.

The invention discloses a large-range collaborative navigation method for an extraterrestrial patroller, and relates to the field of computer vision, visual navigation and map splicing. The method comprises the following steps: performing significance analysis based on spectral residual for a right monocular camera, focusing on sight capture of a typical feature region, establishing a visual odometer model of inertial IMU pre-integration, and performing back-end optimization driven by a factor graph; establishing a plurality of extraterrestrial patroller similar region search models based on an improved bag-of-words model, introducing a similarity relative scoring function based on a norm form, setting a relative threshold value of a similarity suppression function, and obtaining an image overlapping region; according to the method, the calculation capability of an airborne computer is considered, a sparse feature point map and patroller pose splicing model is established, complete splicing of overlapped area maps is completed through splicing of small sparse feature point maps, and the splicing precision and the splicing efficiency are improved.

The invention discloses an underwater semantic self-adaptive binocular vision odometer method capable of resisting dynamic interference. The method comprises the following steps: firstly, acquiring a synchronous stereo image pair through a binocular camera; then extracting features by using a SuperPoint model, and performing field adaptive fine tuning on the model through an unsupervised learning method based on descriptor consistency so as to improve the detection quality of the model in an underwater low-contrast and fuzzy environment; secondly, introducing a semantic segmentation network to identify and filter unreliable feature points on a dynamic target and an unstructured medium in real time; then, a SuperGlue model is adopted and epipolar constraint is combined to carry out efficient and robust feature matching; in the pose estimation stage, rapid initialization of a real scale is realized by using binocular geometric characteristics, and continuous tracking is carried out through a PnP algorithm; and finally, constructing a graph optimization model in a sliding window at the rear end, and by taking minimization of a re-projection error as a target, jointly optimizing a camera pose and a three-dimensional map point to obtain a globally consistent motion trail.

This invention relates to a biomimetic brain-like synchronous localization and environmental perception method for underwater robots. The invention addresses the problems of poor visual odometry, insufficient robustness, and low accuracy in loop closure detection in existing underwater robot navigation systems. The robot first acquires environmental feature information through sonar sensors and its own motion information through navigation sensors. A local scene template is obtained by processing the sonar data using an acoustic-visual processing method. The sensor data undergoes pre-integration processing, and the processed data is used as input to a convolutional neural network to output motion displacement, thus forming the robot's perception of its own position. Finally, an empirical map integrates the above information, and loop closure detection updates the empirical map, reducing drift during robot movement and completing the construction of the empirical map. This invention belongs to the fields of bionics and motion navigation technology.

The invention discloses a continuous learning visual mileage estimation method for plateau mountainous regions, which comprises the following steps: acquiring an image sequence, and obtaining continuous multi-frame image data; constructing a batch image database, and screening samples according to a diversity strategy to improve the diversity of the samples to the greatest extent; the online continuous learning module continuously learns attitude transformation between image sequences online by taking an unsupervised monocular depth estimation network as a branch task; the real-time visual mileage reasoning application module is used for continuously carrying out visual mileage calculation according to network output and providing mileage input of downstream tasks, and by adopting the continuous learning visual mileage estimation method for the plateau mountain land, the adaptability of visual mileage estimation to unknown and changeable environments such as the plateau mountain land can be improved.

The present disclosure relates to methods (200; 300; 400) for use in a moving platform for initialization of a visual-inertial odometry system (100) comprising a camera (1) and an inertial measurement unit (2), IMU. The methods comprise determining (201) point correspondence between at least a first image captured by the camera and a textured 3D model of an environment containing 3D coordinates given in a predetermined global coordinate system. The methods further comprise determining a relative rotation between the predetermined global coordinate system used for the camera and a IMU coordinate reference system, or estimating (210) a relative translation and rotation between the first and second images in a scale of the predetermined global coordinate system using the determined first and second point correspondences, said estimate indicating the scale for movement of the camera (1) for use in initialization of the visual odometry system.

This invention discloses a multimodal tightly coupled synchronous localization and mapping method and system with resistance to dynamic interference. The method simultaneously acquires images, point clouds, and inertial data. The visual front end combines a feature extraction network and a target detection model, using an adaptive extended Kalman filter to track dynamic targets and generate a mask for removing key points in dynamic regions. Dynamic feature points in these regions are removed, and the remaining static key points are used to complete inter-frame matching and relative pose estimation, forming visual odometry constraints. The laser front end uses inertial pre-integration to distort the point cloud and calculates laser odometry based on geometric feature registration. Loop closure detection performs candidate frame retrieval based on geometric descriptors and combines geometric consistency checks to generate loop closure pose constraints. The back end constructs a global factor graph and integrates visual odometry constraints, laser odometry constraints, inertial constraints, and loop closure pose constraints for joint nonlinear optimization. This invention effectively suppresses dynamic environmental interference and significantly improves positioning accuracy and robustness.

This invention provides a visual odometry based on a multi-scale feature aggregation network. Addressing the problem of large pose estimation errors in traditional deep learning visual odometry architectures when vehicles are turning, this invention introduces a feature aggregation module to fully utilize the details and semantic information in the multi-scale feature pyramid of the image, thus improving the accuracy of vehicle pose estimation. The feature aggregation module consists of three sub-modules: a spatiotemporally deformable convolutional fusion sub-module to enhance the extraction of detailed feature changes in the image; a gated attention ConvLSTM sub-module to handle the differences in semantic information of the vehicle in different driving states; and a cross-attention spatiotemporal feature interaction fusion module to enhance the complementary information between adjacent frames of the high-order feature map, further improving the reliability of pose prediction. Compared with traditional deep learning architectures, the method of this invention can effectively improve the accuracy of vehicle pose estimation in urban road scenes.

In an example method of training a neural network for performing visual odometry, the neural network receives a plurality of images of an environment, determines, for each image, a respective set of interest points and a respective descriptor, and determines a correspondence between the plurality of images. Determining the correspondence includes determining one or point correspondences between the sets of interest points, and determining a set of candidate interest points based on the one or more point correspondences, each candidate interest point indicating a respective feature in the environment in three-dimensional space). The neural network determines, for each candidate interest point, a respective stability metric and a respective stability metric. The neural network is modified based on the one or more candidate interest points.

Embodiments of the present application relate to a binocular visual odometry method based on event contrast maximization, compared with the traditional event-by-event tracking method, a contrast maximization algorithm is proposed to solve the data association of events and images, which greatly improves the calculation speed of event stream; since the contrast maximization algorithm is highly dependent on the depth of the scene, a robust Beta-Gaussian distribution depth filter is proposed to obtain more accurate line segment template depth than depth estimation using only triangulation; the evaluation experiment applied to the public event camera dataset can realize better performance and obtain lower delay camera trajectory compared with the visual odometry algorithm of ORB.

Eyewear devices that include two SoCs that share processing workload. Instead of using a single SoC located either on the left or right side of the eyewear devices, the two SoCs have different assigned responsibilities to operate different devices and perform different processes to balance workload. In one example, the eyewear device utilizes a first SoC to operate the OS, a first color camera, a second color camera, a first display, and a second display. A second SoC is configured to run computer vision (CV) algorithms, visual odometry (VIO), tracking hand gestures of the user, and providing depth from stereo. This configuration provides organized logistics to efficiently operate various features, and balanced power consumption.

The invention provides a self-supervision monocular vision odometer method based on optical flow guidance and dynamic masks, and relates to the technical field of computer vision. The method comprises the following steps: S1, constructing a double-branch pose estimation network, a depth estimation network and an optical flow estimation network; s2, inputting all images in the video image sequence into the constructed double-branch pose estimation network, the depth estimation network and the optical flow estimation network, and training the constructed double-branch pose estimation network and the depth estimation network; and S3, estimating the relative camera pose corresponding to each frame of image in the image sequence of the pose to be estimated by using the trained double-branch pose estimation network. According to the invention, the precision and robustness of the relative camera pose estimation result can be effectively improved.

The application relates to the technical field of semiconductor memory and in-memory computing, and discloses a three-dimensional visual perception acceleration method based on a majority decision logic gate of a spin-orbit torque magnetic random access memory (SOT-MRAM), which comprises the following steps: mapping image data to an array; simultaneously activating an odd number of storage units on the same bit line, constructing a majority decision logic gate by using series resistance, and executing a visual perception operator; transversely aligning cross-column data through a shifter; capturing a carry through a carry expansion module to generate a mask, and combining the majority decision logic gate to realize saturation, absolute value, maximum / minimum value operation without branch jump. The application executes the front-end core operator of a visual odometry in a storage array, breaks the storage wall bottleneck, eliminates standby leakage current by using SOT-MRAM non-volatility, and significantly improves the computing efficiency and reduces the power consumption.

The invention discloses a multi-view feature monocular vision odometer pose map attention optimization method, which comprises the following steps: obtaining monocular image sequence data, extracting features of adjacent frame images by using an encoder ResNet of a pose network, and outputting a high-dimensional motion feature map; performing direction transposition of channel number dimension, width dimension and height dimension on the high-dimensional motion feature map by using a motion mode feature reencoder MPFR, inputting the high-dimensional motion feature map into a ConvLSTM network, extracting spatial-temporal features of the feature map, and outputting a high-dimensional feature map; constructing an auxiliary pose optimization module by using a graph attention network (GAT), and optimizing poses of the three view angles by using an optimization algorithm PGAR to generate pose estimation data; and the fusion loss between the pose estimation image and the monocular image of the corresponding time step is calculated, the quality of the pose estimation image is evaluated, and the pose estimation image is output.

This invention discloses a road network map-assisted vehicle localization method based on pose graph optimization, comprising the following steps: S1, loading a regional road network map according to the vehicle's initial global position; S2, converting the regional road network map into a road network primitive map; S3, calculating the connectivity between different road network primitives; S4, determining whether to increase the number of skeleton points within each road network primitive based on the distance between adjacent skeleton points within each road network primitive; S5, calculating the initial transformation matrix T. w g S6, the basic unit of the road network to which the forward-oriented vehicle belongs; S7, according to T w g The visual odometry output pose is converted to the northeast-central coordinate system; S8, discrimination conditions are set to filter map correction points; S9, a pose graph optimization model of the trajectory is constructed to correct historical trajectories; S10, the vehicle pose at the latest moment is predicted; S11, steps S7 to S10 are repeated until the vehicle stops. This invention can improve the positioning accuracy of the vehicle and provide long-term positioning assurance.

An unknown environment adaptive navigation control system based on deep learning comprises a plurality of modules, a heterogeneous sensor array interface acquires spatial features, dynamic target information and self running state data of an unknown environment in real time, a time sequence environment dynamic modeling module is responsible for understanding a current environment structure and dynamic elements, and the time sequence environment dynamic modeling module is responsible for understanding the current environment structure and dynamic elements. The multi-modal perception optimization module is responsible for deeply fusing multi-source perception information, the unknown environment risk prediction module is used for predicting potential collision risks and sensor failure risks, the adaptive navigation decision module is used for generating an initial optimal path, and the dynamic path planning control module is used for realizing dynamic correction and accurate execution of the path. The operation error self-correction module corrects accumulative errors of the visual odometer and other modules in real time, the navigation state visualization module presents the system state through a visual interface and supports manual monitoring and intervention, and the system operation and maintenance management module is responsible for system management.

This invention discloses a localization method based on a lightweight road semantic map. The invention includes the following steps: First, visual odometry information is estimated from binocular images captured by a camera; then, the initial pose T of the camera is estimated based on the visual odometry information. c ; for the initial pose T of the camera c Perform a coordinate system transformation to obtain the initial pose T in the world coordinate system. w Then, extract various semantic targets from the current target image to obtain the corresponding semantic feature points, thereby obtaining the semantic feature point set of the current target image; finally, based on the initial pose T in the world coordinate system... w The method uses a set of semantic feature points and a particle filter to estimate the current pose. Experiments on a self-collected dataset show that, compared with traditional visual SLAM and particle filter methods based on road network maps, the method of this invention achieves higher accuracy and more reliable localization performance.

The invention is applicable to the fields of positioning, multi-sensor fusion and visual odometers, and provides a millimeter-wave radar-assisted monocular visual odometer, which comprises the following steps: body motion estimation and point cloud screening based on millimeter-wave radar point cloud; millimeter wave radar assisted monocular initial scale calculation and three-dimensional map point recovery; millimeter-wave radar-assisted monocular vision odometer scale drift correction is carried out; according to the method, the speed estimation reliability is improved, the problem that a monocular speedometer lacks real speed reference is solved through the processing flow of building a radar speed model, eliminating dynamic points through RANSAC, carrying out speed least square fitting and screening static point cloud, accurate input is provided for initial scale calculation, and the situation that dynamic interference influences a positioning datum is avoided; the platform application limitation is broken through; and the scale drift error is reduced.

The invention relates to a memory efficient visual SLAM method and system based on sparse prior embedding and map rarefaction, and belongs to the technical field of industrial robot positioning and mapping. The method comprises the following steps: extracting key frames in image frame data through a visual odometer, and constructing a key frame set; eliminating redundant key frames by adopting an information matrix retention strategy and an edge residual retention mechanism; then map point quality information is evaluated based on a feature point parallax score and a descriptor aggregation degree score index, and the contribution degree of an observed map point is comprehensively evaluated in a nonlinear function weighted fusion mode; and finally, combining grid discrete constraint and regularization in a sliding window, and realizing sparse selection and dynamic retention of map points through high-score rewards. And performing enhanced loopback detection on the basis of the constructed sparse image, and executing image feature matching and loopback confirmation by constructing a common-view region and extracting information compression features.