828 results about "Monocular camera" patented technology

The invention discloses an end-to-end monocular visual odometer method fusing space-time semantic information. According to the method, continuous image sequence frames are collected through a color monocular camera, and a multi-information fusion end-to-end deep learning framework is constructed; a heterogeneous training domain is adopted to set various data set course sharing parameter fusion training, continuous image sequences are input, and the end-to-end deep learning framework is dynamically coupled with hidden state feature vectors output historically, so that a feature mapping relation of time sequence perception is formed; and interpretable feature decoupling of the static background elements and the dynamic entity objects in the scene is realized. After iterative feature fusion, the system outputs sparse depth and camera motion poses which conform to scene geometric constraints, so that a camera trajectory estimation model with high robustness and strong generalization ability in a complex environment is constructed. According to the method, the positioning precision and stability of the monocular vision odometer are remarkably improved.

The invention provides a multi-source heterogeneous data fusion side slope digital twin modeling method, which comprises the following steps of: acquiring side slope multi-dimensional monitoring data by arranging a GNSS (Global Navigation Satellite System) sensor, a multi-point displacement meter, a distributed optical fiber strain sensor, an accelerometer, an osmometer, a monocular camera and satellite remote sensing image equipment; the collected data is converted into a unified format through time alignment, space registration and standardization processing and serves as modeling input; the method comprises the following steps: constructing an initial digital twinborn model reflecting the real form and physical characteristics of a slope by utilizing a three-dimensional modeling and finite element simulation technology; in combination with real-time sensing data, model evolution is dynamically driven based on a space-time fusion algorithm, boundary conditions and material parameters are automatically corrected through actual measurement deviation feedback, and continuous twin iteration updating of the model is achieved; and finally, extracting a landslide risk index to realize real-time early warning of the side slope. According to the invention, multi-source sensing and digital twinborn fusion is realized, and the accuracy, real-time performance and intelligent level of slope monitoring are improved.

The invention provides a posture recognition algorithm for any object under a monocular camera and an application system, and the algorithm comprises the steps: S1, constructing a target three-dimensional model, carrying out the multi-view annular shooting image collection of a target, and generating a dense grid model through feature extraction, matching, posture calculation and a multi-view geometric method; s2, generating an image depth map, and predicting depth information of a target in a motion process based on a monocular image sequence; s3, extracting a target image mask, and generating a target area mask graph through an image encoder, a prompt encoder and a mask decoder; and S4, executing attitude estimation, performing attitude initialization, correction and screening by combining the three-dimensional model, the depth map and the mask map, and outputting a six-degree-of-freedom attitude result of the target. According to the method, the target is subjected to annular shooting modeling through the method based on multi-view geometry, the three-dimensional model of the target is generated, attitude estimation is achieved in combination with the image mask and the depth map, the generalization ability of an attitude estimation algorithm in an actual scene is improved, and the application range of the attitude estimation algorithm in the actual scene is widened.

The invention discloses a charging robot automobile charging port pose vision measurement method comprising the following steps: 1, controlling a mechanical arm to drive a monocular camera to collect a charging port image, and constructing a charging port image data set; 2, extracting a charging hole contour in each image by adopting an improved Mask R-CNN instance segmentation model; step 3, performing robust ellipse fitting based on each charging hole contour to obtain center coordinates of each charging hole in the two-dimensional image; step 4, establishing a world coordinate system based on charging hole space distribution defined by the automobile charging port standard model, and determining three-dimensional coordinates of each charging hole; and 5, the multi-view two-dimensional center coordinates and the corresponding three-dimensional coordinates are input into the PnP graph optimization model, and the pose of the charging port in the mechanical arm base coordinate system is solved by fusing the re-projection error constraint, the structure prior constraint and the kinematics chain constraint. The invention further discloses a charging robot automobile charging port pose vision measurement charging method and a charging system.

The invention discloses a multi-modal information fusion odometer construction method for star catalogue positioning, and relates to the technical field of star catalogue patroller positioning. The method comprises the following steps: carrying out space joint calibration on a monocular camera, a laser radar and an inertial measurement unit, reconstructing a laser radar point cloud by using a timestamp of a camera image, and realizing time synchronization of the camera image and the laser radar point cloud; establishing an IMU pre-integration error model; and performing motion compensation distortion removal on the laser point cloud by using an IMU pre-integration result, and extracting geometric features of the distorted laser point cloud based on a neighbor region smoothness calculation method of a fixed measurement distance. By researching a multimodal information fusion odometer method, the accumulative error of motion measurement is reduced, the positioning precision and stability are improved, technical support is provided for design and development of a star catalogue navigation system, and the problems that a single-modal star catalogue positioning method is weak in environment adaptive capacity and poor in algorithm generalization are solved.

The invention relates to a robot environment perception method based on single-view three-dimensional scene generation, and the method comprises the steps: collecting a two-dimensional image containing target environment information through employing a monocular camera, generating multi-view information through combining depth estimation, normal prediction and a two-stage semantic guidance diffusion model, and constructing a high-quality three-dimensional scene. And reconstructing three-dimensional point cloud data through the neural radiation field, and performing texture rendering optimization on the point cloud data. A Point Net + + network is used for carrying out semantic analysis on point clouds, a multi-frame time sequence point cloud registration and Kalman filtering tracking method is introduced, modeling is carried out on a dynamic target, and a dynamic semantic map with a motion state is constructed. And finally, structured output environment information is used for robot navigation, path planning and task execution. The problems of high cost, high complexity and insufficient real-time performance and robustness of a three-dimensional scene generation technology in the field of robot environment perception are solved, and the method is high in structuring degree, standard and unified in output format and high in universality and engineering adaptation capacity.

The invention discloses an aircraft target tracking method and system based on compensation prediction, which are used for improving the target tracking precision in an image transmission delay scene. The method comprises the following steps: firstly, acquiring an image frame sequence of a target aircraft by using an airborne monocular camera, extracting a target center coordinate through a small target detection algorithm, and constructing a position sequence; the method comprises the following steps: extracting current high-frequency I MU data aiming at the condition that an image frame has transmission delay, inputting the current high-frequency I MU data into an LSTM-DKF model constructed by fusing LSTM and a delay Kalman filter, and predicting and generating a process noise and observation noise covariance matrix; and initializing a delay Kalman filter by using the matrix, and recursively predicting the target position during the delay period. And when the delayed image frame is received, backtracking and updating the state of the filter, recurring to the current moment again, and outputting the compensated target position. And finally, pixel deviation is calculated according to the compensation position, an aircraft tracking control instruction is generated, and high-precision target tracking is realized.

The invention relates to a three-dimensional reconstruction method based on a visual odometer and deep learning. The method comprises the following steps: obtaining related parameters of a monocular camera through camera calibration; shooting continuous image frames of the target area under the plurality of camera poses through the monocular camera to obtain image frames of the target area after distortion correction under the plurality of camera poses; estimating relative poses of the camera among different image frames by using a visual odometer according to related parameters of the monocular camera; constructing a cost body of the image frame by using a cost body generation model according to the relative pose of the camera; optimizing the cost body generation model by using the geometric consistency loss and the depth consistency loss, and generating a depth map of the image frame through the optimized cost body generation model; performing noise elimination and hole filling on the depth map of the image frame to obtain an optimized three-dimensional map; according to the method, a stable reconstruction effect can be kept in a complex environment, a low-texture region and a scene with a large visual angle span, and compared with an existing single deep learning reconstruction method, the method has higher robustness and reconstruction precision.

The invention provides a crack three-way measurement research method and system based on multi-coordinate system fusion, and relates to the technical field of earth and rockfill dam monitoring. The method comprises the following steps: shooting concentric circle target images at multiple angles through a monocular camera, extracting feature points through preprocessing, and correcting projection distortion of non-parallel shooting; establishing a mapping relation between pixels and a three-dimensional coordinate system, and optimizing camera pose parameters; reconstructing a three-dimensional coordinate point cloud based on the multi-view data, screening outliers and fitting plane optimization data; and registering the three-dimensional point cloud to a world coordinate system through a coordinate system conversion algorithm, and comparing two measurement results to calculate the three-way displacement of the crack. According to the method, concentric circle target constraint feature precision is utilized, projection correction and plane fitting are combined to compensate monocular vision depth deficiency, high-precision displacement detection is achieved, and engineering monitoring requirements are met.

The invention relates to the technical field of cabin loading and unloading automation, and provides a multi-sensor fusion cabin loading and unloading equipment cooperative positioning method and system. A laser radar is used for scanning a cabin, laser radar point cloud data are generated, and a laser radar coordinate system and a point cloud map are constructed; a monocular camera is adopted to obtain an operation area image of loading and unloading equipment, camera image data is generated, and a camera coordinate system is constructed; performing anti-interference processing on the laser radar point cloud data; performing illumination change influence resistance processing on the camera image data; fusing the laser radar point cloud data with the camera image data to obtain fused positioning information; calculating the distance and angle of loading and unloading equipment according to the fused positioning information, generating a moving instruction, and driving the loading and unloading equipment to perform cooperative positioning; a dynamic object in a cabin is detected through a laser radar and a monocular camera, multi-modal verification is carried out on a detection result, the dynamic object is filtered, and a point cloud map is updated in real time. The problems of insufficient positioning precision, GPS failure and large dynamic environment interference of a single sensor are solved.

The invention discloses a visual language navigation method and system based on a monocular camera and a language instruction, and the method comprises the steps: S1, feature field construction: employing a 3DGS to construct a 3DGS feature field by employing visual information obtained by a monocular RGB-D camera; s2, implicit partial complementation: inferring the representation of a missing region through a context feature relationship to generate a complete feature map; s3, waypoint prediction: generating an aerial view feature map based on the 3DGS feature field, and predicting nearby navigable waypoints through a waypoint predictor; s4, based on active perception of uncertainty, when the robot is uncertain in navigation decision, more visual information is acquired and missing visual information is supplemented by rotating a camera; and S5, constructing a topological map based on the constructed panoramic feature map and the predicted path points, and carrying out navigation decision making by using a panoramic visual language navigation planning model in combination with a language instruction. The problem that monocular vision information is incomplete is effectively solved, and the success rate and efficiency of navigation are remarkably improved.

The invention provides a road vehicle real-time twinning method and system based on a monocular camera, which can sense the change of a multi-target vehicle in a dynamic scene in real time by combining target detection, target tracking and depth estimation technologies, and accurately map the position and the feature of the multi-target vehicle to a virtual twinning scene, so that the real-time twinning of the multi-target vehicle is realized. And the defect of dynamic vehicle twinning support in the existing road twinning scene is overcome. The vehicle features are extracted by adopting a visual question and answer large model, dependence on specific task data and scenes is reduced, generative reasoning and multi-round interaction are supported, problems can be dynamically adjusted, and a new scene can be quickly adapted.

The present discloses a lightweight attention mechanism distance estimation method for assisting visual navigation of a vehicle at a container terminal. Firstly, using a depth monocular camera calibrated with the imaging parameters of the planar checkerboard tool to collect RGB-Depth image pairs in the working scenario of the automatic guided vehicle. Secondly, performing depth completion and manual annotation processing on the collected depth image. Thirdly, inputting image pairs into a lightweight monocular metric depth estimation framework which uses an improved lightweight attention mechanism Squeeze Former as the token mixer for training. Finally, fusing the results of relative depth estimation and absolute depth estimation to obtain a prediction of an actual distance between an object in the RGB image and the camera in the real world. The method and model provided by the present invention feature simple equipment, low cost, high timeliness of prediction and accurate results.

The invention relates to the field of mobile robot positioning, in particular to a LiDAR-inertia-vision fusion SLAM switching positioning method applied to a degraded environment. According to the method, through a multi-sensor system composed of a monocular camera, an inertial measurement unit and a laser radar which are installed on a mobile robot, original measurement data such as images, angular velocity, linear acceleration and dense space point cloud are acquired, a system state initialization model is constructed, and the multi-sensor system is used to acquire the measurement data. According to the method, the vision-inertia poses and the laser radar poses or linear interpolation of the vision-inertia poses and the laser radar poses are fused, the degradation state can be effectively recognized in real time, and multi-sensor system information switching can be carried out, so that the problem of positioning failure of the laser radar and the vision-inertia odometer module in the degradation environment can be effectively solved; the overall visual SLAM positioning precision and robustness are improved, and the long-term stable operation requirements under the severe conditions of single structure, complex illumination, sparse texture and the like are met.

The invention discloses a structure three-dimensional displacement monitoring method and device based on a monocular camera and a concentric annulus, and relates to the field of computer vision and structure health monitoring, a concentric annulus target is fixed at a to-be-monitored structure monitoring point, and the monocular camera carries out positioning and lens parameter adjustment; shooting a checkerboard image and calibrating the checkerboard image by adopting a camera calibration algorithm to obtain a distortion parameter and an internal reference matrix; collecting a video sequence containing a structure displacement process of the target; image distortion correction is carried out based on the distortion parameters and the internal reference matrix, a target is identified through a target detection algorithm, and target parameters are calculated through least square circle fitting; realizing multi-target tracking by a target point topology identification algorithm based on Y-X threshold sorting; calculating scale factors according to target parameters and actual physical sizes, and calculating in-plane orthogonal direction displacement and out-of-plane displacement to form three-dimensional displacement information of the structure; according to the invention, the monocular camera is adopted to realize the accurate measurement of the three-dimensional displacement of the structure and reduce the complexity and calibration difficulty of the system.

The invention belongs to the technical field of three-dimensional scene reconstruction, and discloses a three-dimensional scene reconstruction method and system based on monocular depth estimation. The method comprises the following steps: constructing a reconstruction scheme generation model, a two-dimensional image processing model and a three-dimensional scene reconstruction model; generating a reconstruction scheme by using a reconstruction scheme generation model according to the real-time indoor layout data to obtain a real-time reconstruction scheme; acquiring a plurality of real-time two-dimensional images by using a monocular camera according to the real-time reconstruction scheme; according to the real-time reconstruction scheme, processing the plurality of real-time two-dimensional images by using a two-dimensional image processing model to obtain a plurality of processed real-time two-dimensional images; and according to the real-time reconstruction scheme, performing depth estimation and three-dimensional scene reconstruction on the plurality of processed real-time two-dimensional images by using a three-dimensional scene reconstruction model to obtain real-time three-dimensional scene data. According to the method, the problems of high cost investment, low precision, low automation degree and unstable reconstruction result in the prior art are solved.

The invention relates to the technical field of positioning of planar robots, in particular to a wheel type odometer and monocular vision fused SLAM positioning system with three driven omnidirectional wheel sensors. According to the method, firstly, internal reference calibration is conducted on a sensor, then the pose of a vehicle body is calculated according to data of a wheel type odometer sensor, joint initialization of a wheel type odometer and a monocular camera is conducted after timestamp synchronization is completed, and the pose of the vehicle body is calculated through vision-wheel type odometer tight coupling nonlinear sliding window optimization. A dynamic plane constraint self-adaptive method based on local geometric features and meta learning is added, a mixed strategy based on motion component decomposition and residual entropy dynamic adjustment is designed, back-end optimization is carried out, and whether a loop exists or not is detected to optimize a track and reduce errors.

The invention discloses an underwater in-situ fish label-free sample detection method and equipment. The method comprises the following steps: acquiring an in-situ image shot by a monocular camera of an underwater observation platform, and constructing an image-text structured knowledge base; extracting a training and verification data set of the detection model from the knowledge base, and dividing the data set into a closed set category and an open world category; receiving a natural language text instruction which is input by a user and is used for describing the to-be-detected data set, and generating a text feature vector of a target category; a targeting auxiliary model is obtained through vision-text semantic space training, dynamic interactive fusion is carried out on the visual features of the in-situ image and the text features of the category, and end-to-end joint optimization is carried out on the target model; and performing label-free detection on fishes appearing in the underwater in-situ image by using the optimized target model. According to the method, new species which are not seen during training can be effectively identified, cross-domain reasoning can be implemented on a label-free observation sample, and the retraining and maintenance cost is reduced.

The invention relates to a fruit and vegetable picking robot system based on data enhancement and identification precision optimization, and belongs to the field of robots. The system is composed of a visual identification module, a mechanical arm module, a control module and a moving module, and fruits and vegetables are identified and positioned in real time by adopting a YOLOv5 target detection algorithm. The visual module collects images through a monocular camera, and accurately detects the types and positions of fruits and vegetables by using a YOLOv5 algorithm; the control module coordinates the mechanical arm to execute the picking action according to the recognition result, and plans the moving path of the robot to adapt to the complex farmland environment; and the mechanical arm module adopts a multi-degree-of-freedom design and is provided with a flexible clamping jaw, so that efficient picking operation can be completed on the premise of protecting fruits and vegetables. In addition, the system also supports the dynamic updating of the model, and the recognition performance is continuously optimized by adding new fruit and vegetable data. The system can meet the requirements of different crops and environments while improving the picking efficiency and reducing the labor cost, and has wide application prospects.

The invention discloses a height measurement method, a marking method and system, a storage medium and a program product, and relates to the technical field of laser marking, and the method comprises the steps: transmitting first laser through a laser, and projecting a preset first number of first light spots to the surface of a measured object; shooting the first light spot through a monocular camera to obtain a first pixel coordinate of the center of the first light spot in a camera coordinate system; a preset second number of standard scale points on a target laser light path and the height of each standard scale point are determined, the standard scale points are projected to the camera coordinate system, second pixel coordinates of the standard scale points in the camera coordinate system are obtained, and the target laser light path is the light path of the first laser; and calculating the distance between the first pixel coordinate and the second pixel coordinate, determining a target standard scale point with the minimum distance, and taking the height of the target standard scale point as the height of the measured object. According to the invention, the height measurement cost is reduced.

Disclosed are methods, devices, and computer-readable media for detecting lanes and objects in image frames of a monocular camera. In one embodiment, a method is disclosed comprising receiving a sample set of image frames; detecting a plurality of markers in the sample set of image frames using a convolutional neural network (CNN); fitting lines based on the plurality of markers; detecting a plurality of vanishing points based on the lines; identifying a best fitting horizon for the sample set of image frames via a RANSAC algorithm; computing an inverse perspective mapping (IPM) based on the best fitting horizon; and computing a lane width estimate based on the sample set of image frames using the IPM in a rectified view and the parallel line fitting.

The embodiment of the invention provides a barrier gate control method and device, a management and control all-in-one machine and a storage medium, and relates to the technical field of video monitoring, the method is applied to a monocular camera arranged on the front face of the management and control all-in-one machine, and the management and control all-in-one machine further comprises a binocular camera arranged on the side face; the method comprises the following steps: acquiring each abnormal event reported by a binocular camera and the occurrence time of each abnormal event; determining an initial control mode about the barrier gate based on the license plate recognition result; if the initial control mode is to open a barrier gate, whether an abnormal event exists in a lane area indicated by the detection area currently or not is analyzed based on the occurrence time of the currently obtained abnormal event; if the abnormal event exists, correcting the initial control mode based on the current abnormal event to obtain a target control mode, and controlling the barrier gate based on the target control mode; and if not, controlling the barrier gate based on an initial control mode. According to the scheme, the intelligent degree of barrier gate control can be improved.

The invention discloses a rapid robust monocular vision inertial positioning method and system. The method comprises the following steps: configuring IMU and monocular camera sensor parameters; preprocessing the IMU data; iMU attitude initialization is carried out; image features of the monocular camera are extracted, abnormal matching points are removed, and IMU pre-integration is carried out; monocular vision initialization is carried out; odometer attitude optimization: judging whether the feature points are mismatched or not based on an IMU pre-integration result, constructing a feature point re-projection error jacobian matrix, and performing blocking and diagonalization processing on the re-projection error jacobian matrix to respectively optimize inverse depths and image frame attitudes of the feature points; selecting a key frame based on a key frame identification rule after the current frame attitude update is completed; and judging according to the current frame, and removing a certain frame in the sliding window to reserve a sliding window space for adding the latest frame. According to the method, the inverse depth of the feature points and the image frame attitude are optimized respectively based on the counterweight projection error Jacobian matrix partitioning and diagonalization processing, and the calculation complexity is reduced.

The invention relates to the technical field of multi-person posture estimation, in particular to an interactive scene data synthesis and key point visibility updating algorithm based on monocular vision, which comprises the following steps of: 1, arranging single-person picture data under monocular shooting vision, processing an input single-person picture under monocular shooting vision by using a GrondingDINO visual language open type target detection model, and obtaining a target detection result; and the Person is used as a retrieval keyword to identify a character individual in the picture. Through fine processing of the steps, especially introduction of target matching, target position adjustment and key point visibility updating methods, the quality of the synthesized image is greatly improved, the fusion degree of the target and the background is optimized through accurate matching and natural target pasting, and the image quality is improved. The space consistency and the visual naturalness of the synthetic image are enhanced, the problem of overfitting is effectively avoided through the improvement, the diversity of training data is enhanced, and therefore the generalization ability of the model is improved.

The invention provides an oil taking port positioning method and system based on monocular vision and laser positioning. The method comprises the steps that laser point cloud data and a monocular image near an oil taking port are acquired; constructing a three-dimensional model of the oil taking port based on the laser point cloud data, and projecting the three-dimensional model of the oil taking port into a two-dimensional slice image according to an acquisition position label of the monocular camera; matching the two-dimensional slice image with the monocular image to determine the two-dimensional feature position of the oil extraction port; calculating the feature distance between the oil taking port and the camera based on the matched two-dimensional feature position of the oil taking port and the focal length label and the size label of the monocular image; on the basis of the position parameters of the oil taking port in the laser point cloud data, the feature distance is combined, and initial three-dimensional coordinates of the oil taking port are generated; and unifying the pixel coordinates of the monocular vision and the laser point cloud coordinates into the same coordinate system, and carrying out preliminary three-dimensional coordinate data fusion to obtain the final three-dimensional coordinates of the oil extraction port. The positioning precision of the transformer oil taking robot on the oil taking port is improved.

The invention discloses a container lockpin positioning method and system based on combination of a monocular camera and a mechanical arm, and the method comprises the steps: carrying out the target detection of the side holes of a to-be-detected corner fitting in a left visual angle image and a right visual angle image based on a deep learning model, obtaining a left image detection frame pixel set and a right image detection frame pixel set with minimum encirclement; traversing each point in the pixel set of the left image detection frame, calculating the epipolar constraint of each point in the pixel set of the left image detection frame in the pixel set of the right image detection frame, and generating a homonymous pixel point pair set of the corner fitting to be detected in the left and right view angle images according to the corresponding constraint of the homonymous detection frame pixel points of the left and right view angle images; and calculating a depth value of each point in the left image detection frame pixel set according to the homonymous pixel point pair set, reconstructing a 3D detection frame point cloud of the corner fitting to be detected based on the depth value of each point in the left image detection frame pixel set, and calculating to obtain a coordinate and an Euler angle of the center of the 3D detection frame point cloud.

The invention relates to the technical field of automobile detection, in particular to a rim radius visual measurement system and method based on a rim circle common tangent, and the method comprises the following steps: S1, image collection: collecting an image containing two rims on the single side of a vehicle through a monocular camera fixed by a camera support; s2, common tangent extraction and discrimination: fitting two rim ellipses in the image to obtain ellipse parameters and a circle center, solving four common tangents, and determining an outer common tangent to obtain image intersection points of the four common tangents; s3, key point coordinate calculation: expressing a coordinate relationship between a rim circle and an external common tangent point, a circle center and an intersection point in a world coordinate system; s4, rim radius solving, wherein a rim radius equation set is established according to the coordinate transformation relation, and the radius is solved; according to the invention, through the binocular single-frame vision measurement method based on the common tangent geometrical relationship, rapid non-contact measurement of the radius of the rim is realized on the premise that camera calibration and external auxiliary equipment are not needed.

At least one monocular camera is installed in a vehicle in addition to in-cabin camera(s). Images captured by the in-cabin camera(s) at a first frame rate are processed to determine first positions of facial landmark features of a user and first poses of the user's face. Images captured by the at least one monocular camera at a second frame rate, higher than the first frame rate, are processed to identify facial landmark features of the user in the second images. Second positions of the facial landmark features are determined at the second frame rate, by correlating the facial landmark features identified in the second images with the first positions of the facial landmark features determined from the first images. Second poses of the user's face are determined at the second frame rate based on the second positions of the facial landmark features and the first poses of the user's face.

The invention relates to a visual obstacle avoidance method and device, and the method comprises the steps: S1, arranging a monocular camera on a moving carrier, and enabling an image collected by the monocular camera to comprise a preset range at the front lower part of the moving carrier; s2, collecting a sample image through the monocular camera and calibrating the sample image to obtain a relational expression of the distance between a target in the image and the mobile carrier; s3, acquiring a formal image through the monocular camera, and identifying an obstacle in the formal image; and S4, determining the distance between the obstacle and the mobile carrier through the relational expression. According to the invention, the monocular camera is arranged on the mobile carrier and only collects the image in the limited range of the front lower part, and the distance information with the target obstacle can be obtained from the collected image, so that the obstacle avoidance judgment is realized, and compared with the existing monocular vision that the depth-of-field information cannot be obtained, so that the obstacle distance cannot be accurately judged, and the obstacle avoidance accuracy is improved. Good application prospects are realized.

The invention discloses a coarse-to-fine multi-mode multi-sensor external parameter off-line calibration method. The method is mainly completed through two steps of coarse calibration based on a calibration plate and fine calibration based on off-line SLAM. Firstly, a checkerboard calibration plate is adopted to extract two-dimensional feature points in a visible light camera, a PNP algorithm is utilized to calculate external parameters of the two-dimensional feature points, meanwhile, plane parameters of the calibration plate are extracted from laser radar point cloud through region growth and an RANSAC algorithm, and finally, initial external parameters between a laser radar and the camera are optimized. Secondly, in the optimization process, in combination with multiple laser radars and a vision-inertia SLAM system, the robustness of feature points is enhanced through SuperPoint feature extraction, and support is provided for the depth of a monocular camera by using local laser point cloud; and finally obtaining a high-precision multi-sensor extrinsic parameter matrix through feature fusion and global optimization of an error minimization objective function. The process not only improves the registration precision between the sensors, but also provides a solid foundation for subsequent SLAM and automatic navigation tasks.