611 results about "Monocular camera" patented technology

The invention provides a multi-source heterogeneous data fusion side slope digital twin modeling method, which comprises the following steps of: acquiring side slope multi-dimensional monitoring data by arranging a GNSS (Global Navigation Satellite System) sensor, a multi-point displacement meter, a distributed optical fiber strain sensor, an accelerometer, an osmometer, a monocular camera and satellite remote sensing image equipment; the collected data is converted into a unified format through time alignment, space registration and standardization processing and serves as modeling input; the method comprises the following steps: constructing an initial digital twinborn model reflecting the real form and physical characteristics of a slope by utilizing a three-dimensional modeling and finite element simulation technology; in combination with real-time sensing data, model evolution is dynamically driven based on a space-time fusion algorithm, boundary conditions and material parameters are automatically corrected through actual measurement deviation feedback, and continuous twin iteration updating of the model is achieved; and finally, extracting a landslide risk index to realize real-time early warning of the side slope. According to the invention, multi-source sensing and digital twinborn fusion is realized, and the accuracy, real-time performance and intelligent level of slope monitoring are improved.

The invention provides a posture recognition algorithm for any object under a monocular camera and an application system, and the algorithm comprises the steps: S1, constructing a target three-dimensional model, carrying out the multi-view annular shooting image collection of a target, and generating a dense grid model through feature extraction, matching, posture calculation and a multi-view geometric method; s2, generating an image depth map, and predicting depth information of a target in a motion process based on a monocular image sequence; s3, extracting a target image mask, and generating a target area mask graph through an image encoder, a prompt encoder and a mask decoder; and S4, executing attitude estimation, performing attitude initialization, correction and screening by combining the three-dimensional model, the depth map and the mask map, and outputting a six-degree-of-freedom attitude result of the target. According to the method, the target is subjected to annular shooting modeling through the method based on multi-view geometry, the three-dimensional model of the target is generated, attitude estimation is achieved in combination with the image mask and the depth map, the generalization ability of an attitude estimation algorithm in an actual scene is improved, and the application range of the attitude estimation algorithm in the actual scene is widened.

The invention discloses a charging robot automobile charging port pose vision measurement method comprising the following steps: 1, controlling a mechanical arm to drive a monocular camera to collect a charging port image, and constructing a charging port image data set; 2, extracting a charging hole contour in each image by adopting an improved Mask R-CNN instance segmentation model; step 3, performing robust ellipse fitting based on each charging hole contour to obtain center coordinates of each charging hole in the two-dimensional image; step 4, establishing a world coordinate system based on charging hole space distribution defined by the automobile charging port standard model, and determining three-dimensional coordinates of each charging hole; and 5, the multi-view two-dimensional center coordinates and the corresponding three-dimensional coordinates are input into the PnP graph optimization model, and the pose of the charging port in the mechanical arm base coordinate system is solved by fusing the re-projection error constraint, the structure prior constraint and the kinematics chain constraint. The invention further discloses a charging robot automobile charging port pose vision measurement charging method and a charging system.

The invention discloses a multi-modal information fusion odometer construction method for star catalogue positioning, and relates to the technical field of star catalogue patroller positioning. The method comprises the following steps: carrying out space joint calibration on a monocular camera, a laser radar and an inertial measurement unit, reconstructing a laser radar point cloud by using a timestamp of a camera image, and realizing time synchronization of the camera image and the laser radar point cloud; establishing an IMU pre-integration error model; and performing motion compensation distortion removal on the laser point cloud by using an IMU pre-integration result, and extracting geometric features of the distorted laser point cloud based on a neighbor region smoothness calculation method of a fixed measurement distance. By researching a multimodal information fusion odometer method, the accumulative error of motion measurement is reduced, the positioning precision and stability are improved, technical support is provided for design and development of a star catalogue navigation system, and the problems that a single-modal star catalogue positioning method is weak in environment adaptive capacity and poor in algorithm generalization are solved.

The invention relates to a robot environment perception method based on single-view three-dimensional scene generation, and the method comprises the steps: collecting a two-dimensional image containing target environment information through employing a monocular camera, generating multi-view information through combining depth estimation, normal prediction and a two-stage semantic guidance diffusion model, and constructing a high-quality three-dimensional scene. And reconstructing three-dimensional point cloud data through the neural radiation field, and performing texture rendering optimization on the point cloud data. A Point Net + + network is used for carrying out semantic analysis on point clouds, a multi-frame time sequence point cloud registration and Kalman filtering tracking method is introduced, modeling is carried out on a dynamic target, and a dynamic semantic map with a motion state is constructed. And finally, structured output environment information is used for robot navigation, path planning and task execution. The problems of high cost, high complexity and insufficient real-time performance and robustness of a three-dimensional scene generation technology in the field of robot environment perception are solved, and the method is high in structuring degree, standard and unified in output format and high in universality and engineering adaptation capacity.

The invention discloses an aircraft target tracking method and system based on compensation prediction, which are used for improving the target tracking precision in an image transmission delay scene. The method comprises the following steps: firstly, acquiring an image frame sequence of a target aircraft by using an airborne monocular camera, extracting a target center coordinate through a small target detection algorithm, and constructing a position sequence; the method comprises the following steps: extracting current high-frequency I MU data aiming at the condition that an image frame has transmission delay, inputting the current high-frequency I MU data into an LSTM-DKF model constructed by fusing LSTM and a delay Kalman filter, and predicting and generating a process noise and observation noise covariance matrix; and initializing a delay Kalman filter by using the matrix, and recursively predicting the target position during the delay period. And when the delayed image frame is received, backtracking and updating the state of the filter, recurring to the current moment again, and outputting the compensated target position. And finally, pixel deviation is calculated according to the compensation position, an aircraft tracking control instruction is generated, and high-precision target tracking is realized.

The invention relates to the technical field of cabin loading and unloading automation, and provides a multi-sensor fusion cabin loading and unloading equipment cooperative positioning method and system. A laser radar is used for scanning a cabin, laser radar point cloud data are generated, and a laser radar coordinate system and a point cloud map are constructed; a monocular camera is adopted to obtain an operation area image of loading and unloading equipment, camera image data is generated, and a camera coordinate system is constructed; performing anti-interference processing on the laser radar point cloud data; performing illumination change influence resistance processing on the camera image data; fusing the laser radar point cloud data with the camera image data to obtain fused positioning information; calculating the distance and angle of loading and unloading equipment according to the fused positioning information, generating a moving instruction, and driving the loading and unloading equipment to perform cooperative positioning; a dynamic object in a cabin is detected through a laser radar and a monocular camera, multi-modal verification is carried out on a detection result, the dynamic object is filtered, and a point cloud map is updated in real time. The problems of insufficient positioning precision, GPS failure and large dynamic environment interference of a single sensor are solved.

The present discloses a lightweight attention mechanism distance estimation method for assisting visual navigation of a vehicle at a container terminal. Firstly, using a depth monocular camera calibrated with the imaging parameters of the planar checkerboard tool to collect RGB-Depth image pairs in the working scenario of the automatic guided vehicle. Secondly, performing depth completion and manual annotation processing on the collected depth image. Thirdly, inputting image pairs into a lightweight monocular metric depth estimation framework which uses an improved lightweight attention mechanism Squeeze Former as the token mixer for training. Finally, fusing the results of relative depth estimation and absolute depth estimation to obtain a prediction of an actual distance between an object in the RGB image and the camera in the real world. The method and model provided by the present invention feature simple equipment, low cost, high timeliness of prediction and accurate results.

The invention discloses a structure three-dimensional displacement monitoring method and device based on a monocular camera and a concentric annulus, and relates to the field of computer vision and structure health monitoring, a concentric annulus target is fixed at a to-be-monitored structure monitoring point, and the monocular camera carries out positioning and lens parameter adjustment; shooting a checkerboard image and calibrating the checkerboard image by adopting a camera calibration algorithm to obtain a distortion parameter and an internal reference matrix; collecting a video sequence containing a structure displacement process of the target; image distortion correction is carried out based on the distortion parameters and the internal reference matrix, a target is identified through a target detection algorithm, and target parameters are calculated through least square circle fitting; realizing multi-target tracking by a target point topology identification algorithm based on Y-X threshold sorting; calculating scale factors according to target parameters and actual physical sizes, and calculating in-plane orthogonal direction displacement and out-of-plane displacement to form three-dimensional displacement information of the structure; according to the invention, the monocular camera is adopted to realize the accurate measurement of the three-dimensional displacement of the structure and reduce the complexity and calibration difficulty of the system.

The invention discloses an underwater in-situ fish label-free sample detection method and equipment. The method comprises the following steps: acquiring an in-situ image shot by a monocular camera of an underwater observation platform, and constructing an image-text structured knowledge base; extracting a training and verification data set of the detection model from the knowledge base, and dividing the data set into a closed set category and an open world category; receiving a natural language text instruction which is input by a user and is used for describing the to-be-detected data set, and generating a text feature vector of a target category; a targeting auxiliary model is obtained through vision-text semantic space training, dynamic interactive fusion is carried out on the visual features of the in-situ image and the text features of the category, and end-to-end joint optimization is carried out on the target model; and performing label-free detection on fishes appearing in the underwater in-situ image by using the optimized target model. According to the method, new species which are not seen during training can be effectively identified, cross-domain reasoning can be implemented on a label-free observation sample, and the retraining and maintenance cost is reduced.

The invention discloses a height measurement method, a marking method and system, a storage medium and a program product, and relates to the technical field of laser marking, and the method comprises the steps: transmitting first laser through a laser, and projecting a preset first number of first light spots to the surface of a measured object; shooting the first light spot through a monocular camera to obtain a first pixel coordinate of the center of the first light spot in a camera coordinate system; a preset second number of standard scale points on a target laser light path and the height of each standard scale point are determined, the standard scale points are projected to the camera coordinate system, second pixel coordinates of the standard scale points in the camera coordinate system are obtained, and the target laser light path is the light path of the first laser; and calculating the distance between the first pixel coordinate and the second pixel coordinate, determining a target standard scale point with the minimum distance, and taking the height of the target standard scale point as the height of the measured object. According to the invention, the height measurement cost is reduced.

The invention discloses a rapid robust monocular vision inertial positioning method and system. The method comprises the following steps: configuring IMU and monocular camera sensor parameters; preprocessing the IMU data; iMU attitude initialization is carried out; image features of the monocular camera are extracted, abnormal matching points are removed, and IMU pre-integration is carried out; monocular vision initialization is carried out; odometer attitude optimization: judging whether the feature points are mismatched or not based on an IMU pre-integration result, constructing a feature point re-projection error jacobian matrix, and performing blocking and diagonalization processing on the re-projection error jacobian matrix to respectively optimize inverse depths and image frame attitudes of the feature points; selecting a key frame based on a key frame identification rule after the current frame attitude update is completed; and judging according to the current frame, and removing a certain frame in the sliding window to reserve a sliding window space for adding the latest frame. According to the method, the inverse depth of the feature points and the image frame attitude are optimized respectively based on the counterweight projection error Jacobian matrix partitioning and diagonalization processing, and the calculation complexity is reduced.

The invention relates to the technical field of multi-person posture estimation, in particular to an interactive scene data synthesis and key point visibility updating algorithm based on monocular vision, which comprises the following steps of: 1, arranging single-person picture data under monocular shooting vision, processing an input single-person picture under monocular shooting vision by using a GrondingDINO visual language open type target detection model, and obtaining a target detection result; and the Person is used as a retrieval keyword to identify a character individual in the picture. Through fine processing of the steps, especially introduction of target matching, target position adjustment and key point visibility updating methods, the quality of the synthesized image is greatly improved, the fusion degree of the target and the background is optimized through accurate matching and natural target pasting, and the image quality is improved. The space consistency and the visual naturalness of the synthetic image are enhanced, the problem of overfitting is effectively avoided through the improvement, the diversity of training data is enhanced, and therefore the generalization ability of the model is improved.

The invention provides an oil taking port positioning method and system based on monocular vision and laser positioning. The method comprises the steps that laser point cloud data and a monocular image near an oil taking port are acquired; constructing a three-dimensional model of the oil taking port based on the laser point cloud data, and projecting the three-dimensional model of the oil taking port into a two-dimensional slice image according to an acquisition position label of the monocular camera; matching the two-dimensional slice image with the monocular image to determine the two-dimensional feature position of the oil extraction port; calculating the feature distance between the oil taking port and the camera based on the matched two-dimensional feature position of the oil taking port and the focal length label and the size label of the monocular image; on the basis of the position parameters of the oil taking port in the laser point cloud data, the feature distance is combined, and initial three-dimensional coordinates of the oil taking port are generated; and unifying the pixel coordinates of the monocular vision and the laser point cloud coordinates into the same coordinate system, and carrying out preliminary three-dimensional coordinate data fusion to obtain the final three-dimensional coordinates of the oil extraction port. The positioning precision of the transformer oil taking robot on the oil taking port is improved.

At least one monocular camera is installed in a vehicle in addition to in-cabin camera(s). Images captured by the in-cabin camera(s) at a first frame rate are processed to determine first positions of facial landmark features of a user and first poses of the user's face. Images captured by the at least one monocular camera at a second frame rate, higher than the first frame rate, are processed to identify facial landmark features of the user in the second images. Second positions of the facial landmark features are determined at the second frame rate, by correlating the facial landmark features identified in the second images with the first positions of the facial landmark features determined from the first images. Second poses of the user's face are determined at the second frame rate based on the second positions of the facial landmark features and the first poses of the user's face.

The invention relates to a visual obstacle avoidance method and device, and the method comprises the steps: S1, arranging a monocular camera on a moving carrier, and enabling an image collected by the monocular camera to comprise a preset range at the front lower part of the moving carrier; s2, collecting a sample image through the monocular camera and calibrating the sample image to obtain a relational expression of the distance between a target in the image and the mobile carrier; s3, acquiring a formal image through the monocular camera, and identifying an obstacle in the formal image; and S4, determining the distance between the obstacle and the mobile carrier through the relational expression. According to the invention, the monocular camera is arranged on the mobile carrier and only collects the image in the limited range of the front lower part, and the distance information with the target obstacle can be obtained from the collected image, so that the obstacle avoidance judgment is realized, and compared with the existing monocular vision that the depth-of-field information cannot be obtained, so that the obstacle distance cannot be accurately judged, and the obstacle avoidance accuracy is improved. Good application prospects are realized.

Bird's eye view (BEV) semantic mapping systems and methods are provided. A method includes receiving an image captured by a monocular camera having a first point of view (POV) of an environment including a plurality of features. The method further includes processing, by an artificial neural network (ANN), the captured image to generate a semantic map for the captured image, the semantic map associated with a second POV different from the first POV. The features exhibit a uniform scale in the semantic map. Additional methods and associated systems are also provided.

The invention discloses a method and a system for quickly positioning the vertex of a suspension arm based on laser radar point cloud calibration monocular depth estimation, and the method comprises the steps: firstly constructing a power transmission line external damage prevention monitoring and shooting scene in a simulation environment, collecting data, and training a lightweight monocular depth estimation network suitable for the power transmission line scene and a monitoring and shooting visual angle; secondly, constructing a truncated cone containing construction machinery by using rapid projection between the monocular camera and the laser radar point cloud, performing depth scale calibration on a monocular depth estimation result by taking the laser radar point cloud as a reference, and performing rapid densification on sparse point cloud in the truncated cone; and finally, accurately positioning the three-dimensional coordinates of the vertex of the suspension arm in the dense virtual point cloud in the crane truncated cone. The method is suitable for a vision and laser radar integrated power transmission line external damage prevention edge monitoring and shooting device, supports one kind of related applications of accurate distance measurement between the suspension arm vertex and the power transmission line, can overcome the problem of sparse point cloud of a long-distance suspension arm, and improves the three-dimensional positioning precision of the suspension arm vertex.

The invention relates to the technical field of optical three-dimensional measurement and metering, and discloses a single-camera three-dimensional coordinate measurement method and system based on full-field distance measurement, and the method comprises the following steps: S1, building a calibration field containing a control point, shooting an image of the calibration field through a monocular camera, extracting the image coordinate of the control point, carrying out the camera calibration, and obtaining a calibration field; obtaining internal parameters and external orientation parameters of the camera; and S2, controlling the laser range finder to aim at the control point in the calibration field through the non-orthogonal double-axis turntable. Submillimeter-level high-precision measurement can still be realized under the condition of a short baseline, and the problems that the precision in the depth direction is insufficient, the baseline requirement is long, parameters are easy to drift and the like in traditional stereoscopic vision measurement are effectively solved. By introducing a non-orthogonal double-shaft turntable and a visual guidance laser aiming mechanism, autonomous, rapid and accurate measurement of multiple measurement points in a large field of view is realized, and the automation degree and reliability of measurement in a complex environment are remarkably improved.

The invention belongs to the technical field of computer vision, and particularly discloses a monocular 3D object detection method for realizing depth enhancement based on a visual basic model, electronic equipment and a readable storage medium, and the method comprises the steps: S1, building a data set: employing a monocular camera to collect a pavement scene, and obtaining an RGB image in the pavement scene; s2, image preprocessing: preprocessing the RGB image for subsequent feature extraction and depth estimation; s3, performing feature extraction by adopting a dual-backbone network: performing visual semantic feature extraction on the preprocessed RGB image by using DINOv2; performing depth feature extraction on the preprocessed RGB image by using a DPT head; s4, generation of depth perception query points: inputting the visual semantic features and the depth features into a DETR network to generate the depth perception query points; and S5, target detection output: using an MLP-based detection head to obtain information of the category, the size, the center point position, the depth, the 3D size and the direction of the object.

The invention provides a 3D anti-collision detection method and system based on monocular vision, and the method comprises the steps: obtaining the image data of a front environment collected by a vehicle-mounted monocular camera in real time, and carrying out the preprocessing of the image data, and obtaining the standardized image data; generating an optimized absolute depth map by fusing a first depth estimation result and a second depth estimation result based on the standardized image data; detecting a 3D obstacle based on the optimized absolute depth map and scene semantic information to obtain related information of at least one target obstacle; and performing collision risk assessment according to the related information, and triggering early warning when the risk meets an early warning condition. The invention provides a high-precision and high-robustness monocular 3D anti-collision detection scheme, monocular depth estimation based on deep learning, semantic segmentation and a dynamic safety model are combined, and an end-to-end anti-collision system with physical significance is formed.

The invention discloses a low-altitude unmanned aerial vehicle autonomous cruise method and system based on cloud edge basic model collaboration, and the method comprises the following steps: S1, collecting an environment RGB image through an airborne monocular camera of an unmanned aerial vehicle, and carrying out the preprocessing of the image, and obtaining a preprocessed gray image; s2, a neural scheduler based on deep reinforcement learning generates a scheduling instruction according to the environment data and the network state reasoned by the navigation model at the previous moment; s3, generating a preliminary flight instruction; s4, generating an optimized flight instruction; s5, generating a structured flight instruction; and S6, the unmanned aerial vehicle executes the preliminary flight instruction or the optimized flight instruction or the structured flight instruction. According to the invention, through dynamic on-demand cooperation and intelligent scheduling of the end-edge-cloud three-level model, resource consumption and delay are substantially reduced, and high-robustness and high-safety autonomous cruise of the unmanned aerial vehicle in a complex open environment is realized.

In general, disclosed herein are systems and methods of using a monocular camera to determine an estimated time from encounter between an object a region of interest, including receiving an image from a camera, identifying an object from the image, calculating a scaled distance between the object and the region of interest based on the image, calculating a scaled velocity based on the scaled distance, and calculating an estimated time from encounter until the object will encounter the region of interest based on the scaled distance and the scaled velocity.

The invention discloses an excavator operation loading capacity and unloading capacity monitoring method, equipment and a product, and relates to the field of volume monitoring, the method comprises the following steps: using a monocular camera to obtain an operation video of an excavator operation device, and determining images of continuous time points according to the operation video; annotating the images at different time points and in different operation states; acquiring laser point cloud data corresponding to the image at each time point; performing camera internal and external parameter calibration on the marked image, and obtaining a three-dimensional point cloud data set of the excavator operation device according to the corresponding laser point cloud data; according to the three-dimensional point cloud data set of the excavator operation device and the marked image, laser point cloud data of the excavator bucket in different operation states are extracted; and the position and the loading variation of the excavator bucket are determined based on the laser point cloud data of the excavator bucket in different spatial forms in the same operation state period. According to the invention, the quality and precision of engineering machinery operation dynamic monitoring can be improved.

Provided are systems and techniques for automated navigation of vehicles, such as drones. The systems generally include processing unit(s) that, collectively, perform several steps. Such steps include generating metric depth estimates, using a pre-trained model, for each pixel in received image(s) from a monocular camera, or transformed image(s) based on the received image(s). Such steps may also include generating a pose estimate from visual odometry, then generating a truncated signed distance function representation of an environment based on the absolute depth estimates and the pose estimate. The steps may include creating and / or updating a local map based on the truncated signed distance function representation. The steps may include plan a collision-free route towards a goal based on the local map. This may include using motion primitives, which may be generated in a single offline step and stored in a trajectory library.

The invention relates to the technical field of computer vision and image processing, in particular to an end-to-end image registration method based on a monocular camera and micro rendering. The objective of the invention is to solve the problems of low registration precision and insufficient real-time performance caused by complex calculation, serious error propagation and poor adaptability to multi-modal data in a traditional image registration method. According to the main scheme, the method comprises the steps of extracting structured boundary information of a target object in an input image through a contour segmentation model, and generating a binary mask image; based on the contour segmentation result, estimating a 6D pose parameter of the target object through a 6D pose estimation deep network fusing a convolutional neural network CNN and a Transform module; and inputting the 6D pose parameters into a micro-renderable module, generating a prediction image, comparing the prediction image with an original input image, and performing end-to-end optimization and adjustment on model parameters to realize high-precision registration.

The invention provides a large-view-field multi-view camera calibration method, system and device and a medium. The method comprises the following steps: acquiring a multi-view-field target plate image in a large-view-field plane by using a monocular camera; the target plate comprises a plurality of coding targets; obtaining target feature points of the plurality of coding targets based on the target plate image so as to correspondingly obtain target code values based on the target feature points; and obtaining a three-dimensional point coordinate of each target feature point based on each target code value, and carrying out multi-view camera calibration based on the three-dimensional point coordinates. According to the invention, the large-view-field multi-view camera can be calibrated, and the calibration precision is high.

The invention discloses an abnormal gait assessment method and system based on a monocular video, and a computing device. The method comprises the following steps: acquiring a human skeleton point sequence of the monocular video; performing data correction on the obtained human skeleton point sequence; and inputting the corrected human skeleton point sequence into a pre-trained space-time diagram convolutional network model, and obtaining an output result of the space-time diagram convolutional network model, the output result including an evaluation result of the gait anomaly condition of the subject in the monocular video. The method can effectively solve the problems of left and right leg confusion, easy shielding of key points and the like of skeleton point extraction in the monocular video, can enhance capture and understanding of gait space-time dynamic features, realizes accurate extraction of human gait space-time features, improves the accuracy and robustness of monocular video gait analysis, and improves the accuracy and robustness of the monocular video gait analysis. Effective gait analysis can be carried out by using a common monocular camera (such as a smart phone), the cost is lower, and the application scene is wider.

The invention relates to the technical field of computer vision, and discloses a personnel positioning method and device based on monocular vision, equipment and a storage medium, which are used for reducing the hardware deployment cost of personnel positioning and improving the personnel positioning accuracy. The monocular vision-based personnel positioning method comprises the following steps of: estimating depth information by utilizing a depth detection model, and determining an actual vertical distance from a ground contact point to a monocular camera and an actual horizontal distance of a horizontal point pair; fitting the depth information and the actual vertical distance through a polynomial regression equation so as to determine a vertical coordinate mapping relation of the pixel coordinates in a camera reference coordinate system; determining a horizontal coordinate mapping relation according to the actual horizontal distance of the horizontal point pair and the pixel pitch of the horizontal point pair; and determining target positioning information through a coordinate system conversion matrix between the working position and the correction position in combination with the vertical coordinate mapping relation, the horizontal coordinate mapping relation and the relative position information.

The specification discloses an image recognition-based robot charging method, system and electronic equipment, which has the advantages of low cost, high precision, wide recognition range, etc. The method comprises: acquiring a work scene image by using a monocular camera on the robot, and extracting a charging pile image from the work scene image; performing image recognition on the charging pile image to obtain position information of a feature icon on the charging pile in the charging pile image; judging a positional relationship between the robot and the charging pile according to the position information; performing path planning according to the positional relationship, generating charging path parameters, and controlling the robot to move to the charging pile for charging based on the charging path parameters. The system comprises: a charging pile image extraction module, a feature icon positioning module, a robot positioning module, and a path planning control module. The processor in the electronic equipment implements the method when executing a program.