326 results about "Monocular image" patented technology

The invention provides a posture recognition algorithm for any object under a monocular camera and an application system, and the algorithm comprises the steps: S1, constructing a target three-dimensional model, carrying out the multi-view annular shooting image collection of a target, and generating a dense grid model through feature extraction, matching, posture calculation and a multi-view geometric method; s2, generating an image depth map, and predicting depth information of a target in a motion process based on a monocular image sequence; s3, extracting a target image mask, and generating a target area mask graph through an image encoder, a prompt encoder and a mask decoder; and S4, executing attitude estimation, performing attitude initialization, correction and screening by combining the three-dimensional model, the depth map and the mask map, and outputting a six-degree-of-freedom attitude result of the target. According to the method, the target is subjected to annular shooting modeling through the method based on multi-view geometry, the three-dimensional model of the target is generated, attitude estimation is achieved in combination with the image mask and the depth map, the generalization ability of an attitude estimation algorithm in an actual scene is improved, and the application range of the attitude estimation algorithm in the actual scene is widened.

The invention discloses a three-dimensional scene reconstruction method and device based on large model geometric prior, and a medium, and aims to solve the problems that a conventional 3DGS is liable to have artifacts and detail loss in geometric discontinuity, data redundancy and illumination variation scenes, and predicts a dense depth map and a normal map from a monocular image by using a pre-trained large model. The position and form of the Gaussian kernel are constrained as additional geometric priori; a primitive adjustment strategy based on kernel density estimation is introduced in the training stage, small Gaussian primitives with similar structures and adjacent spaces are combined into a large Gaussian primitive, the rendering quality is kept, redundancy is reduced, and the volume of the model is reduced; an exposure coefficient is adaptively estimated for each input image, an exposure compensation image loss function is constructed, and floating artifacts caused by illumination differences at shooting moments are eliminated. Experiments show that compared with the prior art, the method improves the three-dimensional reconstruction precision and real-time rendering quality of complex illumination and less-texture areas in a public data set and an unmanned aerial vehicle aerial photography scene.

The invention provides a depth map generation method and device based on a large model, a three-dimensional reconstruction method and device, electronic equipment and a storage medium, relates to the technical field of artificial intelligence, in particular to the technical fields of computer vision, deep learning, large models and the like, can be applied to real-time road scene depth perception, environment three-dimensional reconstruction and obstacle avoidance, and can be applied to real-time road scene depth perception. And virtual and real scene fusion and other scenes can be realized. The specific implementation scheme is as follows: performing visual coding on a monocular image to obtain a coded image; inputting the coded image and the target text into a pre-trained large language model for fusion to obtain fusion features; generating global guide features based on the fusion features, wherein the global guide features comprise joint semantic information of visual features and text features; adding noise to the color image of the monocular image to obtain a noise feature sequence; de-noising the noise feature sequence under the condition of the global guide feature, and generating an implicit feature matched with the joint semantic information; a depth map is generated based on the implicit features.

The present invention provides a single-image-based three-dimensional face reconstruction and editing method, comprising: extracting features by means of a pre-trained EG3D network, so as to generate a preliminary three-dimensional face; using an inversion module to map a monocular image to a latent code space, and optimizing a latent code to generate a realistic three-dimensional face image; making use of a depth estimation technology and multi-view projection to create a pseudo multi-view image; then, by means of semantic segmentation and expression feature extraction, fusing the features to generate a comprehensive representation; according to editing requirements, optimizing and adjusting the latent code, so as to implement customized editing; and finally, a generator outputting an edited face image. The present invention can enable more efficient, flexible and high-quality three-dimensional face reconstruction and editing, thereby effectively overcoming the defects of high cost, low universality and lack of customized editing in existing face reconstruction technologies.

A method of 3D object detection using an object detection neural network includes: receiving one or more monocular images; extracting 2D feature maps from each one of the one or more monocular images by passing the one or more monocular images through a 2D feature extracting part of the object detection neural network, generating an averaged 3D voxel volume based on the 2D feature maps, extracting a 2D representation of 3D feature maps from the averaged 3D voxel volume by passing the averaged 3D voxel volume through an encoder of a 3D feature extracting part of the object detection neural network, and performing 3D object detection as 2D object detection in a Bird's Eye View (BEV) plane, the 2D object detection in the BEV plane being performed by passing the 2D representation of 3D feature maps through thane outdoor object detecting part of the object detection neural network.

The invention belongs to the technical field of three-dimensional mapping, and particularly relates to an occlusion area three-dimensional voxel reasoning method and system. The method comprises the following steps: S10, acquiring an RGB image containing a semantic segmentation result of an occlusion region and a corresponding depth map, and converting the depth map into an initial sparse three-dimensional voxel; and S20, constructing a three-dimensional voxel inference network, and inferring a three-dimensional voxel occupancy probability graph from the initial sparse three-dimensional voxels through the three-dimensional voxel inference network. The occlusion area can be reconstructed based on the monocular image; multi-format output and navigation interface butt joint are supported, and the method can be integrated to SLAM, 3D mapping or path search systems to serve as spatial feasibility constraints; the method can be used for querying whether any spatial position is passable; the method can be used for extracting a trafficability sub-graph from a designated area for path planning.

The invention relates to the field of robot operation and three-dimensional perception, in particular to a monocular image-based three-dimensional perception robot operation knowledge distillation method, which comprises the following steps of: establishing a strategy learning framework comprising a student model and a teacher model; constructing a strategy prediction model in the strategy learning framework; training the strategy prediction model, wherein a three-level knowledge distillation mechanism is adopted in the training process to complete knowledge migration between the teacher model and the student model; combining the three distillation losses with strategy optimization losses to form a total loss function, and performing end-to-end training on the student model until convergence to obtain a deployable monocular strategy model; the deployable monocular strategy model only retains a student model, and generates a robot operation instruction. The method has the beneficial effects that the robot under monocular RGB input has three-dimensional perception and high-precision operation capabilities while the reasoning efficiency is kept, the task success rate and generalization performance are remarkably improved, and the effectiveness and robustness of the method are verified.

Systems and methods are provided for training a depth model to recover scale factor for self-supervised depth estimation in monocular images. Examples include deriving a depth map for an image based on a depth model. The depth map comprises depth values for pixels of the image. A first scale for the image can be estimated based on the depth values, and depth data captured by a range sensor can be received. The depth data comprises a point cloud comprising depth measures. A second scale for the point cloud can be determined based on the depth measures and a scale factor can be determined based the second scale and the first scale. The depth model can be updated based on the scale factor, wherein the depth model generates metrically accurate depth estimates based on the scale factor.

The invention belongs to the technical field of robot positioning and three-dimensional mapping, and discloses a high-precision map reconstruction method and system based on monocular vision, a medium and equipment. Scene image data are acquired through a monocular image acquisition module, after feature extraction and matching are performed on each frame of image, attitude information acquired by an inertial measurement unit is fused, and pose calculation of a robot is completed through a sparse vision SLAM system; meanwhile, an image dense depth map is generated by a monocular depth estimation model based on an attention mechanism, and the image dense depth map is converted into a single-frame color dense point cloud in combination with image RGB color information. According to the invention, a loose coupling fusion strategy is adopted to carry out spatial registration on a single-frame colored dense point cloud and a robot pose at a corresponding moment, multi-frame data fusion is completed through point cloud splicing and optimization, and a globally consistent three-dimensional dense point cloud map is constructed to realize scene modeling. The method has the characteristics of high robustness and high reconstruction precision, and can be effectively applied to three-dimensional map construction in an outdoor complex environment.

A monocular image depth estimation method and apparatus, and a computer device are provided. The method includes: obtaining a first depth map of a to-be-estimated image; obtaining a dynamic point set and a first pose transformation result of a to-be-estimated millimeter-wave point cloud; obtaining a second depth map of a latter frame of the to-be-estimated image; calculating a projection error between a first depth map and the second depth map of the latter frame of the to-be-estimated image; obtaining a second pose transformation result of the to-be-estimated millimeter-wave point cloud; obtaining a pose estimation error; calculating a depth error of a moving object in two frames of the to-be-estimated image; and obtaining an overall training loss, obtaining a complete depth estimation model, and performing monocular image depth estimation on the to-be-estimated image.

The invention discloses a semantic scene completion method and device, electronic equipment and a readable storage medium. The method comprises the following steps: acquiring a binocular image; performing depth estimation processing on the binocular image to obtain a multi-scale image feature map, a disparity probability distribution map and a disparity map of the monocular image; performing conversion processing from a parallax dimension to a depth dimension on the parallax image of the monocular image to obtain a depth image of the monocular image; processing the multi-scale feature map and the depth map of the monocular image to obtain a first semantic feature map of the monocular image; performing depth feature extraction processing on the parallax probability distribution diagram of the monocular image to obtain a depth probability distribution diagram of the monocular image; and determining voxel occupation data and voxel semantic data of the semantic occupation data based on the depth map of the monocular image, the first semantic feature map and the depth probability distribution map so as to perform semantic scene completion. According to the invention, semantic scene completion can be carried out based on visual image data.

A computer-implemented method includes converting by a pose tokenizer, based on a learned codebook, pose parameters of a body into a sequence of discrete pose tokens; randomly masking a portion of the sequence of discrete pose tokens; predicting the randomly masked sequence of discrete pose tokens based on multi-scale features extracted from a monocular image by an image conditioned masked transformer; optimizing the sequence of discrete pose tokens by aligning a re-projected three-dimensional (3D) pose with an estimated two-dimensional (2D) pose; directly regressing, from the multi-scale features, a shape parameter of the body and a weak perspective camera parameter; and generating a 3D mesh reconstruction of the body based on the shape parameter and the weak perspective camera parameter.

The invention relates to an endoscope monocular image depth estimation method and system, and the method comprises the steps: obtaining the front and rear frame images of an endoscope monocular; an endoscope monocular is trained by using a DA-I CGA model based on a DARES, the DA-I CGA model comprises a PoseNet module and a DAM-LoRA module, and a double-branch geometric perception module and an image-level contrast learning module are creatively introduced; and performing depth estimation on the to-be-detected endoscope monocular image through the trained DA-I CGA model. According to the method, artifacts generated in an overexposure region can be effectively solved, and the monocular depth estimation precision is further improved.

The invention relates to a distance measurement positioning method and system, a storage medium, electronic equipment and a vehicle, and the method comprises the steps: carrying out the processing of a collected initial monocular image through a trained target detection model, and obtaining a detection object; if the target object exists in the detection objects, obtaining initial depth information of the target object; intercepting a target area from the binocular image of the target object based on the initial depth information; and determining target depth information of the target object based on the target area. According to the method, through the improved target detection model, fine positioning of the object is realized through coarse positioning, the positioning efficiency and precision are improved, and the method is suitable for being deployed on edge equipment.

The invention provides a physical driving measurement method for monocular three-dimensional dynamic displacement of a rotary machine, and belongs to the technical field of crossing of computer vision and industrial state monitoring. According to the method, a high-speed dynamic visual acquisition system is constructed, and a time sequence video stream is obtained by using a cooperative marker; constructing a'Gaussian + motion blur 'composite gradient model taking the motion blur width as an endogenous variable, and jointly resolving sub-pixel edge coordinates and ambiguity by adopting a nonlinear optimization algorithm; decoupling the pixel displacement of the radial X axis, the radial Y axis and the axial Z axis by using the width change and the edge displacement of the marker in combination with the geometric principles of sequential robust filtering and monocular imaging; and finally, outputting a three-way physical vibration waveform by using the calibration conversion factor. According to the method, the fuzzy influence is adaptively eliminated from a physical imaging mechanism, micron-level precision three-direction vibration synchronous measurement is realized only by a single camera, and the hardware cost and the deployment difficulty are reduced.

The invention discloses a glaucoma fundus image classification method, which comprises the steps of acquiring existing glaucoma fundus image data information, and performing classification labeling and preprocessing to obtain a training data set; constructing a glaucoma fundus image classification initial model based on a ConvNeXt model, and training to obtain a glaucoma fundus image classification model; and adopting the obtained glaucoma fundus image classification model to classify actual glaucoma fundus images. The invention further discloses an imaging method comprising the glaucoma fundus image classification method. According to the method, the glaucoma fundus image classification network with the feature fusion process is constructed through the ConvNeXt model, so that classification of the glaucoma fundus images can be achieved, the method is suitable for classification of monocular images and binocular images, reliability is high, accuracy is good, the application range is wide, and the method is more objective and scientific.

The invention provides a bas-relief three-dimensional reconstruction method based on a monocular image, and belongs to the technical field of image processing. A digital camera is used for collecting relief images, dense point clouds containing 3D coordinates and RGB values are generated based on a photogrammetry method, and a bas-relief data set containing monocular images and semantics, soft edges and depth labels of the monocular images is established. Training a soft edge enhanced multi-task neural network framework based on the labeled data, simultaneously extracting semantic, edge and depth features in a monocular image in the framework, and taking the semantic and edge features as effective clues of a depth estimation task through a multi-task loss function and an edge comparison module; the depth estimation capability of the model on the detail part is enhanced, and the precision of three-dimensional reconstruction is improved. According to the method, the corresponding soft edge and semantic information can be directly extracted from a monocular relief photo, a depth estimation task is assisted, the capability of capturing details of an edge region is enhanced, and the precision of subsequent bas-relief three-dimensional reconstruction is effectively improved.

The invention discloses a monocular visual odometer method and system based on aerial view representation and differentiable weighted Procrustes solver, and the method comprises the steps: constructing a pose estimation model which can carry out the extraction and matching of key points in an aerial view, and carries out the relative pose estimation through an interpretable weighted Procrustes solver; the method comprises the following steps: constructing a loss function based on pose information, optimizing parameters of a pose estimation model by using the loss function, and performing pose information estimation between two frames of monocular images by using the parameter-optimized pose estimation model to realize the monocular visual odometer, so that the monocular visual odometer is superior in the aspect of reducing scale drift; and stable performance is shown under different weather and illumination conditions.

The invention discloses a monocular image depth estimation method based on multi-scale information fusion, and the method comprises the steps: outputting a first feature map representing local information through a CNN branch, and outputting each second feature map representing global feature correlation at a Transform branch; enhancing the input first feature map and each second feature map from three dimensions of channel, space and cross-scale by using an adaptive fine-grained channel space collaborative gating module to obtain each third feature map; using attention mechanism decoders constructed based on an up-sampling module to sequentially process the input third feature maps to obtain a monocular image depth estimation feature map; according to the invention, the problem that the processing capability of a model on a remote area in a monocular depth estimation scene cannot be improved because local features and global features cannot be effectively combined in the prior art can be solved.

The invention relates to the technical field of monocular depth estimation, and discloses a monocular self-supervision depth estimation method based on credible region depth controllable generation. Firstly, a monocular image is collected and input into a depth estimation module for multi-scale feature extraction, an initial depth map of a whole scene is generated, and relative camera attitude transformation between adjacent frames is estimated through an attitude estimation module; identifying a credible area and an uncredible area according to the obtained initial depth map and relative camera attitude transformation; depth estimation of a dynamic region is guided through a pseudo depth and contrast learning method; then, constructing a multi-task joint loss function, and optimizing depth estimation of the untrusted region by utilizing depth information of the trusted region through a comparison loss mechanism; and iteratively optimizing network parameters of the depth estimation module and the attitude estimation module, and outputting a final optimized depth map. According to the method, the depth estimation precision and robustness of the model in a complex dynamic environment are improved through a credible region depth controllable generation strategy.

The invention provides an oil taking port positioning method and system based on monocular vision and laser positioning. The method comprises the steps that laser point cloud data and a monocular image near an oil taking port are acquired; constructing a three-dimensional model of the oil taking port based on the laser point cloud data, and projecting the three-dimensional model of the oil taking port into a two-dimensional slice image according to an acquisition position label of the monocular camera; matching the two-dimensional slice image with the monocular image to determine the two-dimensional feature position of the oil extraction port; calculating the feature distance between the oil taking port and the camera based on the matched two-dimensional feature position of the oil taking port and the focal length label and the size label of the monocular image; on the basis of the position parameters of the oil taking port in the laser point cloud data, the feature distance is combined, and initial three-dimensional coordinates of the oil taking port are generated; and unifying the pixel coordinates of the monocular vision and the laser point cloud coordinates into the same coordinate system, and carrying out preliminary three-dimensional coordinate data fusion to obtain the final three-dimensional coordinates of the oil extraction port. The positioning precision of the transformer oil taking robot on the oil taking port is improved.

The invention discloses an underwater target identification and positioning method based on fusion of a physical model and deep learning, and belongs to the technical field of computer vision, and the method comprises the steps: carrying out the transmissivity estimation and physical restoration of a collected underwater monocular image according to an underwater light propagation model, and carrying out the image correction; key feature points are extracted, and high-quality matching point pairs are screened in combination with the joint similarity; further deriving a basic matrix through the high-quality matching point pairs meeting the epipolar geometric constraint relation, solving an essential matrix, and obtaining relative attitude parameters between the cameras in combination with weighted re-projection-LM optimization; and finally, refraction correction triangulation is carried out according to the Snell's law, pixel-level fusion is carried out after scale normalization and space alignment are carried out on the refraction correction triangulation and dense depth output by the MiDaS, three-dimensional space coordinates of the target are inverted, and high-precision recognition and positioning of the underwater target are achieved. The system is light in structure, efficient in calculation, suitable for being integrated on various underwater autonomous or remote control robot platforms and used for tasks such as target recognition, tracking and positioning.

The invention provides a fish body length estimation method and system based on a global guidance semantic segmentation network, relates to the technical field of computer vision and artificial intelligence, and aims to solve the problems that in a scene that a fish body in a monocular image and complex background interference coexist, and in a multi-fish-species mixed scene, the model segmentation generalization ability in the prior art is insufficient, and the fish body length estimation accuracy is poor. And high-precision and robust mask segmentation is difficult to realize. The method comprises the following steps: acquiring a fish body monocular image, and performing distortion correction on the fish body monocular image; inputting the corrected fish body monocular image into a trained global guide semantic segmentation network for semantic segmentation to obtain a semantic segmentation mask graph of the fish body and the calibration plate, and constructing a mapping relationship between the physical size and the pixel size of the calibration plate; and performing ellipse fitting on the semantic segmentation mask graph of the fish body, calculating the pixel length of the fish body, and converting the pixel length of the fish body into the actual length of the fish body. The method solves the problems in the prior art, provides the generalization ability of the network model, and achieves the precise estimation of the length of the fish body.

The invention relates to a method and a device for detecting personnel under a suspension arm based on a monocular image and computer equipment. The method comprises the following steps: acquiring a monocular image acquired by a camera installed on a suspension arm construction site, and inputting the monocular image into a yolov8 target segmentation model for segmentation detection to obtain a target area map comprising at least one suspension arm detection area and at least one target personnel detection area, then inputting the target area map into a DehanythingV2 depth estimation model for depth estimation to obtain a target area depth map, based on the target area depth map, obtaining the total map distance of the target area map, the suspension arm depth of each suspension arm detection area and the personnel depth of each target personnel detection area, and finally, according to the total map distance, the suspension arm depth and the personnel depth, determining the personnel depth of each target personnel according to the total map distance, the suspension arm depth and the personnel depth. And generating a detection result of personnel detection under the suspension arm. By means of the combination mode, the data labeling complexity and computing power resource consumption are greatly reduced, and then the detection efficiency of personnel under the suspension arm is improved.

The invention relates to an interpretable three-dimensional reconstruction method and device based on decoupling characterization learning and Gaussian splash, and a storage medium, and the method comprises the following steps: obtaining a monocular image, extracting two-dimensional features, obtaining compression features through convolution coding, obtaining a decoupled low-dimensional potential code through full connection processing, and obtaining a three-dimensional image; respectively converting into conditional representations of a geometric reconstruction branch and an appearance reconstruction branch; based on the two-dimensional coordinates on the predefined grid, three-dimensional points in a three-dimensional space are mapped through a plurality of MLP networks, a three-dimensional point cloud is obtained, and standard deviation-mean value modulation is carried out on the intermediate features by using conditional representation of geometric reconstruction branches; based on the three-dimensional point cloud, coding and projecting the three-dimensional point cloud to a plane to serve as initial three-plane features, and based on conditional representation of appearance reconstruction branches, coding the initial three-plane features into final three-plane features through a stylized U-Net network; and obtaining three-dimensional Gaussian scatter points based on the three-plane features and the three-dimensional point cloud to realize three-dimensional reconstruction.

The invention provides a slope rockfall monitoring and early warning method, device and equipment based on dynamic depth estimation and a medium. Relates to the technical field of civil engineering operation maintenance. The method comprises the following steps: S1, acquiring a video sequence image; s2, performing dynamic target detection on the video sequence image by adopting a frame difference method; s3, based on a pre-trained rockfall identification model, identifying rockfall in the target frame; s4, based on a depth-pro monocular image depth estimation model, performing depth estimation on the rockfall in the target frame to obtain a rockfall position distance; s5, performing feature analysis on the rockfall in combination with the depth data to obtain feature parameters; s6, performing tracking and trajectory identification on the rockfall; and S7, performing alarm reminding according to tracking and trajectory recognition results. According to the method, the extracted features are trained to identify the slope rockfall, the position and the size of the rockfall can be identified more accurately, a sharp measurement depth map is rapidly provided, and the dynamic change of the slope rockfall is captured in time.

The invention relates to a deep-sea mining nonmetal pipeline monocular image three-dimensional sparse reconstruction method and device. The method comprises the following steps that a robot carrying a monocular camera is used for obtaining internal pictures of a to-be-detected nonmetal pipeline to form a data set; performing feature point detection and matching on the data set by using a feature point matching network model based on deep learning; carrying out aggregation and redistribution on the obtained feature points; and based on the aggregated feature points, performing three-dimensional sparse reconstruction by using an improved incremental reconstruction algorithm to obtain internal structure information of the non-metal pipeline to be detected. The method can be widely applied to the technical field of nonmetal pipeline detection.

The invention discloses a miniature intelligent terminal intestinal image processing method and device, a storage medium and computer equipment. The method comprises the following steps: preprocessing an original intestinal tract image set acquired by a monocular endoscope, calculating according to the preprocessed intestinal tract image set to obtain a motion calibration scale factor and a texture calibration scale factor, and adaptively fusing to obtain a conversion coefficient from a pixel to an actual size; extracting polyp region key feature points of each image of the preprocessed intestinal image set, constructing a Delaunay triangulation graph according to each polyp region key feature point, and performing non-rigid deformation matching based on a cross-graph convolutional network and an optimal transmission algorithm to obtain a matched feature point set; and sparse 3D reconstruction is carried out according to the matched feature point set to obtain a polyp point cloud, and the polyp size is calculated according to the Euclidean distance between two farthest points in the polyp point cloud and the conversion coefficient. According to the method, the problems of monocular image scale ambiguity and non-rigid deformation can be solved, and the more accurate polyp size can be obtained.

The invention relates to the technical field of VR, and provides a cloud game display method and equipment based on VR equipment, which are used for improving the fluency of picture display. According to the method, aiming at the condition that a left-eye image and a right-eye image are independently coded and transmitted, when the decoding frame rate of a first-eye image video in a left-eye image video stream and a right-eye image video stream is smaller than a display frame rate, a cached first target image, a current frame of a normally received second-eye image and a binocular difference are used, and a second target image is obtained; the current frame of the first-eye image is predicted respectively, and the current frame of the first-eye image and the current frame of the second-eye image are displayed synchronously, so that the problem of picture jamming when the transmission of the monocular image is abnormal is solved, and the picture display fluency is improved; meanwhile, the field angle corresponding to the left eye image and the field angle corresponding to the right eye image are larger than the field angle of the VR equipment, so that the image black edge problem is reduced, the resolution ratio of the edge area is lower than that of the center area, the coding data volume is effectively reduced, and the transmission efficiency is improved.

The present invention provides a method, device, and apparatus for monocular image depth estimation based on a diffusion model and target prompts, and relates to the field of computer vision technology. The method comprises: inputting a sample detection image into a depth estimation model to be trained, processing the sample detection image through a pre-trained image encoder to obtain a shallow spatial representation; processing the sample detection image through a pre-trained target detection network to obtain a target detection result; processing the target detection result through a target prompt module to be trained to obtain target prompt information; inputting the shallow spatial representation and target prompt information into a denoising network to be trained to obtain multi-scale features; inputting the multi-scale features into an interactive decoder to be trained to obtain a sample depth map; and training the depth estimation model to be trained based on the label depth map and the sample depth map to obtain a trained depth estimation model for depth estimation, thereby improving the accuracy of monocular depth estimation based on the diffusion model.