50 results about "Normal estimation" patented technology

The invention provides a scene reconstruction method based on delayed rendering and three-dimensional Gaussian. The method comprises the following steps: S1, generating initial three-dimensional point cloud data based on a multi-view image; s2, constructing a trainable structural body for three-dimensional Gaussian modeling; s3, normal initialization and residual optimization are carried out on the Gaussian ellipsoid primitives, depth consistency constraint is combined, and a differentiable and learnable normal reconstruction mechanism is realized, so that the geometric expression ability of illumination modeling is enhanced; s4, introducing a reflection training mechanism based on ambient light and a reflection direction, and generating a Gaussian attribute based on a visual angle; and S5, a final image is generated through a differentiable Gaussian sputtering rendering algorithm, and optimization is carried out through pixel loss of the final image and a real image. According to the method, the reality sense and geometric consistency of the Gaussian sputtering model under the complex illumination condition are remarkably improved, and the technical problems of unreal rendering effect, inaccurate surface normal estimation, weak propagation capability and the like of the existing three-dimensional Gaussian sputtering model under the complex illumination condition are solved.

The invention discloses an occlusion perception 3DGS and neural SDF semantic level hybrid reconstruction method, which comprises the following steps: image preprocessing: initially obtaining an image video frame through a camera, starting from image preprocessing, processing an input image, including semantic segmentation, depth estimation, normal estimation and internal and external parameter calibration, a corresponding semantic graph, a depth graph, a normal graph and camera parameters are generated; aligning the 3DGS with the neural SDF, after preprocessing is completed, performing spatial alignment on geometric information generated based on the 3DGS and a neural symbol distance function, and performing restoration in combination with regional densification and a hierarchical depth map; and finishing the final three-dimensional object modeling and rapid reconstruction by adopting a Marking Cubes algorithm. According to the occlusion perception 3DGS and neural SDF semantic level hybrid reconstruction method provided by the invention, each object in a scene can be quickly reconstructed with high quality, and applications such as downstream editing tasks, interaction and the like are supported at the same time.

The invention discloses an intelligent low-altitude unmanned aerial vehicle obstacle avoidance path planning method and system. The method comprises the steps of multi-modal data acquisition, space-time alignment fusion, three-dimensional dynamic trajectory reconstruction, obstacle avoidance path planning and system verification. The invention belongs to the technical field of path planning, and particularly relates to an intelligent low-altitude unmanned aerial vehicle obstacle avoidance path planning method and system.According to the scheme, an incremental dynamic updating mechanism is adopted, and the three-dimensional trajectory reconstruction precision is improved through a double-correction mechanism of hidden space normal estimation and global discretization constraint; bEV feature optimization fusion is adopted to generate a double-enhanced feature map; constructing a mixed bounding box for the trajectory prediction points, calculating a dynamic minimum distance, designing a dynamic safety threshold fusing the real-time speed of the unmanned aerial vehicle and the priority, constructing a space-time coupling risk index in combination with the potential collision time and the priority to complete refined risk grading, and constructing a reinforcement learning model of priority weighted rewards; the problem that the passing right distribution is not uniform in the low-altitude multi-priority unmanned aerial vehicle obstacle avoidance process is solved.

The invention relates to the technical field of industrial nondestructive testing, in particular to an automatic welding seam tracking system of a nondestructive testing robot based on multi-modal sensing. Comprising a data acquisition module, a data preprocessing module, a data fusion module, a track generation module and a motion control module. Point cloud data are obtained through a laser scanner, image information is obtained through an RGB-D camera, attitude data are obtained through an inertial measurement unit, after multi-modal data are subjected to denoising, down-sampling and correction processing, data fusion is achieved through Kalman filtering, and continuous weld space trajectory points are output. And the track generation module generates a tail end pose sequence through three-dimensional B-spline fitting and surface normal estimation, and performs pose smoothing processing by adopting quaternion interpolation. The motion control module is combined with differential path keeping and mixed position-force control, it is ensured that the robot accurately tracks along the center line of the welding seam and maintains constant contact force, meanwhile, the process error correction function is achieved, and the stability and reliability of the system in the complex welding seam environment are improved.

The invention provides a method and system for evaluating the integrated photovoltaic potential of buildings in a high-density urban area, and the method comprises the steps: obtaining the color point cloud data of a city through the photographing and surveying of an unmanned plane, introducing the physical priori knowledge through algorithm design, and assisting a neural network in extracting a feature vector for representing the characteristics of a building group from the data; comprising the steps of point cloud sampling, normal estimation, window-wall ratio estimation and geographic orientation angle calculation, so that the model can learn implicit relation between the urban form and a photovoltaic potential prediction result; aiming at a scene of comprehensively realizing photovoltaic integration popularization of medium and large-scale buildings, the power generation efficiency of building facades and photovoltaic modules mounted at different positions of windows and walls is considered, feature vectors are input into a regression prediction framework formed by a one-dimensional point cloud neural network, evaluation errors are reduced, systematic deviation of resolution is avoided, and the evaluation accuracy is improved. Data acquisition and prediction are facilitated, and the function of predicting the maximum photovoltaic power generation potential after the building is comprehensively paved with the BIPV is realized.

The invention discloses a polishing path planning method and device of a visual language, a storage medium and a terminal, and the method comprises the steps: obtaining a visual language prompt which comprises a prompt word to-be-polished object image and a camera internal reference matrix; inputting the cue word and the image of the object to be polished into a pre-trained visual language model to obtain a semantic region mask; based on a depth map corresponding to a to-be-polished object image, using the camera internal reference matrix to perform spatial back projection on the semantic region mask to obtain a three-dimensional point cloud region; geometric analysis and normal estimation are carried out on the three-dimensional point cloud area to obtain a redundant material thickness sequence, and a normal difference weight coefficient is utilized to carry out priority division on the three-dimensional point cloud area according to the redundant material thickness sequence to obtain a first priority grinding area and a second priority grinding area; and optimizing the local path direction and the residence time of the first priority polishing area to obtain a polishing path. The polishing precision and efficiency of the complex curved surface workpiece are greatly improved.

The invention provides a point cloud normal estimation method and device based on local subspace clustering, electronic equipment and a storage medium, and the method comprises the steps: introducing a feature point weight evaluation mechanism based on covariance features and geometric structure indexes into a local neighborhood, and screening out feature points with high confidence; carrying out multi-subclass division and plane fitting on the local point cloud by adopting low-rank subspace clustering with priori weight constraint in the neighborhood of the feature points, so as to re-estimate the normal direction of the feature points on the subclass level; meanwhile, noise point filtering and a normal diffusion strategy based on adjacent non-noise points are combined, so that the method can more accurately describe a local geometric structure in a complex scene with noise, multi-structure aliasing and non-uniform sampling, the precision and stability of point cloud normal estimation are remarkably improved, and the method is suitable for popularization and application. And sharp features and edge details of the model can be better maintained.

The present disclosure provides a point cloud smoothing method and device based on local feature estimation, electronic equipment and storage medium. By introducing the noise point elimination of local density features, the normal estimation based on weighted local statistical analysis, and the neighbor update weight construction mechanism of fusing spatial distance, normal difference and normal distance, adaptive and structure-preserving smoothing processing of point cloud is realized. The method can effectively eliminate outliers and noise points, reduce the thickness of the point cloud, improve the stability of the normal estimation, and at the same time, preserve the object edges and geometric details in the smoothing process, significantly improving the overall quality and applicability of the point cloud.

The invention provides a rock structure-oriented point cloud normal vector estimation method based on voxel geodesic distance, and the method comprises the steps: obtaining original point cloud data of a three-dimensional rock structure, carrying out the voxelization processing of the original point cloud data, and dividing the point cloud into a plurality of three-dimensional voxel units; a sparse voxel graph is constructed based on voxel space adjacency relations, each voxel is a node of the graph, and adjacent voxels are connected through edges. Furthermore, the voxel geodesic distance between any two points is calculated by using a graph search algorithm, namely the shortest distance along a voxel communication path. On the basis, a target point is taken as a center, a neighborhood point set which is communicated with the target point is screened based on geodesic distance, the distance between the target point and the neighborhood point set is smaller than a threshold value, and a principal component analysis (PCA) method taking voxel geodesic distance as a weighting factor is adopted to estimate a normal vector of the target point. According to the method, pseudo neighborhood point interference caused by cracks, holes and the like can be effectively reduced, and the robustness and precision of normal estimation are improved.

A system for enhancing an image using hand surface normal estimation is provided. In a model training phase, 3D data of the hand at various positions are used to generate a 3D model of the hand. In one embodiment, target normal training data is generated, the target normal training data including a normal of a surface of a 3D model, and synthetic 2D image training data corresponding to the 3D model and the normal is generated. The target normal training data and the composite image training data are used for training a normal estimation model. The interactive application uses normal estimation to generate enhancements applied to the hand image data.

The application discloses a kind of weakly supervised image semantic understanding methods based on multi-task learning, comprising the following steps: obtaining task missing image, constructing multi-level task sharing encoder, extracting high-level semantic information layer by layer, input corresponding decoder branch;Construct public space-task space feature mapping module, through the unaligned task fusion module and task interaction mapping module, update each subtask feature by mapping;Task adaptive feature update module is constructed, and multi-level iterative update unaligned task feature;Task adaptive weakly supervised image semantic understanding framework is constructed, model loss function is established, image data with task missing is input into model, and obtains multi-task prediction result such as semantic segmentation, depth estimation, surface normal estimation.The application is according to the data information of task label unaligned, through the mapping interaction of public space and task space, fully fuses unaligned task feature, iteratively generates high-quality multi-task prediction result, can effectively handle weakly supervised problem with task missing, and simultaneously improves each task prediction accuracy.

The application discloses a kind of based on plane Gauss's color model construction method, system, device and medium, method includes the position parameter of initialization Gauss ball, depth order, shape attribute, color attribute and opacity;According to shape attribute, the shortest axis of Gauss ball is determined, and then flat loss is calculated;According to the shortest axis and observation direction, the local normal of Gauss ball is obtained and then normal alignment loss is calculated;According to opacity, sparsity loss is calculated;Gauss ball is colored and color loss is calculated, according to flat loss, normal alignment loss, sparsity loss and color loss, construct final loss function, and shape attribute, local normal, opacity and color attribute are updated by back propagation algorithm.The application improves normal estimation accuracy by flattening Gauss ball, and then improves the color quality, by combining color model, high-quality rendering of a variety of materials is realized, and can be widely applied in computer vision technical field.

This invention discloses a single-view 3D photometric reconstruction method, system, device, and medium based on machine learning, relating to the field of computer vision technology. The method includes acquiring single-view images of a target object under different light source conditions and light source direction information; preprocessing the images and light source direction information to generate standardized multi-dimensional input data; performing feature extraction and fusion to obtain a fused feature representation; performing preliminary 3D surface normal estimation; outputting preliminary normal vectors; iteratively optimizing the preliminary normal vectors; combining light source reliability assessment and material prior knowledge constraints to obtain accurate normal vectors; and constructing a 3D model of the target object using a 3D geometric reconstruction algorithm. This invention significantly improves the accuracy, robustness, and direct usability of 3D reconstruction in complex lighting and heterogeneous material scenarios by combining multi-scale feature learning with channel attention and incorporating physical optimization based on light source reliability assessment and material prior knowledge.

A real-time photometric stereo vision method and device based on an event camera, belonging to the field of computer vision, includes: lighting pattern design; simultaneously acquiring lighting signals and synchronization signals generated by lighting changes in the test object using an event camera; using the synchronization signals to calibrate the lighting direction when other events occur, achieving synchronous acquisition; constructing null vectors directly related to the surface normals of the test object from each pair of consecutively occurring events; and fusing the null vectors to solve for the surface normals. This invention reconstructs the surface normals of the test object from the basic model triggered by the event camera, reducing the cost of multiple cameras and synchronization; it utilizes the high event resolution of the event camera to acquire data under continuously changing light sources, increasing data sampling density and improving the quality of normal estimation; and it leverages the high dynamic range of the event camera to improve data acquisition speed and quality through native high dynamic range and compressed event representation.

The invention discloses a noise robustness and detail fidelity-oriented point cloud lightening method, which relates to the technical field of three-dimensional point cloud processing, and comprises the following steps: S1, carrying out multi-scale noise suppression on original point cloud data by adopting a self-adaptive sliding window polynomial fitting filtering algorithm to obtain smooth point cloud data; s2, constructing a spatial index structure for the smooth point cloud data by adopting a multi-level axial bounding box adaptive recursive subdivision method, and realizing sparse region memory optimization and high-density region feature retention; s3, extracting a skeleton structure from the point cloud data processed in the step S2 based on an improved central axis transformation algorithm; s4, performing normal estimation and correction on the point cloud data processed in the step S3 so as to retain and enhance sharp features, by means of the adaptive sliding window polynomial fitting filtering method, key details and feature distribution of a local geometric structure of the point cloud are reserved while noise interference is effectively suppressed, and the method has high robustness. And high-fidelity point cloud data is provided for subsequent normal estimation and three-dimensional reconstruction.

The present disclosure provides a point cloud normal estimation method and device based on local subspace clustering, electronic equipment and storage medium. By introducing a feature point weight evaluation mechanism based on covariance characteristics and geometric structure indicators in the local neighborhood, high-confidence feature points are screened out, and a low-rank subspace clustering with prior weight constraint is used in the feature point neighborhood to divide and fit the local point cloud, so as to re-estimate the normal of the feature point at the subclass level. At the same time, combined with the noise point filtering and the normal diffusion strategy based on the adjacent non-noise points, the method can more accurately depict the local geometric structure in the complex scene with noise, multi-structure aliasing and non-uniform sampling, significantly improve the accuracy and stability of the point cloud normal estimation, and better maintain the sharp features and edge details of the model.

The invention discloses a redefined 3D point cloud normal estimation method based on sample selection, and the method comprises the steps: collecting three-dimensional point cloud data, and carrying out the evaluation and screening of the three-dimensional point cloud data through a confidence estimation strategy; performing random sampling on the screened three-dimensional point cloud data through a k-nearest neighbor algorithm and a probability-based sampling strategy to obtain local point cloud blocks and global point cloud blocks; a PCA algorithm is introduced to carry out alignment processing on the point cloud blocks to obtain a point cloud data set; constructing a multi-constraint joint optimization solving network; inputting a point cloud data set for training, optimizing the network through a loss function to obtain an optimized multi-constraint joint optimization solving network, and outputting a predicted point cloud normal; a neural gradient field module is introduced to carry out directional correction on the predicted point cloud normal to obtain a final point cloud normal; according to the method, a network with higher robustness is constructed through multi-constraint integration, data of various noise levels can be processed, and normal estimation results with consistent directions and higher precision are obtained.

The present invention relates to the field of three-dimensional modeling technology, and specifically to a method for establishing a three-dimensional real-scene model with an autonomous updating function, comprising: collecting multi-viewpoint point cloud data of a scene to be modeled to obtain an initial three-dimensional point cloud set; estimating the curvature of the initial three-dimensional point cloud set according to a singular value decomposition algorithm to obtain a curvature value for each data point in the initial three-dimensional point cloud set; and estimating the normal of each data point in the initial three-dimensional point cloud set according to the curvature value of each data point in the initial three-dimensional point cloud set to obtain a normal for each data point in the initial three-dimensional point cloud set. By using a singular value decomposition algorithm for preliminary curvature estimation and normal estimation, the present invention can more accurately analyze details in the three-dimensional point cloud, avoid manual intervention, and improve the degree of automation in the modeling process.

The present application relates to the technical field of industrial nondestructive testing, in particular to a nondestructive testing robot welding seam automatic tracking system based on multi-modal sensing. It comprises a data acquisition module, a data preprocessing module, a data fusion module, a trajectory generation module and a motion control module. Point cloud data is obtained by a laser scanner, image information is obtained by an RGB-D camera, and attitude data is obtained by an inertial measurement unit. After denoising, downsampling and correction processing of multi-modal data, data fusion is realized by Kalman filtering, and continuous welding seam spatial trajectory points are output. The trajectory generation module generates end pose sequences through three-dimensional B-spline fitting and surface normal estimation, and adopts quaternion interpolation for attitude smoothing processing. The motion control module combines differential path keeping and hybrid position-force control to ensure that the robot accurately tracks along the welding seam centerline and maintains constant contact force, while having process correction function to improve the stability and reliability of the system in complex welding seam environment.

The application discloses a kind of multi-storey building point cloud automation segmentation method, system, medium and device based on geometric feature related to building digitization technical field, to realize the high-precision, automation segmentation of multi-storey building point cloud from whole to local.The application first reduces the calculation complexity by voxelization preprocessing, combined with normal estimation screening horizontal voxel, then utilizes Z-axis occupancy rate peak detection to realize accurate multi-storey layering;Second, extract the top point cloud to construct two-dimensional occupancy map, input the improved region growing strategy into two-dimensional occupancy map, realize room segmentation;Finally, based on random sample consensus algorithm, by introducing geometric constraint, and combined with plane merging strategy, realize wall surface segmentation.Consequently, floor-room-wall surface three-level hierarchical segmentation method is constructed, to provide high-precision structured data support for subsequent point cloud data application.

The application discloses a kind of robot multilayer multi-pass welding path adaptive generation method and system, method includes using time flight sensor to the three-dimensional point cloud of weld is collected, and the weld point cloud collected is preprocessed;The preprocessed weld point cloud is segmented to extract the feature of weld by random sample consistent weld segmentation algorithm based on density clustering, and using cubic spline interpolation method is interpolated fitting to weld feature point;According to the geometric information of welding workpiece and the weld feature point after interpolation fitting, multilayer multi-pass welding path is generated;Based on multilayer multi-pass welding path, the normal estimation method of weighted principal component analysis is used to calculate the posture of welding torch end.The method of the application can segment the three-dimensional point cloud of welding workpiece containing complex surface to extract weld feature point, and then generate multilayer multi-pass welding path through weld feature point and the geometric information of welding workpiece to solve the problem of low welding precision, slow speed and narrow application range of medium plate.

The embodiment of the application provides a control method of a feeding robot based on a multimodal large model, and belongs to the technical field of image recognition. The control method comprises the following steps: acquiring a voice instruction issued by a user; acquiring a food position in a dish according to the voice instruction by using a lightened model of an improved YOLOv8n; picking up the food by a mechanical arm according to the food position; detecting a three-dimensional pose of a mouth based on a weighted point cloud normal estimation method; planning a feeding track according to the three-dimensional pose; and executing a feeding operation by the mechanical arm according to the feeding track. The improved YOLOv8n lightened model and the weighted point cloud normal estimation method are used, so that real-time detection of food blocks in the dish and high-precision estimation of the position of the mouth of the user are realized.

The invention provides a face recognition method based on point cloud data and RGB image sequential verification, and the method comprises the following steps: collecting two types of data at the same time, completing the face detection and cutting of an RGB image, combining histogram equalization and illumination compensation, extracting unitized features, comparing the unitized features with a feature library, and carrying out the confidence calibration. And according to the first upper threshold value and the first lower threshold value, primary judgment of passing, re-verification or refusal is made. After the RGB image passes verification, selecting a point cloud corresponding to a human face region, executing voxel downsampling, counting abnormal point removal and normal estimation, calculating local geometric description, performing robust registration, and constructing a single geometric similarity index based on a median absolute deviation of a point-to-surface residual error and quantile statistics; and outputting passing, re-checking or refusing according to the second upper threshold value and the second lower threshold value. Only when the two stages pass, an identity conclusion and confidence information are given; and when any stage is not passed or re-verification is not passed, outputting a rejection result or triggering a warning or auxiliary verification.

This invention discloses a method for determining machining allowance and automatically guiding grinding of curved workpieces based on point clouds. The method includes: performing hand-eye calibration on the robot's end-effector camera to obtain a first transformation matrix; converting the acquired workpiece point cloud to the robot tool coordinate system to form an actual workpiece point cloud; triangulating and sampling the workpiece's 3D model to generate a dense simulation point cloud; extracting the same region of interest (ROI) point cloud from two point clouds, obtaining the initial pose through principal component analysis, and then obtaining the final transformation matrix through ICP fine registration; performing registration transformation on the simulation point cloud; using the registered simulation point cloud as a template to calculate the normal deviation of each point in the actual point cloud relative to the template, and judging the workpiece status as qualified, with allowance, or overcut and scrapped; if it is determined to have allowance, performing surface normal estimation and principal component analysis on the allowance region to generate a grinding trajectory containing discrete path points and corresponding tool directions to guide robot machining; this invention can achieve closed-loop intelligent control of the machining process.

The invention provides a point cloud smoothing method and device based on local feature estimation, electronic equipment and a storage medium, and the method comprises the steps: introducing noise point elimination of local density features, normal estimation based on weighted local statistical analysis, and a neighborhood updating weight construction mechanism fusing a spatial distance, a normal difference and a normal distance. And self-adaptive and structure-preserving smoothing processing of the point cloud is realized. According to the method, outliers and noise points can be effectively eliminated, the thickness of the point cloud is reduced, the stability of normal estimation is improved, meanwhile, the edge and geometric details of an object are reserved in the smoothing process, and the overall quality and applicability of the point cloud are remarkably improved.

To suitably reconstruct both local details and a global three dimensional shape of an object in an image.SOLUTION: The information processing apparatus includes an acquisition unit configured to acquire input data including a plurality of images under a plurality of illumination conditions, an estimation unit configured to execute normal estimation processing and depth estimation processing with reference to the input data, a priority determination unit configured to determine a priority of each of the normal estimation processing and the depth estimation processing, and a generation unit configured to generate output data with reference to a result of the normal estimation processing, a result of the depth estimation processing, and the priority.SELECTED DRAWING: Figure 3

An system for augmenting images using hand surface normal estimation is provided. In a model training phase, 3D models of hands are generated using 3D data of hands in a variety of positions. Target normal training data is generated that includes normals of surfaces of the 3D models and synthetic 2D image training data corresponding to the 3D models and the normals. The target normal training data and the synthetic image training data are used to train a normal estimation model. The normal estimation is used by an interactive application to generate augmentations that are applied to hand image data.

The invention discloses a skeleton guide point cloud segmentation method for building structure identification. The method comprises the following steps: carrying out normal estimation on point cloud data of a target building structure to obtain a point cloud with a direction, calculating rotary symmetry axis points in the point cloud with the direction, generating a point cloud skeleton by utilizing all the rotary symmetry axis points, and storing the point cloud skeleton as an undirected graph; the point cloud skeleton is decomposed into a plurality of branches, each branch comprises a group of edges in the undirected graph, and the edges in one branch have similar directions; points in the point cloud with the directions are distributed to all the branches, corresponding subspaces are formed, and the subspaces corresponding to all the branches form a building structure recognition result. According to the embodiment of the invention, the ROSA rotational symmetry axis extraction method is utilized, the skeleton line reflecting the geometric morphology of the target can be stably extracted from the complex point cloud, and global structure guidance is provided for subsequent segmentation; consistent mapping of the spatial structure and the semantic structure is achieved, the structure understanding ability of segmentation is enhanced, and the overall segmentation robustness is improved.

The invention belongs to the technical field of three-dimensional point cloud processing, and particularly relates to a turbidity-adaptive back scattering weight normal estimation and denoising algorithm. The algorithm comprises the steps of performing background statistics and intensity standardization on a three-dimensional point cloud data set, performing relative extinction estimation, establishing a backscattering / ballistic ratio index, constructing a point-level weighting function, setting an adaptive neighborhood set and outputting a neighborhood scale, performing weighted normal estimation, and performing weighted denoising and outlier suppression. According to the method, normal estimation and denoising can be carried out on the three-dimensional point cloud generated by the underwater laser radar under different turbidity and distance conditions, and the point-level weight is generated through the influence of explicit modeling backscattering and extinction on the observation signal-to-noise ratio and is used for geometric estimation and robust noise suppression, so that the geometric quality of the point cloud and the downstream identification stability are improved.

The invention belongs to the technical field of three-dimensional modeling, and particularly relates to an unsupervised non-watertight point cloud high-fidelity reconstruction method. Aiming at the problems of inconsistent gradient directions, local detail loss, artifact fragmentation and the like in open surfaces, non-watertight point clouds and complex geometric scenes in the existing method, the invention provides a normalization and density adaptive resampling mechanism; symmetric point pairs are generated through local normal estimation, and prior constraints with consistent UDF values and opposite gradients are applied; designing a multi-scale Gabor code with spatial locality and direction selectivity, and combining the combined feature input of a symmetric invariant base and a symmetric variant base; gradually converging from global topology to local details by adopting progressive multi-scale sampling and composite unsupervised loss function optimization; and a UDF curved surface extraction method based on gradient discrimination and improved Marking Cubes is provided.