73 results about "3d mapping" patented technology

The invention belongs to the technical field of three-dimensional mapping, and particularly relates to an occlusion area three-dimensional voxel reasoning method and system. The method comprises the following steps: S10, acquiring an RGB image containing a semantic segmentation result of an occlusion region and a corresponding depth map, and converting the depth map into an initial sparse three-dimensional voxel; and S20, constructing a three-dimensional voxel inference network, and inferring a three-dimensional voxel occupancy probability graph from the initial sparse three-dimensional voxels through the three-dimensional voxel inference network. The occlusion area can be reconstructed based on the monocular image; multi-format output and navigation interface butt joint are supported, and the method can be integrated to SLAM, 3D mapping or path search systems to serve as spatial feasibility constraints; the method can be used for querying whether any spatial position is passable; the method can be used for extracting a trafficability sub-graph from a designated area for path planning.

The invention provides a security situation awareness method and system based on a three-dimensional GIS and a panoramic video, and the method comprises the steps: constructing a three-dimensional scene model of a monitoring region on a three-dimensional GIS platform, accessing a plurality of cameras and an environment sensor, and achieving the space-time alignment and coordinate unification of a video stream and multi-source sensing data; projecting a video frame to a three-dimensional scene based on internal and external parameters of a camera to generate a panoramic video picture, and performing detection and cross-camera tracking on pedestrians and vehicles by adopting a target detection model to obtain a target three-dimensional track; predicting a target position and local density change by using a time sequence prediction model, judging anomalies including congestion, retention and retrograde driving, and sending out an early warning; camera deployment and cooperative tracking are planned through a point location optimization algorithm, and monitoring blind areas are reduced; the system comprises a data acquisition module, a space-time alignment module, a three-dimensional mapping module, a target detection and tracking module, a situation analysis module, a point location planning module and a situation display module, and can improve the automation and precision of security situation awareness in a complex scene.

A system includes a phased array antenna, a signal processing module configured to detect wireless signals received from one or more target devices, a conversion module configured to determine 3D spatial data for the one or more target devices based on the wireless signals, an integration module configured to receive visual data from at least one camera, perform object detection and classification on the visual data to detect and classify one or more objects, and tag the one or more objects with 3D coordinates based on the 3D spatial data for the one or more target devices; and a correlation module configured to synchronize the 3D spatial data from the conversion module with the visual data from the integration module including object detection and classification information for use in a cloud-based augmented reality (AR) content to augment social networking and targeted advertising.

The invention provides a support hanger inclination angle adjusting method based on a 3D laser SLAM unmanned aerial vehicle, and relates to the technical field of unmanned aerial vehicle automatic inspection methods. Acquiring space attitude parameters of the support hanger suspender; processing the original point cloud through a downsampling algorithm to obtain a processed laser point cloud; 3D mapping is used for visual display, laser point clouds of other structures are deleted, and the support hanger suspender laser point clouds needing to be fitted are reserved; a least square fitting method is used for carrying out space straight line fitting calculation processing on the support and hanger suspender laser point cloud, and the axis position and the direction angle of the suspender in the space are obtained; and when the obtained actual inclination angle exceeds the standard, inclination angle data are measured under the cold state working condition and the hot state working condition, the reasonable hanging bracket adjusting distance is calculated and determined, the technical problems that the detection range is limited and the visual inspection error is large are solved, and the technical effects of breaking through the space reachability limitation, eliminating the manual measurement error and supporting the establishment of digital archives are achieved.

In this specification, beam splitters and uses thereof are disclosed. The beam splitter may split a light beam from the light source into a plurality of light beams having a desired shape and intensity, and the beam splitter may be implemented with a simple configuration and process. For example, the beam splitter may be used to construct various sensing devices, facial recognition devices, etc. for 3D mapping.

This invention proposes a multimodal semantically consistent Gaussian mapping and localization method for bridge unmanned aerial vehicle (UAV) inspection. By tightly coupling visual and LiDAR data and introducing semantic consistency constraints, this method achieves high-precision localization and detailed 3D mapping of UAVs in weakly textured environments, providing a reliable technical solution for unmanned bridge inspection and intelligent operation and maintenance in civil engineering.

This invention discloses a method, apparatus, device, and storage medium for constructing a 3D map, applicable to the field of 3D mapping. It involves determining a reference point within the view frustum of a viewpoint, constructing a reference plane containing the reference point and parallel to the view frustum, determining the height of the 3D map rendering interface window, and determining the height of the reference plane window. The unit of the 3D map rendering interface window height is pixels, while the unit of the reference plane window height is length. A distance-pixel ratio is determined based on the ratio of the reference plane window height to the 3D map rendering interface window height. The 3D map scale of the reference point is determined based on the distance-pixel ratio, constructing a 3D map containing the 3D map scale. Based on perspective projection rules, the scale information at any location in the 3D map can be calculated quickly and accurately, improving the efficiency of constructing a 3D map with a high-precision scale. The high-precision scale can assist in obtaining accurate geographic information within the 3D map, thereby improving modeling accuracy.

This invention discloses a tooling and method for precise mapping of deformation defects in subsea pipelines, relating to the field of subsea pipeline inspection technology. It includes an outer frame, a scanning chamber, a scanning chamber drive device, a 3D scanner, a 3D scanning drive device, and a scanning chamber pumping device. The scanning chamber is housed within the outer frame. The pumping device removes seawater from the scanning chamber. The scanning chamber comprises a left scanning chamber and a right scanning chamber, which can be closed by the drive device, sealing the target mapping section of the subsea pipeline within the scanning chamber. The 3D scanner and the 3D scanning drive device are both located within the scanning chamber. The drive device enables the 3D scanner to perform a 3D scan of the target mapping section within the scanning chamber. This invention allows for normal underwater mapping in turbid or turbulent waters, and possesses advantages such as high positioning accuracy and high shape adaptability, enabling non-contact and non-destructive 3D mapping.

This invention discloses a visual-tactile fusion multimodal tactile sensor and a three-dimensional pressure field reconstruction method, relating to the technical field of tactile perception. The sensor includes a visual acquisition unit, a tactile acquisition unit, a gel component, a data processing unit, and a display and interaction unit, achieving dual-modal collaborative perception. The three-dimensional pressure field reconstruction method sequentially executes steps such as force system initialization, synchronous acquisition, image preprocessing, pressure calibration calculation, cross-modal registration, dynamic Gaussian convolution fusion, 2D-to-3D mapping, point cloud and mesh reconstruction, real-time display and interaction, data recording, and reset maintenance. It utilizes dynamic Gaussian convolution kernels to simulate pressure diffusion and generates a three-dimensional pressure field model based on the Open3D library. This invention reduces the cost of the gel component, and the reconstruction delay of the three-dimensional pressure field does not exceed 8 milliseconds, making it suitable for fields such as robot operation, intelligent prosthetics, and human-computer interaction.

The embodiment of the invention discloses a tracking model training method and device, electronic equipment and a storage medium, is applied to the technical field of model training, and can solve the problems that according to an existing 3D target detection method, the problem of shielding and overlapping between targets may be serious, and the detection accuracy of long-distance pedestrians, vehicles or other obstacles is low. The method comprises the following steps: acquiring a training data set, wherein the training data set comprises a plurality of training samples and a pre-marking result corresponding to each training sample; based on the detection frames in the training samples, performing three-dimensional mapping on each training sample to obtain a plurality of initial three-dimensional reference points corresponding to each training sample; and processing the detection frame according to a preset action, screening the plurality of initial three-dimensional reference points according to the position change condition of the processed detection frame, and training the attention network model through the screened three-dimensional reference points and the training data set to obtain a target tracking query model.

Various implementations provide unvarnished video based on adjusting camera parameters according to environmental modeling. Environmental characteristics may be determined based on modeling a physical environment from sensor data captured via one or more sensors. For example, this may involve determining ambient light source optical characteristics, ambient surface optical characteristics, 3D mapping of the environment, user behavior, prediction of optical characteristics of light entering the camera, and the like. The method may involve determining camera parameters of an image captured via an image sensor based on environmental characteristics. For example, the method may determine exposure, gain, tone mapping, color balance, noise reduction, sharpness enhancement. The method may determine camera parameters based on user information (e.g., user preferences, user activity, etc.). The method may involve providing a transparent video of the physical environment based on the determined camera parameters.

The invention discloses an unmanned aerial vehicle tractor path planning method and system based on an ROS2 robot system, the system comprises an RK3566 development board, a laser radar, a shell and a depth vision camera, the RK3566 development board is provided with an ROS2 system, and the laser radar, the development board and the depth camera are installed in the shell. According to the present invention, the use effect is good, the ROS2 system installed on the RK3566 development board is adopted to perform rtabmap vision and laser radar outdoor 3D mapping navigation so as to achieve the autonomous navigation and the automatic driving of the tractor, and the vision camera is adopted to detect and identify the mark and the target so as to substantially improve the automation degree, the maneuverability and the operation effect of the aircraft tractor.

This invention belongs to the field of robot environmental perception and navigation technology, and discloses a method and device for panoramic vision multi-sensor simultaneous localization and mapping based on 3D Gaussian splashing. In the multi-sensor fusion SLAM stage, multi-source observation information acquired by panoramic cameras, LiDAR, and inertial measurement units is used to achieve real-time estimation of the vehicle's pose through tightly coupled state estimation, constructing a globally consistent color point cloud map. In the 3D Gaussian splashing mapping stage, based on pose priors and the color point cloud, a continuous 3D Gaussian scene representation method is introduced to incrementally and densely model the environment in 3D. Differentiable rendering and geometric-photometric joint optimization are used to improve the geometric accuracy and visual consistency of the map. This invention achieves high-quality 3D mapping and representation in complex environments while ensuring system real-time performance and robustness, and can be applied to environmental perception and navigation in mobile robots, autonomous driving, augmented reality, and unmanned systems.

1. The name of this design product: Handheld multi-sensor fusion 3D mapping device. 2. Purpose of this design product: used for 3D scanning and modeling in various venues. 3. The key point of the design of this product lies in its shape. 4. The picture or photo that best illustrates the design points: three-dimensional picture.

A method for constructing a three-dimensional anatomical model of the heart and a three-dimensional mapping system for the heart are provided, wherein an atrial template model and point cloud data of the atrial to be modeled are acquired, the point cloud data is the position coordinate information of some feature points in the atrial to be modeled, some feature points in the atrial to be modeled are mapped to the atrial template model using an affine transformation network, the point cloud data mapped to the atrial template model is input to a prediction network, the displacement of control feature points in the atrial template model is acquired by prediction, and the atrial template model is deformed based on the displacement of the control feature points to acquire a three-dimensional anatomical model of the atrial to be modeled. Thus, the present invention makes it possible to construct a three-dimensional anatomical model of the atrial to be modeled using a trained neural network based on the position coordinate information of some feature points in the atrial to be modeled, and to achieve fully automated construction of a three-dimensional anatomical model of the atrial without requiring manual intervention.

Provided is a method and system for calibrating a three-dimensional (3D) measurement system based on a ray model. The method includes: acquiring corresponding target images at multiple poses, as well as corresponding phase-shifted fringe images and Gray code images; performing center correction on each of the target images to acquire center coordinates of marker points in each of the target images; triangulating the center coordinates of the target images at each target pose to acquire triangular mesh data; calculating object points corresponding to each target pose in a camera coordinate system from each pixel of the target images based on the triangular mesh data, thereby acquiring an object point set corresponding to each pixel, and determining a ray parameter corresponding to the object point set; performing phase extraction on the phase-shifted fringe images and the Gray code images to acquire absolute phase data corresponding to each pixel; and determining a 3D mapping coefficient by coefficient fitting based on the object point set and the absolute phase data corresponding to each pixel, and constructing a lookup table based on the 3D mapping coefficient and the ray parameter to achieve camera calibration.

The present invention discloses a system and method for three-dimensional mapping of building facades based on drones and AI, relating to the field of building surveying technology. The system comprises: a scene parsing module for performing information parsing of calibrated scene information and marking each surveying sub-area as a node to be collected; a collection planning module for calculating the node transition probability of each node to be collected; a surveying search module for configuring a virtual drone and iteratively selecting surveying sub-areas using pheromone concentration and heuristic factors; a clustering and sharing module for performing three-dimensional mapping of building facades and clustering to establish a shared information pool; and a surveying optimization module for performing collaborative decision-making optimization of local paths and updating the collection path based on the collaborative decision-making optimization results. This system solves the technical problem in existing technologies that it is difficult to meet the requirements of large-scale, high-precision surveying and mapping of modern buildings, resulting in low surveying and mapping efficiency and large measurement errors, thereby achieving the technical effect of improving the efficiency and accuracy of building surveying and mapping.

This application discloses a method, system, and medium for rapid mapping of cleaning robots, relating to the field of 3D mapping technology. It is applied to a rapid mapping system for cleaning robots that includes laser devices and image acquisition devices. The system includes: acquiring mapping environment information of the cleaning robot, wherein the mapping environment information includes laser feedback environment information acquired by the laser device and image feedback environment information acquired by the image acquisition device; determining mapping mode information based on the image feedback environment information, wherein the mapping mode information includes a first mapping mode with single feedback information and a second mapping mode with multiple feedback information; when the mapping mode information is the first mapping mode, performing rapid 3D mapping based on either the laser feedback environment information or the image feedback environment information; when the mapping mode information is the second mapping mode, performing rapid 3D mapping based on both the laser feedback environment information and the image feedback environment information. This improves the mapping accuracy of the cleaning robot.

The invention discloses a vehicle-mounted ISP image 3D mapping visual field processing module. The module comprises a camera and an ISP image processing module, the camera is provided with an MIPI output interface, the FPGA is provided with an input MIPI interface and an output MIPI interface, and the FPGA is provided with a CAN bus interface. The ISP visual field processing program comprises an ISP image processing program and an ISP visual field processing program based on a physical rearview mirror visual field. In the prior art, technologies such as electronic rearview mirror projection and projection are not involved, the electronic rearview mirror projection technology can be completed only through complex technical design, and the correction method for the visual field conversion image of the commercial vehicle can effectively improve the image effect and solve the technical problem of distortion of the visual field of a camera.

The invention relates to the technical field of computer vision, and discloses a fast semi-direct SLAM (Simultaneous Localization and Mapping) map construction algorithm which comprises the following steps: acquiring and calculating image features of two adjacent video frames to obtain feature matching points; when a feature matching point reaches a set value, calculating a 3D coordinate of a map point, and performing camera pose estimation; key point extraction is carried out on the video frame image, and feature enhancement is carried out on key points based on the subgraph; back-projecting pixels of the video frame image to a three-dimensional space to obtain point cloud data, and performing primary processing on the point cloud data; and fusing and optimizing the preliminarily processed point cloud data to obtain a 3D reconstruction image. The method comprises two parts of initialization and feature enhancement and 3D mapping reconstruction, and the initialization and feature enhancement aims at obtaining information such as the pose and features of a camera to make preparation for 3D reconstruction; in the 3D mapping reconstruction part, the point cloud information and the like are converted into 3D space information by using the result of the first step, and the space information can be used for robot navigation and other work.

This invention provides a centralized multi-robot cooperative SLAM method based on an FPGA platform. It uses a motion camera to acquire color images and IMU data of the surrounding environment; the color images are input into the FPGA platform of a single robot, converted into grayscale images, and subjected to image pyramid scaling, corner detection, and feature description before being transmitted to the processing system of the single robot; the processing system generates observation information, which in turn generates image frames; keyframes are generated and sent to a central server; the central server performs loop closure detection on the keyframes, performs map fusion on the local maps built by different single robots to obtain a global map, and optimizes the global map; finally, a 3D dense point cloud map is constructed. This method enables cooperative localization and 3D mapping of multiple robots in large-scale and complex environments, solving the problems of limited computing resources and detection range of a single robot in complex scenes, and improving the scale, accuracy, and robustness of map construction.

This application relates to a method and device for environmental perception and 3D mapping based on multi-sensor fusion. The device performs unified time synchronization and spatial calibration on multi-source perception data acquired by LiDAR, inertial measurement unit, image acquisition unit, and satellite navigation system. It then performs multi-sensor data fusion processing on the main control computing platform to obtain temporally and spatially consistent environmental perception information, achieving high-precision positioning and 3D environmental mapping for the device itself. Specifically, the time synchronization module uniformly triggers or aligns timestamps during the multi-sensor data acquisition process, and determines the spatial pose relationships between the sensors through extrinsic parameter calibration. During device movement, LiDAR acquires 3D point cloud data of the environment, and combines this with motion constraints provided by the inertial measurement unit, visual information acquired by the image acquisition unit, and absolute position information provided by the satellite navigation system to construct a device motion model and an observation model. A fusion algorithm is used to jointly optimize the multi-source sensor data to obtain the optimal pose estimation results of the platform at each moment, and the environmental spatial information is cumulatively updated to gradually construct a 3D environmental map. Based on the fusion and mapping results, a three-dimensional reconstruction and visualization of the surrounding environment structure is achieved, and a modular mechanical structure provides reliable support for the mobile platform's environmental perception, positioning, and spatial understanding.

This application relates to a vision-guided loading method and a vision-guided loading device. The method includes: acquiring workpiece image data of a target workpiece and a 3D mapping template; performing feature point detection on the workpiece image data to obtain pixel feature points; matching 3D model feature points in the 3D mapping template based on the pixel feature points to obtain 3D model feature points corresponding to the pixel feature points; determining the initial workpiece pose based on the correspondence between pixel feature points and 3D model feature points; determining the projection error based on the pixel feature points and the corresponding 3D model feature points under the initial workpiece pose; correcting the initial workpiece pose based on the projection error to obtain a corrected workpiece pose; updating the robot's standard grasping pose based on the difference between the corrected workpiece pose and the standard workpiece pose, and grasping the target workpiece to perform the loading action. This method improves the accuracy of workpiece positioning.

3D mapping of a sample in a charged particle microscope. A sample can be positioned on the optical axis of a charged particle microscope based on a 3D map of the sample. The 3D map is generated using back illumination and telecentric imaging of the sample to produce profile images. The profile images are combined to form the 3D map. The 3D map is used to couple a processor to a sample stage to position a selected sample or sample portion for imaging in the charged particle microscope. In some examples, the processor responds to a selection of a sample using a graphical interface so that the sample stage is controlled to safely position the selected sample without further operator intervention.

This specification discloses a beam splitter and its applications. The beam splitter can split a light beam from a light source into multiple beams with desired shapes and intensities, and can be implemented with a simple process to have a simple structure. The beam splitter can be used to construct various sensing devices, such as those for 3D mapping or facial recognition devices.

This invention provides a photovoltaic power plant operation and maintenance method and system based on digital twins, relating to the field of photovoltaic power plant technology. It includes acquiring and registering multi-source 3D mapping data, and performing spatial segmentation and parallel 3D reconstruction based on a cloud-distributed architecture. By matching real-time running data with spatial indexes, the data is dynamically mapped to a digital twin to achieve status updates, and finally, a complete 3D digital twin scene is generated for operation and maintenance monitoring and optimization. This invention improves the efficiency of 3D reconstruction and the real-time update capability of the digital twin model, providing support for refined operation and maintenance of photovoltaic power plants.

This invention discloses a 3D visualization-based intelligent equipment management platform for petrochemical enterprises, relating to the field of intelligent equipment management technology. The platform includes a visualization platform with the following modules: a 3D twin mapping module, which, based on digital twins and image classification algorithms, reconstructs the appearance features and operating status of petrochemical enterprise equipment in 3D visualization, constructing a dynamic 3D mapping model of equipment appearance and operating status. This invention utilizes multi-source perception fusion and dynamic 3D reconstruction technology to spatiotemporally align and fuse multimodal data, constructing a dynamic 3D mapping model of equipment appearance and operating status. This model presents the geometric shape of the equipment and intuitively maps physical attributes to a 3D surface, enabling accurate identification and location of hidden anomalies such as minor leaks and early faults. It overcomes the visual blind spots of traditional manual inspections, providing precise status perception capabilities for equipment management.

A computer-implemented method and apparatus for generating and refining synthetic images of customizable products. The method includes receiving a text prompt describing a product and a desired markup, generating a synthetic image using a generative AI system, validating the image against physical product specifications, mapping the image onto a 3D surface using quadrilateral grid interpolation, applying inpainting techniques to remove the markup while preserving fabric details, incorporating lighting and specular effects to enhance realism, and integrating the enhanced image into an interactive asset for user interaction and customization prior to manufacturing. The apparatus comprises a generative AI system, a validation and correction module, a 3D mapping and image synthesis module, an inpainting module, an interactive asset creation system, depth estimation and mapping tools, and a lighting and specular effects module. The system supports real-time customization and multiple design variations, ensuring conformity with physical specifications.

The application discloses a robot global repositioning method based on a deep neural network and a three-dimensional point cloud, and comprises the following steps: 3D mapping of a target scene is performed by using a laser radar; 3D point cloud key frames are saved and projected on a spherical surface into multi-dimensional arrays; channel number conversion is performed on the obtained n multi-dimensional arrays; a neural network is built; the n multi-dimensional arrays are sent into the neural network for training; the processed key frame point cloud multi-dimensional arrays are sent into the trained neural network; n outputs of an encoder layer are flattened into n descriptors; the robot is placed in any position in a map after being restarted, and the descriptors of the key frames of the robot at the current time are obtained; similarity calculation is performed on the key frame point cloud descriptors photographed by the robot and the key frame point cloud descriptors in the 3D target map, and the descriptors are sorted in descending order; and repositioning is completed. The application can quickly realize repositioning, does not need prior information of a GPS, and enhances the robustness of the system.