317 results about "Lidar data" patented technology

The invention relates to the technical field of artificial intelligence and remote sensing image processing, and particularly provides a hyperspectral image and laser radar data classification method based on a dynamic fusion network. The method comprises the following steps: preprocessing acquired multi-modal data, and constructing multi-scale input; a dual-scale local attention module is designed, and context information of different scales is fused in a self-adaptive weighted mode through gating soft pooling; a dynamic down-sampling feature enhancement module is designed, the down-sampling rate is dynamically adjusted according to the complexity of the feature map, and deep multi-scale interaction is carried out based on a Mama backbone; constructing a directional interactive attention module, extracting features in horizontal, vertical and diagonal directions through directional gating convolution, and capturing an anisotropic structure of a linear ground feature; through the design of a double-path classifier, fusing shallow space details and deep semantic information; and the model is trained, optimized and reasoned to obtain data classification, and the method improves the classification precision and the calculation efficiency.

Embodiments of the present disclosure relate to multi-view LIDAR perception with motion cues for autonomous and semi-autonomous machines and applications. A DNN may be used to detect objects, a navigable space, weather or surface conditions, artifacts, and / or other parts or features of an environment based on multiple views of LIDAR data from multiple time slices. The DNN may include multiple input channels for processing multiple views of sensor data from multiple time slices to provide motion cues, and the extracted features from the different time slices may be geometrically projected from a first 2D view to a second 2D view, combined with features that were extracted from the second 2D view, and applied to a subsequent stage of the DNN. The data generated by the DNN may be provided to the drive stack of an autonomous vehicle or other ego-machine to enable safe planning and control of the vehicle.

The invention discloses a quadruped robot inspection method based on multi-modal sensing fusion. The method comprises the following steps of: 1, initializing a global task, loading a basic navigation map, and generating the basic navigation map comprising topographic features, forbidden areas and parking positions; 2, autonomous navigation and dynamic correction of positioning deviation are realized according to laser radar SLAM data, and the positioning deviation is dynamically corrected according to real-time observation data; 3, dynamically switching or adjusting the gait strategy according to the terrain category, training the gait strategy of the quadruped robot to be matched with the terrain category, and generating a self-adaptive motion control instruction to adapt to the terrain in real time; and step 4, synchronously realizing parking area structured data acquisition and dynamic abnormal information perception through multi-sensor fusion, and realizing target state monitoring and abnormal event response in the inspection task. According to the invention, through collaborative innovation of the bionic motion platform and multi-mode intelligent detection, all-terrain coverage, total-factor perception and full-process autonomous intelligent inspection in a complex parking lot environment is realized.

The invention relates to the technical field of computer vision and multi-sensor data fusion, in particular to a multi-sensor image fusion obstacle real-time detection and tracking system, which comprises the following steps of: firstly, fusing data of a camera, a millimeter wave radar and a laser radar, and extracting and fusing multi-modal features; performing multi-target tracking based on a recurrent neural network: generating a target state through space-time modeling, associating a target with a historical track by using an attention mechanism, and maintaining track consistency; and the system performs semantic classification and interaction analysis on the obstacle, predicts the movement track of the obstacle, realizes deep semantic understanding, and finally outputs the identity label and the complete historical track of the obstacle.

The invention belongs to the technical field of electrical equipment detection, and discloses a defect image enhancement method fusing reasoning and generation, which integrates visible light, infrared and laser radar data through a multi-modal feature fusion network, breaks through the limitation that a contrast file CN114281093A only depends on a visible light image, and improves the detection accuracy. The dynamic attention mechanism can flexibly deploy visual, spatial and semantic feature weights according to defect types, key features can still be captured in complex environments such as strong light and shielding, and meanwhile, the spatial form of the defects is analyzed by means of three-dimensional point cloud; by means of the design, missing detection caused by insufficient characteristics of tiny parts such as hardware fittings and pins is effectively avoided. Aiming at the problem of distortion of a sample generated by a traditional data enhancement method in a comparison file, the sample quality is guaranteed through double mechanisms of reasoning constraint and physical verification, defect features output by a reasoning model directly constrain feature distribution of the generated sample, and meanwhile, a material mechanics rule is introduced to verify the physical rationality of the generated sample.

The disclosure provides an improved canopy height correction method. The method includes: obtaining GEDI LiDAR data, airborne canopy height data, GDEM with high resolution and land cover product within the selected target area and timeframe; performing quality filtering and spatial-scale filtering on GEDI footprints; extracting laser pointing parameters and waveform parameters; extracting reference canopy height from airborne data for each footprint; extracting laser pointing parameters and waveform parameters; preprocessing the GDEM and calculating topographic parameters, including topographic variability index (TVI); constructing the Laser Pointing and Topographic Index (LPTI) according to the 3D forest-ground geometry model; inputting the waveform parameters, even topographic parameters, TVI and LPTI as independent variables, and the reference canopy height as the dependent variable to modeling an improved forest canopy height extraction, and utilizing the improved canopy height extraction model to correct the twofold influence of topographic on GEDI canopy height extraction.

The invention discloses a visibility inversion method based on millimeter wave cloud radar and laser radar data fusion, and relates to the technical field of meteorology, and the method comprises the steps: guiding a millimeter wave cloud radar and a laser radar to carry out data correction and equivalent relation construction through employing a fog drop particle size distribution function measured by a fog drop spectrometer in a sample scene in advance; and then applying the correction coefficient and the equivalent function relationship to a target scene to perform multi-source data correction and fusion on the millimeter wave cloud radar and the laser radar and invert visibility. The method can fuse complementary advantages of the laser radar and the millimeter wave cloud radar; the introduction of the fogdrop particle size distribution function effectively avoids the distribution hypothesis of fogdrop shapes in the traditional model, can weaken the seasonal drift of the traditional empirical coefficient, significantly improves the accuracy and reliability of visibility inversion, and achieves the high-precision visibility monitoring under various weather conditions from clear sky to dense fog.

The invention relates to the technical field of coal mine safety monitoring, and discloses a coal mine tunnel laser radar three-dimensional dynamic monitoring method. Aiming at the problems that in the prior art, data of a laser radar is easy to interrupt or unreliable under a severe working condition, and monitoring continuity is poor due to insufficient multi-source data fusion, the method is characterized in that laser radar surface data and optical strain gauge point data are synchronously acquired; constructing a roadway digital twinborn model; carrying out fusion and assimilation on the multi-source data by utilizing a Transform mechanism to obtain initial roadway deformation data; judging the credibility of the laser radar data based on Bayesian estimation; when the data are missing or uncredible, deducing full-field deformation by utilizing a gating mechanism in combination with optical strain gauge point data and a digital twin model; and finally, generating and outputting a three-dimensional dynamic deformation field. The system is mainly used for real-time, continuous and high-reliability deformation monitoring and safety early warning of the coal mine tunnel.

The invention discloses an SLAM-oriented multi-sensor space-time synchronization and online calibration method, and relates to the technical field of computer vision and robot perception. Comprising the steps that S100, initialization is carried out, and an initial pose and an external parameter transformation matrix initial value are obtained through coarse alignment; s200, data acquisition and preprocessing: acquiring IMU data, LiDAR data and Odom data in real time, and performing data filtering, distortion removal, time sequence correction and point cloud downsampling processing on the acquired data; and S300, performing time synchronization processing, estimating the time offset of the LiDAR and Odom data in real time, and performing unified correction to generate a unified timestamp. According to the method, an online time offset estimation and compensation mechanism with an IMU as a main clock is established, accurate synchronization of sensor data of different frequencies is achieved, and the problem of data alignment caused by sampling frequency difference and clock drift is effectively solved; and meanwhile, tight coupling fusion is realized by using an IMU pre-integration factor, a LiDAR residual factor and an Odom constraint factor, so that the space-time consistency of multi-sensor data is remarkably improved.

The invention discloses a concrete wall full-section deformation monitoring device and method, and relates to the technical field of engineering structure health monitoring, and the device comprises an integrated measurement unit, a passive stable characteristic target group and a processing unit. The integrated measuring unit is composed of an area array solid-state laser radar and a high-resolution binocular vision camera which are rigidly and fixedly connected, and is used for synchronously collecting three-dimensional point cloud and two-dimensional images of a wall and an environment. The target group is arranged on an independent stable object near the wall body; the processing unit identifies coordinates of the target group based on laser radar data, establishes a world coordinate system, calculates self pose drift of the measuring unit by periodically detecting position change of the target, and performs coordinate correction on a wall surface relative displacement field obtained by visual calculation by using the drift distance; and finally, outputting a full-section absolute deformation field under a world coordinate system. According to the invention, high-precision monitoring of full-field continuous deformation of the concrete wall is realized, and the reliability and comprehensiveness of structural health monitoring data are improved.

Embodiments of this application provides a LiDAR data processing method and apparatus, the method comprises: emitting laser beams multiple times; sampling echoes of a laser beam emitted for one time by at least two receiving blocks to obtain sampling data of the at least two receiving blocks for the laser beam emitted for one time respectively, a first receiving block in the at least two receiving blocks is used to sample a laser beam emitted for an Nth time and a laser beam emitted for an (N+1)th time; obtaining a point cloud data of the laser beam emitted for one time; and generating a frame of point cloud data based on the point cloud data of the laser beam emitted for one time corresponding to at least one of the receiving blocks, which can reduce the cost of LiDAR.

The invention belongs to the technical field of satellite remote sensing application, and discloses an active and passive satellite remote sensing fused ocean three-dimensional chlorophyll field reconstruction method and system. According to the method, satellite-borne laser radar data is subjected to correction, signal processing and biological optical model inversion, and a chlorophyll a concentration vertical section along an orbit is obtained; forming a training set by the passive satellite observation optical variable and the marine environment variable of which the profile is matched with the space-time, so as to train a long short-term memory (LSTM) neural network model; and utilizing the trained model to reconstruct a three-dimensional chlorophyll a concentration field of a target area according to passive observation and environment variables of the target area. By fusing the advantages of a satellite-borne laser radar ICESat-2 satellite and a passive optical remote sensing satellite, three-dimensional chlorophyll a concentration field detection based on active and passive fusion remote sensing is developed, the structure and function of a marine ecosystem can be deeply known, and three-dimensional dynamic observation of the ocean is realized.

The invention discloses a quadruped robot space precise positioning method and system based on deep learning. The quadruped robot space precise positioning method specifically comprises the steps that S1, a laser radar data stream and an inertial measurement unit data stream are aligned to generate a synchronous sensing data sequence in a unified mode; s2, determining a mapping pose sequence through consistency evaluation of poses output by the offline point cloud registration algorithm pool; s3, performing parallel loopback detection to generate a unified loopback constraint set; s4, the mapping pose sequence, the loopback constraint set and the pre-integration constraint are written into the factor graph model to generate a three-dimensional point cloud reference map; s5, the local point cloud frame and the three-dimensional point cloud reference map are registered, and a map matching pose is output; s6, the map matching pose and the pre-integration result are written into a factor graph model to continuously solve and output a continuous-time six-degree-of-freedom pose result; and S7, outputting a six-degree-of-freedom pose result in continuous time. According to the invention, direction constraint consistency point cloud registration and dual-channel loopback optimization are introduced, and stable and high-precision positioning of the closed space is realized.

The invention discloses a hyperspectral image and laser radar data fusion classification method based on an improved attention mechanism. The method comprises the following steps: obtaining hyperspectral image data and laser radar elevation data to be classified; processing the hyperspectral image data through a spectral channel attention module to extract spectral features; processing the laser radar elevation data through an elevation space attention module to extract spatial features; inputting the spectral features and the spatial features into a double-feature fusion module for coupling to generate fusion features; image classification is completed based on the fusion features; according to the method, the image features of the hyperspectral image in the spectral dimension and the elevation information of the radar image in the spatial dimension can be fully extracted, and the representation capability of the model for the spatial features and the spectral features is enhanced.

The invention discloses an unmanned aerial vehicle load target tracking system and method based on multi-sensor fusion, and the method comprises the steps: obtaining visible light, infrared and laser radar data, carrying out the time sequence synchronization and attitude calibration, and obtaining a target fusion estimation value through a dynamic weighting Bayesian fusion algorithm; the improved Siamese neural network is used for feature matching, the LSTM trajectory predictor is combined to output a short-term motion state, and finally the adaptive mechanical control module drives the holder to realize stable tracking. According to the invention, the robustness, precision and continuity of target tracking in a complex environment are effectively improved.

The invention discloses an omni-directional mobile robot pose adjustment method based on a laser radar, and the method comprises the steps: collecting and processing multi-frame laser radar data through a system to extract environment features in a working point creation stage after a robot is precisely guided to a target pose, and carrying out the correlation storage of a working point identifier, a global pose and standard feature data after verification. And when the robot navigates to a position near a working point, triggering an accurate adjustment process: calling pre-stored standard features by a system, collecting laser radar data in real time to perform same filtering and feature extraction processing, and performing iterative calculation through a point cloud matching algorithm to obtain X and Y direction and course angle deviations between a current position and a standard position. And finally, carrying out pose correction, and carrying out real-time cyclic feedback until the deviation reaches a millimeter-level precision threshold value. According to the invention, the dependence of a traditional two-dimensional code accurate positioning scheme on illumination, cleanliness and entity marking is overcome, and the method has the advantages of high precision, strong environmental adaptability and high reliability.

Multimodal input data associated with a transportation network environment, including at least one of image data and light detection and ranging (LiDAR) data, is received. Based on an artificial intelligence (AI) component and the multimodal input data, at least one attribute set associated with at least one infrastructure asset within the transportation network environment is identified. Based on the at least one attribute set, output data associated with a multi-objective infrastructure management operation is generated. The output data is provided for display via a graphical user interface rendered by a computing device.

The application provides a multi-target tracking method, device and vehicle, and relates to the field of multi-target tracking.The method comprises the following steps: acquiring laser radar data and millimeter wave radar data collected by a laser radar and a millimeter wave radar at a current time, and determining detection target information at the current time according to the target data; matching and tracking a plurality of obstacles in a preset range based on the detection target information and historical tracking target information; and the historical tracking target information comprises position information, speed information and a heading angle of each obstacle that needs to be tracked in a tracking list.The multi-target tracking method, device and vehicle provided by the application are used for enabling an unmanned working machine to have a stable tracking function for multiple targets, and improving the working efficiency of the working machine.

The invention discloses a natural secondary forest tree species classification method based on multi-modal data fusion. The method comprises the following steps: S1, building a tree species space-spectrum feature extraction module; s2, building a tree canopy global context sensing fusion module; s3, building a tree perception detail optimization module; s4, building a tree species classification network based on multi-modal data fusion; s5, after the tree species classification network with multi-modal data fusion is built, corresponding hyperspectral and laser radar data sets are used for training and testing; according to the method, the problems of fuzzy crown boundary, difficulty in distinguishing tree species with similar structures and low classification precision caused by difficulty in considering long-distance global dependence modeling and fine-grained local detail reservation when an existing tree species classification technology is used for processing a complex forest stand environment are solved.

A LIDAR system concurrently outputs multiple LIDAR output signals that concurrently illuminate the same sample region in a field of view for a data period. The sample region is one of multiple sample regions included in the field of view. The LIDAR system also includes electronics that use the multiple LIDAR output signals to generate LIDAR data for the sample region. The LIDAR data includes a distance and / or a radial velocity between the LIDAR system and an object that reflects the LIDAR output signals.

The invention relates to an automatic acquisition and intelligent processing method and device for power distribution network unmanned aerial vehicle inspection data, visible light, infrared and laser radar data are synchronously acquired through a multi-mode sensor, and the quality of the inspection data is sensed in real time in combination with integrity verification, time alignment and defect marking. Aiming at the problems of blurring, shielding, overexposure, data missing and the like, a cross-modal compensation repair strategy is adopted, and image reconstruction is performed by using point cloud structure prior, infrared temperature distribution or historical data, so that the availability and recognition reliability of original data are remarkably improved; and then fault identification is carried out on the repaired data through a deep learning model, and on the basis of an evaluation mechanism and an anomaly detection thermodynamic diagram containing position coordinates, when an evaluation result is lower than a threshold value, re-flight supplementary collection of the unmanned aerial vehicle is planned, so that the problems of missing detection and false detection caused by a data quality problem in traditional inspection are effectively solved; and the automation level and the operation and maintenance efficiency of power distribution network inspection are greatly improved.

Systems and methods for automatic removal for selected training data from a dataset are provided. Systems and methods are provided for identifying portions of radar data that contain information that is not present in camera and / or lidar data. The identified portions of the radar data can be processed by a neural network to provide the additional information. The identified parts of the radar data can then be processed while the remaining parts of the radar data remain unprocessed. In various examples, features can be extracted from identified regions of the radar data using a neural network and fused with camera and / or lidar features. In some examples, the fused features can be used for object detection.

The invention belongs to the field of laser radar signal processing, and provides a method and system for inverting vertical visibility based on a Mie scattering laser radar, and the method comprises the following steps: S1, obtaining a signal; s2, signal preprocessing, which sequentially comprises background noise correction, overlap factor correction, distance square correction, de-noising processing and normalization; and S3, vertical visibility inversion, including construction, training and use of a DBO-BP optimization neural network model. The system comprises a Mie scattering laser radar, a data preprocessing module and a DBO-BP optimization neural network model module. According to the method, physical characteristics and data characteristics of original echo signals are taken into consideration during inversion of the vertical visibility, the anti-noise performance is remarkably enhanced through multiple times of denoising during signal preprocessing, and efficiency and precision are balanced by adopting a hierarchical iteration strategy during model training.

The invention discloses a digital surface model (DSM)-guided hyperspectral and LiDAR combined unmixing method, relates to the field of multi-modal image processing, and aims to solve the problems of end member confusion and insufficient space structure maintenance caused by spectrum similarity in hyperspectral unmixing. According to the method, a hyperspectral image and LiDAR data of the same area are obtained, a LiDAR elevation map is expanded into a multiband profile through an attribute configuration file method, and a digital surface model (DSM) is generated. Constructing a spectrum and space double-branch auto-encoder, respectively extracting spectrum and space features and carrying out fusion mapping, obtaining an abundance matrix by using a normalized exponential function, and reconstructing a hyperspectral image; a double-branch adaptive mixed channel attention mechanism is designed in a spectrum branch, and a space attention mechanism is introduced in a space branch. In the training process, a self-defined loss function is formed by combining DSM-guided structure entropy regularization, spectral angular distance and root-mean-square error, and the space continuity and boundary retention of the abundance graph are improved. The method is suitable for hyperspectral unmixing and multi-modal remote sensing fine identification under complex terrains.

The invention relates to the technical field of geotechnical engineering, in particular to a simulation calculation method and system based on side slope digital twins. The method comprises the following steps: acquiring airborne laser radar data, generating a slope contour line, and obtaining slope contour line data; performing slope section stability analysis according to the slope contour line data to obtain slope section stability data; obtaining side slope geological data; according to the slope geological data, constructing a slope digital base plate, including constructing a surface simulation layer, a topographic geological layer and a dynamic effect layer, so as to obtain a slope digital base plate architecture; identifying a slope unstable area according to the slope section stability data; reinforcement optimization is conducted based on the slope unstable area, and slope reinforcement optimization data are obtained; and construction sequence optimization is conducted according to the slope reinforcement optimization data, and construction sequence data are obtained. The slope stability evaluation and reinforcement construction efficiency and accuracy are improved based on the geotechnical engineering technology.

The invention belongs to the technical field of forestry information, and particularly relates to a moso bamboo forest individual tree high-precision segmentation method based on unmanned aerial vehicle and foundation LiDAR fusion. The method aims at solving the problems that in the prior art, in a complex bamboo forest scene, canopy information is incomplete, under-forest terrain and bamboo pole structure obtaining is inaccurate due to the fact that a data source is single, and over-segmentation and under-segmentation generally exist due to the fact that a moso bamboo growth mechanism is not considered in a traditional algorithm. Through multi-platform LiDAR data collaborative acquisition, point cloud accurate fusion is realized by adopting a registration algorithm with a bamboo pole vertical structure as a constraint, and a canopy-bamboo pole-terrain integrated three-dimensional model is formed. A comprehensive clustering algorithm fusing tree crown geometric attributes and a canopy growth space competition model is innovatively provided, the moso bamboo growth competition relation is quantified into dynamic constraint, the individual tree boundary is intelligently defined, the over-segmentation and under-segmentation problems in the segmentation process are effectively solved, and the moso bamboo forest individual tree parameter extraction precision and ecological rationality are remarkably improved.

The invention belongs to the technical field of remote sensing image processing, and particularly relates to a multi-modal hyperspectral image collaborative classification method and system oriented to HSI and LiDAR data, by providing a multi-modal Mamba spectral space fusion network model (MMSSF-Net), fusion of multi-scale local space neighborhood features and global spectral sequence features is achieved, and the multi-modal hyperspectral image collaborative classification method and system oriented to HSI and LiDAR data are obtained. And meanwhile, the performance of the classifier is improved by utilizing global-local multi-modal complementary information. The MMSSF-Net model mainly comprises three core components: a multi-modal feature learning module (MMFL), a multi-scale aggregation CNN structure is adopted, multi-modal and multi-scale features are fused, and the learning ability of local spatial features is enhanced; a multi-modal feature fusion module (MMFF) which performs forward and backward selective state space scanning by means of bidirectional Mama, so that the feature of each position can simultaneously fuse global information in front of and behind the context of the feature, thereby further enhancing the fusion effect of the multi-modal complementary information and improving the classification performance; and a decision classification module.

Systems and methods for validating following distance detection models are discussed. A LIDAR unit and an image capture device are installed at a vehicle. LIDAR data from the LIDAR unit is used to identify a lead vehicle and for determination of a true following distance to the lead vehicle. An image-based following distance detection model is applied on image data captured by the image capture device. Following distance as determined by the image-based following distance model is compared to the true distance from the LIDAR data, to assess validity of the image-based following distance model.

In various examples, data collection vehicles or machines may be equipped with one or more LiDAR sensors (and / or other sensors), and the LiDAR sensor(s) may be used to collect frames of LiDAR data representing various real-world conditions. The LiDAR data may be processed using one or more deep neural networks (DNNs) such as a transformer neural network to generate auto-labels representing detected dynamic obstacles of any designated class. Tracking may be applied to generate object tracks (tracklines), estimate velocity, and / or handle occlusions. In some embodiments, the object tracks may be refined based on geometry and / or confidence to improve their accuracy. In some embodiments, the auto-labels are classified to generate an estimated representation of quality, and auto-labels with at least a threshold quality score may be skipped during human labeling. As such, auto-label quality scores may be used to accelerate human validation of auto-labeled scenes by skipping high quality auto-labels.

The invention belongs to the technical field of intelligent traffic Internet of Vehicles, and discloses an Internet of Vehicles big data real-time monitoring and user service optimization platform for an intelligent traffic system, and the platform comprises a preprocessing unit which carries out the real-time denoising of collected data, unifies the format, and extracts core semantic features. Realizing cross-node semantic feature encryption sharing by relying on a federated learning framework, completing semantic consistency verification by combining a traffic scene knowledge graph, and eliminating conflict abnormal data; the transmission unit identifies the data causal correlation degree based on 5G network slices, and high-correlation data is transmitted through low-delay slices, so that the data correlation is guaranteed; the data fusion unit can dynamically adjust the weight according to the traffic event type, for example, the data weight of the laser radar is improved in an accident scene, high-precision timestamps and dual-mode positioning calibration space-time migration are matched, and finally cross-modal data accurate fusion is achieved; the causal inference unit deploys a lightweight causal Markov model at a roadside edge terminal, and quickly locates traffic anomaly core inducements.