1031 results about "Lidar data" patented technology

An agricultural vehicle includes cameras and a LIDAR sensor operably coupled to the agricultural vehicle, and an imaging controller operably coupled to the LiDAR sensor and the cameras. The imaging controller includes at least one processor, and instructions that cause the processor to receive image data from the cameras, receive LiDAR data from the LiDAR sensor, analyze the image data from each of the cameras to generate labeled image data, analyze the LIDAR data to generate labeled LiDAR data, and fuse the labeled image data with the labeled LIDAR data. Related agricultural vehicles and methods are also disclosed.

An agricultural vehicle may include a propulsion system, a steering system, and a vehicle controller. An agricultural vehicle may include a camera sensor having a camera field of view (CFOV), a light detection and ranging (LiDAR) sensor having a LiDAR field of view (LFOV), and an imaging controller in operable communication with the camera sensor and the LiDAR sensor. The imaging controller may include a processor and at least one non-transitory computer-readable storage medium having instructions store thereon that, when executed by the at least one processor cause the imaging controller to, at least, receive image data, segment the image data, fuse LiDAR data and the image data, create a fused instance including depth information; and provide at least the fused instance to the vehicle controller, wherein the vehicle controller is configured to provide at least one navigational command based at least partially on the fused instance.

The invention relates to the technical field of hydraulic engineering intelligent operation and maintenance, and discloses an AI unmanned aerial vehicle-based auxiliary decision-making platform for hydraulic engineering operation and maintenance, and the platform comprises a data collection module which is used for obtaining multi-modal data of a water conservancy facility through an AI unmanned aerial vehicle, the multi-modal data comprises a visible light image, thermal imaging data, multispectral data and laser radar data; the data processing and fusion module is used for carrying out formatting processing, space-time alignment and feature fusion on the multi-modal data; the health assessment and prediction module is used for assessing the health state of the water conservancy facility based on the processed data and predicting the future change trend of the facility state; and the auxiliary decision-making module is used for generating inspection priority planning of the water conservancy facilities. According to the invention, through multi-source data fusion, dynamic risk assessment and intelligent optimization distribution, the effects of accurate monitoring, efficient operation and maintenance and risk visual management of the whole life cycle of the water conservancy facilities are realized.

The invention relates to the technical field of three-dimensional reconstruction, in particular to a multi-mode remote sensing data driven refined city three-dimensional reconstruction method and system, and the method comprises the steps: obtaining an original satellite optical image for providing city surface information; synthetic aperture radar (SAR) data is obtained and used for assisting reconstruction of a point cloud data missing region; the method comprises the following steps: acquiring laser radar data carried by a flight aerial carrier for acquiring high-precision three-dimensional point cloud; performing data preprocessing; carrying out fusion on the preprocessed multi-modal data; on the basis of the fused features, complementing space detail information; on the basis of the refined features, progressive reconstruction is carried out, vertexes and edges are gradually added from the initial vertexes, and a grid model is constructed; in the reconstruction process, performing semantic segmentation on the grid model by using a multi-category segmentation network, and judging the category of each triangular patch; time sequence reconstruction is completed through fusion of multi-temporal data, and a dynamic city change model is generated; and complementation of advantages of different modal data is realized.

The invention discloses an intelligent driving track prediction method, device and equipment and a storage medium, relates to the technical field of intelligent driving, and discloses an intelligent driving track prediction method which comprises the following steps: acquiring environment laser radar data, environment radar data and environment image data; converting the environment laser radar data, the environment radar data and the environment image data into aerial view features; performing feature alignment on the aerial view features based on a sensor field angle identifier to obtain multi-modal sensing features; determining a candidate trajectory according to the multi-modal sensing features; and determining a target trajectory from the candidate trajectories according to a target cost function to complete intelligent driving trajectory prediction. According to the scheme, the complex driving environment can be accurately understood in the intelligent driving process so as to predict the optimal driving track.

The invention provides a water area safety patrol method and system based on multi-modal feature analysis, and relates to the technical field of image recognition. The method comprises the steps of collecting a visible light image, an infrared image and laser radar data corresponding to a target water area, and performing time synchronization and space alignment to obtain visible light alignment data, infrared alignment data and laser radar alignment data; in response to the condition that the definition of the visible light image is higher than a quality reference value, carrying out edge detection and region segmentation processing on the visible light alignment data, and generating a candidate region set containing a target contour; for the infrared alignment data corresponding to the candidate region set, screening out a thermal anomaly region from the candidate region set; and carrying out three-dimensional space feature extraction and motion state feature extraction on the laser radar alignment data corresponding to the thermal anomaly region, and carrying out anomaly identification on the thermal anomaly region. According to the scheme, high-precision safety patrol of the target water area in various environments can be realized.

The invention discloses a navigation matching correction method based on an inspection robot, and the method comprises the steps: building a feature database through the deployment of a physical calibration object and the recognition of a natural feature object, providing a reliable positioning reference for a robot, achieving the coarse positioning through the fusion of visual and laser radar data in a positioning process, and introducing a dynamic credibility evaluation mechanism. The positioning reliability is quantified in real time through an exponential decay model, when the credibility is lower than a threshold value, the system automatically triggers a compensation behavior to re-search features, in the aspect of multi-robot cooperation, secondary positioning correction is achieved through track matching and data fusion, high-confidence-coefficient reference data are screened through a clustering algorithm, the group positioning precision is improved, and the positioning accuracy is improved. Aiming at a key inspection area, multi-angle image matching is adopted to realize fine positioning, a positioning error is dynamically corrected through a sliding window, the continuity and accuracy of robot navigation in a complex environment are remarkably improved through closed-loop correction and self-adaptive optimization, and the method is suitable for intelligent inspection requirements of railway trains.

The invention relates to the technical field of wind power generation, and discloses a wind power plant energy management method based on laser radar wind measurement, comprising the following steps: S1, deploying laser radar equipment, and collecting real-time data of wind speed, wind direction and turbulence intensity in a wind power plant; s2, constructing a three-dimensional dynamic model of a wind field, and modeling wind speed, wind direction and turbulence distribution in combination with laser radar data and historical meteorological data; s3, performing short-term prediction on the future wind speed by using the deep learning model, and outputting minute-level to hour-level wind energy prediction data; s4, optimizing a dynamic scheduling strategy of the wind turbine generator based on the prediction data; and S5, docking an energy storage system, optimizing charging and discharging behaviors of energy storage equipment, and realizing smooth power output. According to the invention, by fusing the laser radar wind measurement data, the deep learning prediction model and the intelligent optimization algorithm, high-precision wind condition monitoring, dynamic scheduling optimization and power output stability improvement of the wind power plant are realized.

The invention discloses a fault detection method and system for a mine unmanned vehicle, and relates to the field of positioning navigation, and the method comprises the steps: collecting the data of each sensor; according to the laser radar data, a slam point cloud map is constructed through a cartographic algorithm; according to the IMU data and the vehicle speed data, performing fusion through a Kalman filtering algorithm to obtain odometer data; the preprocessed laser radar data, IMU data, vehicle speed data, camera data and odometer data are converted to a slam point cloud map coordinate system through coordinate conversion, fusion is carried out through a Kalman filtering algorithm, and state estimation of the unmanned vehicle is obtained; taking the state estimation of the unmanned vehicle as a predicted value, comparing the predicted value with laser radar data, IMU data, vehicle speed data and camera data, and calculating an observation residual error; and performing unmanned vehicle fault detection according to the observation residual error. Aiming at the low positioning precision of the mine unmanned vehicle in a complex environment in the prior art, the positioning precision is improved through multi-sensor fusion, fault-tolerant switching and the like.

The invention discloses a mobile robot positioning and navigation method based on laser SLAM (Simultaneous Localization and Mapping). The method relates to the fields of machine learning, automatic driving, computer vision and the like. Aiming at the balance of real-time performance and calculation efficiency, the method comprises the following steps: firstly, tracking and estimating a motion track of a sensor by using laser radar data in real time, and fusing LiDAR and IMU; secondly, selecting a key frame and expressing the key frame by using a neural network implicit scene; adjusting parameters represented by a key frame pose and a neural scene through joint optimization to minimize a loss function; and finally, dynamically adjusting the loss function margin of laser radar rays by adopting a JS dynamic margin loss function, and converting an implicit neural representation three-dimensional scene into an explicit three-dimensional network model, thereby facilitating visualization, evaluation and use. Compared with other methods, the method has the advantages that the method has continuous and fine scene modeling capability, has real-time performance of positioning and mapping, and can be expanded to the fields of automatic driving, urban planning and the like.

In various examples, feature location identification for autonomous and semi-autonomous systems and applications is described herein. Systems and methods are disclosed that use LiDAR data to determine locations of road markings within an environment. For instance, the LiDAR data may be used to generate one or more images, such as a top-down image, birds-eye-view (BEV) image, and / or an intensity image, representing the environment. The image(s) may then be processed using one or more machine learning models that are configured to determine information associated with the road markings, such as bounding shapes (e.g., bounding boxes) indicating the locations of the road markings within the image(s), classifications associated with the road markings (e.g., the types of road markings), and / or any other information. The information may then be used to perform one or more processes, such as updating a map of the environment and / or navigating a vehicle within the environment.

A method for operating a robot, including: capturing images of a workspace; capturing data indicative of movement of the robot; capturing LIDAR data as the robot moves within the workspace; generating a map of the workspace based on the LIDAR data; actuating the robot to drive; discriminating between an object on a floor surface along a path of the robot and the floor surface based on the captured images; actuating the robot to drive until determining all areas of the workspace are discovered and included in the map; and executing a cleaning function.

A LiDAR apparatus, including: a laser to generate an optical signal; modulation components configured to receive the optical signal as an input and to output at least two corresponding modulated optical signals at respective output ports, wherein each modulated optical signal is modulated by a corresponding pseudo-random bit sequence, and: (i) the optical signals have respective different delays such that the modulations do not overlap in time; or (ii) the pseudo-random bit sequences have low cross-correlation; for each of the at least two modulated optical signals, a corresponding optical transmitter configured to transmit the corresponding modulated optical signal towards a corresponding surface spaced from the LiDAR apparatus by a corresponding distance, and a corresponding optical receiver configured to receive a portion of the transmitted optical signal scattered and / or reflected by the surface, the received portion of the optical signal having a phase shift and / or Doppler shifted angular frequency due to radial motion of the LiDAR apparatus relative to the surface; at least one photodetector to receive the optical signals received by the optical receivers, interfered with a reference beam, and to generate a corresponding output signal; at least one analogue to digital converter to generate a digital signal representing the output signal from the at least one photodetector; and a digital signal processing component configured to process the digital signal to generate LiDAR data representing the distances to the surfaces and / or relative velocities of the surface(s) with respect to the apparatus.

A request signal that indicates a quality for a determination of an orientation of a road user is received. The orientation of the road user is determined based on a) image data when the request signal indicates the quality is below a predetermined quality for the determination of the orientation of the road user, or b) on LIDAR data and image data when the request signal indicates the quality is the predetermined quality.

The invention discloses a crop canopy height estimation method based on laser radar and multispectral fusion, and the method comprises the steps: scanning a crop field through the laser radar, and obtaining the field point cloud data; aerial photography is carried out on a crop field through an airborne multispectral camera to obtain field multispectral image data, and after data image splicing, crop region extraction and land parcel segmentation, two point cloud feature parameters with the strongest correlation with crops are extracted from point cloud data to obtain a point cloud feature parameter; the method comprises the following steps of: extracting a vegetation index with the strongest correlation with the crop height from multispectral image data, solving through multiple linear regression, and establishing a height estimation model; according to the invention, the multispectral vegetation index and the laser radar data are combined, so that the crop canopy height monitoring precision is improved, the crop growth monitoring precision and reliability can be improved, the crop growth condition can be comprehensively understood, and an agricultural decision maker can determine the crop canopy height based on the crop canopy structure data. And a more accurate, timely and effective farmland management decision can be made.

The invention provides a tunnel quality safety detection system and method based on laser radar data, and the method achieves the fusion and complementation of crack detection based on point cloud density change and crack detection based on curvature change, and achieves the quick recognition and precise detection of a crack region. The potential crack area is rapidly positioned through point cloud density change detection, and the boundary of the crack is further accurately positioned in combination with curvature change detection, so that the accuracy and integrity of crack detection are improved. For tiny deformation which is very tiny in change in point cloud density and curvature and is difficult to directly detect, a projection plane is further determined on the basis of the identified crack, and after point cloud data is projected to a two-dimensional plane, the characteristics of the tiny crack in the image are highlighted through image enhancement, so that the crack is identified. And then an edge detection algorithm is used to identify the micro crack so as to make up for the defects of a previous point cloud density and curvature detection method in the aspect of micro crack detection, and the accuracy and integrity of crack detection are further improved.

The invention discloses a robot unknown environment autonomous exploration navigation method based on CTSAC, and the method combines the powerful sequence perception capability of Transform and a Soft Actor-Critic deep reinforcement learning framework, and introduces curriculum learning of a regular review mechanism, laser radar partition optimization processing, and refined reward setting. The state of the robot is represented by laser radar data and gyroscope data which are subjected to laser radar partition optimization processing, meanwhile, after state information is subjected to refined reward setting, corresponding reward values are obtained, corresponding experience sequences are stored in an experience pool, and data obtained through sampling of the experience pool are supplied to a neural network for training. The method is suitable for the robot to perceive the surrounding environment through the sensor in an unknown environment to perform path planning so as to reach a specified target point or establish an environment map, and is used for complex tasks such as space exploration, search and rescue, reconnaissance and the like.

The invention discloses an intelligent lighting management system and method based on multi-source data, and relates to the technical field of intelligent lighting management, and the system comprises a multi-source data collection module, an environment feature anchoring module, a dynamic error modeling and calibration module, an edge-cloud collaborative decision module and a lighting strategy execution module. The multi-source data acquisition module fuses environment sensor, visual perception and laser radar data to construct a'sensor original data + environment characteristic data 'dual-input system; the environment feature anchoring module extracts stable features from the visual / laser radar data to serve as a calibration reference; the dynamic error modeling and calibration module quantifies the error in real time based on the reference and generates calibration parameters; the edge-cloud collaborative decision-making module and cloud long-term optimization form a hierarchical collaborative architecture through edge node real-time control; and the lighting strategy execution module dynamically adjusts lighting parameters according to the calibration data and feeds back an effect to form a closed loop.

The invention relates to a multi-sensor fusion positioning method and device based on environmental characteristics. The method comprises the following steps: respectively acquiring RTK data, laser radar data and IMU data; converting the RTK data into RTK positioning data, converting the laser radar data into laser radar positioning data, and obtaining IMU positioning data according to IMU output; monitoring the quality state of the output signal of each sensor in the current environment, and calculating to obtain a corresponding quality score; constructing and forming an input matrix according to the quality score of each sensor, mapping the input matrix to a high-dimensional feature space through an embedded layer and adding a position code, inputting the input matrix into a Transform model for time sequence modeling and extracting environment dynamic features, and finally outputting a real-time fusion weight of each sensor; and performing weighted fusion by using the real-time fusion weight to obtain final pose output. The method has the advantages of being simple in implementation method, high in positioning precision and stability, high in environmental adaptability and robustness and the like.

The invention discloses a hyperspectral image and LiDAR data collaborative classification method based on double-domain mask and multi-scale local reconstruction, and belongs to the field of remote sensing image classification. According to the method, the problems of scarcity of annotation data and insufficient multi-source feature fusion precision in cross-modal classification of a traditional method are solved. According to the invention, feature learning is carried out through mask random image blocks and channels; a hierarchical multi-scale reconstruction architecture is designed, a lower-layer encoder learns fine-grained features, an upper-layer encoder recovers macroscopic semantic information, and multi-level feature space alignment is realized in combination with deconvolution up-sampling and adaptive pooling. According to the method, a multi-modal feature interaction mechanism and a cross-modal attention module are constructed by fusing the local feature extraction advantages of a convolutional neural network (CNN) and the global modeling capability of Transform, and the complementarity of heterogeneous data is enhanced. According to the method, through multi-level feature dynamic fusion and adaptive weight distribution, the collaborative classification precision of the hyperspectral image and the LiDAR data is improved. The method can be applied to remote sensing image classification.

The invention provides an under-mine video stream dynamic denoising method based on multi-modal fusion, which comprises the following steps: constructing a time sequence synchronous fusion mechanism of visible light, infrared and laser radar data, and realizing time-space alignment of multi-source heterogeneous data; a dynamic noise model is established by introducing a fractional calculus optical flow field concept and combining a Gaussian mixture model, so that a dynamic noise region is accurately identified; an improved self-adaptive wavelet threshold function is constructed, a function threshold parameter can be linked with a dust concentration sensor in real time, and the de-noising intensity is dynamically adjusted according to the actual dust concentration; designing a dual-path feature enhancement neural network to effectively separate and enhance structural features and texture features in the video image; a cascaded detection decision system is created, a lightweight network is used as a primary detector, a high-confidence detection result is directly output, and a low-confidence detection result is input into a Transform correction module for secondary reasoning. According to the invention, dynamic denoising, feature enhancement and target intelligent monitoring of the video stream under the mine can be realized.

The invention relates to the technical field of artificial intelligence and remote sensing image processing, and particularly provides a hyperspectral image and laser radar data classification method based on a dynamic fusion network. The method comprises the following steps: preprocessing acquired multi-modal data, and constructing multi-scale input; a dual-scale local attention module is designed, and context information of different scales is fused in a self-adaptive weighted mode through gating soft pooling; a dynamic down-sampling feature enhancement module is designed, the down-sampling rate is dynamically adjusted according to the complexity of the feature map, and deep multi-scale interaction is carried out based on a Mama backbone; constructing a directional interactive attention module, extracting features in horizontal, vertical and diagonal directions through directional gating convolution, and capturing an anisotropic structure of a linear ground feature; through the design of a double-path classifier, fusing shallow space details and deep semantic information; and the model is trained, optimized and reasoned to obtain data classification, and the method improves the classification precision and the calculation efficiency.

The invention discloses a distributed photovoltaic power station surveying method and system based on an unmanned aerial vehicle, and the method comprises the steps: dynamically planning an obstacle avoidance and supplementary shooting route, and collecting multi-modal data; visible light, thermal infrared and laser radar data are fused and analyzed, missing images are complemented in a generative mode, and defects are diagnosed in a combined mode; generating a dynamic shadow thermodynamic diagram by fusing the historical weather and the real-time cloud picture; and a report is automatically generated and AR interaction visualization is realized. The system correspondingly comprises a route planning module, a defect diagnosis module, a shadow prediction module and an analysis display module. Through the generative AI and spatio-temporal data fusion technology, the exploration efficiency and the defect recognition precision in a complex environment are remarkably improved, and intelligent operation and maintenance decision support of the whole life cycle of the photovoltaic power station is achieved.

The invention discloses an autonomous cognitive algorithm of a patrol robot hidden area based on a 3D laser radar, and relates to the technical field of artificial intelligence. The invention provides an autonomous cognitive algorithm for a patrol robot hidden area based on a 3D laser radar. The autonomous cognitive algorithm comprises the following steps: acquiring a non-ground point cloud by adopting a ground point cloud filtering algorithm; for the obtained non-ground point clouds, dividing concentric circle regions according to Euclidean distances, and performing Euclidean clustering on the point clouds in the concentric circle regions to obtain clustered point cloud clusters; according to the point cloud cluster, calculating an axis alignment bounding box, and obtaining a hidden point coordinate; and according to the point cloud cluster, calculating a directional bounding box, and calculating coordinates of a shielding point in a sight line shielding mode, thereby automatically calculating and obtaining a shielding angle. Therefore, by the adoption of the method, the hidden area is reasonably selected according to the angle information in the patrol process through the 3D laser radar data, the hiding effect is achieved, the patrol task is better completed, and high robustness and practicability are achieved.

The invention discloses a forest height inversion method and system based on multi-source remote sensing data and a Unet network, and particularly relates to the technical field of remote sensing data and deep learning, and the method comprises the steps: obtaining multi-source remote sensing data, including satellite-borne laser radar data and optical remote sensing image data; preprocessing the acquired multi-source remote sensing data, and constructing a network training data set; a Unet deep learning network model is improved, model parameters are optimized through training data, and a forest tree height inversion model is generated; based on a Unet basic deep learning network model, replacing a conventional convolution module with a two-dimensional dense connection module 2D DenseBlock, and adding an attention mechanism module CBAM; and utilizing the generated forest tree height inversion model to invert the forest tree height, and evaluating the inversion precision.

The present invention belongs to the field of indoor and outdoor robot application scene reconstruction, specifically, a deep learning 3D reconstruction method based on FMCW laser radar and visual fusion, comprising the following steps: 1) setting multiple laser radars and cameras to ensure time synchronization; 2) calibrating multiple laser radars and cameras; 3) real-time collection of FMCW laser radar data and camera image data in each scene and angle, corresponding to timestamps; 4) processing the collected data as a training data set; 5) the data are input into the deep learning model network for training to obtain a 3D Gaussian model of dynamic and static objects in the scene; 6) after coordinate conversion and velocity calculation of the collected FMCW laser radar data, combined with the collected camera image data, input into the 3D Gaussian model, and the present invention obtains scene reconstruction information. The present invention uses the native velocity information of the FMCW laser radar, combined with visual information, to better characterize the characteristics of dynamic objects and achieve native dynamic and static distinction.

The invention relates to the technical field of laser radars, in particular to an unmanned vehicle road obstacle sensing system based on a laser radar, and the system comprises a laser radar data collection module which adjusts the scanning frequency of the laser radar, collects point cloud data from static and dynamic obstacles, and obtains preliminary scanning data; and filtering the preliminary scanning data, and removing noise data. According to the invention, by calculating the dynamic characteristics of the point cloud data, the key point and the moving track of each obstacle are extracted, and the ability of sensing the obstacles in the dynamic environment is optimized. By adopting a time sequence analysis method, the moving state and track of the obstacle can be directly deduced according to the point cloud change, so that the ability of tracking the moving obstacle in a complex dynamic scene is improved. In the spatial reconstruction process, geometric subdivision is utilized to reconstruct point cloud data, and dynamic information is combined with a static point cloud structure, so that the analysis capability of spatial data is enhanced.