14068 results about "Lidar" patented technology

A system and method are disclosed for generating hyperspectral images from multi-modal sensor data including RGB, LiDAR, thermal, and near-infrared inputs. Training data includes hyperspectral images and corresponding multi-modal measurements. Spectral band grouping is performed based on correlation coefficients. A multi-modal decomposition network with cross-modal attention mechanisms generate reconstructed hyperspectral images by fusing complementary sensor information. A fine-tuning network creates reconstructed RGB images. A comprehensive quality assurance system analyzes spectral consistency, cross-modal coherence, and fusion artifacts to generate quality metrics. Missing data compensation strategies handle corrupted sensor inputs using information from other modalities. The system includes temporal integration for video sequences and multi-resolution processing for different sensor resolutions. Quality metrics guide network weight adjustments to improve reconstruction accuracy while maintaining robustness to sensor failures and environmental variations.

Disclosed are a high-performance loosely-coupled multi-modal data fusion system for a smart driving environmental perception system and a vehicle-mounted device, comprising: a fusion detection model based on a modality-independent feature interaction strategy, which is configured for converting a LiDAR point cloud, a camera image, and a millimeter-wave radar point cloud into a unified bird's-eye view representation, and performing multi-modal fusion; and a fusion tracking model based on a motion-appearance feature cascaded coupling data association strategy, which is configured for performing subsequent trajectory tracking and matching according to multi-modal fusion feature information. A VoD data set and a K-Radar data set are selected for training, verifying, and testing the comprehensive performance of the models, and a TensorRT accelerated inference model is applied, then quantized, and deployed to a vehicle-mounted computational testing platform. The present invention is compatible with mainstream sensor deployment solutions, and achieves the efficient complementary fusion of multi-source heterogeneous sensor information, significantly improving the reliability, accuracy, and adaptability of vehicle-mounted perception systems, thereby effectively responding to extreme operating conditions such as complex traffic scenarios and inclement weather.

The invention relates to the technical field of unmanned aerial vehicle control, in particular to an unmanned aerial vehicle autonomous obstacle avoidance decision-making method and system based on multi-source sensor fusion, and the method comprises the steps: achieving the time-space synchronization of a laser radar, a visual camera and a millimeter-wave radar through timestamp alignment and coordinate mapping, and constructing a dynamic obstacle grid map; fusing multi-source data based on a dynamic Bayesian network, and dynamically adjusting the confidence coefficient weight of the sensor in combination with the environment illumination intensity and the barrier surface material; and adopting a reinforcement learning model to generate an incremental obstacle avoidance strategy, and triggering a grading response instruction according to the risk assessment grade. The problem of fusion errors caused by spatial-temporal asynchronization of multi-source sensor data is solved, and the real-time obstacle avoidance success rate of dynamic obstacles is increased.

The invention discloses a logistics robot path planning method based on multi-modal perception, and relates to the technical field of robot path planning. Laser radar, visual camera and IMU data are fused, and environment state feature vectors are generated through multi-modal data synchronization and space-time alignment; the method comprises the following steps: analyzing environmental semantics by using models such as PointPill and YOLOv8, extracting dynamic characteristics, and identifying obstacles; constructing a space-time risk field, searching a path by a space-time algorithm, converting path points into a continuous trajectory, and optimizing the continuous trajectory; deviation is evaluated in real time, dynamic re-planning is triggered, and multi-robot cooperation and environment semantic understanding are included. Through multi-mode perception fusion, hierarchical planning, multi-target optimization and a cooperation mechanism, the obstacle detection accuracy and the obstacle avoidance success rate are improved, the path planning time is shortened, the energy consumption is reduced, the multi-robot conflict is reduced, the task efficiency is improved, the environment semantic understanding and task adaptive ability is enhanced, and the method is suitable for scenes such as intelligent storage and the like and has wide application prospects. The automation level is improved.

The invention relates to the technical field of rail traffic safety detection, in particular to a railway freight train key part online monitoring system based on an unmanned aerial vehicle, which comprises an unmanned aerial vehicle control module, a multi-modal data acquisition module, an edge calculation module, a central processing module and a feedback execution module. A dynamic three-dimensional grid flight path is generated by fusing a train Beidou positioning signal and a millimeter wave radar sensing result, a system is provided with an infrared thermal imager, a laser radar and a high-speed polarization camera, bearing temperature, train body point cloud and a train coupler image sequence are obtained, and temperature rise area identification, structural deformation extraction and coupling state modeling are completed on the edge side. The central processing module outputs a multi-dimensional safety assessment result, and the feedback module generates a compensation control instruction and a graded early warning signal based on the risk fusion index. The method has the advantages of being high in autonomy degree, high in recognition precision and high in response speed, and is suitable for full-time structural intelligent inspection and early warning of the freight train in a high-speed operation environment.

The invention relates to the technical field of unmanned aerial vehicle flight path planning, in particular to an unmanned aerial vehicle real-time path planning system and method based on dynamic weight distribution and multi-source data fusion. The system comprises a multi-source data fusion module, an integrated laser radar, a millimeter wave radar, a visual sensor and a Beidou positioning unit. The dynamic weight distribution module dynamically adjusts the weight coefficient of each sensor according to the environmental complexity, the threat level and the state of the unmanned aerial vehicle by adopting a mixed decision-making mechanism combining fuzzy logic and reinforcement learning; an improved RRT * algorithm and a Markov decision process are built in the real-time path planning module, and a global optimal path and a local obstacle avoidance track are generated by adopting a layered planning architecture; the unmanned aerial vehicle cooperative control module comprises a dual-redundancy flight control system and a dynamic obstacle avoidance unit; and the communication relay module supports 5G and low-orbit satellite dual-mode communication, updates an environment cognitive model of each unmanned aerial vehicle through federated learning, and realizes multi-source fusion real-time path planning based on dynamic weight distribution and the unmanned aerial vehicles.

The invention relates to the technical field of tunnel engineering, in particular to a tunnel surrounding rock grading method and system, comprising intelligent sensing and data acquisition, multi-source data fusion and modeling, hybrid model dynamic grading, real-time decision and support optimization, online learning and dynamic feedback, and risk early warning and emergency response. Compared with the prior art that a geological data acquisition mode combining manual drilling coring and low-resolution geophysical prospecting is adopted, efficiency is low, subjective errors are large, and a complex geological structure is difficult to cover, unmanned aerial vehicle LiDAR scanning, intelligent rock core image analysis and a high-density IoT sensor network work cooperatively, and the working efficiency is greatly improved. Real-time dynamic acquisition of full-section geological information is achieved, manual intervention errors are eliminated in combination with a multi-source data fusion algorithm, the automation level and three-dimensional space representation precision of data acquisition are remarkably improved, and a high-resolution holographic data base is provided for surrounding rock classification.

The invention discloses a three-dimensional environment reconstruction optimization method based on multi-sensor fusion data, and relates to the field of three-dimensional environment reconstruction optimization, and the three-dimensional environment reconstruction optimization method based on the multi-sensor fusion data comprises the following steps: S1, collecting multi-source sensor data, and constructing a data set under a unified coordinate system; s2, generating dense visual point cloud, and extracting laser point cloud features to construct a model; s3, establishing a local three-dimensional model, and generating a local environment image; s4, shadow parameters are extracted through shadow geometric analysis, and time sequence optimization is carried out; s5, consistency verification and correction are carried out, and three-dimensional reconstruction data are output; and S6, comparing the reconstruction data with the navigation map database, and carrying out map optimization updating. According to the method, time synchronization and space calibration are carried out on data acquired by the depth camera and the laser radar, complete and accurate three-dimensional information modeling of the target environment is realized, and the geometric precision of environment reconstruction and the image detail reduction capability are improved.

The invention discloses an intelligent photoelectric theodolite aerial target positioning and tracking system, which relates to the technical field of photoelectric detection, and comprises a multi-mode photoelectric sensor module integrating visible light, infrared thermal imaging, a laser radar and a polarized light sensor and supporting spectrum adaptive switching; the dynamic noise suppression processing module is used for eliminating environmental interference based on a time-space domain hybrid filtering algorithm; the multi-target tracking control module adopts a time-sharing partition scanning strategy and a graph neural network data association algorithm; the anti-interference servo driving module is used for realizing stable tracking under strong disturbance through inertial navigation-visual fusion compensation; and the edge computing platform is used for deploying a lightweight deep learning model to complete target recognition and trajectory prediction. According to the invention, through interdisciplinary collaboration of quantum dot materials, graph neural networks and physical equation constraints, the bottleneck of a single technology is broken through; and through closed-loop optimization of dynamic anti-interference and edge intelligence, full-link enhancement of'perception-decision-execution 'is realized.

The invention relates to the technical field of road crack detection, in particular to a road crack detection method and system based on a fused image. The method comprises the following steps: acquiring road multi-source monitoring data including a visible light image, infrared thermal imaging data and laser radar point cloud data, and performing multi-modal image fusion and road three-dimensional point cloud reconstruction to generate a fused road image and road three-dimensional modeling data; performing crack curvature analysis based on the fused road image to generate crack curvature data; performing reflection crack contour recognition and positioning on the fused road image through the crack curvature data to generate reflection crack initial positioning data; obtaining road base material data; and performing reflection crack stress field reconstruction on the road area according to the reflection crack initial positioning data to obtain a reflection crack stress field. According to the invention, through multi-modal fusion, curvature identification, stress field modeling and crack channel analysis, the accuracy and strain of road reflection crack detection are improved.

The invention provides a flood control project virtual simulation and risk rehearsal method based on digital twinning, and the method comprises the following steps: obtaining drainage basin multi-source data collected by a space-air-ground integrated monitoring network, the multi-source data comprising a satellite remote sensing image, an unmanned aerial vehicle LiDAR point cloud and ground sensor monitoring data; inputting the multi-source data into a pre-constructed digital twin model, wherein the model comprises a spatio-temporal data fusion layer, a physical process modeling layer, a multi-scale simulation deduction layer and a risk assessment decision-making layer which are connected in sequence; the spatio-temporal data fusion layer is used for performing spatio-temporal registration and feature fusion on input multi-source data, and constructing a total element digital backplane; through the space-air-ground integrated monitoring network and the spatio-temporal data fusion technology, high-precision digital mapping of basin total elements is realized. Compared with a traditional single data source, time-space consistency of flood routing simulation is remarkably improved through multi-source data fusion, and prediction errors of key physical processes such as river channel scouring are greatly reduced.

The invention discloses a multi-source information collaborative power equipment three-dimensional temperature field construction method. Firstly, an infrared camera, an IMU and a laser radar are utilized to obtain an accurate external parameter relation through joint calibration, point cloud distortion is eliminated, angular points and plane points are extracted, a re-projection residual error, a distance sampling residual error and an IMU pre-integration residual error are constructed, an error state iteration Kalman filter is adopted to optimize a global pose, and positioning is achieved. Providing a self-supervised depth completion network, combining an infrared temperature image and a sparse depth map generated by a laser radar as input, adopting a depth completion strategy guided by an infrared image, estimating relative motion of adjacent frames by using pose information, introducing a feature alignment module to reduce alignment errors, and combining the depth map, the infrared image and IMU data to obtain a self-supervised depth completion algorithm; and efficient construction of the three-dimensional temperature field of the power equipment is realized. According to the invention, the three-dimensional temperature field of the power equipment is constructed more accurately, and the capability of the substation inspection robot for state monitoring and fault diagnosis of the power equipment is improved.

The invention relates to the technical field of industrial nondestructive testing and computer vision, and particularly discloses an intelligent detection method for surface defects of a container bottom plate based on deep learning. The method comprises the following steps: synchronously acquiring data through a laser radar and a line scanning camera, complementing a shielding area of a point cloud, and executing coordinate normalization to generate preprocessed data; constructing a lightweight feature alignment network to realize cross-modal feature mapping and pixel-level error correction; segmenting a defect area by adopting an improved PointNet + + network and reconstructing a three-dimensional model; extracting defect geometric parameters and combining with material attributes to perform stress simulation and life prediction; aggregating multi-port safety life data to construct a federated framework to update model parameters; and synthesizing physically real defect samples based on false detection cases, and injecting the defect samples into the network for training. According to the method, the bottleneck of missing detection of internal defects in traditional two-dimensional detection is overcome, the defect detection rate and quantification precision are remarkably improved, full-life-cycle safety evaluation of the container is supported, and the efficient requirement of automatic port inspection is met.

The invention relates to the technical field of building appearance defect detection, in particular to a building appearance defect detection method and system based on an unmanned aerial vehicle, and the method comprises the steps: obtaining the building information of a target building, and generating a hierarchical scanning path and a three-dimensional obstacle avoidance flight path, a visible light image, an infrared thermodynamic diagram and laser radar point cloud information are collected for space-time alignment processing, and an attention mechanism neural network is used for extracting multi-scale features to generate a detection report containing defect three-dimensional coordinates, damage levels and safety risk assessment. The method achieves the purpose of efficiently and accurately detecting the building appearance defects, can adapt to complex building structures and environmental conditions, supports defect trend prediction and maintenance decision, and remarkably improves the building safety management efficiency.

The invention relates to the technical field of computer vision and three-dimensional reconstruction, and discloses an exhibition hall three-dimensional modeling intelligent optimization system based on multi-modal data fusion, and the system comprises a data collection module which is configured to synchronously obtain laser radar point cloud data, a multispectral image sequence and inertial measurement unit data; the preprocessing module is used for receiving the output of the data acquisition module, aligning a multi-source sensor coordinate system through a space-time calibration algorithm, and separating a static scene from a dynamic interference element by using a dynamic segmentation network; and the multi-modal fusion module is used for receiving the preprocessed data and carrying out adaptive weighted fusion on the geometric features of the laser radar and the visual texture features through a cross-modal attention mechanism. According to the invention, through multi-modal data fusion and a dynamic scene adaptive mechanism, the modeling precision and the real-time updating capability in a complex exhibition hall environment are significantly improved.

The invention relates to the technical field of unmanned aerial vehicle remote sensing and artificial intelligence crossing, in particular to an intelligent monitoring system and method based on multi-modal remote sensing data and deep learning, and the system comprises an unmanned aerial vehicle cluster networking subsystem, a mixed feature matching subsystem and a multi-modal fusion and continuous learning subsystem. The method comprises the following steps: constructing an unmanned aerial vehicle cluster carrying a multispectral sensor and a laser radar LiDAR, and carrying out wireless networking among a plurality of unmanned aerial vehicles to realize sharing of acquired images; feature point extraction is carried out on collected images of different time phases, the extracted feature points are input into the generative adversarial network, and the feature points are matched; and receiving the matched collected images, dynamically fusing data of visible light, infrared and other multi-modal images through a space-time attention mechanism, and realizing high-precision target recognition and dynamic environment self-adaption in a small sample scene. According to the method, unmanned aerial vehicle multi-source remote sensing data acquisition, feature fusion and deep reinforcement learning are combined, and the method is used for intelligently monitoring a dynamic environment.

According to the invention, based on a low-line-number road side laser radar, real-time extraction and optimization of multi-vehicle tracks are researched, and a track completion method and system combined with a road topological map are provided for solving the problem of track interruption caused by mutual shielding of vehicle targets in a complex environment. According to the method, track completion is carried out by obtaining a full-time motion mode, context information and environment characteristics of a vehicle in a scene and combining a road topological map. Specifically, the topological map is coded by adopting a gating loop unit and a map attention network, and missing trajectory data is generated by utilizing a diffusion probability model based on trajectory context and map condition constraints. By integrating point cloud data acquired by a roadside laser radar, a vehicle track is complemented, and a high-precision continuous track conforming to road constraints, traffic rules and vehicle kinematics characteristics is generated. The trajectory completion method has important significance in improving the intelligent traffic decision-making level and promoting the application of the intelligent driving technology.

The invention relates to a multi-sensor cross-scene dynamic preferential fusion positioning and mapping method, and the method comprises the steps: obtaining the data of a plurality of sensors, and completing the unification of the time-space relation of the data of the plurality of sensors; processing the data, carrying out loopback detection on image key frame data acquired by a camera, constructing to obtain an I MU pre-integration factor, a visual inertial odometer factor, a laser radar odometer factor, a GPS inertial odometer factor, a UWB factor, a GNSS factor and a loopback detection factor, and adding the factors into a factor graph for optimization; a global positioning pose and a map are obtained; and optimizing the multi-sensor data fusion strategy based on a deep fuzzy neural network. According to the multi-sensor cross-scene dynamic preferential fusion positioning and mapping method provided by the invention, high-precision positioning and navigation of an agricultural robot in different scenes are realized through real-time fusion of various sensor data, and the problems of scene dependence and insufficient precision of an existing single sensor scheme are solved.

The invention discloses a multi-modal visual fusion complex scene small target detection tracking method and system, and relates to the technical field of unmanned aerial vehicle target tracking, and the method comprises the steps: employing a visible light camera, an infrared thermal imager and a laser radar sensor which are carried on an unmanned aerial vehicle platform, and synchronously collecting RGB images, thermal infrared images and point cloud data; the consistency of the multi-modal data is ensured through data preprocessing and space-time alignment; constructing a lightweight double-branch network to extract multi-scale features, generating a fusion feature map by adopting adaptive weighted fusion, and generating depth information by utilizing point cloud to assist in scale estimation; a small target detection head is designed based on the fusion feature map, and precise detection is realized in combination with a feature pyramid network, adaptive scale prediction and a context awareness suppression mechanism; furthermore, through multi-mode cooperative tracking, including target association, spatio-temporal context modeling, trajectory prediction and a re-detection mechanism, tracking continuity is ensured.

The invention discloses an ultra-wideband laser radar inertial navigation cooperative SLAM (Simultaneous Localization and Mapping) method and system, which are suitable for robot positioning and mapping tasks in a GPS (Global Positioning System)-free environment. According to the method, high-frequency motion priori of an IMU, relative pose constraint of LiDAR and absolute ranging information of UWB are fused, a unified factor graph optimization model is constructed, and multi-sensor cooperative positioning is realized; in the system initialization stage, IMU bias calibration, UWB base station coordinate configuration and LiDAR initial attitude estimation are completed; in the operation process, IMU pre-integration is utilized to predict the pose, LiDAR point cloud registration is utilized to obtain the relative motion between key frames, UWB ranging is combined to construct a residual item, a self-adaptive weight mechanism is introduced to suppress ranging abnormity, and the fusion robustness is improved; the system supports geometric feature loopback detection, cross-frame constraints are constructed in combination with UWB to carry out closed-loop optimization, and an optimization result is used for real-time incremental updating of a global map; the method has the advantages of high positioning precision, strong anti-interference capability, wide application scene and the like.

The invention discloses a data center machine room AI energy-saving control method and system, a digital twin model of a machine room operation state is constructed through a holographic perception and heterogeneous data fusion technology, centimeter-level monitoring of an equipment state and environmental parameters is realized, and the system integrates a laser radar array, an acoustic sensor and a gas sensor network. The time-space alignment of multi-modal data is completed by combining edge computing nodes, holographic mapping including thermodynamic characteristics, vibration characteristics and gas leakage risks is formed, historical temperature control strategy characteristics are extracted by adopting a variational auto-encoder based on a dynamic strategy generation mechanism of generative artificial intelligence, and a load trend is predicted by combining a long-short-term memory network. Constructing a self-adaptive strategy pool; the multi-agent reinforcement learning framework enables temperature control, equipment scheduling and power grid response to form game optimization, the strategy robustness in a complex scene is improved, and the system innovatively fuses power grid real-time electricity price and carbon transaction data so as to establish a multi-target decision system.

The invention relates to a multi-source remote sensing image classification method based on a key band retrieval attention mechanism, and belongs to the technical field of remote sensing image processing. The method comprises the steps of firstly performing preprocessing and data set division on multi-source remote sensing data, then constructing a key band retrieval attention network, performing training and evaluation on a model by utilizing a training set and a verification set after division, and finally performing visual analysis on a model result by utilizing a test set. According to the method, the features of the hyperspectral image and the laser radar / synthetic aperture radar image are effectively extracted and fused, redundant information interference is effectively reduced, the retention rate of hyperspectral key information is improved, the complementary expression ability among multi-source data is enhanced, and the remote sensing image classification precision and calculation efficiency are remarkably improved. The method is suitable for application scenes of multi-source remote sensing data fusion and classification, can meet efficient intelligent processing requirements of complex earth surface information, and provides an accurate and efficient remote sensing image classification solution.

The invention discloses a multi-modal image automatic labeling system and method, and relates to the technical field of image data processing. According to the multi-modal image automatic labeling system and method, time sequence alignment is carried out on video streams and laser radar point cloud data through an asymmetric dynamic time warping algorithm, semantic and geometric features are extracted, and the elastic coefficient of the algorithm is dynamically adjusted. And combining a modal perception attention mechanism, dynamically allocating fusion weights of the video stream and the laser radar according to the features, generating a cross-modal joint feature vector, and outputting a preliminary labeling result. And generating an annotation robustness index by calculating the prediction entropy and the three-dimensional intersection-to-union ratio confidence of the target detection frame, and iteratively optimizing the annotation result. And mapping the cross-modal features and the labeling result into a space-time correlation map, and displaying the three-dimensional positioning, motion trail and modal contribution degree thermodynamic diagram of the target in real time. The problems of time alignment, feature fusion and labeling robustness are effectively solved, and a high-precision and interpretable automatic labeling solution is provided.

The present invention relates to a method and system for estimating forest carbon storage that combines artificial intelligence algorithms and multimodal remote sensing data. This approach comprehensively utilizes laser radar satellites, multi- / hyperspectral satellites, radar satellites, high-resolution optical imagery, etc. A hybrid technical system is employed for different forest coverage areas, resulting in high-precision forest carbon storage mapping with a resolution of 10 meters and area coverage. This provides technical support and assurance for assessing global forest carbon storage and supporting forestry carbon sequestration transactions.

The invention provides a forest land tree height measurement and determination method and system based on laser radar point cloud data. Stress wave signals are collected based on a trunk base acoustic emission sensor array to generate an acoustic characteristic parameter set, the digital twin model is driven to complete forest stand structure topological optimization, and a three-dimensional growth vector model reflecting the internal mechanical state of a trunk is formed. And synchronously fusing high-precision slope point cloud data returned by the unmanned aerial vehicle laser radar in real time, correcting a terrain distortion error through a dynamic splicing algorithm in combination with stress distribution characteristics, and generating a crown segmentation boundary constrained by physical characteristics. And finally outputting a tree height parameter corrected by the abrupt slope topography through model iterative optimization and space vector analysis. According to the technical scheme, synchronous sensing of the three-dimensional shape and the mechanical state of the tree in the complex terrain environment is achieved, and the tree height measurement error is reduced.

The invention relates to a self-localization and motion perception method based on deep learning, and the method comprises the following steps: multi-modal perception: collecting RGB frames and event streams through employing a DAVIS346 event camera, obtaining texture and depth information through employing an RGB-D camera, and supplementing 3D structure data through a laser radar; hybrid optical flow driven motion perception: adopting a double-branch architecture of an improved RAFT network and an event optical flow private network; multi-modal 3D detection and trajectory management: fusing multi-modal data based on VoxelNeXt-Lite to generate a 3D detection frame, filtering false detection by combining point cloud density clustering and an event density threshold, then constructing a space-time diagram associated trajectory through a Spatio-Temporal Graph Transformer, and optimizing pedestrian trajectory prediction continuity by using an LSTM (Long Short Term Memory) model perceived by a gait cycle; multi-view semantic graph matching: constructing a 3D semantic voxel map containing vertical features, calculating scene similarity in combination with top view NetVLAD features and deformable graph matching, and dynamically adjusting semantic weight by using a situation encoder and knowledge graph reasoning; and carrying out multi-sensor fusion and robust positioning.

The invention provides an intelligent water conservancy digital twinborn simulation system based on multi-source data, and belongs to the technical field of digital monitoring. Minute-level data acquisition and transmission are realized by constructing a space-air-ground three-dimensional sensing network and combining edge intelligent preprocessing, data acquisition, noise reduction and abnormity identification are realized by utilizing sensors such as millimeter wave radar and laser radar, and the system is used for realizing data acquisition and transmission. Dynamic data fusion and intelligent calibration are carried out, and second-level alignment and credible verification of data are realized by means of a space-time calibration algorithm, Kalman filtering and a block chain evidence storage technology; a hybrid simulation and intelligent decision model is established, a physical model, a machine learning architecture and a dynamic threshold decision tree are adopted, flood routing minute-level prediction and emergency response are realized, and the system improves water conservancy monitoring prediction precision and emergency response efficiency.

The invention discloses an unmanned aerial vehicle laser and vision fusion inspection method and system for bridge bottom disease detection, and the method comprises the steps: carrying out the synchronous data collection through employing a calibrated laser radar, a camera and an IMU, and obtaining a three-dimensional laser point cloud and a two-dimensional visual image of the appearance of a bridge; sharpening the image containing the motion blur and completing brightness self-adaption of the image; stable feature points are extracted, multi-frame matching is carried out, the corresponding poses of the images are estimated, and bridge dense point cloud reconstruction is completed; performing geometric component segmentation on the point cloud to generate a geometric prior region; component segmentation is carried out on a support area in the image, and a continuous and accurate component segmentation result is obtained in combination with a geometric prior area; screening the image, calling a targeted disease detection model in a corresponding component area, and generating a segmentation mask for the disease; obtaining a real disease three-dimensional point cloud, and carrying out quantitative calculation on the physical size of the disease; and displaying the real disease three-dimensional point cloud data and the physical size of the disease. The method is high in efficiency and precision.

The invention belongs to the technical field of computer vision, and relates to a dynamic scene online calibration method and system for three-dimensional target detection. The method comprises the following steps: firstly, acquiring a 3D point cloud around a vehicle, an initial external parameter and a synchronous image, and processing the 3D point cloud, the initial external parameter and the synchronous image by a bimodal static mask generation module to obtain a static region probability graph; calculating a multi-scale residual error of the image and the LiDAR edge image in a static region through a static region fusion module; a historical time sequence modeling module is used to generate time sequence fusion features; performing external parameter correction through a coarse-fine multi-stage external parameter correction module; and finally, according to the predicted external parameters, performing three-dimensional target detection through an image-point cloud bidirectional enhancement module in combination with Transform. Full-scene self-adaptive calibration is achieved, the invalid calibration rate is reduced by 67%, permanent deformation and temporary interference of the sensor can be effectively distinguished, the diagnosis accuracy rate reaches 92%, the false alarm rate is lower than 0.5%, and the three-dimensional target detection performance is improved.

The invention relates to a man-machine hybrid autonomous navigation system in an unknown dynamic environment, which comprises an environment sensing module for acquiring environment information by using sensors such as a laser radar, a camera and an inertial measurement unit and performing data preprocessing; the local target point selection module selects a local target point set meeting requirements in the environment according to the real-time information provided by the environment sensing module; the decision planning module is used for generating an optimal path and a navigation strategy based on mixed training of a deep reinforcement learning algorithm and artificial experience; and the learning optimization module improves the generalization ability of the algorithm through online learning and transfer learning technologies, and realizes effective introduction and strategy adjustment of human experience in combination with the man-machine interaction module. The man-machine interaction module can work in cooperation with the local target point selection module so as to dynamically adjust target point selection according to external input and optimize path planning. According to the method, high decision stability can be kept in an unseen environment, the training cost is reduced, and the method is more suitable for autonomous navigation application in the real world.