15149results about "Scene recognition" patented technology

Disclosed are a high-performance loosely-coupled multi-modal data fusion system for a smart driving environmental perception system and a vehicle-mounted device, comprising: a fusion detection model based on a modality-independent feature interaction strategy, which is configured for converting a LiDAR point cloud, a camera image, and a millimeter-wave radar point cloud into a unified bird's-eye view representation, and performing multi-modal fusion; and a fusion tracking model based on a motion-appearance feature cascaded coupling data association strategy, which is configured for performing subsequent trajectory tracking and matching according to multi-modal fusion feature information. A VoD data set and a K-Radar data set are selected for training, verifying, and testing the comprehensive performance of the models, and a TensorRT accelerated inference model is applied, then quantized, and deployed to a vehicle-mounted computational testing platform. The present invention is compatible with mainstream sensor deployment solutions, and achieves the efficient complementary fusion of multi-source heterogeneous sensor information, significantly improving the reliability, accuracy, and adaptability of vehicle-mounted perception systems, thereby effectively responding to extreme operating conditions such as complex traffic scenarios and inclement weather.

A small-size vehicle detection deep learning model based on feature fusion of multi-scale modules is provided, which solves the problem of small-size vehicle image detection. The model includes a Backbone network, a Neck layer and a Head network, wherein a C2f_DCNv3 module based on the combination of deformable convolution v3 (DCNv3) and a cross stage feature fusion (C2f) module and an SPPF_LSKA module based on the combination of a spatial pyramid pooling fast (SPPF) layer and a large separable kernel attention (LSKA) module are introduced into the Backbone network; a C2f_SCConv module based on the combination of spatial and channel reconstruction convolution (SCConv) and a C2f module is introduced into the Neck layer; and a multi-scale kernel detection (MSK_Detect) module is introduced into the Head network.

The invention relates to the technical field of red tide anomaly detection, in particular to an ocean red tide anomaly detection method and system fusing multi-source remote sensing and a graph neural network. The method comprises the following steps: acquiring remote sensing image data, unmanned aerial vehicle image data and monitoring data of a monitoring point; performing data preprocessing on the acquired remote sensing image data and unmanned aerial vehicle image data; performing feature extraction and feature fusion on the remote sensing image and the unmanned aerial vehicle image to obtain remote sensing feature data; constructing a space-time diagram structure based on the monitoring data of the monitoring points to obtain diagram structure data; based on a cross-modal comparison self-supervised learning mechanism, carrying out consistency representation learning on a remote sensing feature mode and a graph structure feature mode; by introducing multi-source heterogeneous data and fusing a graph neural network modeling means, the limitation of a single data driving method in the aspects of coarse red tide recognition granularity, low space-time precision and the like is effectively broken through, and the meticulous property and global perception ability of red tide feature modeling are remarkably improved.

The invention discloses a vehicle multi-modal trajectory prediction method based on an improved attention network, and belongs to the technical field of intelligent vehicle trajectory prediction, and the method comprises the steps: collecting historical trajectory data of a target vehicle and surrounding vehicles as an input sequence; secondly, constructing a vehicle multi-modal trajectory prediction model which comprises a motion feature extraction module, a space-time interaction module, a space-time fusion module and a trajectory output module; the motion feature extraction module uses a multi-scale convolution attention network and a gating circulation unit for processing, the space-time interaction module uses a dynamic graph attention network for extracting vehicle interaction information, and the space-time fusion module splices and fuses target vehicle motion features and space-time interaction features to obtain space-time fusion features; the track output module inputs the fusion features into a gating circulation unit, decodes the fusion features and then inputs the fusion features into a mixed density network, and multi-mode output of vehicle tracks is achieved; and finally, a proper loss function is selected for training, so that the prediction precision and the convergence speed of the model are improved.

The invention discloses a robot real-time potential safety hazard recognition system based on multi-modal sensor fusion, and particularly relates to the technical field of intelligent inspection and safety monitoring, the system comprises five parts of data acquisition, information fusion, behavior response, trajectory analysis and risk output, and the potential safety hazard recognition system is used for recognizing potential safety hazards through image acquisition, thermal imaging, gas concentration and temperature and humidity information. Carrying out numerical value normalization and feature extraction, identifying potential abnormity and generating early warning; triggering data enhanced acquisition and track recording in the target area, and constructing a space-time path model to analyze an abnormal evolution trend; and finally, outputting a potential safety hazard assessment result according to a risk level classification rule by combining the enhanced information and the trajectory features. According to the method, high-precision early warning is realized through multi-source data acquisition and normalization fusion, the local recognition capability is improved based on dynamic enhanced acquisition of a behavior response mechanism, an abnormal development trend is tracked by combining track evolution modeling, risk level assessment is output according to the abnormal development trend, and accurate recognition and dynamic management and control of hidden dangers are realized.

The invention relates to the technical field of remote sensing monitoring, in particular to a remote sensing image ocean green tide monitoring method and system fusing physical prior and spatio-temporal evolution. The method comprises the following steps: acquiring a multi-modal remote sensing monitoring image; performing multi-modal feature extraction on the acquired image, wherein the multi-modal feature extraction comprises spectral reflectivity feature extraction, ocean dynamics feature extraction and feature alignment and unified representation; establishing a physical prior of a green tide characteristic wave band by using an ocean optical radiation transmission model; constructing a dynamic space-time diagram based on the extracted multi-modal features to obtain a node global feature vector and a dynamic adjacency matrix; carrying out adaptive graph convolution feature coding based on physical prior and a dynamic space-time diagram; through fusion of multi-spectral images of multiple platforms such as satellites and unmanned aerial vehicles and ocean dynamic data and combination of atmospheric correction and wave band resampling, consistency processing and high-precision extraction of multi-source features are realized, and comprehensiveness and reliability of green tide feature recognition are remarkably improved.

The invention provides a river water quality monitoring method based on multi-source remote sensing data, and belongs to the technical field of river water quality monitoring. The method comprises the following steps: establishing an inherent optical characteristic model of a river water body based on a Hybrid water body radiation transmission model, optimizing a calculation path by adopting a shortest path algorithm, and carrying out water body optical component inversion by adopting a quasi-analysis algorithm and a generalized inherent optical characteristic algorithm in combination with a multispectral characteristic enhancement model to obtain absorption coefficients of all components; and constructing a multi-band combination index to realize optical coupling effect decoupling, and applying the fine-tuned water quality parameter inversion basic model to a target river area to output a suspended matter concentration distribution diagram, a chlorophyll concentration distribution diagram and a transparency distribution diagram. The technical problem of low precision of remote sensing inversion of water quality parameters caused by mutual coupling of multiple optical active components in a river water body is solved.

The invention discloses a multi-modal semantic and physical law driven remote sensing image generation method, belongs to the technical field of computer vision and remote sensing image generation, and aims to solve the problems of insufficient cross-modal semantic alignment, low reliability of a generation result and insufficient physical mechanism fusion. The four-stage method comprises the following steps: firstly, rejecting low-quality samples from original data and unifying a spatial scale; then, extracting a multi-modal semantic vector by adopting a BLIP model and a CLIP model, and introducing a remote sensing physical rule to carry out vector optimization; then position coding and physical constraint conditions are embedded in the submerged space, and multi-source information joint modeling is achieved through a cross-modal encoder; and finally, by taking text description, physical priori knowledge and diffusion time steps as joint conditions, performing de-noising reasoning based on a Transform architecture, and completing back diffusion reconstruction by means of a trans-attention mechanism. According to the method, physical rationality and semantic consistency are improved, and a more reliable technical normal form is provided for remote sensing image generation in the fields of disaster monitoring, military simulation and the like.

The invention relates to the technical field of natural resource monitoring and spatial information processing, in particular to a method and a system for feeding back land utilization change based on a land spatio-temporal model. The method comprises the following steps: deploying multi-source land monitoring equipment, carrying out collaborative data acquisition and standardization processing, and constructing a land space-time reference data set; performing triple mapping on the land space-time reference data set to obtain a land semantic association graph; constructing a land utilization knowledge graph based on the land semantic association graph; constructing a land change detection initial model by using the land utilization knowledge graph; meanwhile, in a high-frequency change scene, such as an urban and rural ecologic zone or an ecological sensitive area, a traditional model is slow in response to short-term land utilization disturbance, automatic adjustment cannot be carried out through deviation feedback between historical errors and model output, and the reliability of the model in actual application scenes such as resource regulation and control is limited.

The invention relates to the field of remote sensing image processing, in particular to a high-resolution remote sensing image semantic segmentation method based on multi-scale feature fusion, which comprises the following steps: acquiring a public remote sensing image data set, preprocessing the image, and constructing a training and testing set of semantic segmentation; a CTMFNet is designed, an encoder is composed of a lightweight residual module and an MS-Transform, and local space details and global context information are extracted; rID is adopted to reduce spatial information loss, LSFE is introduced to improve spatial positioning capability, and feature calibration is carried out in space and channel dimensions through DecoderAttn to realize boundary fine segmentation; inputting the training sample into the network for training to obtain a converged optimal semantic segmentation model; and inputting the test set into the model to obtain a semantic prediction map, and outputting a fine segmentation result of the remote sensing image through multi-scale fusion and boundary restoration. According to the method, the precision and robustness of ground feature extraction are effectively improved, the calculation cost is remarkably reduced while high segmentation precision is kept, and the method has good practical value and popularization prospects.

The invention relates to the technical field of data security management, and discloses a blockchain-based forestry data security management system and method, and the system comprises a data collection unit which is used for comprehensively obtaining a multi-dimensional information flow of forest ecology and resource states; the data processing unit is used for carrying out deep cleaning, standardized regulation and feature value extraction on the originally collected mass heterogeneous data; the block chain storage unit is used for constructing a permanent, tamper-proof and distributed secure storage infrastructure platform of the forestry core data; a verification unit; the safety control unit is used for constructing a covering data transmission, storage and access full-chain depth defense system structure model; a user interface unit; an auditing unit; and a network communication unit. The method is reasonable in design, and the intelligent analysis module is used for processing to form a forest interannual growth trend prediction function distribution map state visual output result set.

The present invention relates to a method and system for estimating forest carbon storage that combines artificial intelligence algorithms and multimodal remote sensing data. This approach comprehensively utilizes laser radar satellites, multi- / hyperspectral satellites, radar satellites, high-resolution optical imagery, etc. A hybrid technical system is employed for different forest coverage areas, resulting in high-precision forest carbon storage mapping with a resolution of 10 meters and area coverage. This provides technical support and assurance for assessing global forest carbon storage and supporting forestry carbon sequestration transactions.

The invention relates to the technical field of drainage basin water scheduling, and discloses a drainage basin water regulation and control optimization method based on ecological element changes. The method comprises the following steps: deploying a drainage basin monitoring system, and collecting ecological element real-time data such as a hydrological parameter sequence and a remote sensing image; after the data is cleaned and converted, hydrological trend features and spatial distribution features are extracted by adopting a feature learning model, and the hydrological trend features and the spatial distribution features are fused into unified ecological representation through a cross-modal alignment mechanism; inputting the unified ecological representation into a physically constrained neural network prediction model, and outputting a water regimen dynamic prediction value; and finally, based on the predicted value, a water resource regulation and control instruction is generated and executed by using a multi-objective decision algorithm so as to optimize the watershed water circulation process. According to the method, feature extraction comprehensiveness is improved through multi-source data fusion and cross-modal analysis, prediction reliability is enhanced in combination with physical constraints, reasonable allocation of water resources is achieved by means of multi-target decision, the ecological condition of a drainage basin can be improved, and the water utilization efficiency is improved.

The invention proposes a land utilization monitoring method and system based on remote sensing and big data, and relates to the technical field of land monitoring, and the method comprises the steps: dividing sub-regions, and obtaining the multi-temporal and multi-resolution remote sensing data of the sub-regions; performing feature extraction and classification on the remote sensing data based on a deep learning model to generate a land utilization classification map; based on the dual-temporal difference attention network, identifying a change area and constructing a change driving factor library fusing meteorological data and human activity data; detecting an abnormal area based on the driving factor library, and generating an abnormal type label and an attribution analysis report in combination with a dynamic early warning threshold; performing visual rendering on the monitoring result, and outputting an abnormal region early warning map and a disposal suggestion; high-precision feature extraction and classification are realized, change areas and driving factors are deeply analyzed, abnormal areas are effectively detected and early warning is performed, and the accuracy, timeliness and practicability of land utilization monitoring are improved.

The invention discloses a real-time single-stage remote sensing image correction target detection method based on YOLOV8, and relates to the technical field of remote sensing image processing. According to the method, a deformable convolution dynamic prediction local geometric distortion parameter is embedded based on a YOLOv8 backbone network, an adaptive deformation field is generated, pixel-level real-time correction is realized, shallow details and high-level semantic features are fused through a bidirectional path aggregation network, and channel attention and a space gating mechanism are combined, so that the real-time correction of the image is realized. The small target detection capability is enhanced, background noise is suppressed, angle prediction is divided into discrete classification and continuous residual error regression tasks through a decoupling type rotation detection head, angle periodic errors are eliminated in combination with a direction sensitive loss function, and the rotation frame positioning precision is improved. And constructing a dynamic multi-task collaborative loss function, introducing gradient distribution consistency constraint to jointly optimize correction and detection tasks, and realizing feature semantic alignment and model self-enhancement through end-to-end closed-loop training. And the rotating target detection precision and the complex scene robustness are obviously improved.

The invention relates to the technical field of hydrological monitoring, in particular to a multi-satellite collaborative hydrological monitoring system. The method comprises the following steps that a water level height measurement module obtains radar pulse recovery time delay data through a height measurement satellite and calculates the water level height, abnormal points are removed to generate river water level data, a water remote sensing module collects river water images through a remote sensing satellite to extract water boundary features, water area change data are deduced, and the water level height measurement module calculates the water level height. The meteorological wind shear analysis module obtains river channel meteorological data through a meteorological satellite, microwave scattering measurement and wind speed inversion are carried out, wind-induced shear stress parameters are generated, and the flow estimation module carries out section dynamic analysis and estimates the water flow in combination with river channel water level and water area data. And the flow correction module corrects the river water flow based on the wind-induced shear stress parameter and carries out real-time monitoring, and data are synchronously uploaded to the control terminal, so that comprehensive dynamic monitoring of the hydrological state of the river is realized. According to the invention, a more efficient multi-satellite collaborative hydrological monitoring system is realized.

The invention provides a remote sensing coastline automatic extraction method and system based on residual space pyramid segmentation and two-dimensional attention, and relates to the technical field of space analysis. The method comprises the steps of high-resolution remote sensing image acquisition and preprocessing, sea-land segmentation network reasoning, probability graph thresholding and edge extraction and vectorization processing. According to the method, the segmentation precision is improved through multi-scale feature aggregation and attention enhancement, coastline vector data with geographic coordinates are generated in combination with edge detection and topological repair, and the method is suitable for spatial analysis and coastline monitoring.

The invention relates to the field of unmanned aerial vehicle environment perception, in particular to a multi-mode environment perception method and system for a low-altitude medical unmanned aerial vehicle. The method comprises the following steps: collecting a multi-modal data stream, carrying out adaptive data optimization processing, and constructing a multi-modal fusion data set; performing environment multi-level obstacle identification and evaluation on the multi-modal fusion data set to generate a threat mapping environment map; multi-dimensional environment parameters are collected based on the unmanned aerial vehicle, wind field time-varying prediction and safe flight area calculation are performed based on the threat mapping environment map, and a flight area map is constructed; performing multi-position collision risk assessment based on the multi-modal fusion data set to generate collision risk coefficients of different positions; and carrying out safe flight constraint analysis on the flight area map according to the collision risk coefficient, and extracting an optimal flight path. According to the invention, in combination with real-time environment data, comprehensive flight path planning is provided, and the flight safety and task completion efficiency of the unmanned aerial vehicle are improved.

The invention relates to the technical field of soil pollution identification, and discloses a soil heavy metal pollution identification system. A multispectral remote sensing sensing module of the system obtains surface reflectance data and soil in-situ spectral data through a satellite load and a vehicle-mounted mobile platform respectively, and the surface reflectance data and the soil in-situ spectral data are processed by a heterogeneous data fusion gateway to generate multiband spectral response signals. In the pollution risk assessment module, a spatial distribution analysis unit outputs a heavy metal spatial distribution map, a migration risk prediction unit generates a pollution migration probability cloud map in combination with meteorological and hydrological data, and a pollution threshold defining unit outputs a soil remediation safety threshold. In the treatment decision execution module, an in-situ remediation execution unit adjusts passivator injection parameters, a pollution source management and control unit regulates pollution source blocking equipment and collects monitoring signals, and a three-dimensional dynamic early warning platform generates a comprehensive pollution risk index. According to the system, the cooperative operation of soil heavy metal pollution identification, evaluation and treatment is realized.

The invention discloses a forest prevention and control method based on disease and pest monitoring, and belongs to the technical field of forestry prevention and control, and the method comprises the steps: dividing a target forest into regions according to vegetation types, collecting historical multi-source data, filling the missing data, analyzing a time sequence and causal relationship to construct a time sequence causal diagram, and simulating a disease and pest spatio-temporal dynamic state in combination with current data. And determining a prevention and control risk level and a key induction factor of each region, executing a prevention and control strategy according to the level, and updating a causal diagram through reinforcement learning according to forest health response data. According to the method, a closed loop is formed from data processing to model construction, accurate prevention and control are achieved, efficiency and scientificity are improved, forest ecological changes can be dynamically optimized and adapted, and the prevention and control effect is guaranteed.

Processing circuitry may configure a system to implement a CAR-OnMap (“CAROM”) air framework for vehicle localization and traffic scene reconstruction using the aerial video of the traffic scene. Such a system may obtain aerial video of a traffic scene including vehicles that traverse the traffic scene and a satellite map image of the traffic scene as a distinct reference image. In such an example, processing circuitry may determine aerial image reference points within the aerial image which correspond to reference points in the satellite map image of the traffic scene. Processing circuitry may responsively generate calibrated images of the traffic scene from individual frames of the aerial video and determine unique keypoints on the vehicles in the traffic scene. In such an example, processing circuitry may track the vehicles across the individual frames of the aerial video utilizing the unique keypoints. Processing circuitry may output vehicle metrics for the vehicles.

The invention discloses a cascade reservoir optimization regulation and control method based on nitrogen and phosphorus circulation of a water body, and relates to the technical field of water resource management optimization. Multi-dimensional data of a gradient reservoir is collected, and a time-space aligned multi-modal data set is constructed; according to a biogeochemical process of nitrogen and phosphorus circulation, a physical equation of the nitrogen and phosphorus circulation is deduced, and a physical constraint loss item is designed and is subjected to weighted combination with data driving loss to form a mixed loss function. According to the method, the physical constraint neural network model is constructed, and the physical equation of nitrogen and phosphorus circulation is embedded into the neural network loss function, so that the strong fitting capability of the data driving model is utilized, and the model output is ensured to accord with the biogeochemical law through the physical constraint; the key parameters of the physical equation are calibrated through the Bayesian optimization algorithm, the prediction precision of the model on the nitrogen and phosphorus migration and transformation process is further improved, the dependence on big data is effectively reduced, and a reliable prediction result can still be provided especially under the condition that the temporal-spatial resolution of data is low.

The invention discloses a natural resource dynamic monitoring system and method based on a multi-source remote sensing image. The system comprises a multi-source data acquisition module, a preprocessing module, a feature extraction module, a dynamic monitoring module, a visual output module and a distributed storage module. According to the method, the precision and timeliness of natural resource change detection are remarkably improved by constructing a multi-source remote sensing image intelligent fusion framework. The system adopts a parallelization preprocessing assembly line to effectively eliminate radiation and geometric differences among multi-source data; a feature fusion module based on deep learning automatically extracts multi-scale change features, and overcomes the environmental adaptability defect of a traditional threshold method; and the dynamic monitoring module realizes automatic extraction and classification of sub-meter change pattern spots, and the monitoring efficiency is improved. Meanwhile, the system supports collaborative analysis of multi-platform data of satellites, aviation, unmanned aerial vehicles and the like, the cloud shielding influence is greatly reduced, and total-factor and high-frequency decision support is provided for natural resource supervision.

The invention discloses a multi-modal remote sensing semantic segmentation method and system for learning frequency domain fusion. The method comprises the following steps: respectively extracting multi-scale features of two modal input images by adopting a double-branch encoder; sequentially executing frequency domain decoupling and fusion, mutual information constraint-based feature optimization and low-frequency guided cross-modal fusion processing on each scale feature to generate a fused semantic feature; and performing up-sampling and feature refining on the fused features through a decoder, and outputting a full-resolution segmentation prediction map. According to the multi-modal remote sensing image semantic segmentation method, modal sharing information and specific details are effectively separated through frequency domain decoupling, feature representation is optimized through mutual information constraint, adaptive feature fusion is achieved in combination with an attention mechanism, and the accuracy and robustness of multi-modal remote sensing image semantic segmentation are remarkably improved.

The invention discloses a battlefield target behavior prediction method capable of being guided by micro-physical representation and a thinking chain, and the method comprises the steps: obtaining satellite images, radar / communication detection and open source text multi-source time sequence data, completing the entity recognition, relation extraction and event detection, and constructing a dynamic space-time knowledge graph; the maneuverability, sensor detection, weapon range and terrain accessibility mechanism are micronized to serve as a physical consistency constraint embedded prediction model; generating an intention-action-result causal priori chain and parameterizing the causal priori chain into a computable structure; performing multi-branch long-time-sequence situation deduction, and outputting a future target behavior track and a scene probability; and evaluating and explaining by integrating the causal confidence coefficient, the physical consistency and the data goodness of fit, and giving a key event probability and situation evolution report. According to the method, unified modeling of semantic causal and physical constraints is realized, and the method has explainable, verifiable and robust prediction capabilities, and is suitable for target behavior prediction and command information system decision support in a complex environment.

The invention provides a natural resource analysis method and system combined with multi-source data, and the method comprises the steps: firstly obtaining a multi-source data set of remote sensing images, geographical monitoring, environment monitoring and the like of a target region, and then carrying out the spatial-temporal feature extraction processing of the multi-source data set, thereby obtaining a resource spatial distribution feature set and a time change feature set; according to the method, features of resources under different time-space dimensions are accurately described, then, based on a preset dynamic association rule set, dynamic association processing is performed on a resource space distribution feature set and a resource time change feature set, a resource dynamic association relationship set is generated, and interaction and evolution rules among the resources are revealed. According to the resource dynamic association relationship set, a resource state evaluation strategy is generated, a resource management optimization direction is determined, finally, the resource management optimization direction is fed back to a resource management system, resource scheduling operation is triggered, scientific, accurate and dynamic management of natural resources is achieved, and resource management efficiency and sustainability are improved.

The invention discloses a plateau mountain road disaster identification method and system based on multi-source remote sensing image restoration and super-resolution reconstruction. The method comprises the following steps: acquiring and preprocessing a multi-source remote sensing image of a plateau mountain region, and extracting landform measurement parameters based on a digital elevation model; a super-resolution reconstruction network fusing deformable convolution and Transform is constructed, and a low-resolution image is reconstructed by using constraint training of a composite loss function containing geomorphic measurement parameters; performing feature extraction and adaptive weighted fusion on the preprocessed image and the reconstructed high-resolution image; based on the fused image, utilizing a multi-task deep learning model to identify landslide, debris flow and roadbed subsidence disasters along the highway; and carrying out morphological optimization and boundary refinement under GIS constraint on an identification result, and outputting a disaster thematic map. According to the invention, the precision and reliability of road disaster identification in a complex terrain environment are effectively improved.

The invention provides a multi-fusion seaweed field ecosystem observation method, system, device and medium, and belongs to the technical field of ecological monitoring, the method comprises the following steps: obtaining chlorophyll a concentration, seaweed canopy spectrum and three-dimensional biomass point cloud; aligning the chlorophyll a concentration of the target sea area with the seaweed canopy spectrum, and fusing the three-dimensional biomass point cloud to generate a three-dimensional biomass model; constructing an in-situ sampling network to monitor water quality parameters, benthic organism video streams and eDNA metagenome sequencing data, calibrating a three-dimensional biomass model, executing anomaly detection through a lightweight LSTM model, and identifying benthic organism species in real time through an improved YOLOv5 model; constructing a graph neural network, outputting a carbon sink prediction value, generating a brown tide early warning signal when the carbon sink prediction value is lower than a dynamic threshold value, optimizing a patrol path of the unmanned aerial vehicle based on reinforcement learning, and improving the sampling frequency of the water quality sensor. According to the invention, multi-fusion monitoring of the seaweed field is realized, the ecological condition is accurately evaluated, and abnormity is warned in advance.

The invention relates to the field of rail detection, in particular to a rail transit vehicle obstacle detection method based on multi-modal data fusion, which comprises an environmental perception classification step, a credibility calculation and mode decision step, a detection data acquisition step and an obstacle detection step. A full-scene coverage detection system is constructed by integrating a visual image sensor, a millimeter-wave radar sensor and an ultrasonic radar sensor and utilizing complementary advantages of the three sensors in different environments. When a certain sensor is interfered by the environment, other sensors can still provide effective data support, a detection blind area caused by failure of a single mode is avoided, and the robustness and the detection accuracy of obstacle detection are improved.

The invention discloses an end-to-end automatic driving method based on dynamic multi-modal fusion in a complex scene, and belongs to the technical field of automatic driving. In order to solve the problems of sensor perception deficiency, cross-modal feature mismatching, unstable trajectory planning and the like easily occurring in night, low-illumination and complex dynamic environments in the existing end-to-end automatic driving method, texture details of a camera mode and geometric structure features of a laser radar mode are respectively enhanced through a double-flow feature refining mechanism; the characteristic difference between different modes is relieved; an information-driven dynamic fusion strategy is designed, the fusion weight is adaptively adjusted according to scene factors such as environment illumination and obstacle density, and the scene sensitivity and discrimination ability of the model are improved; asymmetric convolution and a low-rank-sparse decoupling technology are introduced, multi-order reconstruction of key channels is carried out on the multi-modal features, and the path change modeling capability is enhanced; and in combination with time sequence dependence of waypoints, outputting a future trajectory through an autoregression decoder to realize high-precision trajectory prediction and stable decision control.