4575 results about "Wave radar" patented technology

The invention provides a home abnormal state signal detection method and system based on multi-modal sensing, and relates to the technical field of detection, and the method comprises the steps: collecting human body motion, acoustics and environment parameters through a millimeter wave radar, an acoustic sensor and an environment sensor, inputting the parameters into a deep fusion network, a spatio-temporal attention mechanism and a multi-scale convolutional neural network are used to extract a spatio-temporal feature sequence, and joint probability features are formed in combination with an acoustic feature spectrum analysis result. And inputting the joint probability features into a time sequence knowledge graph, and obtaining scene adaptive features after processing by a graph attention network and a double-flow auto-encoder network. And finally, the multi-task learning network is combined with the risk propagation neural network and the recursive neural network to carry out anomaly prediction and risk level evaluation, and is combined with historical early warning information to output an early warning strategy. According to the invention, multi-modal data can be effectively fused, the accuracy and reliability of home abnormal state detection are improved, and a more accurate risk assessment and early warning strategy is provided.

Disclosed are a high-performance loosely-coupled multi-modal data fusion system for a smart driving environmental perception system and a vehicle-mounted device, comprising: a fusion detection model based on a modality-independent feature interaction strategy, which is configured for converting a LiDAR point cloud, a camera image, and a millimeter-wave radar point cloud into a unified bird's-eye view representation, and performing multi-modal fusion; and a fusion tracking model based on a motion-appearance feature cascaded coupling data association strategy, which is configured for performing subsequent trajectory tracking and matching according to multi-modal fusion feature information. A VoD data set and a K-Radar data set are selected for training, verifying, and testing the comprehensive performance of the models, and a TensorRT accelerated inference model is applied, then quantized, and deployed to a vehicle-mounted computational testing platform. The present invention is compatible with mainstream sensor deployment solutions, and achieves the efficient complementary fusion of multi-source heterogeneous sensor information, significantly improving the reliability, accuracy, and adaptability of vehicle-mounted perception systems, thereby effectively responding to extreme operating conditions such as complex traffic scenarios and inclement weather.

The invention belongs to the technical field of path planning, and discloses an unmanned driving dynamic path planning method and system based on multi-source data fusion, and the method comprises the steps: collecting an ice and snow pavement friction coefficient, a curve curvature and an obstacle point cloud, constructing a sensor confidence coefficient matrix, generating a fused semantic map, and constructing a dynamic environment semantic model. Outputting a real-time friction coefficient field and a risk thermodynamic map layer; the roadside unit broadcasts coordinates of opposite vehicles in a blind area of a curve to a vehicle end, constructs an ice and snow pavement offset crowdsourcing map, and generates a global-local fusion topology; fusing the real-time friction coefficient field and the global-local fusion topology to generate a smooth trajectory set, and further generating a risk optimal path instruction set; a steering angle and torque instruction is decomposed, positioning drift is compensated in real time, and a normal mode for updating the vehicle positioning state and a degradation mode when the millimeter wave radar fails are constructed; and generating execution logs and health state vectors, and aggregating the execution logs and the health state vectors of multiple vehicles to form closed-loop iterative update.

The invention relates to the technical field of unmanned aerial vehicle control, in particular to an unmanned aerial vehicle autonomous obstacle avoidance decision-making method and system based on multi-source sensor fusion, and the method comprises the steps: achieving the time-space synchronization of a laser radar, a visual camera and a millimeter-wave radar through timestamp alignment and coordinate mapping, and constructing a dynamic obstacle grid map; fusing multi-source data based on a dynamic Bayesian network, and dynamically adjusting the confidence coefficient weight of the sensor in combination with the environment illumination intensity and the barrier surface material; and adopting a reinforcement learning model to generate an incremental obstacle avoidance strategy, and triggering a grading response instruction according to the risk assessment grade. The problem of fusion errors caused by spatial-temporal asynchronization of multi-source sensor data is solved, and the real-time obstacle avoidance success rate of dynamic obstacles is increased.

The invention discloses an intelligent driving multi-sensor fusion data processing system, which belongs to the technical field of fusion data processing and comprises a data preprocessing unit, a data fusion unit and a data post-processing unit. The data preprocessing unit normalizes the original data of the laser radar, the camera, the millimeter wave radar and the ultrasonic sensor through a standardization, cleaning, noise reduction and calibration synchronization module; the feature level fusion module deeply digs the characteristics of each sensor, and fuses geometric, visual, motion and close-range obstacle features by using a deep neural network; the decision-making level fusion module generates a driving instruction through weighted voting and fuzzy logic reasoning through classification and situation evaluation; and the data post-processing unit combines the vehicle and road condition optimization instruction, evaluates the risk, and stores the data feedback optimization system. The system can improve the sensing precision and decision reliability, enhances the expansion adaptability of the system, and provides guarantee for safe and efficient operation of intelligent driving.

The invention relates to the technical field of rail traffic safety detection, in particular to a railway freight train key part online monitoring system based on an unmanned aerial vehicle, which comprises an unmanned aerial vehicle control module, a multi-modal data acquisition module, an edge calculation module, a central processing module and a feedback execution module. A dynamic three-dimensional grid flight path is generated by fusing a train Beidou positioning signal and a millimeter wave radar sensing result, a system is provided with an infrared thermal imager, a laser radar and a high-speed polarization camera, bearing temperature, train body point cloud and a train coupler image sequence are obtained, and temperature rise area identification, structural deformation extraction and coupling state modeling are completed on the edge side. The central processing module outputs a multi-dimensional safety assessment result, and the feedback module generates a compensation control instruction and a graded early warning signal based on the risk fusion index. The method has the advantages of being high in autonomy degree, high in recognition precision and high in response speed, and is suitable for full-time structural intelligent inspection and early warning of the freight train in a high-speed operation environment.

The invention relates to the technical field of unmanned aerial vehicle flight path planning, in particular to an unmanned aerial vehicle real-time path planning system and method based on dynamic weight distribution and multi-source data fusion. The system comprises a multi-source data fusion module, an integrated laser radar, a millimeter wave radar, a visual sensor and a Beidou positioning unit. The dynamic weight distribution module dynamically adjusts the weight coefficient of each sensor according to the environmental complexity, the threat level and the state of the unmanned aerial vehicle by adopting a mixed decision-making mechanism combining fuzzy logic and reinforcement learning; an improved RRT * algorithm and a Markov decision process are built in the real-time path planning module, and a global optimal path and a local obstacle avoidance track are generated by adopting a layered planning architecture; the unmanned aerial vehicle cooperative control module comprises a dual-redundancy flight control system and a dynamic obstacle avoidance unit; and the communication relay module supports 5G and low-orbit satellite dual-mode communication, updates an environment cognitive model of each unmanned aerial vehicle through federated learning, and realizes multi-source fusion real-time path planning based on dynamic weight distribution and the unmanned aerial vehicles.

The invention relates to the field of intelligent driving, in particular to a driver dangerous behavior intervention system and method based on a space-time diagram neural network, and the method comprises the steps: obtaining a physiological signal, a driving posture and control behavior data through a multi-modal data collection module; constructing a dynamic graph structure and evaluating a driver risk state by using a space-time diagram neural network risk perception engine; the intelligent intervention execution module adopts a tactile, visual or vehicle control intervention strategy according to the evaluation result; the closed-loop optimization module monitors an intervention effect and updates a risk model and a strategy; according to the system, millimeter wave radar and multi-channel visual analysis are fused, the recognition accuracy is improved to 98.6%, dangerous events are predicted 15-30 seconds in advance through physiological signals and micro-expression analysis, and sufficient response time is provided for a driver.

The invention provides a multi-target tracking and interference cooperation method and system based on a millimeter wave radar. The method comprises the following steps: firstly, acquiring an original detection signal in real time by using a millimeter-wave radar carried by an unmanned aerial vehicle, establishing a track correlation sequence of each moving target, extracting interference waveform characteristics formed by signal coupling among a plurality of moving targets in the original detection signal, and then, according to a spatial distribution parameter of the interference waveform characteristics, determining a track correlation sequence of each moving target; correcting the space-time continuity judgment rule of the trajectory correlation sequence, and finally outputting an anti-interference cooperative tracking result based on the corrected space-time continuity judgment rule. According to the technical scheme provided by the invention, the industry pain point that waveform distortion interference of the millimeter wave radar in a dense target scene cannot be eradicated is solved, and the error correlation rate in a dense formation scene is also reduced.

The invention relates to the technical field of robot autonomous navigation and control, and discloses a robot system based on a sensing system, which comprises a multi-mode sensing module for outputting data to a dynamic environment modeling module; the dynamic environment modeling module is used for transmitting the model to the path planning module; the path planning module outputs the path sequence to the neural motion control module; the neural motion control module outputs the control signal to the dynamic execution module; the dynamic execution module is used for receiving the pulse control signal and driving each joint of the robot to execute the path in combination with inverse kinematics solution and a feedback control mechanism; and the collaborative optimization module is in two-way communication with the modeling module, the path planning module and the neural motion control module. According to the method, the contradiction between real-time performance and integrity of environment modeling in a complex scene is solved by fusing cross-modal information complementary characteristics of binocular vision, millimeter-wave radar and inertial data and combining mathematical representation of a hypergraph topology model on a dynamic obstacle interaction relationship.

The invention provides an intelligent water conservancy digital twinborn simulation system based on multi-source data, and belongs to the technical field of digital monitoring. Minute-level data acquisition and transmission are realized by constructing a space-air-ground three-dimensional sensing network and combining edge intelligent preprocessing, data acquisition, noise reduction and abnormity identification are realized by utilizing sensors such as millimeter wave radar and laser radar, and the system is used for realizing data acquisition and transmission. Dynamic data fusion and intelligent calibration are carried out, and second-level alignment and credible verification of data are realized by means of a space-time calibration algorithm, Kalman filtering and a block chain evidence storage technology; a hybrid simulation and intelligent decision model is established, a physical model, a machine learning architecture and a dynamic threshold decision tree are adopted, flood routing minute-level prediction and emergency response are realized, and the system improves water conservancy monitoring prediction precision and emergency response efficiency.

The invention belongs to the technical field of intelligent perception and mode recognition, and particularly relates to a human body behavior recognition method based on millimeter wave radar and multi-task learning. The method comprises the steps that millimeter wave radar point cloud data and RGB video streams are synchronously collected, and a five-dimensional point cloud scene is generated through three-dimensional analysis and dynamic target extraction; a hierarchical Point Transform network is constructed to extract radar space-time features, and human body key point features are generated by using visual auxiliary attitude estimation; radar and key point features are dynamically fused through a cross-modal attention mechanism, a multi-task joint optimization strategy is combined, behavior classification serves as a main task, attitude estimation serves as an auxiliary task, and a behavior recognition result is output. According to the method, through multi-task cooperation and cross-modal feature interaction, on the premise of ensuring privacy security, the accuracy and robustness of human behavior recognition in a complex scene are remarkably improved, and efficient and reliable technical support is provided for the fields of intelligent monitoring, human-computer interaction and the like.

The invention discloses an unmanned aerial vehicle flight control system and method with precise positioning and autonomous obstacle avoidance, and particularly relates to the field of control, and the system comprises a positioning module, an obstacle detection module, a data processing and analysis module, an autonomous obstacle avoidance algorithm module, and a flight control module. Static and dynamic obstacles are detected in real time by integrating the laser radar, the millimeter wave radar, the visual sensor and the ultrasonic sensor, and an obstacle information report is generated through a data fusion algorithm based on machine learning; based on the processed environment data, the system uses deep reinforcement learning and a model prediction control algorithm to generate a safe flight path in real time, and an obstacle avoidance decision is optimized; the flight control part adopts a hierarchical control framework, and precise flight of the unmanned aerial vehicle is realized through trajectory tracking, attitude control and actuator control.

The present invention relates to the technical field of environmental perception in confined spaces, and discloses a multi-modal coupled perception method for target recognition and region segmentation in a confined space. The method comprises: firstly, constructing a 3D point cloud data processing network, a 2D point cloud data processing network, and an image data processing network; next, constructing a feature fusion device, and outputting a bird's-eye view incorporating multi-sensor information; then constructing a multi-scale feature fusion extraction network and a network output head, outputting a target recognition and region segmentation result, and designing a loss function to train a network weight; and finally, inputting 3D point cloud data, 2D point cloud data, and camera image data into a network model, performing inference to obtain a target recognition and region segmentation prediction result, and performing visual rendering on the prediction result. In the present invention, in view of the characteristics of a millimeter-wave radar, a multi-modal coupled perception network is constructed, so that effective information in mass point cloud data of the millimeter-wave radar can be deeply mined, thereby effectively improving the detection accuracy of target recognition.

The invention discloses a millimeter wave radar human body tumble detection method based on multi-feature fusion, and the method comprises the steps: obtaining an original point cloud frame from each radar, and carrying out the timestamp calibration and space coordinate system conversion; performing noise filtering, ground segmentation and human body point cloud extraction on each frame of point cloud; dividing the preprocessed point cloud into a static cluster and a dynamic cluster; filtering false dynamic clusters; extracting a residual dynamic cluster set, and identifying and tracking the human body dynamic clusters in continuous frames by adopting a tracking algorithm; extracting a corrected time sequence feature from the tracked human body cluster; inputting the time sequence characteristics into a pre-trained deep time sequence network, learning a falling time sequence dependency relationship, and outputting an abnormal index; and performing multi-source fusion with the abnormal index to obtain a comprehensive index to judge whether to trigger an alarm. The method can adapt to a complex home environment, improves the detection accuracy and real-time performance, and reduces the false alarm and missing alarm.

The invention provides a vehicle surrounding obstacle detection method and system based on a multi-modal sensor, and relates to the technical field of intelligent driving, and the method comprises the steps: obtaining ultrasonic radar point clouds and millimeter wave radar point clouds at two sides of a vehicle, and image data at the rear side of the vehicle; the radar point clouds are clustered, and the spatial position, the movement speed and the reflection section characteristics of an obstacle are extracted; performing target detection on the image data, and outputting to obtain a bounding box and texture features of an obstacle; generating preliminary multi-modal data, inputting the multi-modal data into a collision detection model, respectively entering a radar branch network and a visual branch network to extract point cloud spatial-temporal features and image semantic features, and performing weighted fusion on the point cloud spatial-temporal features and the image semantic features through a Transform mechanism to obtain a collision detection result; outputting the three-dimensional position, the movement track, the type label and the collision risk probability of the obstacle; and when the collision risk probability exceeds a preset threshold value, triggering a corresponding early warning or control instruction.

The invention relates to the field of heart rate monitoring, and discloses a millimeter wave radar breath and heart rate synchronous monitoring method and system, and the method comprises the steps: transmitting a linear frequency modulation continuous wave signal according to a millimeter wave radar, and collecting original echo data reflected by a target region; baseband signal demodulation and phase information extraction are carried out on the original echo data to obtain original phase time sequence data, and the original phase time sequence data comprise thoracic cavity micro-motion features; and according to a Butterworth band-pass filter, preprocessing the original phase time sequence data through human body physiological signal frequency band characteristics to obtain a breathing frequency band signal and a heart rate frequency band signal. According to the method, the apnea event triggering threshold value and the arrhythmia early warning index are updated in real time through Kalman filtering and extended Kalman filtering, so that the monitoring system can dynamically adjust the health parameters, which means that the monitoring system can be optimized in real time and the abnormal health event can be accurately responded in different physiological states.

The invention discloses an expressway tunnel monitoring method, system and device and a medium, and relates to the technical field of tunnel monitoring. Tunnel environment data are collected in real time through a multi-source heterogeneous sensor array, and the data at least comprise video images, millimeter wave radar point cloud, laser radar three-dimensional coordinates, temperature and humidity and CO concentration parameters. According to the system, an improved YOLOv7 algorithm is used for carrying out target real-time detection, an optical flow method is combined to predict the motion trail of a target object, a tunnel dynamic characteristic spectrum is constructed, real-time monitoring of the tunnel environment is achieved, by deploying a risk level assessment engine, the system can extract space-time correlation characteristics, and the risk level assessment efficiency is improved. A comprehensive risk index is generated in combination with a fuzzy logic decision tree, and when the risk index exceeds a dynamic threshold value, the system triggers a grading early warning mechanism, links tunnel emergency equipment, synchronously generates an emergency plan and pushes the emergency plan to an operation and maintenance terminal to guarantee tunnel operation safety.

The present application discloses a multifunctional health monitoring method and system based on a millimeter wave radar, and a computer readable storage medium. The method comprises: receiving an echo signal in a space, and on the basis of the echo signal, calculating point cloud data of a user and phase change data caused by the respiration and heartbeat of the user; calculating behaviors of the user on the basis of the point cloud data; calculating vital signs of the user on the basis of the phase change data, wherein the vital signs comprise respiration rate and heart rate; and on the basis of the behaviors and / or the vital signs, determining that the user triggers a set alarm event, and on the basis of the type of the triggered alarm event, performing a corresponding alarm operation.

The invention belongs to the technical field of intelligent driving, and particularly relates to an intelligent driving method and system with a body, and the method comprises the steps: fusing the V2X data of a vehicle-mounted laser radar, a millimeter wave radar and a road side unit, and obtaining the vehicle driving information; constructing a centimeter-level dynamic environment model by adopting a space-time alignment algorithm, and realizing multi-source data space-time synchronization and obstacle real-time prediction; designing a hierarchical reward function including security, efficiency and comfort rewards based on a user risk cognition mechanism, and generating an end-to-end reinforcement learning decision in combination with a Transform architecture; optimizing the dynamic environment model through reinforcement learning decision; inputting vehicle driving information into the optimized centimeter-level dynamic environment model, and predicting obstacles and vehicle tracks in a vehicle driving path; an MPC-Hybrid feedforward-feedback control system is constructed, a feedforward module is adopted to calculate a front wheel turning angle and acceleration parameters according to obstacles in a vehicle driving path and a vehicle track, and a feedback module is adopted to optimize transverse and longitudinal errors; controlling the driving direction of the vehicle according to the front wheel turning angle and the acceleration parameter transverse and longitudinal errors; according to the invention, through deep fusion of vehicle-road collaborative perception and reinforcement learning decision, the reliability and adaptability of the automatic driving system are significantly improved.

The invention relates to the field of rail detection, in particular to a rail transit vehicle obstacle detection method based on multi-modal data fusion, which comprises an environmental perception classification step, a credibility calculation and mode decision step, a detection data acquisition step and an obstacle detection step. A full-scene coverage detection system is constructed by integrating a visual image sensor, a millimeter-wave radar sensor and an ultrasonic radar sensor and utilizing complementary advantages of the three sensors in different environments. When a certain sensor is interfered by the environment, other sensors can still provide effective data support, a detection blind area caused by failure of a single mode is avoided, and the robustness and the detection accuracy of obstacle detection are improved.

The invention discloses a vehicle-mounted monitoring system based on deep learning, particularly relates to the technical field of vehicle-mounted multi-mode cooperative sensing, and is used for solving the problems of space-time progressive deviation accumulation and high-speed scene target state estimation distortion caused by hardware synchronization limitation in existing vision and millimeter wave radar fusion. According to the method, high-precision alignment and self-adaptive correction of multi-modal data are realized through dynamic space-time baseline compensation and a closed-loop feedback optimization mechanism; a dynamic compensation coefficient is generated based on sensor space-time distribution mode prediction, and a nonlinear correlation model of a multi-modal data stream is constructed to eliminate sampling frequency difference and transmission fluctuation interference; visual motion artifacts and radar positioning jump noise are separated through a cross-modal feature decoupling technology, and high-confidence feature fragments are extracted in combination with historical trajectory screening and real-time residual analysis; and a closed-loop architecture fusing weight distribution and compensation parameter reverse correction is established, and dynamic calibration and error traceability of multi-source data collaborative sensing are realized.

The invention relates to the field of unmanned aerial vehicle monitoring, in particular to a low-altitude unmanned aerial vehicle flight monitoring method and system. A low-altitude unmanned aerial vehicle flight monitoring system comprises an obstacle sensing module, a confidence coefficient calculation module, a risk assessment module, an obstacle avoidance distance adjustment module and a flight mode switching module. According to the invention, by quantifying the rainfall interference index, the visibility index, the illumination intensity index, the obstacle complex index and the electromagnetic interference index, the confidence coefficient weight of the millimeter wave radar, the visual sensor and other multi-element sensing equipment is calculated in real time, so that the limitation of traditional fixed priority fusion is broken through; the optimal sensor data can be automatically selected as an obstacle avoidance decision basis according to actual environmental conditions, misjudgment or delayed response caused by sensor conflicts is avoided, the obstacle avoidance decision precision is improved, and the method is particularly suitable for high-reliability flight in complex environments such as urban canyons.

The invention discloses a multi-dimensional anti-bird intelligent recognition method and system based on thermal imaging, and relates to the technical field of intelligent monitoring and ecological protection. Multi-modal data is collected through thermal imaging, visible light, millimeter wave radar and a voiceprint sensor, after PTP protocol synchronization and Kalman filtering preprocessing, 3-5-second tracks of birds are predicted by using an LSTM network, and the anti-bird intelligent recognition method and system based on the thermal imaging are obtained. And the cross-modal features are fused through a Transform architecture, so that 95% of classification accuracy is realized. A bird repelling strategy is generated in real time through edge calculation, and the model is updated through cloud federal learning. The system integrates an oil-electric hybrid unmanned aerial vehicle and a ground device, supports dynamic path planning based on a thermodynamic diagram and differentiated repelling of directional sound waves, laser stroboflash and the like, has the night recognition accuracy rate of 92% and the bird repelling response time of 0.8 second, and is suitable for scenes of electric power, airports and the like. Through multi-dimensional perception, dynamic modeling and eco-friendly expelling, the problems of poor environmental adaptability, single strategy and the like of a traditional scheme are solved, and the anti-bird efficiency and the ecological safety are improved.

The invention provides a lightweight AI personnel perception method and system based on a millimeter wave radar, and relates to the technical field of personnel perception, and the method comprises the steps: carrying out the phase compensation, amplitude calibration and bidirectional combined filtering processing of a millimeter wave echo signal, and constructing three-dimensional space point cloud data. Then, constructing a spatial distribution matrix based on the point cloud data, calculating the center position of a point cloud cluster, establishing an adaptive sampling radius to extract point cloud features, and calculating a motion vector and attitude parameters; and finally, inputting the motion vector and the attitude parameter into a double threshold judgment module, judging whether a person exists in the target area, generating a person position thermodynamic diagram based on the center position of the point cloud cluster, and outputting a person perception result. According to the method, the calculation complexity is reduced, the real-time performance and accuracy of personnel perception are improved, and the method is suitable for smart home, security monitoring and other scenes.

The invention discloses a multi-mode perception and reinforcement learning engineering machinery intelligent regulation and control method and system, and belongs to the technical field of intelligent control and industrial automation. According to the system, stratum data within the range of 20-50 meters in front of a shield tunneling machine are collected in real time through multi-mode sensing equipment such as a distributed optical fiber sensor, a cutterhead vibration sensor and an electromagnetic wave radar, and original data are processed by adopting a wavelet-Fourier combined noise reduction algorithm; and inputting the processed multi-modal data into a reinforcement learning intelligent decision-making model based on a CNN-LSTM hybrid network architecture. The model is trained through a dynamic reward function, and weight coefficient combinations can be automatically switched according to different construction scenes. According to the system, parameters such as the rotating speed, the thrust and the grouting amount of a shield cutter head are regulated and controlled in real time through the self-adaptive control module, and the slurry utilization rate is increased through the gradient pulse grouting technology. The precision, efficiency and safety of shield construction are remarkably improved, and meanwhile energy consumption and construction cost are reduced.

The invention discloses a collaborative unmanned aerial vehicle cluster path planning and scheduling system, and particularly relates to the technical field of unmanned aerial vehicle intelligent control, and the system comprises a multi-mode sensing unit which is composed of a heterogeneous sensor array composed of LiDAR, binocular vision and millimeter wave radar, and an output dynamically updated three-dimensional Gaussian mixture map; the decision control unit is used for implementing double-layer optimization of mixed integer programming task allocation and artificial potential field path planning; the dynamic communication network adopts a hybrid networking protocol of TDMA backbone nodes and 802.11 ax terminal nodes; an energy management module; aiming at the insufficient environment perception and dynamic modeling capability in the prior art, the method achieves the effects that the centimeter-level positioning precision and the dynamic obstacle recognition rate are greater than 92%, the environment model is delayed and compressed to be within 200ms, the response speed is increased by 5 times by setting multi-modal sensor fusion, constructing a dynamically updated 3D Gaussian mixture map and combining an LSTM network to predict the obstacle trajectory in real time, and the dynamic obstacle recognition rate is greater than 92%. And the obstacle avoidance reliability in a complex scene is obviously enhanced.

The invention relates to the technical field of image processing, and discloses a multispectral fusion night low-illumination image enhancement and shielding compensation method, which comprises the following steps of: cooperatively acquiring multi-modal data through a visible light camera, an infrared sensor, a thermal imaging sensor and a millimeter wave radar; comprising a low-illumination basic image, dark light texture details, target temperature distribution and contour and motion information of an object behind the shelter; fusing visible light and infrared textures by adopting a multi-scale transformation algorithm to generate a transition fusion image, and performing illumination compensation based on temperature distribution; predicting texture details of the occlusion area through a u-Het deep learning network in combination with detection data of the millimeter wave radar; and filling the missing texture according to the shielding contour, and generating a final target image through edge optimization and illumination smoothing technologies. According to the invention, the definition of a night low-illumination image can be improved.

The invention discloses an unmanned aerial vehicle autonomous obstacle avoidance system and method based on millimeter wave radar and multi-mode vision fusion, and the system comprises a millimeter wave radar module which is used for obtaining the distance, speed and point cloud data of an obstacle; the multispectral vision module comprises an RGB camera and an infrared camera and is used for extracting the texture, the category and the thermal characteristics of the obstacle; the embedded AI calculation unit is used for real-time data processing and fusion; the time-space synchronization module is used for aligning timestamps of radar and visual data through hardware trigger signals and unifying space coordinates based on joint calibration and an SLAM (Simultaneous Localization and Mapping) technology; and the feature level fusion module is used for fusing the radar point cloud features and the visual semantic features by adopting a lightweight network. Aiming at single sensor defects, complex environment challenges and hardware computing power limitation, a more efficient unmanned aerial vehicle autonomous obstacle avoidance system is constructed through multi-modal hardware fusion, lightweight feature processing, a dynamic adaptive mechanism and intelligent algorithm optimization, and all-weather and full-scene autonomous operation requirements are met.

The invention relates to the technical field of radar sensing, and discloses a sensing method and system based on millimeter radar waves. The method comprises the steps that millimeter wave radar receiving signals are subjected to low-noise amplification, frequency bands are screened through a band-pass filter and then mixed with local oscillation signals to obtain distance and speed information, after analog-to-digital conversion digitization, features are extracted to distinguish a human body target in a closed / open space, and finally multi-scene adaptability analysis is executed to generate human body position state information. According to the invention, through construction of a signal processing link, from signal amplification, filtering, frequency mixing and digital processing to target identification and multi-scene adaptability analysis, comprehensive perception of a human body target is realized, a static human body and a non-human body interference source can be effectively distinguished, a processing strategy is automatically adjusted according to different space environment characteristics, and the processing efficiency is improved. And the accuracy and reliability of human body perception are greatly improved.