483 results about "Motion prediction" patented technology

The invention provides an anti-unmanned aerial vehicle intelligent identification and tracking system based on multi-source data fusion, and relates to the technical field of anti-unmanned aerial vehicle detection, and the system comprises a multi-source data preprocessing module which is used for outputting preprocessed multi-source data; the target detection module is used for carrying out unmanned aerial vehicle target detection on visual data in the preprocessed multi-source data and outputting a detection result containing a bounding box position, confidence and morphological characteristics; the target tracking module is used for performing unmanned aerial vehicle target tracking based on the target detection result and outputting a tracking result; and the fusion decision module is used for confirming the target identity based on the tracking result and the preprocessed multi-source data and outputting a final recognition result. The technical problems of low detection precision of small targets, difficulty in distinguishing similar targets, inaccurate 3D motion prediction, difficulty in re-identification after long-time shielding and the like in the prior art can be solved, and accurate identification, stable tracking and intelligent decision making of the unmanned aerial vehicle target are realized.

The invention relates to the technical field of safety monitoring, and particularly discloses an operation safety distance management method based on laser radar point cloud data, and the method comprises the steps: generating a dangerous point cloud set based on an initial three-dimensional point cloud model of a complete field environment; performing coordinate transformation on the point cloud data currently collected by the three-dimensional laser radar equipment based on the rigid body transformation matrix to obtain current point cloud data; registering the current point cloud data with the initial three-dimensional point cloud model and identifying all independent dynamic targets; determining a danger approaching trend of each independent dynamic target based on the motion prediction trajectory of each independent dynamic target; determining a dynamic safety distance threshold value of each dangerous dynamic target based on the danger approaching trend and the response delay of each independent dynamic target and the additional safety distance of the corresponding dangerous source area; triggering a corresponding alarm mechanism based on the dynamic safety distance threshold and the distance between the point cloud set of each independent dynamic target and the dangerous point cloud set; and the overall safety of a working site is improved.

The invention discloses an unmanned aerial vehicle intrusion real-time alarm and countering guidance system based on RID information. The system comprises a multi-source information acquisition and fusion module, an intrusion detection and threat evaluation module, a real-time alarm and countering strategy generation module, a feedback evaluation and strategy iteration module, a signal loss prediction and compensation module, a comprehensive processing terminal module and the like. The system establishes a three-dimensional geofence model by analyzing remote identification information of the unmanned aerial vehicle and fusing radar, photoelectric, electromagnetic and other multi-source data, and performs dynamic judgment and multi-factor threat grading on a target intrusion state. Multi-mode alarm is automatically triggered according to the threat level, and a hierarchical countering scheme from broadcast alarm and electromagnetic interference to physical capture is generated; and when the RID signal is lost, the system realizes continuous tracking and state recovery of the target by using a motion prediction and probability association algorithm.

The invention relates to the technical field of computer video processing, and discloses a low-delay video stream real-time processing method and device, and the method comprises the steps: obtaining original video stream data, and processing the original video stream data through employing a lightweight motion prediction method; processing the macro block data set and the predicted coding configuration parameter by adopting multi-thread assembly line coding to obtain a coded data block; establishing a data transmission mechanism to perform data flow control on the unified memory access interface; a heterogeneous task scheduling strategy is adopted to distribute task division results; a lightweight neural network is adopted to carry out parameter adaptive adjustment, and an optimized video stream processing result is obtained; according to the method, a zero-copy data transmission technology is adopted, and optimal configuration and efficient utilization of computing resources are achieved.

The invention discloses a target detection method and device based on an aerial view, equipment, a medium and a product. The method comprises the following steps: extracting multi-scale features for an input image of a multi-view-angle surround-view camera; converting the point cloud data of the laser radar into a sparse voxel structure and generating BEV features for voxel features; fusing the multi-scale features and the BEV features to obtain fused BEV features; road topology and semantic information are extracted according to the fused BEV features, and a map is generated; and determining a three-dimensional bounding box and a motion state of the traffic participant according to the fused BEV features and a prediction result of the position of the traffic participant in the map. According to the technical scheme, multi-scale geometric features, BEV features and semantic information are fused, time sequence features of dynamic target detection and static map segmentation are synergetic, the reliability of target detection based on aerial view is improved, and the method is suitable for multi-task BEV perception of three-dimensional object detection, high-precision map segmentation and motion prediction.

The invention discloses an automatic obstacle avoidance method and system for an unmanned ship based on Beidou artificial intelligence, and relates to the field of computer vision, and the method comprises the steps: collecting a visible light image and a near-infrared image of a water surface environment, and carrying out the reflection suppression processing; performing feature fusion on the image to generate an environment perception graph; obstacle features are extracted through a cascade cavity convolution structure, and an obstacle pixel-level semantic segmentation result is obtained; constructing an obstacle motion prediction model, and obtaining a predicted obstacle motion track; acquiring motion track data of the unmanned ship based on Beidou positioning and electronic chart data; establishing a space-time collision detection model, calculating collision parameters, and generating a collision data set; an obstacle avoidance path is generated through a dynamic situation field method, and a rudder angle instruction and a propeller rotating speed instruction are obtained. The unmanned ship has the advantages that accurate sensing, motion prediction and intelligent obstacle avoidance of the unmanned ship on water surface obstacles are realized through Beidou positioning and technology, and the sailing safety and intelligent level are improved.

The invention discloses a seamless switching method and system for a high-speed terminal of a ground-non-ground convergence network based on trajectory prediction and resource pre-reservation, and belongs to the technical field of space-ground convergence communication. The core of the method is that a switching area is accurately judged through bidirectional motion prediction of a terminal and an NTN node; a dynamic switching window is calculated in a self-adaptive mode by combining factors such as terminal speed and network time delay; intelligently selecting an optimal target link by using an AI enabled link scoring model; and cooperative pre-reservation and uplink and downlink synchronization of resources are completed before the window. The method has the advantages that the problem of failure of traditional RSRP judgment in an NTN dynamic environment is solved, the switching success rate and prediction accuracy in a high-speed moving scene are remarkably improved, the service interruption time and signaling overhead are greatly reduced, the continuity and reliability of communication are effectively guaranteed, and the method is suitable for large-scale popularization and application. The method is suitable for high-dynamic service scenes such as intelligent network connection vehicles and unmanned aerial vehicles in a 6G network.

The invention discloses a visual labeling mechanism device and an automatic labeling method thereof, and the method comprises the steps: obtaining a workpiece surface image in motion in real time through a visual assembly, extracting a pose parameter of a workpiece in combination with an image processing algorithm, generating a pose data set containing a coordinate parameter and a deflection angle, and inputting a motion prediction model. Calculating a future movement track of the workpiece in combination with the speed parameter of the conveying line, and generating an adjustment instruction set of the labeling head; the movement track of the multi-axis mechanical arm is controlled based on the adjusting instruction set, the attaching pressure of the label contact face is adjusted through the pressure feedback module, and labeling operation of the curved surface of the workpiece is completed; a line scanning camera module is used for scanning the geometric position of the labeled label, a real-time detection result is generated by comparing the geometric position with a preset quality standard, a compensation signal is triggered if position deviation exists, and compensation operation is conducted through a compensation pressure head so as to ensure that the labeling quality meets the preset standard; and through cooperative control of visual identification, track prediction and compensation, the stability and the fitting precision of dynamic labeling are effectively improved.

The invention relates to a multi-modal perception fusion target following method and system. The method comprises the following steps: adopting an improved KCF algorithm to realize a closed-loop process of target tracking, multi-scale space construction, feature fusion, response calculation, optimal scale decision and model updating; designing a four-level shielding processing mechanism fusing motion prediction and depth verification, and realizing tracking recovery in a shielding scene through shielding judgment, motion prediction, fine search and template protection; constructing a target distance mapping function fusing geometric distortion correction and attitude compensation, and realizing high-precision distance estimation based on monocular vision; laser radar point cloud information is integrated, an obstacle threat degree model is constructed, and cooperative path planning of following and obstacle avoidance is realized in combination with an improved TEB algorithm; and designing a linear velocity control law and an angular velocity control law based on the distance deviation and the azimuth angle deviation, and driving the robot to complete target following motion. According to the invention, high-precision and robust following of the robot to the target can be realized.

The invention relates to a collision early warning method based on a laser radar in a non-structural environment, belongs to the technical field of vehicle driving safety, and solves the technical problem of low accuracy and effectiveness of collision early warning in the non-structural environment. Comprising the following steps: in a vehicle body coordinate system, collecting point cloud data in a non-structural environment by using a laser radar, estimating a vehicle motion pose by using the point cloud data, and obtaining a real-time pose and speed of a vehicle; generating a vehicle motion prediction trajectory based on the motion pose and speed of the vehicle, and performing ground point cloud filtering and obstacle clustering detection on the point cloud data in the vehicle prediction driving area to obtain an obstacle clustering result; carrying out obstacle geometric collision check based on an obstacle clustering result, determining an obstacle vertex closest to the vehicle in a vehicle prediction trajectory range area, estimating the movement speed of the obstacle vertex, and calculating an RSS distance; if the RSS distance is smaller than the preset safety distance threshold value, collision early warning is triggered, and accurate early warning in the non-structural environment is achieved.

The invention relates to the technical field of navigation assistance, in particular to a dynamic obstacle prediction and obstacle avoidance path planning method based on multi-sensor fusion, which realizes omnibearing perception of a dynamic obstacle through multiple sensors, provides a rich and reliable data basis for subsequent processing, and improves the accuracy of the dynamic obstacle prediction and obstacle avoidance path planning. Meanwhile, motion prediction is carried out on a dynamic obstacle through data processing fusion and model prediction, the motion inertia of the obstacle is fully considered, track prediction of the obstacle is closer to a real physical rule, a global path search problem is converted into a local but large enough space search problem through a mode of delimiting a final obstacle avoidance target area, and the search efficiency is improved. According to the method, the number of grids needing to be processed and the search calculation amount are remarkably reduced, the harsh requirement of the mobile device for the real-time performance is met, the area possibly occupied by the obstacle in the future is avoided in real time in the mode of constructing the cost function, then the obstacle avoidance planning path is generated, and the path safety is improved.

The invention belongs to the field of ocean robots, and particularly relates to an underwater acousto-optic fusion sensing method and system based on multi-scale space-time alignment. According to the invention, environment parameter data, a target distance and a low-resolution image are acquired through the data acquisition layer; establishing an acoustic effective distance model and an optical effective distance model, and generating acoustic features, optical features and a distance-confidence mapping function; adjusting the focal length and the wavelength and generating a 4K image; carrying out acousto-optic feature fusion through the acoustic features, the optical features and a confidence coefficient function, and outputting fusion features; performing observation updating by using space-time asynchronous compensation and motion prediction; according to the method, starting point and detection are utilized, system parameters are updated in real time through a multi-target algorithm, a sensing-optimization-feedback-iteration closed loop is formed, and the full-distance environment adaptability is improved. According to the invention, the problem of data fusion fault caused by inconsistent sensing distances of acoustic and optical sensors is solved, and high-precision target identification and positioning with full-distance coverage are realized.

The invention discloses a car following control method and a related product. According to the scheme, multi-source data are acquired, and the multi-source data are fused to obtain a fusion result; based on the fusion result, performing multi-modal trajectory prediction on the driving behavior of the preceding vehicle to obtain a prediction result of the driving behavior of the preceding vehicle; the multi-modal trajectory prediction comprises longitudinal motion prediction and transverse motion prediction; quantifying a car following risk by using a multi-dimensional risk index to obtain a risk level; constructing a multi-target cost function based on the fusion result, the front vehicle driving behavior prediction result and a control vector, and solving the multi-target cost function by using a preset constraint condition to obtain an initial control parameter of the vehicle; and on the basis of the risk level, initial control parameters of the vehicle are adjusted, and target control parameters of the vehicle are obtained. Compared with the problem of response lag in adaptive cruise control in the prior art, the adaptive cruise control method has obvious advantages.

The invention relates to the technical field of robot control, in particular to a complex distribution network operation environment-oriented mechanical arm track obstacle avoidance control method, which comprises the following steps of: acquiring geometric information of an operation environment, a mechanical arm body posture and a grabbed target object through a multi-modal sensor; performing time synchronization and coordinate system calibration on the information to obtain environment data under a unified reference system; on the basis of the environment data, discretizing a work space into a three-dimensional voxel grid, calculating the occupancy probability of each voxel, and generating a probability occupancy model in combination with dynamic obstacle recognition and motion prediction; according to the method, through multi-modal sensing and unified reference system modeling, the problem that dynamic obstacle avoidance fails due to the fact that most of traditional mechanical arm track control methods adopt static environment hypothesis and cannot sense dynamic environment changes in real time and cannot unify multi-source information reference is solved.

The invention discloses a multi-level and multi-mode target identification and dynamic tracking method based on an unmanned aerial vehicle, and relates to the technical field of target identification and tracking, and the method comprises the steps: obtaining target information in the tracking process of the unmanned aerial vehicle, processing visual features and semantic features through a double-flow heterogeneous network, and generating a fusion feature map; recognizing a search object in the fused feature map, locking a tracking target based on a target evaluation and motion prediction result, and starting real-time dynamic tracking of the unmanned aerial vehicle; and periodically calculating the coordinate position of the tracking target relative to the unmanned aerial vehicle, adaptively adjusting the calculation frequency through regional warning monitoring, and continuously tracking the tracking target. According to the method, different levels of target identification are established, the object range is defined through the target contour, the target to be tracked is determined through the color, the vehicle mark, the animal hair color, the dressing and vehicle color and the like, effective combination of target identification and a dynamic tracking technology is realized, and automatic searching and automatic tracking of the target needing to be searched can be completed.

The invention discloses a dynamic submarine cable response extremum prediction method and device fusing a physical mechanism and a dynamic time sequence characteristic, and the method comprises the steps: deeply fusing a dynamic submarine cable control equation and data driving training through a physical information neural network, and converting the mechanical mechanism of a dynamic submarine cable into a residual constraint term; the defect of poor generalization of a pure data driving method is overcome; amplitude characteristics of six-degree-of-freedom motion of the floating body are extracted through an LSTM time sequence prediction model, and a long time sequence dynamic coupling effect is compressed into physical interpretable characteristics; a structure type judgment module is designed, so that two types of mainstream submarine cable structures can be covered without reconstructing the model, and the engineering applicability is greatly enhanced; dynamically balancing data fitting loss, physical residual loss and motion prediction loss by adopting a time-varying weight strategy, accelerating convergence and ensuring that a prediction result accords with a physical rule; dynamic submarine cable strength failure early warning is provided for offshore wind power by outputting key node response extreme values and motion amplitude characteristics in real time, and the operation and maintenance cost of the offshore wind power is greatly reduced.

The invention discloses an unmanned aerial vehicle flight obstacle avoidance method for realizing autonomous crossing of a dynamic obstacle by adopting intelligent flight control, relates to the technical field of unmanned aerial vehicle flight control, and enables an unmanned aerial vehicle to autonomously recognize a crossing window and effectively cope with challenges caused by high-speed rotating obstacles by sensing the motion state of a gear in real time and establishing an accurate rotating motion model. In combination with motion prediction and time window analysis, the planned spatio-temporal trajectory has high timeliness and spatial precision, the collision risk is significantly reduced, and safe execution of a flight task is guaranteed. The system constructs a sensing-prediction-planning-control closed-loop architecture, can respond in milliseconds, adapts to complex and high-speed task scenes, and has high robustness and real-time performance.

A method for pose and gait classification and motion prediction using spatio-temporal relationships between body joints includes capturing a sequence of images or video frames of a subject; applying a neural network-based pose estimation algorithm to the sequence of images or video frames to detect landmark positions of anatomical joints; constructing a spatio-temporal graph from the detected landmark positions of the one or more anatomical joints, wherein nodes of the spatio-temporal graph correspond to the anatomical joints and the landmark positions, spatial edges of the spatio-temporal graph represent anatomical connections between the anatomical joints within a single image or frame, and temporal edges connect the one or more anatomical joints across successive images or frames of the sequence of images or video frames; and inputting the constructed spatio-temporal graph into a spatio-temporal graph convolutional network (ST-GCN) to classify pose and gait patterns and predict motion or stability states.

The invention provides a low-line-number 3D laser radar elevation constraint SLAM method based on ground curvature fitting in order to solve the problems that a laser radar-inertial measurement unit SLAM system lacks effective constraint in the elevation direction and accumulative drifting is likely to be generated due to the fact that a low-line-number 3D laser radar is sparse in point cloud and insufficient in resolution in the vertical direction. The method comprises the following steps: firstly, carrying out preprocessing and ground point segmentation on point cloud data; then, in a vehicle real-time position neighborhood, quadric surface fitting is carried out on local ground points by using a least square method, ground normal vectors of the points are accurately extracted, and an average curvature representing the overall bending degree of the terrain is calculated; on the basis, normal vector constraint and average curvature correction are combined, and a new motion prediction and factor graph optimization model is constructed. Through the method, the observation capability of the low-line-number laser radar SLAM system in the elevation direction is remarkably enhanced, elevation drift is effectively inhibited, the positioning precision is improved, and compared with a mainstream open source algorithm, the average error is reduced by 46.3% to the maximum through actual measurement.

The invention provides a three-dimensional degree-of-freedom clamping jaw cooperative belt conveying access method and system, and the method comprises the steps: obtaining and analyzing point cloud data of an object on a conveying belt, and generating geometric center features and motion feature parameters of the object; inputting the motion characteristic parameters into an inertia compensation mechanism of the clamping jaw to generate counter-acting force parameters; combining the geometric center characteristics and the reaction force parameters to generate stable position data of the object; motion prediction is conducted on the continuous change characteristics of the stable position data, and three-dimensional moving track parameters and a conveying belt speed adjusting instruction are generated; and the three-dimensional moving track parameters are used for controlling the clamping jaw to conduct object grabbing operation in the space displacement process, and meanwhile the speed of the conveying belt is adjusted through the conveying belt speed adjusting instruction so that object placing operation can be conducted in the adjusting process. Millimeter-level synchronization precision of grabbing and placing operation in a high-speed dynamic access scene is achieved, and the problems of positioning misalignment and sequential disorder caused by vibration are solved.

The invention discloses an unmanned aerial vehicle cluster area sealing control method and system in a dynamic game scene. The method comprises the following steps: in a region exploration stage, realizing coverage and detection search of a task region based on a collaborative search algorithm of multi-agent partition perception and path planning; in the collaborative interception stage, based on the established target motion prediction model and the self-adaptive allocation mechanism of the enemy target, dynamic allocation is realized, the optimal interception path of the unmanned ship is planned, and meanwhile, energy charging and secondary flying of the unmanned ship are carried out in time; in the confrontation game stage, based on a decision tree model, in combination with multi-dimensional battlefield evaluation data, decision execution and evaluation are carried out, attack, avoidance or other tactical instructions are generated, and an optimal instruction and global target distribution are waited and cyclically made; and after an attack instruction is sent out, performing a safe predefined tactical attack in combination with tactical action execution logic. According to the method, the problems of poor unmanned cluster cooperative perception, difficulty in resource scheduling and untimely decision response in a complex dynamic environment are effectively solved, and the interception efficiency and comprehensive combat capability of regional sealing and control are improved.

The invention relates to an unmanned aerial vehicle multi-dimensional environment perception obstacle avoidance method and system, and belongs to the field of unmanned aerial vehicle control, and the obstacle avoidance method comprises the steps: constructing a multi-dimensional environment perception data cube with an unmanned aerial vehicle body coordinate system as a reference, and carrying out the feature decoupling processing of the multi-dimensional environment perception data cube, static obstacle geometric features, dynamic obstacle kinematics features, environment disturbance features and unmanned aerial vehicle state features are separated; processing the four features to respectively obtain an obstacle space occupation probability graph, an obstacle future motion prediction envelope surface, a local airflow disturbance compensation factor and a maneuverability boundary function, inputting into a lightweight obstacle avoidance decision neural network, and outputting three candidate obstacle avoidance paths and corresponding path safety scores; and sorting the three candidate obstacle avoidance paths, and selecting the path with the highest score as a final obstacle avoidance instruction. The autonomous obstacle avoidance success rate and the task execution efficiency of the unmanned aerial vehicle in an unknown airspace are remarkably improved.

The invention provides a universal dexterous hand action redirection method based on modular residual reinforcement learning, and the method comprises the steps: S110, receiving MANO parameters subjected to structural splitting through a finger sub-strategy network, and outputting initial action prediction; s120, simultaneously receiving the initial motion prediction and the complete MANO parameter representation as input through a residual error coordination network, correcting the local prediction of each finger under the guidance of the global hand intention, and outputting residual error correction, so as to obtain a final motion; s130, executing two-stage training: in the first stage, training each finger sub-strategy network in parallel; in the second stage, parameters of the finger sub-strategy network are frozen, and only the residual error coordination network is trained. According to the technical scheme, the real-time performance of a learning type method and the accuracy of an optimization type method are both achieved, data dependence is reduced, the generalization ability is improved, and meanwhile the equipment universality and action flexibility are both considered.

The invention discloses a low-altitude defense scene low-slow small target identification method, a terminal, a medium and a product. According to the method, firstly, a video frame is processed through a multi-scale feature adaptive target detection network, the network integrates a global self-attention mechanism to capture a remote dependency relationship, shallow details and deep semantic information are fused by adopting a dynamic weighting multi-scale feature fusion structure, and a target bounding box and a category are output in combination with an optimized loss function. And then, a time sequence level multi-target identity keeping and trajectory generating module is used, stable association of cross-frame target identities is realized through fusion of Kalman filtering motion prediction and appearance feature matching, and a target life cycle is managed in cooperation with a trajectory maintenance mechanism. According to the method, the problems of low-speed small target feature weakening, complex background interference, unstable multi-target tracking and the like are effectively solved, the recognition precision, the anti-interference capability and the tracking continuity are remarkably improved, and meanwhile, the low-altitude defense real-time requirement is met.

The invention relates to the technical field of automatic driving environment perception, and discloses a target detection method, device and equipment based on multi-modal fusion, and a storage medium, which are used for improving the accuracy of three-dimensional target detection. The multi-modal fusion-based target detection method comprises the steps of obtaining a two-dimensional detection frame and a three-dimensional detection frame of a current frame; performing data association on the two-dimensional detection frame and the three-dimensional detection frame through a dynamic adaptive bidirectional matching strategy to obtain a matching result, and calculating a matching score based on target geometric consistency, category confidence and spatial distribution similarity through the matching strategy; performing multi-stage optimization processing based on the matching result to obtain an optimized three-dimensional detection result; and performing track consistency verification on the optimized three-dimensional detection result in combination with the motion prediction information of the historical frame to obtain a fused three-dimensional target detection result.

The invention discloses a tippler dynamic unhooking robot control method based on visual identification, and belongs to the field of industrial robots, and the method comprises the steps: deploying a visual camera array in a tippler operation area, collecting tippler-carriage image data, and carrying out the dynamic target identification of the tippler-carriage image data; constructing a compartment claw instance segmentation network to perform segmentation positioning on the dynamic instance target set; acquiring claw relative motion state data to predict claw motion, compensating a claw motion deviation prediction parameter to a vehicle claw connection position, and determining an unhooking operation position parameter; and performing smooth unhooking control on an execution mechanism of the dynamic unhooking robot of the car dumper through the fuzzy logic controller based on the unhooking operation position parameters. The technical problems that in the prior art, unhooking of a car dumper depends on manual assistance or a fixed program, the adaptability to dynamic conditions such as claw vibration and image interference is poor, and unhooking stability and accuracy are caused are solved.

The invention provides a camera-based AI visual recognition and prediction control method and related equipment, and the method comprises the steps: obtaining continuous image frames of a moving object, and recognizing the skeleton key point coordinates and appearance feature information of the moving object; inputting the skeleton key point coordinates and the appearance feature information into a preset time sequence neural network model to obtain a motion prediction track; performing space mapping on the centroid coordinates of the moving object according to the motion prediction trajectory, and determining focal length control parameters of the camera; meanwhile, angle mapping is carried out on the position offset of the moving object, and pan-tilt control parameters are determined; and tracking and shooting the moving object according to the focal length control parameter and the holder control parameter. Through the mode, the camera can complete the pre-adjustment of the focusing mechanism and the composition direction before the target motion trend occurs, thereby effectively reducing the delay error in the focal length switching process, and improving the clear imaging rate and composition stability of a high-speed motion main body.

A decision generating device is configured to execute the following operations. A driving image of a vehicle and a driving status corresponding to the driving image are obtained. An image recognition is performed on the driving image by using an image recognition model to generate object information of objects in the driving image. A predictive driving information is generated based on the object information and the driving status by using a prediction model, the predictive driving information includes object movement predictions of the objects and a movement prediction of the vehicle, and the prediction model is generated based on a trained generative content model. A driving decision is generated based on the predictive driving information.

The invention relates to the technical field of image processing, and discloses an interventional navigation method and system based on multi-modal image fusion. The method comprises the following steps: segmenting CT angiography data to form a three-dimensional vascular skeleton model, calculating MRI perfusion data to generate a vascular flow velocity field, and constructing a vascular geometric constraint matrix; performing diffusion reconstruction on the constraint matrix and the instrument position information, and outputting an instrument motion prediction track; estimating the postures of the tips of the X-ray perspective imaging instruments to form instrument-blood vessel space correlation features; safety navigation constraint parameters are generated through distance field calculation; and generating a real-time navigation instruction containing a propulsion distance and a rotation angle by adopting a path planning algorithm. The technical problems that in a traditional interventional navigation system, multi-modal image data fusion is insufficient, instrument motion prediction is inaccurate, and a safety constraint mechanism is incomplete are solved, and the precision, safety and real-time performance of interventional operation navigation are improved.

The invention discloses a lightweight multi-agent motion prediction method combined with physical information, and the method comprises the steps: carrying out the vectorization coding of historical tracks and map information of traffic participants, and defining a plurality of local regions according to the position of each agent; introducing physical information to carry out local region feature aggregation on each agent; comprehensively considering road information, and carrying out global interaction on local areas; and outputting a future trajectory of the multiple agents based on a global interaction result. According to the method, the hierarchical design of local interaction modeling and global interaction modeling is used, the calculation redundancy is reduced, the calculation complexity is reduced, and the method is more suitable for large-scale multi-target prediction tasks. During interactive learning, physical priori and physical constraints are introduced, so that wrong learning of invalid relationships is avoided.