2341 results about "Camera image" patented technology

Disclosed are a high-performance loosely-coupled multi-modal data fusion system for a smart driving environmental perception system and a vehicle-mounted device, comprising: a fusion detection model based on a modality-independent feature interaction strategy, which is configured for converting a LiDAR point cloud, a camera image, and a millimeter-wave radar point cloud into a unified bird's-eye view representation, and performing multi-modal fusion; and a fusion tracking model based on a motion-appearance feature cascaded coupling data association strategy, which is configured for performing subsequent trajectory tracking and matching according to multi-modal fusion feature information. A VoD data set and a K-Radar data set are selected for training, verifying, and testing the comprehensive performance of the models, and a TensorRT accelerated inference model is applied, then quantized, and deployed to a vehicle-mounted computational testing platform. The present invention is compatible with mainstream sensor deployment solutions, and achieves the efficient complementary fusion of multi-source heterogeneous sensor information, significantly improving the reliability, accuracy, and adaptability of vehicle-mounted perception systems, thereby effectively responding to extreme operating conditions such as complex traffic scenarios and inclement weather.

The invention relates to the technical field of unmanned vehicle control, and discloses a smart city environmental sanitation unmanned vehicle path planning and real-time monitoring method. The method comprises the following steps: firstly, acquiring a multi-source environment sensing data set such as laser radar point cloud data, a camera image sequence and real-time traffic flow information; local road network features are extracted based on laser radar point cloud data, a dynamic target motion prediction map is generated according to a camera image sequence, and real-time traffic flow information is processed to generate a regional traffic efficiency evolution curve. And inputting the data into a path optimization model to generate an initial path sequence, dividing cleaning task priorities, fusing related data and generating a final path planning scheme through a reinforcement learning algorithm. In addition, operation state data of the unmanned vehicle are collected in real time, and a path correction instruction set is generated through an anomaly detection model to update the strategy network. According to the method, the rationality and the operation efficiency of the path planning of the environmental sanitation unmanned vehicle can be improved, real-time monitoring is realized, and the operation safety and the management intelligence level are enhanced.

In a first aspect, a computer-implemented method of fusing visual features obtained with multiple sensor modalities comprises extracting from a camera image using a visual feature extractor a plurality of camera feature vectors at respective camera plane locations in a camera image plane; receiving sensor data of a non-camera sensor modality; processing the sensor data to compute a plurality of second feature vectors in a target plane lying non- parallel to the camera image plane in 3D space; and computing a plurality of fused feature vectors at a plurality of target plane locations in the target plane, without applying monocular depth detection to the camera image. In a second aspect, the fused feature vectors are computed using a transformer neural network cross-attention between the camera feature vectors and the second feature vectors.

The invention discloses a cradle head tracking method and device based on a binocular camera and a storage medium, and relates to the technical field of computer vision, and the method comprises the steps: processing image data based on a binocular parallax principle, and generating a three-dimensional coordinate of a center point of a tracking target; based on the three-dimensional coordinates of the camera coordinate system and the offset from the optical center of the camera to the rotation center of the holder, generating three-dimensional holder coordinates through coordinate transformation solution; determining a historical track based on the tracking target feature information and a historical target feature matching result, and outputting an identifier and a three-dimensional position observation value through correlation verification of a three-dimensional holder coordinate and the historical track; inputting an observation updating equation correction state through the identifier and the three-dimensional position observation value, and outputting a three-dimensional prediction position; based on the three-dimensional prediction position and a deviation formula, calculating the angle deviation with the camera image center under the holder coordinate system, and driving the holder to center the target in the picture center according to the angle deviation. The problem that the target tracking effect is poor is solved, and the robustness of target tracking in a complex scene is improved.

The invention relates to the technical field of image recognition, in particular to a license plate recognition system and method based on an image technology and a medium. The method comprises the following steps: acquiring area sensing data and a camera image set, and performing deformation effect compensation to obtain an environment compensation image set; performing image diffusion reverse enhancement on the environment compensation image set to obtain a license plate area enhanced image set; performing character region high-dimensional topological mapping based on the license plate region enhanced image set to obtain a character segmentation matrix; extracting character morphological characteristics according to the character segmentation matrix, and performing character recognition on the character morphological characteristics to obtain a character recognition result; and carrying out cross-character semantic compensation on the character recognition result to obtain a semantic compensation license plate character vector, and carrying out multi-target cross verification on the semantic compensation license plate character vector to obtain a license plate recognition result. According to the invention, the accuracy and robustness of license plate recognition can be improved.

The present invention relates to the technical field of environmental perception in confined spaces, and discloses a multi-modal coupled perception method for target recognition and region segmentation in a confined space. The method comprises: firstly, constructing a 3D point cloud data processing network, a 2D point cloud data processing network, and an image data processing network; next, constructing a feature fusion device, and outputting a bird's-eye view incorporating multi-sensor information; then constructing a multi-scale feature fusion extraction network and a network output head, outputting a target recognition and region segmentation result, and designing a loss function to train a network weight; and finally, inputting 3D point cloud data, 2D point cloud data, and camera image data into a network model, performing inference to obtain a target recognition and region segmentation prediction result, and performing visual rendering on the prediction result. In the present invention, in view of the characteristics of a millimeter-wave radar, a multi-modal coupled perception network is constructed, so that effective information in mass point cloud data of the millimeter-wave radar can be deeply mined, thereby effectively improving the detection accuracy of target recognition.

The invention discloses a multi-modal information fusion odometer construction method for star catalogue positioning, and relates to the technical field of star catalogue patroller positioning. The method comprises the following steps: carrying out space joint calibration on a monocular camera, a laser radar and an inertial measurement unit, reconstructing a laser radar point cloud by using a timestamp of a camera image, and realizing time synchronization of the camera image and the laser radar point cloud; establishing an IMU pre-integration error model; and performing motion compensation distortion removal on the laser point cloud by using an IMU pre-integration result, and extracting geometric features of the distorted laser point cloud based on a neighbor region smoothness calculation method of a fixed measurement distance. By researching a multimodal information fusion odometer method, the accumulative error of motion measurement is reduced, the positioning precision and stability are improved, technical support is provided for design and development of a star catalogue navigation system, and the problems that a single-modal star catalogue positioning method is weak in environment adaptive capacity and poor in algorithm generalization are solved.

The invention relates to the technical field of synchronous positioning and map construction, and discloses a multi-sensor fusion SLAM method and system based on data quality comprehensive evaluation, and the method comprises the steps: carrying out the multi-dimensional quality evaluation of laser radar point cloud data and camera image data, and enabling the evaluation indexes to comprise the scene degradation degree, the view field integrity and the intensity information entropy of a point cloud, according to the point cloud data and the definition, self similarity and natural quality of the image, calculating the comprehensive weight of each index by adopting a subjective and objective weight fusion method, and dynamically selecting a laser radar pose, a visual sensor pose or an IMU integral pose increment as an initial value according to the comprehensive quality score of the point cloud data and the image data; and matching optimization of the point cloud and the map and global pose map optimization are carried out. According to the method, the sensor data quality is evaluated in real time, and the fusion strategy is dynamically adjusted, so that the positioning precision and robustness of the SLAM system in a complex environment are remarkably improved, and meanwhile, the calculation complexity is reduced.

The invention discloses a high-performance low-cost multi-modal data fusion model and equipment for complete driving environment state perception, and the model comprises a feature extraction module which is used for extracting initial potential features of a camera image and a laser radar point cloud; a probability cross attention mechanism constructs a feature fusion model of uncertainty perception; a query initialization model capable of reducing uncertainty is adopted, and the initialization process of query is enhanced by utilizing two-dimensional target detection priori; and based on a robustness optimization training strategy of query denoising, stable optimization is realized under uncertainty. A nuScenes data set is selected for training, verifying and testing the comprehensive performance of the model, and the TensorRT realizes CUDA acceleration and is quantitatively deployed on a vehicle-mounted calculation experiment platform. According to the multi-modal data fusion system, a perception state uncertainty explicit measurement calculation theory based on data and a model is innovated, a perception model for coping with environment uncertainty is established, the completeness, reliability and reconfigurability requirements of a driving environment are met, and driving safety and decision accuracy are ensured.

The invention relates to a wharf sensing method and system based on data fusion of a laser radar and a camera, and the method comprises the steps: firstly employing a fisheye camera to shoot a checkerboard calibration plate at multiple angles, employing a Zhang's calibration method to calculate an internal reference matrix and a distortion coefficient of the fisheye camera, and synchronously collecting the data of the laser radar and the camera in real time; the method comprises the following steps: performing distortion removal processing on a camera image, calculating an external parameter matrix between a laser radar and a camera through a feature extraction algorithm, a feature point matching algorithm and a nonlinear optimization algorithm, and performing data fusion through an internal parameter matrix, the external parameter matrix and a bilinear interpolation method to generate fusion point cloud data with color information; and finally, identifying a key target of the wharf by marking and training a CNN model, and calculating three-dimensional parameters of the key target by adopting a specific calculation method, so that a sailor can quickly distinguish the target in the fused point cloud data, the accuracy and efficiency of wharf environment perception are remarkably improved, and accurate environment perception and decision support are provided for ship berthing and departing operation.

The invention provides a cross-modal adaptive matching online external parameter calibration method and system for a laser radar and a camera, and belongs to the technical field of online external parameter calibration of the laser radar and the camera, initial external parameters are obtained by using a static calibration result, a laser radar point cloud is projected to a visual angle of a virtual camera, an LIP image aligned with a camera image is generated, and a calibration result is obtained. Performing cross-modal segmentation on the LIP image and the RGB image, extracting a semantic mask, generating a cross-modal feature point pair through a strategy, and performing preliminary external parameter calculation in combination with a PnP algorithm; global optimization is carried out through a multi-modal loss function, and an inter-frame smoothing and increment optimization strategy is introduced to ensure accurate estimation of external parameters. Finally, the optimized external parameter matrix is applied to projection and matching of subsequent frames, and can adapt to environment change and data dynamic update in real time. The method has high precision and real-time performance, overcomes the defects that off-line calibration depends on a specific calibration target and cannot cope with dynamic environment changes, and adapts to application requirements in high-speed motion and complex environments.

The invention discloses an intelligent car coupler identification method and system of a car dumper based on a picking and re-hooking robot, and relates to the technical field of intelligent identification, the method comprises the following steps: multi-modal data is collected and preprocessed, and the multi-modal data comprises laser radar point cloud data, camera images and infrared data; fusing the processed multi-modal data, identifying coupler information by adopting a convolutional neural network and an attitude estimation algorithm, and generating a coupler identification result; a path planning algorithm is adopted, a grabbing point path of the car coupler is calculated, interferents are avoided, the grabbing path is optimized, and an optimal grabbing path is generated; and the robot moves to the position of the car coupler according to the optimal grabbing path, the grabbing posture is corrected in real time through image perception, and a mechanical arm tail end executor is driven to clamp the car coupler for grabbing. Through multi-modal data fusion and path planning, the coupler is accurately recognized, the grabbing path is optimized, and the grabbing precision and stability of the robot in a complex environment are improved.

Methods and processors for rendering a 3D object are disclosed. The method includes acquiring multi-camera image input including first image frames of the 3D object generated by a first camera and second image frames of the 3D object generated by a second camera, acquiring an initial 3D Gaussian Splatting (3DGS) model having a plurality of initial parameters including an initial frame-wise GS parameter and an initial camera-wise GS parameter, generating an adjusted 3DGS model by adjusting, based on the multi-camera image input, at least one of: the initial frame-wise GS parameter, the initial camera-wise GS parameter, generating, by the adjusted 3DGS model, a 3DGS output and rendering a 2D image of the 3D object using the 3DGS output.

The invention belongs to the technical field of data processing, and particularly discloses a dynamic path planning method and system for an inspection robot, and the method comprises the steps: collecting field data through a sensor, processing a laser scanning point cloud and a camera image through a positioning mapping algorithm, and fusing multi-source information to generate a real-time map. Obtaining a three-dimensional environment model containing a dynamic obstacle position; according to the three-dimensional environment model, obstacle features are extracted, a deep learning model is adopted to identify moving personnel and randomly placed goods, obstacle types and movement tracks are judged, and a classified obstacle data set is obtained; through the classified obstacle data set, the distance and the relative speed between the obstacle and the current position of the robot are calculated, and if the distance is smaller than a preset threshold value and the speed is larger than zero, high-priority interference is marked; according to the invention, the problem of inaccurate dynamic obstacle identification and obstacle avoidance in a complex scene in the prior art is solved.

The invention relates to the technical field of security and protection management, in particular to a campus security and protection management system based on deep learning, which improves the accuracy and robustness of identity recognition by acquiring access control card numbers, face images or fingerprint features and generating standardized identity authentication data. And on the basis of a comparison result of the identity authentication data and the campus database, a behavior chain initialization identifier is generated, and accurate identity binding of the school entering personnel is realized. Furthermore, by collecting multi-camera image stream data, pedestrian re-identification and similarity calculation are executed by using a deep feature matching network, and a cross-camera continuous trajectory data set is generated. And matching the behavior track data set with the conventional path template to generate a behavior offset feature vector. And carrying out joint modeling on the behavior offset characteristics and the identity information through a graph neural network model containing an attention mechanism, and outputting a behavior purpose label and a risk grade score. And a graded security response instruction is generated based on the risk score, so that the missing report rate and the false report rate are effectively reduced.

The invention discloses a slope collapse rockfall monitoring method, and relates to the technical field of slope rockfall monitoring, and the method comprises the following steps: arranging multiple types of monitoring sensors in a slope region, and covering a key monitoring region in combination with a camera to form a multi-source data collection network; according to the method, false alarms caused by natural interference and non-dangerous interference are effectively reduced through real-time monitoring of a multi-source sensor and a signal filtering technology based on an environmental noise model, and the authenticity of risk signals is verified from multiple dimensions by combining sensor data with camera images through a multi-modal data fusion algorithm, so that the reliability of the system is improved. In combination with a dynamically updated slope collapse feature library and an intelligent algorithm, the system can accurately identify key parameters of slope instability and predict risks, flexibly responds based on a graded early warning mechanism, improves the disaster prevention and control capability, reduces resource waste caused by false alarm, reduces threats of disasters to life and property, and improves the safety and reliability of the system. And sufficient time is provided for managers to take measures.

The invention relates to the technical field of motion capture data processing, and discloses a robot reinforcement learning model construction method based on optical inertial fusion whole body motion capture, and the method comprises the steps: measuring and recording a plurality of key parameters of a target human body; carrying out primary calibration and recording on all inertial sensors worn on a target human body, and calibrating all cameras arranged around at the same time; in the process that the target human body executes the specified action, synchronously acquiring action sampling data of the target human body; the action sampling data comprises motion data corresponding to the inertial sensor and camera image data corresponding to the camera; the motion data and the camera image data are processed, fused and optimized, and optimized joint posture quaternary data and human body motion sequence data are obtained; and inputting the human body action sequence data into the reinforcement learning model of the humanoid robot, and configuring the preset reward function to perform reinforcement learning training, so that the learning efficiency of the robot and the accuracy of action execution can be improved.

The invention discloses a dynamic object identification and tracking method based on a laser radar. The method comprises the following steps: S1, collecting data; s2, extracting features from the optical image data, extracting speed and distance information from the millimeter wave radar, and enhancing the expression ability of the features by using a deep learning method and the like; s3, fusing the point cloud data of the laser radar with the camera image, and further processing the fused data; s4, fusing a plurality of target lists to obtain reliable and real target information, and avoiding misinformation and missing detection; s5, comprehensively combining the decisions obtained by the multiple sensors by adopting a fusion strategy, and obtaining a fusion result; and S6, estimating and updating the state of the sensor carrier in real time by using a Bayesian filtering method. According to the invention, identification and tracking of the dynamic object are realized. The system has the beneficial effects that the sensing capability of the system is improved, false alarm and missing detection are reduced, and the reliability and the accuracy of the system are enhanced.

The invention discloses an unmanned aerial vehicle dynamic environment 3D target detection method and device based on multi-modal fusion, and the method comprises the steps: calibrating a 4D millimeter-wave radar and a camera, and obtaining a pose relation between the millimeter-wave radar and the camera; the method comprises the following steps: preprocessing 4D millimeter wave radar point cloud data to obtain a denoised point cloud image, and performing feature extraction on the denoised point cloud image; using hidden space normal estimation to obtain a pose matrix between adjacent frames, and then through global discretization analysis, realizing multi-frame point cloud pose correction and fusion in a motion scene, and enhancing point cloud data; processing the radar point cloud and the camera image, and obtaining a BEV feature map through a feature extraction network; using a double-end channel attention mechanism and a cross attention mechanism to fuse BEV features; a detection head module based on an anchor frame is designed, and differentiation setting is carried out for different detection targets; and verifying by adopting different data sets and different test scenes to obtain a 3D target detection result. According to the invention, the hidden space normal estimation method is provided to correlate spatio-temporal information, multi-frame splicing is carried out, the density of point clouds is increased, and the performance of target features is improved.

The invention belongs to the field of data analysis and processing, and discloses a virtuality and reality combined automobile automatic driving test method. Comprising the following steps: connecting a sensor suite and an automatic driving system of an automobile with a virtual twin center by using a CAN bus or an Ethernet interface, operating the automatic driving system, and generating a performance report; the method comprises the following steps: initializing raindrop attributes, setting the transparency of raindrops based on real environment data, introducing a complex wind field model and a surface ponding dynamic evolution mechanism, carrying out collision detection on all raindrop pairs by using a space segmentation algorithm, and simulating dynamic behaviors of the raindrops in the air based on a discrete element method and a smoothed particle fluid dynamics method. Visual rendering is carried out on the water spray effect; under the influence of raindrop physical characteristic modeling, output of laser radar point cloud scattering modeling, camera image fuzzy modeling, millimeter wave radar signal attenuation and scattering modeling and ultrasonic sensor interference modeling is iteratively updated, and automatic driving testing of an automobile under virtual-real combination is achieved.

The invention discloses an external parameter calibration method for a laser radar and camera system, and the method specifically comprises the steps: synchronously collecting the laser radar point cloud and camera image data of a circular plane calibration plate through employing the circular plane calibration plate with a two-dimensional coding pattern printed at the center; segmenting laser radar point cloud subsets belonging to the calibration plate from the laser radar point cloud through a plane extraction algorithm, and calculating centroid coordinates of the point cloud subsets as three-dimensional feature points; detecting angular points of the two-dimensional coding pattern in the camera image, and calculating geometric centers of the angular points as two-dimensional feature points; multiple groups of three-dimensional-two-dimensional feature point corresponding relations are established through multiple times of scanning, and initial external parameters are solved by using a PnP algorithm; and minimizing a re-projection error through a nonlinear optimization algorithm to obtain a final external parameter. The method provided by the invention is suitable for application scenes such as robots, automatic driving or three-dimensional reconstruction, and accurate, efficient and automatic external parameter calibration can be realized.

The invention provides a high-altitude operation risk early warning method and system based on camera image recognition, and relates to the technical field of computer vision, and the method comprises the steps: firstly collecting a video image sequence of a high-altitude operation scene, and generating a fusion feature map containing environment and operation main body features through multi-level feature extraction; performing spatial dimension segmentation and regional feature comparative analysis on the fusion feature map to obtain a spatial risk distribution map containing risk region identification information, processing the spatial risk distribution map of continuous frames based on a time sequence feature fusion rule to generate a dynamic risk evolution map, and calling a risk decision model to perform mode recognition to obtain a dynamic risk evolution map; and generating a risk level classification result and a risk position coordinate set according to the risk level classification result and the risk position coordinate set, and finally generating a risk early warning signal and sending the risk early warning signal to the monitoring terminal, thereby comprehensively, accurately and dynamically monitoring the high-altitude operation risk, and improving the accuracy and timeliness of risk early warning.

The present invention belongs to the field of indoor and outdoor robot application scene reconstruction, specifically, a deep learning 3D reconstruction method based on FMCW laser radar and visual fusion, comprising the following steps: 1) setting multiple laser radars and cameras to ensure time synchronization; 2) calibrating multiple laser radars and cameras; 3) real-time collection of FMCW laser radar data and camera image data in each scene and angle, corresponding to timestamps; 4) processing the collected data as a training data set; 5) the data are input into the deep learning model network for training to obtain a 3D Gaussian model of dynamic and static objects in the scene; 6) after coordinate conversion and velocity calculation of the collected FMCW laser radar data, combined with the collected camera image data, input into the 3D Gaussian model, and the present invention obtains scene reconstruction information. The present invention uses the native velocity information of the FMCW laser radar, combined with visual information, to better characterize the characteristics of dynamic objects and achieve native dynamic and static distinction.

The invention relates to the technical field of settlement observation, in particular to a non-contact monocular vision high-precision automatic settlement monitoring method, which comprises the following steps: firstly, performing target monitoring by using a YOLOv10s model, improving the real-time reasoning speed and recognition precision of target recognition, and solving the target recognition problem under a complex background; secondly, a digital image processing technology and an edge-based least square ellipse fitting algorithm are combined to realize accurate positioning of the target; thirdly, deriving based on a camera imaging principle, and introducing an improved world coordinate calculation method to solve world coordinates corresponding to the pixel coordinates so as to obtain three-dimensional world coordinates; and finally, calculating a settlement value of the monitoring area by taking the first frame image as a reference, predicting a future settlement value by using a deep learning model, and making an early warning for structure safety in advance. According to the invention, the installation demand and the calculation overhead are reduced, and the settlement monitoring precision is improved.

The invention discloses a projection single attitude calibration method, system and device and a medium, and relates to the technical field of computer vision and projection interaction.The method comprises the steps that a plurality of calibration patterns are generated and projected to a calibration plane, gray level images are collected, corner point detection and sub-pixel refinement processing are carried out, and a projection single attitude calibration result is obtained. The method comprises the following steps: acquiring an angular point corresponding relation between a projector image and a camera image, carrying out iterative solution on a projection matrix of a projector and a projection matrix of a camera, separating an internal parameter matrix and an external parameter matrix, calculating a basic matrix and an essential matrix, and decomposing to obtain a rotation matrix and a translation vector of the camera relative to the projector; camera imaging distortion is estimated, a distortion correction coefficient is obtained, and distortion correction is carried out; and counting a re-projection error generated in the correction process, and adjusting parameters of the projector and the camera until the error meets a preset precision requirement. Projector parameter calculation is completed through projection single posture calibration, posture adjustment is not needed in the calibration process, the automation degree is improved, and meanwhile manual intervention is reduced.

The invention discloses a mechanical arm visual servo control method and equipment based on target condition reinforcement learning, and belongs to the technical field of mechanical arm control. A mechanical arm environment is built, a camera is additionally arranged at the tail end of a mechanical arm, a computer is used for designing a safety reinforcement learning model, then online learning is carried out according to a depth deterministic strategy gradient algorithm and a post-experience playback algorithm, and the mechanical arm environment is matched. The position of the feature point in the camera picture serves as input to control the mechanical arm to move and lock the two-dimensional code attached to the surface of the object, and finally the visual servo task is completed. The method is simple in step, low in environment requirement and rapid in response of the mechanical arm, and compared with a traditional control method based on pseudo-inverse, the method has remarkable effect improvement in the aspects of the number of iterations and robustness.

Provided is a system or method configured to present reactions exhibited by participants in an online event to users such that individual participants are not identified, thereby preventing the participants from being conscious that their reactions are being sensed, and thus enabling them to naturally exhibit their reactions, which allows the users to acquire information on the participants' naturally-exhibited reactions. The system includes a user terminal provided with a camera; and an analysis server configured to acquire camera images of each participant from the terminal device, and perform a reaction sensing operation to sense reactions exhibited by the each participant based on the camera images. The analysis server aggregates results of the reaction sensing operation for all the participants to thereby generate reaction information on the all the participant's reactions (e.g., level information and proportion information), and generates display information that visualizes the reaction information.

The invention discloses a multi-view underwater three-dimensional point cloud splicing method based on sparse and dense point cloud fusion, and relates to the technical field of structure underwater detection. The method comprises the steps that a binocular camera surface image of an underwater structure is received, the binocular camera comprises a left camera and a right camera, dense point clouds are generated based on stereo matching and triangulation, and the corresponding relation between the dense point clouds and image pixels of the left camera is output; and based on the received left camera image, a sparse point cloud is generated by adopting a motion structure recovery algorithm and utilizing multi-view triangulation, and a corresponding relation between the sparse point cloud and the pixels of the left camera image is output. According to the method, when the point cloud of multi-view reconstruction is spliced, the surface features of the underwater structure can be well restored, feature extraction is not needed, the method has high accuracy and robustness, and high-precision point cloud splicing can be achieved in the underwater complex environment, so that the method has good application prospects in actual underwater structure detection and image splicing tasks.

The invention relates to the technical field of autonomous navigation of inspection robots, and provides a field inspection robot autonomous navigation and path planning method based on multi-modal sensing. The method comprises the following steps: collecting a laser radar point cloud, a camera image, inertial measurement and positioning data, obtaining an environment semantic feature set through spatio-temporal feature fusion, and generating a semantic occupation map; inputting the current position of the robot, the target point and the semantic occupation map into a trajectory generation network to obtain candidate trajectories meeting obstacle avoidance and path smoothness constraints, and completing task sorting and trajectory splicing in combination with inspection task points to form a global path; in the operation process, the reinforcement learning control model adjusts the linear speed and the angular speed in real time, and path tracking and dynamic obstacle avoidance are achieved. The navigation precision and the operation safety of the inspection robot in the complex field area are improved.

The invention provides a touch and visual perception integrated micro-nano robot magnetic field control system and method, and the system comprises a camera which collects the image of a micro-nano robot in a pipeline in real time; the magnetic field generating device generates a gradient magnetic field for driving the micro-nano robot to move through a permanent magnet; the tail end of the mechanical arm is connected with the permanent magnet and adjusts the three-dimensional pose to change the magnetic field distribution; the force feedback equipment is used for providing a touch signal for the micro-nano robot to be in contact with the pipeline boundary; and the computer is configured to identify the aggregation state of the micro-nano robot according to the camera image, increase the resistance of the force feedback equipment and reduce the rotation speed of the permanent magnet when the dispersion state is detected, and reduce the resistance and recover the conventional magnetic field control after the aggregation state is recovered. According to the system, accurate control and abnormal state self-adaptive response of the micro-nano robot in a complex experiment scene are realized through touch-vision collaborative perception and dynamic magnetic field regulation and control.