22315 results about "Point cloud" patented technology

Disclosed are a high-performance loosely-coupled multi-modal data fusion system for a smart driving environmental perception system and a vehicle-mounted device, comprising: a fusion detection model based on a modality-independent feature interaction strategy, which is configured for converting a LiDAR point cloud, a camera image, and a millimeter-wave radar point cloud into a unified bird's-eye view representation, and performing multi-modal fusion; and a fusion tracking model based on a motion-appearance feature cascaded coupling data association strategy, which is configured for performing subsequent trajectory tracking and matching according to multi-modal fusion feature information. A VoD data set and a K-Radar data set are selected for training, verifying, and testing the comprehensive performance of the models, and a TensorRT accelerated inference model is applied, then quantized, and deployed to a vehicle-mounted computational testing platform. The present invention is compatible with mainstream sensor deployment solutions, and achieves the efficient complementary fusion of multi-source heterogeneous sensor information, significantly improving the reliability, accuracy, and adaptability of vehicle-mounted perception systems, thereby effectively responding to extreme operating conditions such as complex traffic scenarios and inclement weather.

The invention discloses a three-dimensional environment reconstruction optimization method based on multi-sensor fusion data, and relates to the field of three-dimensional environment reconstruction optimization, and the three-dimensional environment reconstruction optimization method based on the multi-sensor fusion data comprises the following steps: S1, collecting multi-source sensor data, and constructing a data set under a unified coordinate system; s2, generating dense visual point cloud, and extracting laser point cloud features to construct a model; s3, establishing a local three-dimensional model, and generating a local environment image; s4, shadow parameters are extracted through shadow geometric analysis, and time sequence optimization is carried out; s5, consistency verification and correction are carried out, and three-dimensional reconstruction data are output; and S6, comparing the reconstruction data with the navigation map database, and carrying out map optimization updating. According to the method, time synchronization and space calibration are carried out on data acquired by the depth camera and the laser radar, complete and accurate three-dimensional information modeling of the target environment is realized, and the geometric precision of environment reconstruction and the image detail reduction capability are improved.

The invention relates to an optimization method and device for sparse view angle three-dimensional Gaussian splash, and belongs to the technical field of three-dimensional reconstruction in computer vision, and the method comprises the steps: collecting a sparse view angle image; a multi-view stereoscopic vision model based on deep learning generates a geometrically consistent depth map for the sparse view image, converts the depth map into point clouds and fuses the point clouds to obtain dense point clouds; sampling dense point clouds by adopting voxel-guided farthest point sampling to obtain initialized point clouds, and constructing a three-dimensional Gaussian field; rendering the three-dimensional Gaussian field through an enhanced geometric renderer to obtain a rendering depth and a rendering normal; constructing a multi-level geometric regularization loss function, and optimizing the three-dimensional Gaussian field; and performing optimization adjustment on the three-dimensional Gaussian field based on a shape-scale constraint criterion and a two-stage adaptive opacity constraint strategy to obtain an optimized three-dimensional Gaussian field. According to the method, the problems of initialization failure, insufficient geometric supervision and element out-of-control of 3D Gaussian splashing under the sparse view angle are solved.

The invention discloses a mechanical arm dynamic deviation correction method and system based on visual driving and a medium, and relates to the technical field of mechanical arm control. The method comprises the steps that when the tail end of a mechanical arm enters a preset machining space, an integrated 3D visual sensor is triggered to collect 3D point cloud of a workpiece to be machined; after pose recognition is carried out on the point cloud, the offset is recognized according to the teaching pose and the actual pose, and the initial offset is output; calling the multi-dimensional perception data, performing fusion correction, and outputting a correction offset; performing interference correction through an offset compensation model, and outputting a target offset; parameter adjustment and optimization are carried out according to the target offset, and a joint angle adjustment instruction is output; and performing correction closed-loop feedback according to the updated pose data. The technical problems of precision errors and low efficiency caused by deviation in the operation process of the mechanical arm are solved, and the technical effects that through dynamic deviation correction and multi-sensor data fusion, the operation precision and efficiency of the mechanical arm are improved, and stable operation in a complex environment is ensured are achieved.

The invention discloses a multi-modal visual fusion complex scene small target detection tracking method and system, and relates to the technical field of unmanned aerial vehicle target tracking, and the method comprises the steps: employing a visible light camera, an infrared thermal imager and a laser radar sensor which are carried on an unmanned aerial vehicle platform, and synchronously collecting RGB images, thermal infrared images and point cloud data; the consistency of the multi-modal data is ensured through data preprocessing and space-time alignment; constructing a lightweight double-branch network to extract multi-scale features, generating a fusion feature map by adopting adaptive weighted fusion, and generating depth information by utilizing point cloud to assist in scale estimation; a small target detection head is designed based on the fusion feature map, and precise detection is realized in combination with a feature pyramid network, adaptive scale prediction and a context awareness suppression mechanism; furthermore, through multi-mode cooperative tracking, including target association, spatio-temporal context modeling, trajectory prediction and a re-detection mechanism, tracking continuity is ensured.

The invention relates to the technical field of civil engineering tunnel structure safety monitoring and intelligent detection, in particular to a multi-modal fusion tunnel structure apparent disease identification and risk assessment system, which comprises an image acquisition module used for acquiring continuous images of the inner wall of a tunnel lining; a laser point cloud acquisition module; a structure sensor acquisition module; a data synchronization and preprocessing module; the multi-modal feature extraction module is used for performing depth feature extraction on the image, the point cloud and the sensor data; the heterogeneous feature fusion and disease identification module is used for fusing each modal feature and outputting a disease type identification result; and the risk assessment module is used for carrying out size estimation and parameterized expression on the identified diseases. The problems that in an existing tunnel inspection technology, the detection means is single, appearance and internal information cannot be considered, and the disease size is difficult to quantify automatically are solved.

The invention discloses a tunnel blasting quality evaluation and optimization method based on multi-source data fusion, and belongs to the field of tunnel blasting quality evaluation, and the method comprises the steps: collecting and preprocessing multi-source data of a tunnel blasting region; based on the preprocessed multi-source data, performing blasting quality evaluation according to local back break, a blasting contour line, average linear back break and point cloud extraction to obtain a blasting quality evaluation result; according to the blasting quality evaluation result, the blasting quality is graded, and a comprehensive blasting quality score is calculated and graded; and establishing a database containing geological parameters, surrounding rock response parameters and blasting process parameters, training through a convolutional neural network model to generate a blasting parameter optimization scheme, and dynamically adjusting blasting parameters of the next cycle according to the comprehensive blasting quality score. According to the method, the blasting parameter optimization and the quality evaluation process are closely combined to form a closed-loop system, the specific situation in the construction can be reflected in real time, and the accuracy of the blasting effect is ensured.

The present disclosure relates to the field of computer vision and discloses a three-dimensional dynamic scene reconstruction method and apparatus, and a storage medium. The three-dimensional dynamic scene reconstruction method comprises: acquiring synchronized videos of a plurality of viewpoints of a dynamic scene; computing matching points between video images of different viewpoints, and estimating intrinsic and extrinsic parameters of each camera; obtaining a Gaussian splatting point set {p0} on the basis of a sparse point cloud constructed according to the depth of each matching point; for the first image frame of each video, using {p0} to perform static training thereon, to obtain a Gaussian splatting point set {p}; for the remaining image frames, dividing {p} into a static point set {S} and a dynamic point set {D}, performing dynamic training on {D}, and constructing a dynamic Gaussian splatting point set {P} from {p}, {S}, and the final {D}; and, in view of the intrinsic and extrinsic parameters of each camera, rendering {P} using a Gaussian splatting rendering pipeline, to obtain rendered images at different moments from new viewpoints.

The invention discloses an unmanned ship multi-agent collaborative obstacle avoidance method and system for complex sea conditions, and relates to the field of unmanned ship multi-agent collaborative obstacle avoidance, and the method comprises the steps: obtaining the real-time data of each unmanned ship and the surrounding environment; generating a candidate obstacle target point cloud cluster list based on a density clustering algorithm; on the basis of a Kalman filter, real-time absolute motion state estimation of the candidate obstacles is obtained, and an obstacle feature list is output; determining a safety radius compensation amount required by autonomous obstacle avoidance of each unmanned ship; obtaining the safe sailing space of each unmanned ship at the current moment; constructing a global synthetic potential field, and generating a group of optimal alternative paths of the unmanned ship from the current position to the target point; on the basis of adopting a consensus binding algorithm, an unmanned ship multi-agent collaborative obstacle avoidance path reaching a consensus is obtained. The method has the advantages that safe, efficient, cooperative and consistent intelligent obstacle avoidance of a multi-unmanned-ship cluster in a real complex marine environment is realized.

The invention relates to an underwater robot navigation positioning method and system. The method comprises the following steps: S1, acquiring angular velocity and acceleration signals through an inertial measurement unit; s2, resolving a three-dimensional velocity observation value according to the beam radial velocity vector signal in combination with the angular velocity signal, and extracting environment feature point cloud data according to the acoustic image signal; s3, multi-source data time synchronization is carried out, and a fusion input signal with time-space alignment is generated; s4, constructing an adaptive factor graph optimization model, and dynamically adjusting an inertial navigation solution node based on a real-time weight coefficient; inputting the environment feature point cloud data into a closed-loop detection module to generate a loopback factor node, and adaptively correcting the weight of the node according to the feature matching degree; and S5, solving the adaptive factor graph optimization model through a nonlinear optimization algorithm. According to the underwater robot navigation positioning method and system, the problem that the fusion positioning precision of a multi-source heterogeneous sensor is insufficient in an underwater GPS-free environment can be solved.

The invention discloses a dynamic path planning and self-adaptive control method and system for a coating robot, and relates to the technical field of coating automation. The method comprises the following steps: acquiring point cloud data through three-dimensional scanning equipment, constructing a dynamically updated workpiece curved surface model, and extracting curvature, edge and high-curvature mutation region features; a spraying path is generated based on a reinforcement learning algorithm, and path density, speed and coating supply are dynamically adjusted for a high-curvature area; distance, force feedback and environment parameters are fused, and mechanical arm and spray gun parameters are dynamically adjusted; dividing operation sub-areas and distributing tasks based on robot capability characteristics to realize multi-machine cooperation; and fault redundancy control and multispectral imaging are added to optimize the coating quality. The system comprises a sensing module, a decision-making module, an execution module, a redundancy control module and a communication module. According to the invention, the uniformity of the complex curved surface coating, the multi-machine cooperation efficiency and the system anti-interference capability are improved, and the method is suitable for spraying large workpieces such as aircraft fuselages and high-speed rail vehicle bodies.

The invention relates to the technical field of quality monitoring, and discloses a diamond high-strength micro-powder quality detection method and system based on artificial intelligence. The method comprises the steps of obtaining a two-dimensional projection image sequence of diamond micro-powder particles, calculating a projection matrix based on camera calibration parameters and geometric constraints, obtaining a multi-view image data set of the particles, establishing a pixel-level corresponding relation, extracting three-dimensional space coordinates of the surfaces of the particles, and reconstructing dense point cloud data of the particles. Establishing a local coordinate system based on the dense point cloud data, determining attitude parameters of particles in a three-dimensional space, if the attitude parameters deviate from a normal range, performing attitude compensation processing to obtain standardized point cloud data, and performing three-dimensional grid model construction on the standardized point cloud data; and calculating geometrical characteristic parameters of the particles based on the three-dimensional grid model, performing defect detection on the surfaces of the particles, and generating a crystal integrity evaluation report of the particles. The quality detection accuracy of the diamond high-strength micro-powder particles is improved.

The invention discloses an unmanned aerial vehicle laser and vision fusion inspection method and system for bridge bottom disease detection, and the method comprises the steps: carrying out the synchronous data collection through employing a calibrated laser radar, a camera and an IMU, and obtaining a three-dimensional laser point cloud and a two-dimensional visual image of the appearance of a bridge; sharpening the image containing the motion blur and completing brightness self-adaption of the image; stable feature points are extracted, multi-frame matching is carried out, the corresponding poses of the images are estimated, and bridge dense point cloud reconstruction is completed; performing geometric component segmentation on the point cloud to generate a geometric prior region; component segmentation is carried out on a support area in the image, and a continuous and accurate component segmentation result is obtained in combination with a geometric prior area; screening the image, calling a targeted disease detection model in a corresponding component area, and generating a segmentation mask for the disease; obtaining a real disease three-dimensional point cloud, and carrying out quantitative calculation on the physical size of the disease; and displaying the real disease three-dimensional point cloud data and the physical size of the disease. The method is high in efficiency and precision.

The present invention belongs to the technical field of digital mine safety, and particularly relates to a blasting area surface morphology inversion method based on an unmanned aerial vehicle. The method comprises: S01, on-site data collection; S02, blasting area morphology inversion; S03, a muck pile throw distance; and S04, a muck pile surface fragmentation distribution. In the present invention, oblique photography by an unmanned aerial vehicle is used, and three-dimensional model reconstruction is performed by capturing a blasting area image, so that a feasible aerial survey scheme is formulated, and the integrity of collected data is good; and reverse modeling of the blasting area is performed by means of the steps of feature point extraction, spatial information conversion, point cloud generation, grid generation, etc.; voxel grid downsampling, point cloud matching and error detection are used to perform registration on point clouds of a target area before and after blasting, so that a good effect is achieved; and a color-based region growing algorithm is used to perform coarse segmentation of point cloud features on an ore-rock block on the surface of a muck pile, and a PointNet++ algorithm is used to perform fine segmentation of point cloud features on the ore-rock block on the surface of the muck pile, so that the muck pile throw distance and the muck pile surface fragmentation distribution are calculated.

The invention discloses a power transmission line inspection unmanned aerial vehicle obstacle intelligent detection and obstacle avoidance method. The method comprises the steps that multi-source sensing data is acquired, and alignment is completed through calibration and timestamp matching; heterogeneous data preprocessing and feature enhancement; constructing a high-precision environment fusing a geometric structure and a semantic tag, mapping a two-dimensional target detection result output by the recognition network to a three-dimensional coordinate system through spatial transformation, and fusing the two-dimensional target detection result with a point cloud structure to construct a semantic occupation grid map; performing preliminary route planning according to a preset power grid topological structure and task coverage requirements, and generating a barrier-free flight path covering the whole inspection area; reinforcing learning of a dynamic obstacle avoidance strategy; track dynamic reconstruction and energy consumption optimization scheduling are carried out; the technical problems that an existing technical system has defects in the aspects of obstacle recognition accuracy, complex environment adaptability, data fusion capacity and obstacle avoidance strategy intelligence, and the requirements for high-reliability, low-energy-consumption and high-efficiency unmanned aerial vehicle power transmission line inspection are difficult to meet are solved.

A method and apparatus for reconstructing a three-dimensional (3D) scene from a two-dimensional (2D) input image of the scene using a fully-differentiable transformer-based encoder-decode. A 2D input image encoded into a set of image features using a pre-trained vision transformer model, wherein the vision transformer model is pre-trained with multi-view RGB image supervision and point cloud supervision. The set of image features is projected onto a 3D triplane representation using a transformer decoder to obtain output triplane tokens. A triplane representation is created from the tokens and queried. 3D point features of color and density for volumetric rendering re predicted using a multi-layer perceptron. The geometry of the generated 3D asset is represented with a surface mesh including vertices and triangular faces. A texture map by is created with a multichannel image in UV space. Multiple views of the 3D scene are simultaneously generated based on the surface mesh.

The invention discloses a distribution network tree barrier real-time analysis method and system based on dynamic vision and SLAM. The method comprises the following steps: generating a dynamic visual baseline by cooperatively controlling the translation and flight displacement of an unmanned aerial vehicle holder, and constructing a bionic binocular model to simulate a time sequence image into a binocular image pair; generating a depth point cloud through epipolar correction and stereo matching; key targets are recognized and extracted through a semantic segmentation network, and semantic point clouds are generated; establishing a dimensionality reduction motion model by utilizing pan-tilt stability augmentation, and fusing a visual inertial odometer and RTK data by adopting a filtering or optimization algorithm to realize centimeter-level pose estimation; and finally, performing optimization processing on the semantic point cloud, completing three-dimensional reconstruction based on multi-modal fusion, and outputting a risk assessment result through tree line spacing calculation and safety margin analysis. According to the invention, accurate, efficient and automatic routing inspection and risk early warning of the distribution network tree obstacles are realized.

The invention relates to the technical field of tooth modeling, and discloses a three-dimensional tooth modeling system based on computer vision, computer equipment and a readable storage medium. According to the method, mirror reflection, diffuse reflection and subsurface scattering components in an original image are separated, mirror reflection intensity is normalized in combination with a dynamic truncation algorithm, pixel saturation is eliminated, groove and nest textures are reserved, a complete point cloud is obtained based on a two-dimensional texture image and cubic spline repair, and a multi-exposure point cloud sequence is obtained through bimodal calibration. The method comprises the following steps: solving the problem of data dislocation, carrying out weight assignment and data fusion on three-dimensional points in a plurality of exposure point cloud sequences to obtain three-dimensional fusion feature data, combining layered optical modeling and photon tracking compensation deviation, fusing clinical constraints, finally dynamically adjusting parameters, feeding back and optimizing, and outputting a micron-sized precision model. The modeling defect caused by difficulty in effectively coordinating feature contribution degrees under different exposure conditions is overcome, and high-precision modeling is realized.

The invention provides an intelligent real-time detection method and system for PCCP welding quality, and relates to the technical field of online detection and intelligent evaluation of pipeline welding quality through machine learning. Light energy data and multi-light-source images of a spiral weld pool are collected, exposure parameters are dynamically adjusted through the energy difference of visible light near-infrared bands, and the real-time detection of the PCCP welding quality is achieved. Inhibiting strong light interference and generating a weld surface image; a stress concentration area is positioned by scanning a welding seam thermal deformation area and combining speckle pattern change, sound frequency change and the elastic characteristic of the thin-wall steel cylinder; inputting the surface image and the deformation data into a space-time convolutional neural network, fusing light energy change, image details and spatial features to construct a weld joint space structure diagram, and adaptively correcting the position of a sensor; and comparing the sinking depth of the three-dimensional point cloud reconstruction, analyzing the correlation between the sinking degree and the stress, and generating a probability thermodynamic diagram to output the pressure-bearing failure risk level, so that the probabilistic early warning of the pressure-bearing failure risk can be realized.

The embodiment of the invention discloses an intelligent surveying and mapping method and system based on AI and BIM fusion. The method comprises the steps that an unmanned aerial vehicle platform carrying a laser radar, an RGB camera and a positioning system is used for scanning ancient building cultural relics and surroundings in a multi-angle flight mode, point cloud data, multi-view image data and position and attitude data are synchronously collected, and the three are associated through timestamps; after the point cloud data and the multi-view image data are preprocessed, cross-modal registration is completed through feature matching and pose estimation in combination with the position and pose data, and a registration data set is obtained; semantic segmentation is carried out on the point cloud data and the image data in the registration data set, and semantic segmentation results are fused based on the incidence relation; classifying and aggregating the original point cloud components according to category labels, constructing a topological relation reasoning assembly relation, calling corresponding BIM template instantiation model components based on the assembly relation, and hooking a segmentation result to generate a semantic enhanced BIM model; and integrating the BIM model and the GIS base map to form a fusion model so as to plot the historic building cultural relics.

The invention relates to the technical field of printed circuit board assembly quality detection, and discloses a PCBA anomaly detection method and system based on three-dimensional modeling and AI fusion, and a medium. The method comprises the following steps: acquiring three-dimensional point cloud data of a PCBA board to be detected; generating a reference three-dimensional digital twin model according to a standard PCBA design drawing; carrying out spatial registration on the three-dimensional point cloud data and the reference three-dimensional digital twin model, obtaining the three-dimensional point cloud data, carrying out hierarchical processing on the obtained three-dimensional point cloud data, extracting geometric features of a welding spot region, contour features of an element region and surface features of a substrate region, and carrying out fusion to generate a feature vector group; constructing a generative adversarial model based on a preset semi-supervised learning framework and the normal PCBA sample vector group; and inputting the feature vector group into a generative adversarial model, and examining the abnormal vectors, the corresponding three-dimensional coordinates and the abnormal types in the feature vector group by the generative adversarial model to complete the abnormal detection of the PCBA board. The method is suitable for quality control of a high-density and miniaturized PCBA.

The invention discloses a tunnel back break intelligent finishing and surrounding rock stability dynamic evaluation method and system, relates to the field of tunnel construction, and provides a tunneling-oriented back break intelligent finishing and surrounding rock stability dynamic evaluation method which integrates point cloud sensing, geological modeling, finite element analysis and deep learning prediction. And a closed-loop control process from identification, control, modeling, prediction and feedback is constructed. An over-excavation and under-excavation area is identified through a high-precision point cloud and a neural network, a trimming instruction is generated, a three-dimensional model and a support time sequence model are dynamically constructed in combination with geologic features, a variable parameter reduction method is introduced to realize more real stability analysis, and future displacement and a safety coefficient of surrounding rock are predicted by using deep learning. The method can improve the intelligent level of tunneling control and construction safety, and has remarkable engineering application value.

The invention discloses an urban building three-dimensional automatic modeling and visualization method, and belongs to the technical field of building three-dimensional modeling. The method comprises the steps that point cloud data, high-resolution images and geographic information system data of urban buildings are acquired, data cleaning, registration and alignment are carried out, and preliminary building digital representation is formed; accurately segmenting each building, and identifying the contour and main structural features of the building; based on the data integrity and the building complexity, adaptively selecting a proper reconstruction strategy to carry out three-dimensional reconstruction; in the reconstruction process, the geometric structure is analyzed and optimized in real time, and potential topological problems are repaired; automatically generating missing details based on a predefined architectural style library and a component library, and performing material inference and texture mapping; a graph structure is used for representing the relation between the buildings, and the positions and orientations of the buildings are adjusted through a global optimization algorithm; a rendering engine supporting multi-level detail switching is developed, and smooth visualization and interaction of a large-scale city scene are achieved.

The invention discloses a BIM model automatic generation method and system based on point cloud data, and belongs to the field of building information modelling, and the transmission method comprises the steps: obtaining original point cloud data, and employing a filtering algorithm based on point cloud density adaptive adjustment to carry out the preprocessing of the point cloud data; constructing a voxel octree structure for the preprocessed point cloud data, and performing semantic classification on the point cloud; geometric modeling is carried out based on the segmented point cloud subsets, and a fitting algorithm is adopted to carry out shape completion on a point cloud area; semantic annotation is carried out on the components subjected to geometric reconstruction, a corresponding relation between component types and spatial attributes is constructed, fusion features based on a point feature histogram and a local curvature are adopted, and classification is carried out; a standard BIM component family is converted, and a three-dimensional BIM model is constructed through the mapping relation. According to the method, the voxel octree data structure and the deep semantic segmentation neural network model are combined, division and semantic recognition are performed on the point cloud data, the intelligent degree of the model is improved, and manual intervention is reduced.

The invention discloses a storage checking method based on a multi-modal sensing technology, a robot and a storage system, and belongs to the technical field of storage management and intelligent sensing fusion. Multi-modal data such as a visual image, space depth, radio frequency sensing and infrared temperature are collected, and an image feature vector, a three-dimensional point cloud model, a radio frequency response matrix and a temperature map are constructed; generating a fusion recognition vector through a multi-channel fusion network based on an attention mechanism, and dynamically adjusting a modal weight; constructing an article space distribution map, and marking a perception missing region; automatically complementing low-confidence region data based on a priority scheduling algorithm; performing joint verification on the original fusion result and the completion result to form a final inventory list; if the confidence coefficient of a certain article is lower than an early warning threshold continuously for multiple times, triggering an abnormal alarm and generating a traceable sensing sequence; the method is suitable for a high-precision inventory task in a complex storage scene, and has the advantages of high recognition robustness, intelligent completion mechanism, traceable abnormity and the like.

The invention relates to a tower crane operation control system based on complex scene three-dimensional real-time modeling. According to the system, a lifting hook is coarsely positioned through a lifting hook positioning and state sensing unit, a real-time position is positioned by combining a laser radar point cloud clustering algorithm with historical pose data, and visual tracking is synchronously performed by means of a tower top camera AI; converting the real-time point cloud data into a 3D voxel grid map, generating a global path by using a 3DA algorithm, and outputting a hoisting track after smooth processing and track optimization; establishing a sling-lifting hook double-pendulum dynamic model, predicting a state sequence based on a model prediction control algorithm, and adjusting a control signal through a feedforward compensation item and a feedback correction item; and the man-machine interaction and monitoring unit is used for displaying the cantilever angle, the lifting hook height and the three-dimensional map of the tower crane in real time and remotely intervening the operation state of the tower crane. According to the system, multi-source data are fused to construct a high-precision three-dimensional map, lifting hook positioning and full-view tracking are achieved, and lifting safety and trajectory tracking precision are improved through path planning and dynamics control.

The invention provides an end-to-end automatic driving decision control method and system, and belongs to the technical field of automatic driving. The invention relates to an end-to-end automatic driving decision control method based on multi-modal perception and hierarchical trajectory optimization, and the method comprises the steps: constructing an end-to-end decision closed loop through combining the zero sample migration capability of a vision-language-action (VLA) model with a hierarchical optimization architecture: analyzing multi-modal input (vision, language and point cloud) by using a pre-trained VLA model to generate path points; the vehicle pose is dynamically adjusted through upper-layer optimization to expand a feasible solution space, a smooth track meeting dynamics and collision avoidance constraints is solved in real time through lower-layer optimization, and finally a vehicle control instruction is output. According to the method, a multi-modal sensing and hierarchical trajectory optimization mechanism is fused, and the sensing generalization ability, the path planning feasibility and the control execution robustness of the system in a complex traffic environment are effectively improved.

The invention provides a multi-view three-dimensional point cloud reconstruction method and device based on DPE-SE depth estimation, and relates to the technical field of computer vision and three-dimensional reconstruction. The method comprises the following steps: acquiring multi-view image data; preprocessing the image; inputting the preprocessed image into a DPE-SE-based depth estimation model, carrying out key constraint on an edge region through a semantic edge guiding mechanism, carrying out adaptive propagation updating on a weak texture region, realizing accurate depth estimation, and generating a multi-view depth result; then geometric consistency check and multi-scale depth fusion are performed on a multi-view depth result, and a dense depth map is constructed; and finally, performing three-dimensional back projection reconstruction and point cloud optimization processing, and outputting high-quality point cloud data containing three-dimensional coordinates and confidence information. According to the method, the problems of edge mismatching and depth voids are remarkably improved in complex illumination, weak texture and shielding environments, the continuity and structural integrity of the point cloud are improved, and technical support is provided for unmanned aerial vehicle surveying and mapping, building detection and digital twin modeling.

The invention relates to the technical field of intelligent agricultural robots, and provides a multi-agent collaborative anti-collision picking method based on digital twinning and deep reinforcement learning, which comprises the following steps: constructing a digital twinning model of a picking scene, and generating environmental geometric parameters, agent kinetic parameters and fruit position parameters through three-dimensional point cloud reconstruction; acquiring environment state data in real time and inputting the environment state data into the digital twin model for space-time alignment processing to generate synchronous state data; based on the synchronous data, a collaborative strategy containing a collision avoidance priority matrix, a path planning sequence and a task allocation weight is generated through a deep reinforcement learning network; an action instruction set is generated according to the strategy, and multiple agents are controlled to execute a picking task after virtual-physical space bidirectional verification of the digital twin model. According to the invention, efficient collision avoidance and accurate picking of multiple agents in a dynamic environment can be realized, and the picking efficiency, safety and system robustness are improved.

The invention discloses a multi-modal information fusion odometer construction method for star catalogue positioning, and relates to the technical field of star catalogue patroller positioning. The method comprises the following steps: carrying out space joint calibration on a monocular camera, a laser radar and an inertial measurement unit, reconstructing a laser radar point cloud by using a timestamp of a camera image, and realizing time synchronization of the camera image and the laser radar point cloud; establishing an IMU pre-integration error model; and performing motion compensation distortion removal on the laser point cloud by using an IMU pre-integration result, and extracting geometric features of the distorted laser point cloud based on a neighbor region smoothness calculation method of a fixed measurement distance. By researching a multimodal information fusion odometer method, the accumulative error of motion measurement is reduced, the positioning precision and stability are improved, technical support is provided for design and development of a star catalogue navigation system, and the problems that a single-modal star catalogue positioning method is weak in environment adaptive capacity and poor in algorithm generalization are solved.