Dynamic positioning management system and method based on visual laser collaboration

Abstract

The invention particularly relates to a dynamic positioning management system and method based on visual laser collaboration. The system comprises a hardware sensing module; a multi-sensor fusion module; the positioning and map building module is used for realizing real-time positioning of the system in an unknown environment by utilizing an SLAM (Simultaneous Localization and Mapping) technology and combining vision and laser radar data, and building a geometric map; and a data processing and control module. According to the invention, multi-mode fusion of vision and laser radar data is combined with an advanced SLAM technology and a dynamic positioning tracking algorithm, so that the defect of a single sensor is effectively overcome, high-precision positioning is realized in a complex dynamic environment, and the positioning error is remarkably reduced; according to different application scenes and environmental conditions, sensor configuration, data fusion modes and algorithm strategies are flexibly selected, and the method has good adaptability to illumination changes, texture feature differences, dynamic obstacles and the like.

Description

Technical Field

[0001] The present invention relates to the field of dynamic positioning management technology, and in particular to a dynamic positioning management system and method based on visual laser collaboration. Background Art

[0002] Accurate dynamic positioning management is crucial in areas such as intelligent robotics, autonomous driving, and industrial automation. Traditional positioning technologies, such as single-use visual positioning or LiDAR positioning, have numerous limitations. Visual positioning accuracy significantly decreases in environments with poor lighting conditions and unclear texture features. LiDAR positioning, on the other hand, has limited recognition capabilities for small and low-profile targets in complex and dynamic scenes, and is also costly.

[0003] Furthermore, existing multi-sensor fusion positioning systems still need improvement in terms of temporal and spatial synchronization accuracy, data fusion efficiency, and accuracy, making them difficult to meet the stringent requirements for positioning accuracy and real-time performance in complex dynamic environments. Therefore, a system that effectively integrates the advantages of vision and lidar to achieve high-precision, high-reliability dynamic positioning management is urgently needed. Summary of the Invention

[0004] The purpose of the present invention is to propose a dynamic positioning management system and method based on visual laser collaboration in order to solve the above problems.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions: A dynamic positioning management system and method based on visual laser collaboration, comprising: Hardware perception module: uses visual sensors to collect image data for target detection and motion estimation, and lidar to obtain point cloud data for environment modeling and ranging; Multi-sensor fusion module: aligns visual and lidar data using temporal and spatial synchronization technology and integrates the information from both. Positioning and Mapping Module: Utilizes SLAM technology, combined with vision and lidar data, to achieve real-time positioning of the system in unknown environments and build geometric maps; Data processing and control module: Based on the positioning results and map data, the motion path is generated through the path planning algorithm, and the actuator is driven through navigation control.

[0006] Preferably, the hardware perception module specifically includes: a visual sensor unit and a laser radar unit; Visual sensor unit: captures environmental images and provides raw data for subsequent target detection, scene understanding, and motion estimation; LiDAR unit: By emitting laser pulses and receiving reflected signals, it constructs a three-dimensional point cloud model of the environment, providing the system with accurate distance information and spatial structure data.

[0007] Preferably, the multi-sensor fusion module specifically includes: a time and space synchronization unit and a data fusion algorithm unit; Temporal and spatial synchronization unit: This is the foundation of multi-sensor fusion, ensuring temporal and spatial consistency between visual and lidar data, and avoiding positioning errors and target misjudgment caused by asynchrony. Data fusion algorithm unit: organically combines visual and lidar perception data to leverage the complementary advantages of the two sensors and improve overall performance.

[0008] Preferably, the positioning and mapping module specifically includes: a SLAM (Simultaneous Localization and Mapping) unit and a dynamic positioning tracking unit; SLAM (Simultaneous Localization and Mapping) unit: enables the system to determine its own position in an unknown environment in real time and simultaneously build a map of the environment, providing a basis for subsequent navigation and decision-making; Dynamic positioning tracking unit: In a dynamic environment, it tracks changes in its own position and surrounding dynamic targets in real time.

[0009] Preferably, the data processing and control module specifically includes: a path planning and navigation unit and a status monitoring and feedback unit; Based on the system's current positioning results and environmental map information, it plans a safe and efficient motion path from the starting point to the target point, and generates corresponding control instructions to drive the actuator movement; Path planning algorithms are divided into global path planning and local path planning; Global path planning: Finding the optimal path from the starting point to the end point on a known global map; commonly used algorithms include A algorithm and Dijkstra algorithm; Local path planning: In a dynamic environment, the system needs to respond to sudden obstacles in real time, and this is when the local path planning algorithm comes into play; Navigation control: Generate accurate control instructions based on the planned path, drive the actuator, and make the system move accurately along the path; the control algorithm includes PID control and pure tracking algorithm; Feedback regulation: real-time monitoring of the deviation between the actual position and the planned path, and dynamic adjustment of control parameters through feedback regulation; Status monitoring and feedback unit: monitors the system's operating status in real time, identifies potential problems promptly and takes corresponding measures to ensure reliable system operation; Condition monitoring indicators: Positioning accuracy evaluation: Evaluate positioning accuracy by estimating the covariance matrix of positioning results or calculating reprojection error; Sensor health check: Real-time monitoring of the sensor's operating status. If the LiDAR signal strength is below the normal range, it is considered that the LiDAR may be faulty or blocked. For cameras, the health status is determined by testing indicators such as image clarity and the presence of noise. System resource monitoring: monitors the CPU usage, GPU usage, and memory usage of computing units to prevent system performance degradation or crashes due to insufficient resources. Feedback and decision-making: Based on the status monitoring results, the system automatically adjusts the working parameters; User interaction: Displays real-time system status information to users through a visual interface, including positioning trajectory, map, and sensor data; Users can perform manual intervention based on this information.

[0010] Preferably, the communication and interface module: uses a communication protocol to realize internal and external data transmission, and connects various sensors, actuators and cloud platforms through interfaces; External interface unit: provides the system with the ability to connect to external devices and systems to achieve function expansion and system integration.

[0011] Preferably, the power supply and hardware support module is composed of a power adapter, a battery pack and a power management module, and provides stable power for each module.

[0012] A dynamic positioning management method based on visual laser collaboration, comprising: Hardware perception: Vision sensors collect image data for target detection and motion estimation, while lidar acquires point cloud data for environment modeling and ranging. Multi-sensor fusion: Aligning visual and lidar data and integrating their information using temporal and spatial synchronization techniques; Positioning and Mapping: Using SLAM technology, combined with vision and LiDAR data, the system can achieve real-time positioning in unknown environments and build geometric maps; Data processing and control: Based on the positioning results and map data, the motion path is generated through the path planning algorithm, and the actuator is driven through navigation control.

[0013] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are: 1. The present invention effectively compensates for the shortcomings of a single sensor through the multi-mode fusion of visual and lidar data, combined with advanced SLAM technology and dynamic positioning and tracking algorithms, and achieves high-precision positioning in complex dynamic environments, with significantly reduced positioning errors. According to different application scenarios and environmental conditions, the invention flexibly selects sensor configuration, data fusion methods and algorithm strategies, and has good adaptability to lighting changes, texture feature differences, dynamic obstacles, etc.

[0014] 2. The present invention uses a complete status monitoring and feedback mechanism to monitor the operating status of each module of the system in real time, and promptly discover and handle faults and abnormal situations; the reliable design of the power supply and hardware support modules ensures the stable operation of the system in harsh environments, thereby improving the overall reliability and stability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Further details, features and advantages of the present application are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which: Figure 1 is a flow chart of the present invention; DETAILED DESCRIPTION Several embodiments of the present application will be described in more detail below with reference to the accompanying drawings so that those skilled in the art can implement the present application. The present application can be embodied in many different forms and for many different purposes and should not be limited to the embodiments described herein. These embodiments are provided to make the present application comprehensive and complete and to fully convey the scope of the present application to those skilled in the art. The embodiments do not limit the present application.

[0016] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. It will be further understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the relevant art and / or the context of this specification, and will not be interpreted in an idealized or overly formal sense unless expressly defined as such herein.

[0017] See also Figure 1 As shown, the present invention provides a technical solution: A dynamic positioning management system and method based on visual laser collaboration, comprising: Hardware perception module: uses visual sensors to collect image data for target detection and motion estimation, and lidar to obtain point cloud data for environment modeling and ranging; Specifically including: visual sensor unit and lidar unit; Visual sensor unit: captures environmental images and provides raw data for subsequent target detection, scene understanding, and motion estimation; Hardware composition: Camera Selection: Monocular cameras offer low cost and a simple structure, enabling object detection and depth estimation through deep learning algorithms. Binocular cameras utilize the principle of parallax to directly acquire depth information, offering excellent ranging accuracy. Multi-cameras combine the advantages of the first two, expanding the field of view and making them suitable for complex scenarios. For example, in autonomous driving, multiple cameras with different viewing angles are often used to provide 360-degree coverage around the vehicle. Furthermore, to cope with varying lighting conditions, some cameras are equipped with infrared filters, enabling operation at night or in low-light environments. Global shutter cameras can prevent image distortion of moving objects. Image acquisition card: responsible for converting the analog signal captured by the camera into a digital signal and transmitting it to the subsequent processing module. Its performance (such as sampling rate and resolution support) directly affects the image quality. Image processing flow: Preprocessing: The original image may contain noise, lens distortion, and other issues, so denoising (such as median filtering and Gaussian filtering) is required to remove random noise. Distortion correction is performed using techniques such as the Zhang Zhengyou calibration method to restore the image's true shape. Color space conversion (such as converting RGB to HSV) helps separate color information, facilitating subsequent target recognition. Feature extraction and target detection: Deep learning-based target detection algorithms are widely used in visual sensor units. For example, the YOLO (You Only Look Once) algorithm can quickly detect multiple targets in an image through a single convolutional neural network calculation, with high speed and accuracy. Faster R-CNN uses a region proposal network (RPN) to generate regions where targets may be present, and then performs fine classification and positioning, performing well in complex scenes. In addition, traditional feature point extraction algorithms (such as ORB and SIFT) can be used to estimate motion in image sequences, by tracking changes in feature points between adjacent frames and calculating the camera's motion parameters. LiDAR unit: By emitting laser pulses and receiving reflected signals, it constructs a 3D point cloud model of the environment, providing the system with accurate distance information and spatial structure data; Hardware type and features: Mechanical LiDAR: 360-degree scanning is achieved through mechanical rotating parts. The higher the number of lines (such as 16 lines, 32 lines, 64 lines, 128 lines), the denser the generated point cloud and the richer the environmental details, but the cost and volume are relatively high. Solid-state lidar: It has no mechanical rotating parts and has the advantages of small size, high reliability, and low cost. It is suitable for large-scale applications, but its scanning range and resolution still need to be improved. Hybrid solid-state lidar: It combines the advantages of mechanical and solid-state technologies, strikes a balance between performance and cost, and is a popular choice in the current market. Point cloud data processing: Preprocessing: The original point cloud data contains a large amount of redundant information and noise, so voxel filtering is required to reduce the data volume and remove outliers through statistical filtering. Clustering algorithms such as DBSCAN can segment the point cloud into different objects, facilitating subsequent obstacle detection and target recognition. Positioning and Environment Modeling: Positioning algorithms based on point cloud registration are the core of LiDAR positioning. For example, the Iterative Closest Point (ICP) algorithm calculates the sensor's pose by continuously iterating to find the best match between the current point cloud and the map point cloud. The Normal Distribution Transform (NDT) algorithm converts the point cloud into a probability distribution and achieves rapid registration by comparing distribution differences. Furthermore, LiDAR can construct high-precision 3D point cloud maps or 2D occupancy grid maps, visually displaying the distribution of obstacles in the environment and providing a basis for path planning. Multi-sensor fusion module: aligns visual and lidar data using temporal and spatial synchronization technology and integrates the information from both. Specifically including: time and space synchronization unit and data fusion algorithm unit; Temporal and spatial synchronization unit: This is the foundation of multi-sensor fusion, ensuring temporal and spatial consistency between visual and lidar data, and avoiding positioning errors and target misjudgment caused by asynchrony. Time synchronization: Hardware synchronization: GPIO (general purpose input and output) signals are used to trigger the camera and lidar to collect data synchronously, achieving precise synchronization at the hardware level and controlling the time error to the microsecond level. Timestamp alignment: Due to data transmission and processing delays, timestamps from different sensors need to be interpolated or filtered. For example, linear interpolation can be used to deduce the exact acquisition time of the data from adjacent timestamps to ensure data alignment on the time axis. Space synchronization: External parameter calibration: Using a specially designed calibration plate (such as a checkerboard pattern combined with a lidar calibration frame), by collecting images and point cloud data from multiple angles, and using techniques such as Zhang Zhengyou's calibration method, the relative rotation matrix and translation vector between the vision sensor and lidar are calculated to accurately determine the spatial positional relationship between the two. The calibration process requires strict control of environmental conditions to ensure the accuracy of the calibration results; Data fusion algorithm unit: organically combines visual and lidar perception data, leveraging the complementary advantages of the two sensors to improve the overall performance of the system; EarlyFusion: Principle: Projecting a laser point cloud onto an image plane generates a pseudo-color image or depth image, fusing the point cloud data with the image data on the same dimension. The powerful feature extraction capabilities of a convolutional neural network (CNN) are then leveraged to jointly process the fused data and extract richer feature information. Application example: The PointPainting algorithm maps point cloud features to image pixels, adds point cloud geometric information to each pixel, and then performs target detection through CNN, effectively improving the detection accuracy of small and occluded targets. LateFusion: Principle: Vision and LiDAR independently perform positioning and target detection. After obtaining their respective results, they are fused using algorithms such as the Extended Kalman Filter (EKF), Undestructive Kalman Filter (UKF), or Particle Filter. These filtering algorithms establish a system state model and observation model, performing a weighted fusion of measurements from different sensors to estimate the optimal system state. Application example: In mobile robot navigation, the visual odometry (VO) and laser odometry (LO) respectively calculate the robot's position and posture. By fusing their outputs through the EKF, the accumulation of their respective errors can be effectively suppressed, thereby improving positioning accuracy. Tight Fusion: Principle: During state estimation, the constraints of visual feature points and the laser point cloud are directly incorporated into a unified optimization framework. The system pose and map are jointly solved using algorithms such as graph optimization or nonlinear optimization (such as Bundle Adjustment). This fusion approach fully utilizes the raw data from both sensors, enabling higher-precision positioning and mapping. Application example: The Lego-LOAM algorithm combines high-precision geometric information from lidar with visual texture information. Through a tightly coupled optimization strategy, it achieves stable SLAM (Simultaneous Localization and Mapping) in complex dynamic environments. Positioning and Mapping Module: Utilizes SLAM technology, combined with vision and lidar data, to achieve real-time positioning of the system in unknown environments and build geometric maps; Specifically including: SLAM (Simultaneous Localization and Mapping) unit and dynamic positioning tracking unit; SLAM (Simultaneous Localization and Mapping) unit: enables the system to determine its own position in an unknown environment in real time and simultaneously build a map of the environment, providing a basis for subsequent navigation and decision-making; SLAM technology classification: Visual SLAM (VSLAM): Relying solely on visual sensors, it extracts feature points from images, calculates the camera's pose using feature matching and motion estimation algorithms, and constructs sparse or semi-dense maps. It is suitable for texture-rich indoor and outdoor scenes, such as the ORB-SLAM3 algorithm, which offers excellent real-time and robustness, and can process monocular, binocular, and RGB-D camera data. Laser SLAM (LSLAM): Based on lidar point cloud data, it generates high-precision 3D point cloud maps or 2D occupancy grid maps through point cloud registration and map construction algorithms. For example, the Cartographer algorithm uses submap stitching and loop closure detection to build a globally consistent map. The LOAM algorithm focuses on real-time performance and high precision, making it suitable for real-time-critical scenarios such as mobile robots. Multi-sensor SLAM: Fusion of visual and lidar data leverages the strengths of both. Lidar provides reliable geometric information in low-texture environments, while visual sensors can better identify and track moving objects in dynamic scenes. For example, the V-LOAM algorithm fuses the results of visual and laser odometry, achieving stable SLAM in complex environments through a tightly coupled optimization strategy. Map types and applications: Geometric map: includes point cloud map and occupancy grid map. Point cloud map intuitively displays the 3D structure of the environment and is suitable for obstacle detection and path planning; occupancy grid map divides the environment into grids, each grid representing the probability of occupancy in that area, facilitating local path planning and obstacle avoidance for robots. Semantic Map: Combined with object detection algorithms, semantic labels (such as "person," "car," and "chair") are added to map elements, enabling the system to understand the semantics of the environment and support more advanced decision-making tasks, such as cargo sorting and delivery in smart warehousing. Dynamic positioning and tracking unit: In a dynamic environment, it tracks changes in its own position and surrounding dynamic targets in real time to ensure the stability and reliability of the system in complex scenarios; Motion Modeling and Prediction: Motion Model: Common motion models include the uniform velocity model, uniform acceleration model, and IMU (Inertial Measurement Unit) pre-integration model. The uniform velocity model assumes that the object maintains a uniform velocity over a short period of time and is suitable for scenarios where the motion state is relatively stable. The uniform acceleration model takes into account the acceleration changes of the object and is closer to the actual motion situation. The IMU pre-integration model predicts the motion state of the object by integrating the IMU data and can quickly respond to changes in the object's posture and position. Prediction algorithm: Based on the motion model, the system's next state is predicted using algorithms such as Kalman filtering or particle filtering, providing prior information for subsequent data association and state updates. Data association and dynamic target processing: Data association: Accurately matching identical targets in sensor data at different times is key to dynamic positioning. Algorithms such as the Hungarian algorithm and the nearest neighbor (NN) algorithm are commonly used to solve data association problems. These algorithms calculate the similarity between feature points or point cloud clusters to determine whether they belong to the same target. Dynamic target detection and tracking: Identify and track dynamic targets using point cloud density changes, optical flow, or deep learning target tracking algorithms (such as SORT and DeepSORT). When a dynamic target is detected, the system adjusts its positioning strategy to avoid misidentifying it as a static obstacle. It also predicts the target's trajectory, providing a reference for path planning. Improved anti-interference and robustness: Robust loss function: During data processing, a robust loss function (such as Huber loss) is introduced to reduce the impact of abnormal data (such as mismatched feature points or noise points) on the positioning results and improve the system's anti-interference ability. Relocalization mechanism: When the accumulated positioning error becomes too large or sensor data is lost, the system uses a relocalization algorithm (such as appearance-based relocalization or geometry-based relocalization) to re-determine its own position and resume normal operation. Data processing and control module: Based on the positioning results and map data, the motion path is generated through the path planning algorithm, and the actuator is driven through navigation control; Specifically including: path planning and navigation unit and status monitoring and feedback unit; Based on the system's current positioning results and environmental map information, it plans a safe and efficient motion path from the starting point to the target point, and generates corresponding control instructions to drive the actuator to ensure that it can accurately reach the target position; Path planning algorithms are divided into global path planning and local path planning; Global path planning: Finding the optimal path from the starting point to the end point on a known global map. Common algorithms include the A algorithm and the Dijkstra algorithm. Learning-based path search algorithms can also be used. Taking the A algorithm as an example, its core is to evaluate the function To select the optimal path node, the evaluation function formula is: ; in represents the actual cost from the starting point to node n; represents the estimated cost from node n to the end point (heuristic function); Algorithm A maintains two sets: an open list (storing nodes to be evaluated) and a closed list (storing evaluated nodes). Each time a node is selected from the open list, The node with the smallest value is expanded until the end is found or the open list is empty; Learning-based path-finding algorithms, such as the Deep Q-Network (DQN), approximate the Q-value function by building a neural network. During training, the agent takes actions in the environment and receives rewards. The neural network parameters are updated through the backpropagation algorithm to make the Q-value function approach the optimal policy. The Q value update formula is as follows ; in and are the state and action at time t, is the reward value, is the learning rate, is the discount factor; Local path planning: In a dynamic environment, the system needs to respond to sudden obstacles in real time, and this is when the local path planning algorithm comes into play; Taking the Dynamic Window Algorithm (DWA) as an example, its basic idea is to sample different combinations of speed and angular velocity within the robot's kinematic constraints, evaluate whether the motion trajectory under each combination is safe, and select the optimal motion command; Navigation control: Generates accurate control instructions based on the planned path, drives actuators (such as motors and servos), and ensures the system moves accurately along the path. Control algorithms include PID control and pure tracking algorithms. Feedback regulation: real-time monitoring of the deviation between the actual position and the planned path, and dynamic adjustment of control parameters through feedback regulation; A closed-loop control strategy is usually used to continuously update the control instructions based on the deviation, so that the system gradually approaches the planned path. For example, in PID control, the control variable is continuously adjusted based on the error calculated in real time to reduce the position deviation. Status monitoring and feedback unit: monitors the system's operating status in real time, identifies potential problems promptly and takes corresponding measures to ensure reliable system operation; Condition monitoring indicators: Positioning accuracy evaluation: Evaluate positioning accuracy by estimating the covariance matrix of positioning results or calculating reprojection error; Sensor health check: Real-time monitoring of the sensor's operating status. If the LiDAR signal strength is below the normal range, it is considered that the LiDAR may be faulty or blocked. For cameras, the health status is determined by testing indicators such as image clarity and the presence of noise. System resource monitoring: monitors the CPU usage, GPU usage, and memory usage of computing units to prevent system performance degradation or crashes due to insufficient resources. Feedback and decision-making: Based on the status monitoring results, the system automatically adjusts the working parameters; User interaction: Displaying real-time system status information to users through a visual interface (such as RViz, MATLAB), including positioning trajectory, map, and sensor data; Users can perform manual intervention based on this information, such as manually adjusting path planning parameters (such as the heuristic function weight in the A* algorithm) or starting the system self-check program to enhance the flexibility and operability of the system; Communication and interface module: uses communication protocols to realize internal and external data transmission, and connects various sensors, actuators and cloud platforms through interfaces; Data transmission unit: realizes high-speed and stable data communication between modules within the system and between the system and external devices; Communication protocol: LAN communication: Within the system, data is transmitted using the TCP / IP or UDP protocols. The TCP / IP protocol provides reliable, connection-oriented communication and is suitable for scenarios requiring high data integrity, such as transmitting map data and control commands. The UDP protocol, with its fast transmission speed and low overhead, is suitable for transmitting sensor data with high real-time requirements (such as LiDAR point clouds and camera images). In the Robot Operating System (ROS), topics and services are widely used to implement inter-node communication. Bus communication: The CAN bus is commonly used to connect sensors and actuators. It has strong anti-interference capabilities and high real-time performance, making it suitable for industrial environments. The USB 3.0 interface enables high-speed data transmission and is commonly used to connect cameras, computing units, and other devices. The Ethernet interface supports long-distance, high-speed data transmission and is suitable for communication between the system and remote servers or other network devices. Wireless communication: Wireless communication technologies such as Wi-Fi, 5G, and Bluetooth enable remote monitoring and data upload. For example, in smart logistics, AGVs can transmit their location data and operating status to a cloud management platform in real time via 5G networks, enabling remote scheduling and monitoring. Bluetooth can be used for fast device pairing and short-distance data transmission, such as configuring sensor parameters. External interface unit: provides the system with the ability to connect with external devices and systems to achieve function expansion and system integration; Sensor expansion interface: Multiple standard interfaces (such as GPIO, I2C, and SPI) are reserved to support the connection of auxiliary sensors such as IMU, GPS, and ultrasonic sensors to further enrich the system's perception capabilities. For example, in autonomous driving scenarios, GPS can provide global positioning information, which complements the visual laser fusion positioning results to improve positioning accuracy and reliability. Actuator interface: Connects actuators such as robotic arms, mobile chassis, and industrial robots through PWM (pulse width modulation) interfaces and CAN bus interfaces to achieve automated operations under positioning guidance. For example, in industrial inspection, after a mobile robot equipped with a visual laser system locates a workpiece, it controls the robotic arm through the actuator interface for precise inspection and assembly. Cloud interface: Provides RESTful API, WebSocket and other interfaces to enable data interaction with the cloud management platform. The system can upload positioning data, operation logs, etc. to the cloud, and use cloud computing and big data analysis technologies for data mining and optimization. At the same time, the cloud platform can also issue task instructions and configuration parameters to the system, enabling remote management and intelligent decision-making. Power supply and hardware support module: consists of a power adapter, battery pack and power management module, providing stable power for each module; A dynamic positioning management method based on visual laser collaboration, comprising: Hardware perception: Vision sensors collect image data for target detection and motion estimation, while lidar acquires point cloud data for environment modeling and ranging. Multi-sensor fusion: Aligning visual and lidar data and integrating their information using temporal and spatial synchronization techniques; Positioning and Mapping: Using SLAM technology, combined with vision and LiDAR data, the system can achieve real-time positioning in unknown environments and build geometric maps; Data processing and control: Based on the positioning results and map data, the motion path is generated through the path planning algorithm, and the actuator is driven through navigation control.

[0018] The above formulas are obtained by collecting a large amount of data and performing software simulation, and a formula close to the actual value is selected. The influencing weight factors and specific coefficient values ​​in the formula are set by technical personnel in this field according to actual conditions, and can be adjusted and modified later.

[0019] The above description of the embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A dynamic positioning management system based on visual laser collaboration, characterized in that: include: Hardware perception module: uses visual sensors to collect image data for target detection and motion estimation, and lidar to obtain point cloud data for environment modeling and ranging; Multi-sensor fusion module: aligns visual and lidar data using temporal and spatial synchronization technology and integrates the information from both. Positioning and Mapping Module: Utilizes SLAM technology, combined with vision and lidar data, to achieve real-time positioning of the system in unknown environments and build geometric maps; Data processing and control module: Based on the positioning results and map data, the motion path is generated through the path planning algorithm, and the actuator is driven through navigation control.

2. A dynamic positioning management system based on visual laser collaboration according to claim 1, characterized in that: The hardware perception module specifically includes: visual sensor unit and lidar unit; Visual sensor unit: captures environmental images and provides raw data for subsequent target detection, scene understanding, and motion estimation; LiDAR unit: By emitting laser pulses and receiving reflected signals, it constructs a three-dimensional point cloud model of the environment, providing the system with accurate distance information and spatial structure data.

3. A dynamic positioning management system based on visual laser collaboration according to claim 2, characterized in that: The multi-sensor fusion module specifically includes: time and space synchronization unit and data fusion algorithm unit; Temporal and spatial synchronization unit: This is the foundation of multi-sensor fusion, ensuring temporal and spatial consistency between visual and lidar data, and avoiding positioning errors and target misjudgment caused by asynchrony. Data fusion algorithm unit: organically combines visual and lidar perception data to leverage the complementary advantages of the two sensors and improve overall performance.

4. A dynamic positioning management system based on visual laser collaboration according to claim 3, characterized in that: The positioning and mapping module specifically includes: SLAM (Simultaneous Localization and Mapping) unit and dynamic positioning tracking unit; SLAM (Simultaneous Localization and Mapping) unit: enables the system to determine its own position in an unknown environment in real time and simultaneously build a map of the environment, providing a basis for subsequent navigation and decision-making; Dynamic positioning tracking unit: In a dynamic environment, it tracks changes in its own position and surrounding dynamic targets in real time.

5. A dynamic positioning management system based on visual laser collaboration according to claim 4, characterized in that: The data processing and control module specifically includes: path planning and navigation unit and status monitoring and feedback unit; Based on the system's current positioning results and environmental map information, it plans a safe and efficient motion path from the starting point to the target point, and generates corresponding control instructions to drive the actuator movement; Path planning algorithms are divided into global path planning and local path planning; Global path planning: Finding the optimal path from the starting point to the end point on a known global map; commonly used algorithms include A algorithm and Dijkstra algorithm; Local path planning: In a dynamic environment, the system needs to respond to sudden obstacles in real time, and this is when the local path planning algorithm comes into play; Navigation control: Generates accurate control instructions based on the planned path, drives the actuators, and enables the system to accurately move along the path; control algorithms include PID control and pure tracking algorithms; Feedback regulation: real-time monitoring of the deviation between the actual position and the planned path, and dynamic adjustment of control parameters through feedback regulation; Status monitoring and feedback unit: monitors the system's operating status in real time, identifies potential problems promptly and takes corresponding measures to ensure reliable system operation; Condition monitoring indicators: Positioning accuracy evaluation: Evaluate positioning accuracy by estimating the covariance matrix of positioning results or calculating reprojection error; Sensor health check: Real-time monitoring of the sensor's operating status. If the LiDAR signal strength is below the normal range, it is considered that the LiDAR may be faulty or blocked. For cameras, the health status is determined by testing indicators such as image clarity and the presence of noise. System resource monitoring: monitors the CPU usage, GPU usage, and memory usage of computing units to prevent system performance degradation or crashes due to insufficient resources. Feedback and decision-making: Based on the status monitoring results, the system automatically adjusts the working parameters; User interaction: Displays real-time system status information to users through a visual interface, including positioning trajectory, map, and sensor data; Users can perform manual intervention based on this information.

6. A dynamic positioning management system based on visual laser collaboration according to claim 5, characterized in that: Communication and interface module: uses communication protocols to realize internal and external data transmission, and connects various sensors, actuators and cloud platforms through interfaces; External interface unit: provides the system with the ability to connect to external devices and systems to achieve function expansion and system integration.

7. A dynamic positioning management system based on visual laser collaboration according to claim 6, characterized in that: Power supply and hardware support module: consists of a power adapter, battery pack and power management module, providing stable power for each module.

8. A dynamic positioning management method based on visual laser collaboration, according to any one of claims 1-7, characterized in that: include: Hardware perception: Vision sensors collect image data for target detection and motion estimation, while lidar acquires point cloud data for environment modeling and ranging. Multi-sensor fusion: Aligning visual and lidar data and integrating their information using temporal and spatial synchronization techniques; Positioning and Mapping: Using SLAM technology, combined with vision and LiDAR data, the system can be positioned in real time in unknown environments and a geometric map can be constructed. Data processing and control: Based on the positioning results and map data, the motion path is generated through the path planning algorithm, and the actuator is driven through navigation control.

Images