Security inertia, vision and laser fusion positioning method and system for unmanned vehicle at wharf
By adopting multi-sensor fusion positioning method of satellite inertia, lidar and vision in the dock unmanned vehicles, combined with Kalman filtering and ICP matching algorithm, the problems of positioning accuracy and robustness in complex port scenarios are solved, and high Accuracy and flexibility for autonomous dock positioning.
Patent Information
- Application Number
- CN202411234187.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-04
- Publication Date
- 2024-11-26
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing dock unmanned vehicles are unable to meet the requirements of automated operations in complex port scenarios. The accuracy and reliability of a single navigation positioning technology in harsh environments are reduced, especially the cost of magnetic nail positioning and poor flexibility. GNSS The signal is affected by obstacles.
Using satellite inertial positioning, lidar and visual fusion positioning methods, the satellite inertial positioning information, lidar odometer information and visual precise positioning information are integrated through the distributed Kalman filtering fusion positioning framework, combined with the ICP matching algorithm and the extended Kalman filtering algorithm. Achieve multi-sensor fusion positioning to improve positioning accuracy and robustness.
It improves the positioning accuracy and stability of the dock unmanned vehicles, reduces civil engineering costs, adapts to complex environmental scenarios, reduces calculation errors and processor computing power requirements, and enhances positioning flexibility and adaptability.
Smart Images

Figure CN119022936A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of unmanned driving control of automated docks, and in particular to a method and system for integrating satellite, visual and laser positioning of unmanned vehicles at docks. Background Art
[0002] With the rapid growth of global trade, the operational efficiency and safety of terminals are receiving more and more attention. Traditional manually driven terminal vehicles can no longer meet the growing demand, especially during peak hours and in bad weather conditions, where the limitations and safety risks of manual operation are particularly prominent. Therefore, the development and application of unmanned horizontal transport equipment at terminals are becoming more and more extensive.
[0003] At present, many automated terminals have been built at home and abroad. The main way to locate their unmanned horizontal transport equipment is magnetic nail positioning, that is, the magnetic nails are pre-buried underground, and the positioning antenna detects the magnetic nail signals to obtain the number and location information of the magnetic nails, thereby achieving precise positioning and navigation. However, with the continuous advancement of technology and the support of policies and regulations, more and more terminals have begun to try to use unmanned positioning technologies such as Beidou positioning and laser positioning. The use of magnetic nail positioning technology has problems such as high civil engineering costs, poor vehicle driving flexibility, and large limitations in the operating area. Single positioning technology faces challenges in the road sections blocked by large equipment and containers at the terminal and in complex weather conditions. For example, GNSS signals may be affected by surrounding obstacles and limited, resulting in reduced positioning accuracy and reliability; visual recognition and laser scanning have advantages in close distance and fine positioning, but their performance degrades in insufficient light or complex environments. Therefore, in complex port scenarios, a single navigation and positioning technology is difficult to meet the centimeter-level position accuracy requirements of vehicles. Therefore, combining multiple positioning technologies to achieve complementary advantages is an important direction for the development of positioning technology. Summary of the invention
[0004] In view of this, it is necessary to propose a satellite, visual and laser fusion positioning method and system for terminal unmanned vehicles to address the above-mentioned problems, so as to overcome some shortcomings in the above-mentioned background technology and solve the following technical problems: how to improve the positioning robustness and positioning accuracy of terminal unmanned vehicles.
[0005] To achieve the above object, the present invention adopts the following technical solutions:
[0006] The present invention proposes a method for integrating satellite inertial positioning, vision and laser positioning of a dock unmanned vehicle, which is applied to a dock unmanned vehicle equipped with a satellite inertial positioning device, a laser radar device, a vehicle central control unit, a speed sensor, an angle sensor and a camera. The dock unmanned vehicle travels in a dock equipped with a plurality of positioning poles. The positioning method includes the following steps performed in sequence:
[0007] S1, based on the distributed Kalman filter fusion positioning framework, build a multi-sensor fusion positioning system model;
[0008] S2, integrating the satellite positioning information received from the satellite inertial positioning device, the inertial navigation position information received from the satellite inertial positioning device and the vehicle speed information, thereby obtaining the initial position of the terminal unmanned vehicle and the satellite inertial positioning information;
[0009] S3, integrating the laser radar odometer information received from the laser radar device, the pole point cloud matching positioning information received from the laser radar device, and the vehicle mileage information obtained from the vehicle central control unit, and then obtaining the laser radar positioning result through the extended Kalman filter algorithm fusion framework;
[0010] S4, collects images taken by the binocular camera for target detection, obtains visual precise positioning information through the binocular ranging model, and outputs the distance from the center of the vehicle to the left and right lane lines;
[0011] S5, transmitting the satellite inertial positioning information, the laser radar positioning result, the visual precise positioning information, the sensor information obtained from the speed sensor and the sensor information obtained from the angle sensor to the multi-sensor fusion positioning system model for integration to obtain the final fusion positioning result of the vehicle.
[0012] Further, step S3 includes the following sub-steps executed in sequence:
[0013] S301, matching the front and rear frame data of the laser radar through the ICP matching algorithm, calculating the displacement and rotation within the specified time, and then continuously matching the front and rear frames of the laser radar data through the ICP matching algorithm, continuously accumulating the translation and rotation data, and finally outputting the laser radar odometer information;
[0014] S302, using a total station to mark the location of the positioning poles, forming a pole point cloud format map in the dock coordinate system, and then using point cloud clustering and recognition algorithms to identify all the positioning poles, and finally using an ICP matching algorithm to match the pole map with the point cloud of the laser scan to obtain a laser positioning result;
[0015] S303, using the extended Kalman filter algorithm to fuse the lidar odometer information, the pole point cloud matching positioning information, and the vehicle mileage information to obtain the final lidar positioning result.
[0016] Further, step S4 includes the following sub-steps executed in sequence:
[0017] S401, storing the precise coordinates of the diamonds of each terminal driving lane in the terminal coordinate system, and using the diamond solid pattern of the terminal driving lane as a landmark;
[0018] S402, collecting images through a shooting device;
[0019] S403, performing target detection by using a gradient histogram detection method, an image pyramid detection method, and a sliding window detection method, that is, determining whether a designated area of the image contains an object to be identified;
[0020] S404, performing visual precise positioning by comparing the stored coordinates of the diamond pattern with the binocular ranging result of the similar triangle model.
[0021] Further, step S303 includes the following sub-steps executed in sequence:
[0022] Step S3031, using a nonlinear state transfer equation to predict the state of the next time step;
[0023] Step S3032, through covariance prediction: update the prediction covariance matrix to reflect the uncertainty of the prediction state;
[0024] Step S3033, observation fusion: The final LiDAR positioning result is obtained by fusing the observations of the satellite inertial positioning device, the LiDAR device and the camera device.
[0025] The present invention further proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the satellite, visual and laser fusion positioning method for a terminal unmanned vehicle as described in any of the above items are implemented.
[0026] The present invention further proposes an electronic device, comprising a storage device, a processor, and a computer program stored in the storage device and executable by the processor. When the processor executes the computer program, the steps of the satellite, visual and laser fusion positioning method for the terminal unmanned vehicle as described in any of the above items are implemented.
[0027] The present invention further proposes a satellite, visual and laser fusion positioning system for an unmanned dock vehicle, the positioning system comprising:
[0028] Model building module, used to build a multi-sensor fusion positioning system model based on the distributed Kalman filter fusion positioning framework;
[0029] The satellite-inertial positioning module is used to integrate the received satellite positioning information, inertial navigation position and vehicle speed information to obtain the initial position of the vehicle;
[0030] Laser positioning module, which is used to integrate laser radar odometer information, pole point cloud matching positioning, and vehicle odometer information, and obtain laser radar positioning results through the extended Kalman filter algorithm fusion framework;
[0031] The visual positioning module is used to collect images taken by the binocular camera for target detection, obtain visual precise positioning information through the binocular ranging model, and output the distance from the center of the vehicle to the left and right lane lines;
[0032] The fusion positioning module is used to transmit the satellite inertial positioning information, the laser radar positioning result, the visual precise positioning information, the sensor information obtained from the speed sensor and the sensor information obtained from the angle sensor to the multi-sensor fusion positioning system model for integration to obtain the final fusion positioning result of the vehicle.
[0033] Furthermore, the positioning system communicates with multiple terminal driverless vehicles traveling in a terminal equipped with several positioning poles. Each terminal driverless vehicle is equipped with a satellite inertial positioning device, a laser radar device, a vehicle central control unit for obtaining vehicle mileage information, and a shooting device.
[0034] Furthermore, the laser positioning module includes the following submodules:
[0035] The submodule for obtaining the positioning result of the laser radar odometer information matches the laser radar front and back frame data through the ICP matching algorithm, calculates the displacement and rotation within the specified time, and then continuously matches the front and back frames of the laser radar data through the ICP matching algorithm, continuously accumulates the translation and rotation data, and finally outputs the laser radar odometer information;
[0036] The matching positioning result acquisition submodule uses the total station to mark the location of the positioning poles to form a pole point cloud format map in the dock coordinate system. Then, the positioning poles are identified through point cloud clustering and recognition algorithms. Finally, the ICP matching algorithm is used to match the pole map with the point cloud of the laser scan to obtain the laser positioning result.
[0037] The LiDAR positioning result acquisition submodule uses the extended Kalman filter algorithm to fuse the LiDAR odometer information, pole point cloud matching positioning information, and vehicle mileage information to obtain the final LiDAR positioning result.
[0038] Furthermore, the visual positioning module includes the following submodules:
[0039] The marker determination submodule stores the precise coordinates of the diamonds of each terminal driving lane in the terminal coordinate system, and uses the diamond solid pattern of the terminal driving lane as a marker;
[0040] An image acquisition and processing submodule, which acquires images through a shooting device;
[0041] The target detection processing submodule performs target detection by using the detection methods of gradient histogram, image pyramid and sliding window, i.e., determines whether the specified area of the image contains the object to be identified;
[0042] The visual precise positioning processing submodule performs visual precise positioning by comparing the coordinates of the stored diamond pattern with the binocular ranging results of the similar triangle model.
[0043] The beneficial effects of the present invention are:
[0044] The present invention overcomes the shortcomings of poor positioning robustness, inability of accuracy and stability of unmanned vehicles with single positioning technology to meet the operational requirements of automated terminals in complex port scenarios. It saves more civil engineering costs than the existing technical solutions of magnetic nail positioning, and has the advantages of higher flexibility, strong stability, high positioning accuracy, and adaptability to different complex environmental scenarios of ports. The present invention is suitable for the fusion positioning method of unmanned vehicles in ports. The final laser radar positioning result is obtained by fusing the laser radar odometer information, the pole point cloud matching positioning information, and the vehicle mileage information through the extended Kalman filter algorithm, that is, by first fusing the two sets of laser radar data and the vehicle odometer data, and then outputting the laser positioning, and fusing the output laser positioning with other positioning results again, thereby further improving the positioning robustness. The present invention can obtain accurate positioning of the radar in the real world, reduce complex dynamic mapping, key frame and current frame matching processes, reduce calculation errors, improve positioning accuracy, and reduce processor computing power requirements. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] Figure 1 The present invention is a structural principle diagram of a satellite, visual and laser fusion positioning system for an unmanned dock vehicle. DETAILED DESCRIPTION
[0046] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be further clearly and completely described in combination with the embodiments of the present invention. It should be noted that the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.
[0047] It should be understood that the directions or positional relationships indicated by terms such as "upper", "lower", "front", "back", "left", and "right" are based on the directions or positional relationships shown in the accompanying drawings and are only for the convenience of describing the present invention and simplifying the description. They do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operated in a specific direction. Therefore, they should not be understood as limitations on the present invention.
[0048] The terms "first", "second", "third", "fourth", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the definition of "first", "second", "third", "fourth" features may explicitly or implicitly include one or more of the features.
[0049] Example 1
[0050] This embodiment proposes a method for integrating satellite inertial positioning, vision and laser positioning of a dock driverless vehicle, which is applied to a dock driverless vehicle equipped with a satellite inertial positioning device, a laser radar device, a vehicle central control unit, a speed sensor, an angle sensor and a camera. The dock driverless vehicle travels in a dock equipped with a plurality of positioning poles. The positioning method includes the following steps performed in sequence:
[0051] S1, based on the distributed Kalman filter fusion positioning framework, build a multi-sensor fusion positioning system model;
[0052] S2, integrating the satellite positioning information received from the satellite inertial positioning device, the inertial navigation position information received from the satellite inertial positioning device and the vehicle speed information (the satellite inertial positioning device has the speed measurement capability, and the vehicle speed information is derived from the satellite inertial positioning device), thereby obtaining the initial position of the terminal unmanned vehicle and the satellite inertial positioning information;
[0053] S3, integrating the laser radar odometer information received from the laser radar device, the pole point cloud matching positioning information received from the laser radar device, and the vehicle mileage information obtained from the vehicle central control unit, and then obtaining the laser radar positioning result through the extended Kalman filter algorithm fusion framework;
[0054] S4, collects images taken by a binocular camera (a binocular camera is a special shooting device with two cameras arranged on the same horizontal line) to perform target detection, obtains visual precise positioning information through a binocular ranging model, and outputs the distance from the center of the vehicle to the left and right lane lines;
[0055] S5, transmitting the satellite inertial positioning information, the laser radar positioning result, the visual precise positioning information, the sensor information obtained from the speed sensor and the sensor information obtained from the angle sensor to the multi-sensor fusion positioning system model for integration to obtain the final fusion positioning result of the vehicle.
[0056] Specifically, the satellite inertial positioning information integrates satellite positioning information, inertial navigation position information and vehicle speed information.
[0057] Optimally, step S3 includes the following sub-steps performed in sequence:
[0058] S301, matching the front and rear frame data of the laser radar through the ICP matching algorithm, calculating the displacement and rotation within the specified time, and then continuously matching the front and rear frames of the laser radar data through the ICP matching algorithm, continuously accumulating the translation and rotation data, and finally outputting the laser radar odometer information;
[0059] S302, using a total station to mark the location of the positioning poles, forming a pole point cloud format map in the dock coordinate system, and then using point cloud clustering and recognition algorithms to identify all the positioning poles, and finally using an ICP matching algorithm to match the pole map with the point cloud of the laser scan to obtain a laser positioning result (the laser positioning result is the position of the unmanned vehicle in the pole point cloud format map in the dock coordinate system);
[0060] S303, using the extended Kalman filter algorithm to fuse the lidar odometer information, the pole point cloud matching positioning information, and the vehicle mileage information to obtain the final lidar positioning result.
[0061] Specifically, S301 utilizes the characteristic that the environment of the dock remains fixed within a very short period of time (0.1 s).
[0062] Specifically, in S302, some positioning poles are placed at the corresponding wharf, and the positions of the poles are marked by a total station to form a point cloud format map of the poles in the wharf coordinate system. Then, the positioning poles are identified through point cloud clustering and recognition algorithms, and finally the ICP matching algorithm is used to match the pole map and the point cloud to obtain the positioning result.
[0063] Optimally, step S4 includes the following sub-steps performed in sequence:
[0064] S401, storing the precise coordinates of the diamonds of each terminal driving lane in the terminal coordinate system, and using the diamond solid pattern of the terminal driving lane as a landmark;
[0065] S402, collecting images through a shooting device;
[0066] S403, performing target detection by using a Histogram of Oriented Gradient (HOG) detection method, an image pyramid detection method, and a sliding window detection method, that is, determining whether a specified area of the image contains an object to be identified;
[0067] S404, performing visual precise positioning by comparing the stored coordinates of the diamond pattern with the binocular ranging result of the similar triangle model.
[0068] Specifically, the feature descriptor obtained from the gradient histogram can become important information for feature matching and target detection; the image pyramid is a structure that interprets images at multiple resolutions. By performing multi-scale pixel sampling on the original image, N images of different resolutions are generated to solve target detection at different scales; the sliding window includes sliding window detection and multi-scale detection of each part using an image pyramid, decomposing the image into multiple parts, then discarding those parts that are unlikely to contain objects, and classifying possible areas. The sliding window solves the positioning problem by scanning smaller areas of a larger image and then repeatedly scanning the same image at different scales.
[0069] Further optimized, step S303 includes the following sub-steps:
[0070] ① Status prediction:
[0071] Use the nonlinear state transfer equation f to predict the state of the next time step. The formula for the prediction step is:
[0072]
[0073] in, is the predicted state at time k based on the information at time k-1, u k is the control input (may include vehicle speed, steering angle, etc.), f is the nonlinear state transfer function;
[0074] ② Covariance prediction:
[0075] Update the forecast covariance matrix P to reflect the uncertainty of the forecast state:
[0076] P k∣k-1 =F k P k-1∣k-1 F k T +Q k
[0077] Among them, F k is f in The Jacobian matrix at (for the linearization process in practical applications), Q k is the process noise covariance matrix;
[0078] ③Observation fusion, that is, obtaining the final lidar positioning result by fusing the observations of the satellite inertial positioning device, lidar device and camera device;
[0079] For the laser radar odometer, pole point cloud matching positioning, and vehicle odometer corresponding observation equation z i =h i (x)+v i , where zi is the observed value of the sensor, h i is the observation function, v i is the observation noise;
[0080] Define an extended observation vector z and the corresponding observation matrix H to fuse the observation data of multiple sensors:
[0081]
[0082] Among them, z1, z2, z3 come from the observation values of the laser radar odometer, the pole point cloud matching positioning and the vehicle odometer respectively, and H1, H2, H3 are their respective The Jacobian matrix at ;
[0083] The general formula for observation fusion is:
[0084] Observation residuals: Covariance update:
[0085] P k∣k =(IK k H k ) k∣k-1 (IK k H k ) T +K k R k K k T
[0086] After all the sensor observations are integrated, we get This is the final lidar positioning result, which contains the optimal state estimate of the vehicle at the current moment.
[0087] The present invention uses a laser radar to scan the surrounding environmental features and landmark features, and matches them with the pre-entered pole point cloud format map through point cloud clustering and recognition algorithms, thereby accurately positioning the radar in the real world, reducing complex dynamic mapping, key frame and current frame matching processes, reducing calculation errors, improving positioning accuracy, and reducing processor computing power requirements.
[0088] Example 2
[0089] This embodiment proposes a computer-readable storage medium, which stores a computer program, characterized in that when the computer program is executed by a processor, the steps of the satellite, visual and laser fusion positioning method for the unmanned terminal vehicle as described in any one of Embodiment 1 are implemented.
[0090] Example 3
[0091] This embodiment proposes an electronic device, including a storage device, a processor, and a computer program stored in the storage device and executable by the processor. When the processor executes the computer program, the steps of the satellite, visual and laser fusion positioning method for the unmanned terminal vehicle as described in any one of Embodiment 1 are implemented.
[0092] Example 4
[0093] like Figure 1 As shown:
[0094] This embodiment proposes a satellite, inertial, visual and laser fusion positioning system for an unmanned dock vehicle, the positioning system comprising:
[0095] Model building module, used to build a multi-sensor fusion positioning system model based on the distributed Kalman filter fusion positioning framework;
[0096] The satellite-inertial positioning module is used to integrate the received satellite positioning information, inertial navigation position and speed information to obtain the initial position of the vehicle;
[0097] Laser positioning module, which is used to integrate laser radar odometer information, pole point cloud matching positioning, and vehicle odometer information, and obtain laser radar positioning results through the extended Kalman filter algorithm fusion framework;
[0098] The visual positioning module is used to collect images taken by the binocular camera for target detection, obtain accurate visual positioning through the binocular ranging model, and output the distance from the center of the vehicle to the left and right lane lines;
[0099] The fusion positioning module is used to transmit the satellite inertial positioning information, the laser radar positioning result, the visual precise positioning information, the sensor information obtained from the speed sensor and the sensor information obtained from the angle sensor to the multi-sensor fusion positioning system model for integration to obtain the final fusion positioning result of the vehicle.
[0100] Optimally, the positioning system communicates with multiple terminal driverless vehicles traveling in a terminal equipped with several positioning poles. Each terminal driverless vehicle is equipped with a satellite inertial positioning device, a laser radar device, a vehicle central control unit for obtaining vehicle mileage information, and a camera.
[0101] Optimally, the laser positioning module includes the following submodules:
[0102] The submodule for obtaining the positioning result of the laser radar odometer information matches the laser radar front and back frame data through the ICP matching algorithm, calculates the displacement and rotation within the specified time, and then continuously matches the front and back frames of the laser radar data through the ICP matching algorithm, continuously accumulates the translation and rotation data, and finally outputs the laser radar odometer information;
[0103] The matching positioning result acquisition submodule uses the total station to mark the location of the positioning poles to form a pole point cloud format map in the dock coordinate system, and then uses the point cloud clustering and recognition algorithm to identify the positioning poles. Finally, the ICP matching algorithm is used to match the pole map and the point cloud of the laser scan to obtain the laser positioning result (the laser positioning result is the position of the unmanned vehicle in the pole point cloud format map in the dock coordinate system);
[0104] The LiDAR positioning result acquisition submodule uses the extended Kalman filter algorithm to fuse the LiDAR odometer information, pole point cloud matching positioning information, and vehicle mileage information to obtain the final LiDAR positioning result.
[0105] Optimally, the visual positioning module includes the following submodules:
[0106] The marker determination submodule stores the precise coordinates of the diamonds of each terminal driving lane in the terminal coordinate system, and uses the diamond solid pattern of the terminal driving lane as a marker;
[0107] An image acquisition and processing submodule, which acquires images through a shooting device;
[0108] The target detection processing submodule performs target detection by using the detection methods of gradient histogram, image pyramid and sliding window, i.e., determines whether the specified area of the image contains the object to be identified;
[0109] The visual precise positioning processing submodule performs visual precise positioning by comparing the coordinates of the stored diamond pattern with the binocular ranging results of the similar triangle model.
[0110] Further optimized, the satellite inertial, visual and laser fusion positioning system for a dock unmanned vehicle of this embodiment applies a satellite inertial, visual and laser fusion positioning method for a dock unmanned vehicle as described in any technical solution of Example 1.
[0111] Example 5
[0112] Embodiment 5 is a further optimization of Embodiment 1;
[0113] In S5, the positioning is based on sensor perception information, laser radar, and the modeling of the visual camera device is complex, and the system is always in a nonlinear state. Therefore, a distributed Kalman filter fusion positioning framework is used as the basis, and the kinematic model of the vehicle is reasonably modeled. Sensors such as IMU, gyroscope, and encoder are used to estimate the position and posture of the IGV, and the positioning results of different positioning sources such as satellite inertial positioning, laser positioning, and visual positioning are used as observation values to participate in fusion to obtain the final positioning result of the vehicle;
[0114] Multiple sensors of the same or different types obtain different local and category information respectively. This information may complement each other or may be redundant and contradictory. The control center ultimately only needs one correct result, which requires the fusion of the information obtained by multiple sensors. Through reasonable fusion, multi-sensor fusion can significantly improve the redundancy and fault tolerance of the system.
[0115] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.
Claims
1. A method for integrating satellite inertial positioning, vision, and laser positioning for a dockless vehicle, applied to a dockless vehicle equipped with a satellite inertial positioning device, a laser radar device, a vehicle central control unit, a speed sensor, an angle sensor, and a camera. The dockless vehicle travels within a dock equipped with several positioning poles. The method is characterized by: The positioning method includes the following steps performed in sequence: S1, based on the distributed Kalman filter fusion positioning framework, builds a multi-sensor fusion positioning system model; S2, integrating the satellite positioning information received from the satellite inertial positioning device, the inertial navigation position information received from the satellite inertial positioning device, and the vehicle speed information, thereby obtaining the initial position of the terminal unmanned vehicle and the satellite inertial positioning information; S3 integrates the lidar odometry information received from the lidar device, the pole point cloud matching positioning information received from the lidar device, and the vehicle mileage information obtained from the vehicle's central control unit, and then obtains the lidar positioning result through the extended Kalman filter algorithm fusion framework; S4 collects images taken by the binocular camera for target detection, obtains visual precise positioning information through the binocular ranging model, and outputs the distance from the vehicle center to the left and right lane lines; S5, transmitting the satellite inertial positioning information, the lidar positioning result, the visual precise positioning information, the sensor information obtained from the speed sensor, and the sensor information obtained from the angle sensor to the multi-sensor fusion positioning system model for integration to obtain the final fusion positioning result of the vehicle.
2. The method for integrating satellite, visual and laser positioning for unmanned terminal vehicles according to claim 1 is characterized in that: Step S3 includes the following sub-steps performed in sequence: S301, using the ICP matching algorithm to match the front and back frames of the LiDAR data, calculate the displacement and rotation within a specified time, then continuously use the ICP matching algorithm to match the front and back frames of the LiDAR data, continuously accumulate the translation and rotation data, and finally output the LiDAR odometry information; S302: Mark the positions of the positioning poles using a total station to form a point cloud map of the poles in the dock coordinate system. Then, all the positioning poles are identified using a point cloud clustering and recognition algorithm. Finally, the ICP matching algorithm is used to match the pole map with the point cloud of the laser scan to obtain the laser positioning result. S303: Using the extended Kalman filter algorithm to fuse the lidar odometer information, the pole point cloud matching positioning information, and the vehicle mileage information, the final lidar positioning result is obtained.
3. The method for integrating satellite, visual and laser positioning for unmanned terminal vehicles according to claim 1 is characterized in that: Step S4 includes the following sub-steps performed in sequence: S401, storing the precise coordinates of the diamonds of each terminal driving lane in the terminal coordinate system, and using the solid diamond patterns of the terminal driving lanes as landmarks; S402, capturing images through a camera; S403, performing target detection by using a gradient histogram detection method, an image pyramid detection method, and a sliding window detection method, i.e., determining whether a specified area of the image contains an object to be identified; S404 , performing visual precise positioning by comparing the stored coordinates of the diamond pattern with the binocular ranging result of the similar triangle model.
4. The method for integrating satellite, inertial, visual and laser positioning for unmanned terminal vehicles according to claim 2 is characterized in that: Step S303 includes the following sub-steps executed in sequence: Step S3031, using a nonlinear state transfer equation to predict the state of the next time step; Step S3032, through covariance prediction: update the prediction covariance matrix to reflect the uncertainty of the prediction state; Step S3033, observation fusion: The final LiDAR positioning result is obtained by fusing the observations of the satellite inertial positioning device, the LiDAR device and the camera device.
5. A computer-readable storage medium storing a computer program, wherein: When the computer program is executed by a processor, the steps of the satellite, vision and laser fusion positioning method for a terminal unmanned vehicle are implemented as described in any one of claims 1 to 4.
6. An electronic device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable by the processor. When the processor executes the computer program, the steps of the satellite, vision, and laser fusion positioning method for a terminal unmanned vehicle are implemented as described in any one of claims 1 to 4.
7. A satellite, inertial, visual and laser fusion positioning system for unmanned dock vehicles, characterized by: The positioning system includes: Model building module, used to build a multi-sensor fusion positioning system model based on the distributed Kalman filter fusion positioning framework; The satellite-inertial positioning module is used to integrate the received satellite positioning information, inertial navigation position and speed information to obtain the initial position of the vehicle; The laser positioning module is used to integrate the laser radar odometry information, the pole point cloud matching positioning, and the vehicle odometry information, and obtain the laser radar positioning results through the extended Kalman filter algorithm fusion framework; The visual positioning module is used to collect images taken by the binocular camera for target detection. Through the binocular ranging model, it obtains precise visual positioning and outputs the distance from the vehicle center to the left and right lane lines. The fusion positioning module is used to transmit satellite inertial positioning information, lidar positioning results, visual precise positioning information, sensor information obtained from the speed sensor and sensor information obtained from the angle sensor to the multi-sensor fusion positioning system model for integration to obtain the final fusion positioning result of the vehicle.
8. The satellite, inertial, visual and laser fusion positioning system for unmanned terminal vehicles according to claim 7 is characterized in that: The positioning system communicates with multiple terminal unmanned vehicles traveling in a terminal equipped with several positioning poles. Each terminal unmanned vehicle is equipped with a satellite inertial positioning device, a lidar device, a vehicle central control unit for obtaining vehicle mileage information, and a camera device.
9. The satellite, inertial, visual and laser fusion positioning system for unmanned terminal vehicles according to claim 8 is characterized in that: The laser positioning module includes the following submodules: The LiDAR odometry information positioning result acquisition submodule uses the ICP matching algorithm to match the LiDAR front and back frame data, calculates the displacement and rotation within a specified time, and then continuously matches the front and back frames of the LiDAR data using the ICP matching algorithm, continuously accumulating translation and rotation data, and finally outputs the LiDAR odometry information. The matching positioning result acquisition submodule uses a total station to mark the location of the positioning poles, forming a point cloud map of the poles in the dock coordinate system. Then, through point cloud clustering and recognition algorithms, all positioning poles are identified. Finally, the ICP matching algorithm is used to match the pole map with the point cloud of the laser scan to obtain the laser positioning result. The LiDAR positioning result acquisition submodule uses the extended Kalman filter algorithm to fuse the LiDAR odometer information, the pole point cloud matching positioning information, and the vehicle mileage information to obtain the final LiDAR positioning result.
10. The satellite, inertial, visual and laser fusion positioning system for unmanned terminal vehicles according to claim 8 is characterized in that: The visual positioning module includes the following submodules: The landmark determination submodule stores the precise coordinates of the diamonds of each terminal driving lane in the terminal coordinate system and uses the diamond solid pattern of the terminal driving lane as a landmark; An image acquisition and processing submodule, which acquires images through a shooting device; The target detection processing submodule performs target detection through gradient histogram detection, image pyramid detection, and sliding window detection, that is, determining whether the specified area of the image contains the object to be identified; The visual precise positioning processing submodule performs visual precise positioning by comparing the coordinates of the stored diamond pattern with the binocular ranging results of the similar triangle model.
Citation Information
Patent Citations
AGV transport system based on vision guidance and driving control method thereof
CN107422730A
Vehicle positioning method and device, electronic equipment and storage medium
CN109059906A
Terrain sensing method and system thereof, storage medium and computer equipment
CN113465728A
Millimeter wave radar self-positioning and mapping method based on ground passive positioning beacon
CN114942414A
KR20200036405A
Cited By
Four-section type alignment parking method for transport vehicle and gantry crane
CN119705426A
Multi-target tracking method and system based on laser radar
CN122330905A
