A multi-sensor fusion multi-scenario unmanned vibratory roller

Abstract

The present invention discloses a multi-sensor fusion multi-scenario unmanned vibratory roller, GPS, used to obtain the positioning accuracy of the roller's location and the roller's location information; visual camera, used to obtain image information of the construction area where the roller is located; laser radar, used to obtain laser point cloud information of the construction area where the roller is located; wheel odometer, used to obtain the travel distance information of the roller during construction; inertial measurement unit, used to obtain the posture information of the roller during construction; vehicle-mounted computing unit, used to select the first positioning navigation mode when the positioning accuracy is within the first preset accuracy range; select the second positioning navigation mode when the positioning accuracy is within the second preset accuracy range; select the third positioning navigation mode when the positioning accuracy is within the third preset accuracy range. The present invention realizes the autonomous switching mode of the unmanned roller, and autonomously plans the rolling operation with high precision and high efficiency in multiple scenarios.

Description

Technical Field

[0001] The present invention belongs to the technical field of road rollers, and in particular relates to a multi-sensor fusion multi-scenario unmanned vibratory road roller. Background Art

[0002] In recent years, my country's investment in large projects such as highways, railways, and airports has increased year by year, and higher requirements have been put forward for construction quality and efficiency. As an indispensable engineering machinery, rollers are responsible for compacting roadbeds and pavements. The level of construction of vibratory rollers during road construction directly determines the quality of roadbeds and pavements. At present, the compaction quality problem of construction roads in my country is still prominent. The roller operators cannot strictly follow the prescribed compaction process to perform rolling operations. The main manifestations are: the driver will feel tired after a long period of construction, use small amplitude instead of large amplitude operation, arbitrarily speed up the rolling speed, reduce the number of rolling times, and cause problems such as underpressure, underpressure, and overpressure on the roadbed and pavement; the driver cannot accurately control the starting speed of the roller, and too fast or too slow will lead to substandard compaction quality; during the construction process, the roller needs to change lanes and ensure an overlap of about 20cm after changing lanes. This link will involve steering problems. As an articulated engineering vehicle, the steering angle of the roller is difficult to control, and it is completely completed by the driver's visual ability and operating experience; the driver cannot control the compaction degree in real time, and technical personnel are required to repeatedly measure the compaction degree, which seriously affects the construction efficiency; the harsh construction environment and frequent safety accidents are problems that rollers often face, which pose a serious threat to the personal safety of the driver. Therefore, it is urgent to design and develop unmanned vibratory rollers to replace traditional vibratory rollers for compaction operations.

[0003] With the rapid development of sensors, big data, the Internet of Things, and artificial intelligence technologies, major construction machinery manufacturers have begun to apply these emerging technologies to the research and development of unmanned rollers. Unmanned driving has the advantages of high efficiency, low error, informatization of construction data, and real-time online monitoring of construction quality. It can maximize the guarantee of construction quality and avoid the above problems caused by manual driving. At present, there are unmanned rollers based on GPS technology, autonomous operation of rollers based on machine vision, and detection of obstacles around unmanned rollers based on millimeter-wave radar and ultrasonic radar. For example: unmanned rollers based on GPS technology can only be used in operating scenarios with good GPS signals, and will immediately fail in tunnels or mountains; unmanned rollers based on machine vision can only work in tunnel operating scenarios, but they cannot achieve high-precision unmanned construction operations with multiple scenes switched at will for a single roller, and cannot fundamentally improve construction efficiency and give full play to the greatest advantages of unmanned driving. Summary of the invention

[0004] In response to the problems existing in the prior art, the present invention provides a multi-scenario unmanned vibratory roller with multi-sensor fusion, which realizes the autonomous switching mode of the unmanned roller and autonomously plans the rolling operations with high precision and high efficiency in multiple scenarios, thereby avoiding the problem that previous unmanned rollers could only perform unmanned compaction operations in a single scenario, thereby increasing the use scenarios and application scope of the unmanned roller, thereby ensuring the safety of construction personnel and improving construction quality and efficiency.

[0005] In order to solve the above technical problems, the present invention is implemented by the following technical solutions:

[0006] A multi-sensor fusion multi-scenario unmanned vibratory roller, comprising:

[0007] GPS, used to obtain the positioning accuracy of the roller's location and the roller's location information;

[0008] A visual camera is used to obtain image information of the construction area where the roller is located;

[0009] LiDAR, used to obtain laser point cloud information of the construction area where the roller is located;

[0010] Wheel odometer, used to obtain the travel distance information of the roller during construction;

[0011] Inertial measurement unit, used to obtain the position information of the roller during construction;

[0012] The vehicle-mounted computing unit is used to select a first positioning navigation mode, a second positioning navigation mode or a third positioning navigation mode according to the positioning accuracy, and when the positioning accuracy is within a first preset accuracy range, the first positioning navigation mode is selected; when the positioning accuracy is within a second preset accuracy range, the second positioning navigation mode is selected; when the positioning accuracy is within a third preset accuracy range, the third positioning navigation mode is selected;

[0013] The first positioning and navigation mode is to directly obtain a satellite map corresponding to the construction area where the roller is located, and perform positioning and navigation according to the location information of the roller and the satellite map corresponding to the construction area where the roller is located;

[0014] The second positioning and navigation mode is to obtain first fused position information of the road roller by fusing the position information of the road roller, the image information, the laser point cloud information, the travel distance information and the posture information, and to construct an octree map using the image information and the laser point cloud information, and to perform positioning and navigation according to the first fused position information of the road roller and the octree map;

[0015] The third positioning and navigation mode is to obtain the second fused position information of the roller by fusing the image information, the laser point cloud information, the driving distance information and the posture information, and to construct an octree map using the image information and the laser point cloud information, and to perform positioning and navigation based on the second fused position information of the roller and the octree map.

[0016] Furthermore, the step of fusing the position information of the road roller, the image information, the laser point cloud information, the travel distance information and the posture information to obtain first fused position information of the road roller includes:

[0017] Continuous image frames are read from the image information, feature points of each frame are extracted using the ORB operator, and feature points of two adjacent frames are matched to construct a least squares problem. The least squares problem is solved using the ICP algorithm to obtain a posture transformation matrix of two adjacent frames. Based on the EKF algorithm, the driving distance information, the laser point cloud information and the posture information are fused to obtain a priori posture of the roller. According to the posture transformation matrix and the prior posture at different times, nonlinear optimization is performed using a method combining BA with a Ceres library, and loop detection is performed using the position information of the roller to finally obtain the first fused position information of the roller.

[0018] Furthermore, the fusing of the image information, the laser point cloud information, the travel distance information and the position information to obtain the second fused position information of the road roller includes:

[0019] Continuous image frames are read from the image information, feature points of each frame are extracted using the ORB operator, and feature points of two adjacent frames are matched to construct a least squares problem. The least squares problem is solved using the ICP algorithm to obtain a posture transformation matrix of two adjacent frames. Based on the EKF algorithm, the driving distance information, the laser point cloud information and the posture information are fused to obtain a priori posture of the roller. According to the posture transformation matrix and the prior posture at different times, nonlinear optimization is performed using a method combining BA with a Ceres library to finally obtain second fused position information of the roller.

[0020] Furthermore, the constructing an octree map using the image information and the laser point cloud information includes:

[0021] Set the global map coordinate system;

[0022] Performing TF transformation on the image information and the laser point cloud information to obtain a point cloud map based on a global map coordinate system;

[0023] The point cloud map calls the Octomap library to obtain an octree map.

[0024] Furthermore, the positioning and navigation based on the location information of the road roller and the satellite map corresponding to the construction area where the road roller is located includes:

[0025] Obtain compaction parameter information and electronic fence information set on a satellite map corresponding to the construction area where the roller is located;

[0026] Generate rolling path information according to the compaction parameter information and the electronic fence information;

[0027] The position information of the roller is compared with the rolling path information to generate action instructions for positioning navigation.

[0028] Furthermore, the positioning and navigation according to the first fused position information of the road roller and the octree map includes:

[0029] On the octree map, rolling path information is generated according to the length and width of the construction area;

[0030] The first fused position information of the roller is compared with the rolling path information to generate an action instruction for positioning navigation.

[0031] Furthermore, the positioning and navigation according to the second fused position information of the road roller and the octree map includes:

[0032] On the octree map, rolling path information is generated according to the length and width of the construction area;

[0033] The second fused position information of the road roller is compared with the rolling path information to generate an action instruction for positioning navigation.

[0034] Furthermore, the first preset accuracy range is not less than 1 mm and not more than 30 mm, the second preset accuracy range is more than 30 mm and not more than 200 mm, and the third preset accuracy range is more than 200 mm.

[0035] Furthermore, the visual camera adopts an RGB-D camera.

[0036] Furthermore, the laser radar adopts a single-line laser radar.

[0037] Compared with the prior art, the present invention has at least the following beneficial effects:

[0038] The present invention provides a multi-sensor fusion multi-scenario unmanned vibratory roller. Different positioning and navigation modes are selected according to the positioning accuracy of GPS. When the positioning accuracy is within a first preset accuracy range, the first positioning and navigation mode is selected, that is, a satellite map corresponding to a construction area where the roller is located is directly obtained, and positioning and navigation are performed according to the position information of the roller and the satellite map corresponding to the construction area where the roller is located; when the positioning accuracy is within a second preset accuracy range, the second positioning and navigation mode is selected, that is, the position information, image information, laser point cloud information, driving distance information and posture information of the roller are fused to obtain the first fused position information of the roller, and an octree map is constructed by using the image information and the laser point cloud information, and positioning and navigation are performed according to the first fused position information of the roller and the octree map; when the positioning accuracy is within a third preset accuracy range, the third positioning and navigation mode is selected, that is, the image information, laser point cloud information, driving distance information and posture information are fused to obtain the second fused position information of the roller, and an octree map is constructed by using the image information and the laser point cloud information, and positioning and navigation are performed according to the second fused position information of the roller and the octree map. Through this improved design, high-precision unmanned construction operations with arbitrary switching of multiple scenes for a single roller are achieved, the usage scenarios and scope of the unmanned roller are increased, and intelligent operations of the unmanned roller in multiple scenes are realized, thereby improving construction quality and efficiency, reducing manual errors, and getting rid of dependence on skilled operators. It can accurately locate in tunnels, areas with large obstacles and other areas without GPS signals, thereby realizing high-precision navigation of the unmanned roller.

[0039] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the specific implementation modes of the present invention, the drawings required for use in the description of the specific implementation modes will be briefly introduced below. Obviously, the drawings described below are some implementation modes of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0041] Figure 1 A schematic diagram of the system composition of a multi-scenario unmanned vibratory roller with multi-sensor fusion according to the present invention;

[0042] FIG2 is a flowchart of an algorithm for image matching and nonlinear optimization;

[0043] Figure 3 This is a schematic diagram of the actual construction of a road roller in multiple scenarios;

[0044] Figure 4This is a schematic diagram of the whole machine modification of the multi-sensor fusion multi-scenario unmanned vibratory roller of the present invention.

[0045] In the figure: 1-GPS base station; 2-single-line laser radar; 3-GPS mobile station; 4-vehicle controller; 5-RGB-D camera; 6-GPS radio receiver; 7-cab; 8-vehicle computing platform; 9-laser radar; 10-steering angle sensor; 11-articulated shaft; 12-wheel odometer; 13-inertial measurement unit. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] The multiple scenarios mentioned in the present invention mean that the roller can automatically select and switch the unmanned driving mode in any operating scenario, including: tunnels, mountains, open areas and other construction sites, so as to achieve the highest positioning accuracy and realize the highest precision unmanned rolling operation.

[0048] As a specific embodiment of the present invention, Figure 1 As shown, a multi-sensor fusion multi-scenario unmanned vibratory roller includes a global positioning system (GPS), a visual camera, a laser radar, a wheel odometer, an inertial measurement unit (IMU), and a vehicle-mounted computing unit installed on the roller body, as follows:

[0049] GPS, used to obtain the positioning accuracy of the roller's location and the roller's location information;

[0050] A visual camera, used to obtain image information of the construction area where the road roller is located; preferably, the visual camera is an RGB-D camera;

[0051] A laser radar is used to obtain laser point cloud information of the construction area where the road roller is located; preferably, the laser radar is a single-line laser radar;

[0052] Wheel odometer, used to obtain the travel distance information of the roller during construction;

[0053] Inertial measurement unit, used to obtain the position information of the roller during construction;

[0054] The vehicle-mounted computing unit is used to select the first positioning navigation mode, the second positioning navigation mode or the third positioning navigation mode according to the positioning accuracy. When the positioning accuracy is within the first preset accuracy range, the first positioning navigation mode is selected. Preferably, the first preset accuracy range is not less than 1mm and not more than 30mm; when the positioning accuracy is within the second preset accuracy range, the second positioning navigation mode is selected. Preferably, the second preset accuracy range is greater than 30mm and not more than 200mm; when the positioning accuracy is within the third preset accuracy range, the third positioning navigation mode is selected. Preferably, the third preset accuracy range is greater than 200mm. More specifically, the positioning navigation mode is selected based on the GPS positioning accuracy of the current location of the unmanned roller, and is divided into three positioning navigation modes for open, uncovered construction sites, semi-sheltered construction sites, and fully-sheltered construction sites. Before the unmanned roller starts rolling, it first uses GPS to obtain the positioning accuracy of the roller's location. If the GPS positioning accuracy is within 1mm-30mm, that is, an open and unobstructed construction site with a good GPS signal, the unmanned roller selects the first positioning and navigation mode (single GPS positioning mode); if the GPS positioning accuracy is within 30mm-200mm in a semi-obstructed construction site, the unmanned roller selects the second positioning and navigation mode (multi-sensor fusion positioning mode); if the positioning accuracy is infinite, that is, a fully obstructed construction site that cannot receive GPS signals at all, the third positioning and navigation mode (fusion positioning mode without GPS) is selected. Without the intervention of the operator, the three positioning and navigation modes can be independently selected by the on-board computing unit, but the planning software in the host computer also provides a way to manually modify the positioning mode.

[0055] The first positioning and navigation mode is to directly obtain the satellite map corresponding to the construction area where the roller is located (that is, directly obtain the satellite map corresponding to the construction area where the roller is located through the network), and perform positioning and navigation according to the location information of the roller and the satellite map corresponding to the construction area where the roller is located, as follows:

[0056] Obtain compaction parameter information and electronic fence information set on a satellite map corresponding to the construction area where the roller is located;

[0057] Generate rolling path information according to the compaction parameter information and the electronic fence information;

[0058] The position information of the roller is compared with the rolling path information to generate action instructions for positioning navigation.

[0059] In layman's terms, the principle of the first positioning and navigation mode is as follows: a high-precision map is loaded in the remote planning software, and the unmanned roller obtains the electronic fence information of the construction area through GPS positioning, and combines the compaction parameter information manually input by the planner (including compaction speed, compaction times, vibration frequency and amplitude, etc.) to autonomously generate rolling path information. The rolling path information consists of two parts, one is the driving trajectory of the unmanned roller, and the other is the compaction parameters to be executed under the current trajectory. During the driving process, the unmanned roller obtains its own position information (including longitude, latitude and heading angle) in real time, and transmits it to the on-board computing unit. The on-board computing unit uniformly transforms its coordinates and compares it with the rolling path information autonomously generated by the rules, generates corresponding action instructions and sends them to the on-board controller, thereby controlling the roller to perform corresponding actions (including acceleration, deceleration, parking, steering and vibration, etc.), corrects its speed and direction, and enables the roller to perform unmanned rolling construction. In this positioning and navigation mode, the driving accuracy of the unmanned roller is completely dependent on the positioning accuracy of the GPS. Therefore, only when the positioning accuracy of the GPS is high enough, reaching 1mm-30mm, can the driving accuracy during the compaction process be met.

[0060] Among them, the second positioning and navigation mode is to fuse the position information of the roller, the image information, the laser point cloud information, the driving distance information and the posture information to obtain the first fused position information of the roller, and use the image information and the laser point cloud information to construct an octree map, and perform positioning and navigation based on the first fused position information of the roller and the octree map.

[0061] Preferably, the first fused position information of the road roller is obtained by fusing the position information of the road roller, the image information, the laser point cloud information, the travel distance information and the posture information, as follows:

[0062] Continuous image frames are read from the image information, feature points of each frame are extracted using the ORB operator, and feature points of two adjacent frames are matched to construct a least squares problem. The least squares problem is solved using the ICP algorithm to obtain a posture transformation matrix of two adjacent frames. Based on the EKF algorithm, the driving distance information, the laser point cloud information and the posture information are fused to obtain a priori posture of the roller. According to the posture transformation matrix and the prior posture at different times, nonlinear optimization is performed using a combination of BA and Ceres, and loop detection is performed using the position information of the roller to finally obtain the first fused position information of the roller.

[0063] Preferably, the positioning and navigation according to the first fused position information of the road roller and the octree map is specifically as follows:

[0064] On the octree map, rolling path information is generated according to the length and width of the construction area;

[0065] The first fused position information of the roller is compared with the rolling path information to generate an action instruction for positioning navigation.

[0066] In layman's terms, the principle of the second positioning and navigation mode is: when rolling operations are carried out in semi-sheltered construction scenarios such as large buildings and mountains, the GPS signal is relatively general, and it is impossible to completely rely on GPS data to control the compaction driving of the roller. At this time, the data obtained by the visual camera, wheel odometer, IMU and lidar are also used as input and input into the on-board computing unit for subsequent fusion processing: continuous image frames are read from the visual camera, and the feature points of each frame are extracted using the ORB operator and the feature points of two adjacent frames are matched to construct a least squares problem. The problem is solved using the ICP algorithm to obtain the pose transformation matrix of two adjacent frames. Based on the EKF algorithm, according to the characteristics of each sensor, the data from the wheel odometer, lidar and IMU sensors are fused to obtain the prior pose of the roller, providing the prior pose for subsequent autonomous mapping and positioning. The back end receives the pose transformation matrix and prior pose estimated at different times, uses the BA combined with Ceres method for nonlinear optimization, and uses GPS data for loop detection, and finally obtains the first fused position information of the globally consistent unmanned roller, and establishes an octree map in real time. Based on this map, the on-board computing unit will autonomously generate rolling path information according to the length and width of the construction area. During the compaction process, the first fused position information of the roller obtained by multi-sensor fusion is compared with the rolling path information. After calculation, the on-board computing unit generates corresponding action instructions and sends them to the on-board controller, which then controls the roller to perform corresponding actions and complete the final rolling task.

[0067] Among them, the third positioning and navigation mode is to obtain the second fused position information of the roller by fusing the image information, the laser point cloud information, the driving distance information and the posture information, and use the image information and the laser point cloud information to construct an octree map, and perform positioning and navigation based on the second fused position information of the roller and the octree map.

[0068] Preferably, the second fused position information of the road roller is obtained by fusing the image information, the laser point cloud information, the travel distance information and the posture information, as follows:

[0069] Continuous image frames are read from the image information, feature points of each frame are extracted using the ORB operator, and feature points of two adjacent frames are matched to construct a least squares problem. The least squares problem is solved using the ICP algorithm to obtain a posture transformation matrix of two adjacent frames. Based on the EKF algorithm, the driving distance information, the laser point cloud information and the posture information are fused to obtain a priori posture of the roller. According to the posture transformation matrix and the prior posture at different times, nonlinear optimization is performed by combining BA (Bundle Adjustment) with the Ceres library to finally obtain the second fused position information of the roller.

[0070] Preferably, the positioning and navigation according to the second fused position information of the road roller and the octree map is specifically as follows:

[0071] On the octree map, rolling path information is generated according to the length and width of the construction area;

[0072] The second fused position information of the road roller is compared with the rolling path information to generate an action instruction for positioning navigation.

[0073] In layman's terms, the principle of the third positioning and navigation mode is: in fully shielded construction scenarios such as tunnels, GPS is completely ineffective. The only difference between the third positioning and navigation mode and the second positioning and navigation mode is that there is no GPS data. At this time, GPS data cannot be used for loop detection. At this time, the data of the wheel odometer is used as a supplement for loop detection. The rest of the algorithm processing is consistent with the third positioning and navigation mode, and will not be repeated here.

[0074] Preferably, the image information and the laser point cloud information are used to construct an octree map, specifically as follows:

[0075] (1) Configure the startup parameters of the launch file, including: configuring the octree map resolution to initialize the map object; setting the global map coordinate system to construct the coordinate system of the global octree map;

[0076] (2) Performing TF transformation on the image information and the laser point cloud information to obtain a point cloud map based on a global map coordinate system.

[0077] (3) The point cloud map calls the Octomap library to obtain and save the octree map for subsequent planning and navigation of the roller.

[0078] Multi-sensor fusion is to transmit all the data of the above sensors to the on-board computing unit in the cab, process all the sensor data in real time, perform data fusion, and output the real-time positioning information of the roller.

[0079] When an unmanned roller encounters an unexpected and uncontrollable situation, the operator can press the emergency brake button in the remote planning software. At this time, the unmanned roller will stop urgently, stop all construction tasks, and wait for the on-site construction personnel to eliminate the safety accident. If the emergency brake button is pressed in the remote planning software, the system will directly let the on-board controller receive the emergency stop command.

[0080] For example, Figure 4 The figure shows the whole machine modification diagram of the unmanned roller of the present invention, including installing a set of GPS mobile station 3 and GPS radio receiver 6 on the vibratory roller, interacting with the GPS base station 1 outside the construction area to obtain high-precision GPS positioning data; installing an RGB-D camera 5 on the top of the roller, installing a single-line laser radar 2 at the front and rear, an inertial measurement unit 13 under the single-line laser radar 2, and installing a wheel odometer 12 at the center of the side of the front steel wheel. These sensors are used for multi-sensor data acquisition, and after fusion, high-precision relative positioning data without GPS is output; since the roller is an articulated structure, a steering angle sensor 10 is installed at the articulated shaft 11, and the actual front and rear frame steering angles obtained by the steering angle sensor 10 are compared with the to-be-steering angle issued by the on-board computing unit, and PID feedback control is performed to make the actual steering angle gradually approach the to-be-steering angle, thereby reducing the steering error in unmanned driving.

[0081] Combination Figure 2a As shown in Figure 2b, the multi-sensor fusion process is described in more detail below.

[0082] In the multi-sensor fusion positioning mode and the fusion positioning mode without GPS, the image feature matching method based on the ORB operator is used for pose estimation. Figure 2a As shown in the figure, the process is as follows: The first step is to extract the Oriented FAST corner points from the image information collected by the visual camera at the current moment. Select pixel p in the image information, whose brightness is I p , set a threshold T, such as I p 20%; take pixel p as the center, select 16 pixels on a circle with a radius of 3, if there are N consecutive points on the selected circle with a brightness greater than I p + or less than I p-, then the pixel p is considered to be an Oriented FAST corner point, and the above steps are repeated for each pixel to obtain all the Oriented FAST corner points in the image information at the current moment; the second step is to calculate the BRIEF descriptor based on the corner point, which is a 128-dimensional vector composed of 0 and 1; the third step is to repeat the first and second steps for the image information at the next moment to obtain the Oriented FAST corner point and BRIEF descriptor of the image information at the next moment; the fourth step is to perform brute force matching on the BRIEF descriptors in the image information at the above two moments. If the Hamming distance between the BRIEF descriptors is less than twice the minimum distance, the corner point is matched successfully and saved as a feature point pair.

[0083] The Ceres library function is used to solve the ICP problem, that is, the pose transformation matrix of the current image information relative to the next image information, providing a good initial estimate for backend optimization. The process is as follows Figure 2b As shown in the figure, the first step is to construct a calculation model of the residual function and calculate the residual; the second step is to construct the least squares problem based on the residual function; the third step is to configure the solver, set the solution method of the incremental equation, the maximum number of iterations, the longest running time, etc.; finally, the optimization result is output to obtain the relative pose change matrix of the image information at two adjacent moments after optimization. In multi-sensor fusion positioning, the front end considers the image information at adjacent moments, and inevitably accumulates the error to the next moment. After a long period of error accumulation, the theoretical path and the actual path differ greatly, and drift occurs. In the multi-sensor fusion positioning method, the roller position information obtained by GPS is combined with the appearance detection method for loop detection, thereby improving the accuracy and robustness of the entire positioning system.

[0084] like Figure 3As shown, the multi-scenario unmanned vibratory roller rolling operation process of the multi-sensor fusion of the present invention is as follows: the roller is parked at the starting point of the compaction operation, the power is turned on, the system is initialized, and the real-time data information obtained by GPS, RGB-D camera, single-line laser radar, wheel odometer and IMU is obtained, and the data information is input into the on-board computing unit, and a judgment is made on whether each sensor is working normally, and the result is displayed on the remote end planning software interface. When all sensors are working normally, the on-board computing unit obtains the positioning accuracy of the GPS and determines which of the three positioning and navigation modes the unmanned vibratory roller needs to use in this scenario. After selecting the positioning and navigation mode, the unmanned vibratory roller begins to perform the corresponding steps for the mode, and the remote end planning software displays the current positioning and navigation mode for the operator to supervise and adjust. When the unmanned vibratory roller starts normal construction and rolling, the remote end planning software displays its operation parameter information in real time, including the real-time position of the roller, movement speed, number of rolling passes, whether vibration is turned on, vibration frequency, amplitude and steering angle, etc. When the roller reaches the end of the operation, the operator decides whether to continue with the subsequent construction tasks or exit.

[0085] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A multi-sensor fusion multi-scenario unmanned vibratory roller, characterized in that: include: GPS, used to obtain the positioning accuracy of the roller's location and the roller's location information; A visual camera is used to obtain image information of the construction area where the roller is located; LiDAR, used to obtain laser point cloud information of the construction area where the roller is located; Wheel odometer, used to obtain the travel distance information of the roller during construction; Inertial measurement unit, used to obtain the position information of the roller during construction; The vehicle-mounted computing unit is used to select a first positioning navigation mode, a second positioning navigation mode or a third positioning navigation mode according to the positioning accuracy, and when the positioning accuracy is within a first preset accuracy range, the first positioning navigation mode is selected; when the positioning accuracy is within a second preset accuracy range, the second positioning navigation mode is selected; when the positioning accuracy is within a third preset accuracy range, the third positioning navigation mode is selected; The first positioning and navigation mode is to directly obtain a satellite map corresponding to the construction area where the roller is located, and perform positioning and navigation according to the location information of the roller and the satellite map corresponding to the construction area where the roller is located; The second positioning and navigation mode is to obtain first fused position information of the road roller by fusing the position information of the road roller, the image information, the laser point cloud information, the travel distance information and the posture information, and to construct an octree map using the image information and the laser point cloud information, and to perform positioning and navigation according to the first fused position information of the road roller and the octree map; The third positioning and navigation mode is to obtain the second fused position information of the roller by fusing the image information, the laser point cloud information, the driving distance information and the posture information, and to construct an octree map using the image information and the laser point cloud information, and to perform positioning and navigation based on the second fused position information of the roller and the octree map.

2. The multi-sensor fusion multi-scenario unmanned vibratory roller according to claim 1, characterized in that: The step of fusing the position information of the road roller, the image information, the laser point cloud information, the travel distance information and the posture information to obtain first fused position information of the road roller comprises: Continuous image frames are read from the image information, feature points of each frame are extracted using the ORB operator, and feature points of two adjacent frames are matched to construct a least squares problem. The least squares problem is solved using the ICP algorithm to obtain a pose transformation matrix of two adjacent frames. Based on the EKF algorithm, the driving distance information, the laser point cloud information and the pose information are fused to obtain a priori pose of the roller. According to the pose transformation matrix and the prior pose at different times, a nonlinear optimization is performed using a combination of BundleAdjustment and Ceres library, and loop detection is performed using the position information of the roller to finally obtain the first fused position information of the roller.

3. The multi-sensor fusion multi-scenario unmanned vibratory roller according to claim 1, characterized in that: The step of fusing the image information, the laser point cloud information, the travel distance information and the position information to obtain the second fused position information of the road roller includes: Continuous image frames are read from the image information, feature points of each frame are extracted using the ORB operator, and feature points of two adjacent frames are matched to construct a least squares problem. The least squares problem is solved using the ICP algorithm to obtain a posture transformation matrix of two adjacent frames. Based on the EKF algorithm, the driving distance information, the laser point cloud information and the posture information are fused to obtain a priori posture of the roller. According to the posture transformation matrix and the prior posture at different times, a nonlinear optimization is performed using a combination of BundleAdjustment and Ceres library to finally obtain the second fused position information of the roller.

4. The multi-sensor fusion multi-scenario unmanned vibratory roller according to claim 1, characterized in that: The method of constructing an octree map by using the image information and the laser point cloud information includes: Set the global map coordinate system; Performing TF transformation on the image information and the laser point cloud information to obtain a point cloud map based on a global map coordinate system; The point cloud map calls the Octomap library to obtain an octree map.

5. The multi-sensor fusion multi-scenario unmanned vibratory roller according to claim 1, characterized in that: The positioning and navigation according to the location information of the road roller and the satellite map corresponding to the construction area where the road roller is located includes: Obtain compaction parameter information and electronic fence information set on a satellite map corresponding to the construction area where the roller is located; Generate rolling path information according to the compaction parameter information and the electronic fence information; The position information of the roller is compared with the rolling path information to generate action instructions for positioning navigation.

6. The multi-sensor fusion multi-scenario unmanned vibratory roller according to claim 1, characterized in that: The positioning and navigation according to the first fused position information of the road roller and the octree map includes: On the octree map, rolling path information is generated according to the length and width of the construction area; The first fused position information of the roller is compared with the rolling path information to generate an action instruction for positioning navigation.

7. The multi-sensor fusion multi-scenario unmanned vibratory roller according to claim 1, characterized in that: The positioning and navigation according to the second fused position information of the road roller and the octree map includes: On the octree map, rolling path information is generated according to the length and width of the construction area; The second fused position information of the road roller is compared with the rolling path information to generate an action instruction for positioning navigation.

8. The multi-sensor fusion multi-scenario unmanned vibratory roller according to claim 1, characterized in that: The first preset accuracy range is not less than 1 mm and not more than 30 mm, the second preset accuracy range is more than 30 mm and not more than 200 mm, and the third preset accuracy range is more than 200 mm.

9. The multi-sensor fusion multi-scenario unmanned vibratory roller according to claim 1, characterized in that: The visual camera adopts an RGB-D camera.

10. The multi-sensor fusion multi-scenario unmanned vibratory roller according to claim 1, characterized in that: The laser radar adopts a single-line laser radar.

Images