A road maintenance unmanned patrol vehicle system based on drone and working method

Abstract

The present invention relates to a road maintenance unmanned patrol car system and working method based on drones. The system includes a patrol car, a four-rotor drone and a vehicle-mounted control terminal, wherein the vehicle-mounted control terminal is mounted on the patrol car, and the patrol car includes a drone parking and fixing interface, an alarm device, a visual sensor, an information receiving and control module, a laser ranging module and a visual positioning module arranged on the top; the four-rotor drone includes an alarm device, a visual sensor, an automatic driving controller and a laser ranging module. The present invention not only realizes the unmanned operation of the entire process of road maintenance and sealing operations and the isolation of people and vehicles, thus ensuring the safety of the operation, but also only requires the provision of an inertial navigation module, a laser ranging radar and a monocular camera, without the need for expensive RTK positioning antennas and 3D laser radars, and the automation solution has a low cost.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned patrol vehicles, and in particular to a road maintenance unmanned patrol vehicle system based on an unmanned aerial vehicle and a working method. Background Art

[0002] With the continuous development of science and technology, artificial intelligence technology has attracted more and more attention. Unmanned patrol vehicles based on the above technology are also becoming the theme of our research.

[0003] When carrying out road maintenance operations such as road cleaning, pavement maintenance, street lamp repair, vehicle rescue, and road obstacle removal, road maintenance workers need to be exposed to high-speed traffic to perform manual operations. The construction road surface will be isolated only by isolation safety cones, which poses a great safety hazard and is prone to major safety accidents involving collisions between people and vehicles. This is a problem that we urgently need to solve. Summary of the invention

[0004] In view of the above problems, the present invention provides a road maintenance unmanned patrol vehicle system and working method based on drones, which not only realizes the unmanned operation of the entire process of road maintenance and control operations and the isolation of people and vehicles, ensuring the safety of operations, but also only requires the installation of inertial navigation modules, laser ranging, and visual recognition modules, without the need for expensive RTK positioning antennas and 3D laser radars, and the automation solution has low cost.

[0005] In order to achieve the above-mentioned purpose and other related purposes, the technical solution provided by the present invention is as follows:

[0006] A road maintenance unmanned patrol car system based on drones, the system comprises a remote control, a patrol car with an image return function, the working patrol car is equipped with a vehicle-mounted controller based on a PID control module, the system also comprises a quad-rotor drone, the patrol car and the quad-rotor drone are both provided with a monocular camera, the quad-rotor drone and the patrol car perform data communication and status feedback through a Mavlink protocol, a warning sign is fixed at the bottom of the quad-rotor drone, triangular legs are provided at both ends of the bottom of the warning sign, a fixing frame is provided at the rear of the patrol car, an interface slot for drone parking is provided at the top of the fixing frame, a display screen is fixed at the front of the patrol car, the bottom of the sign is inserted into the interface slot, a pressure plate is provided in the interface slot, an electric push rod is fixed outside the interface slot, a telescopic rod of the electric push rod passes through the side of the interface slot and is connected to the pressure plate, the pressure plate commands the side of the sign to be locked or released under the action of the electric push rod, and a long-distance single-point laser ranging radar is also provided on the top of the patrol car display screen.

[0007] Further optimized, the display screen of the patrol car and the sign of the quad-rotor drone are at the same height from the ground and are both within the viewing angle of the laser ranging radar.

[0008] Further optimized, the remote controller has the function of remote controlling the quad-rotor UAV and the patrol car, and adopts one-key switching to realize the remote controlling of the quad-rotor UAV and the patrol car respectively.

[0009] Further optimized, the quad-rotor drone has a one-key return function.

[0010] In order to achieve the above-mentioned and other related purposes, the present invention also provides a working method of a road maintenance unmanned patrol vehicle based on a drone, comprising the following steps:

[0011] S1: The operator controls the patrol car through the remote control, drives along the right side of the lane marking on the highway, reaches the preset position A operation point and lights up the display screen;

[0012] S2: Start the quadrotor drone, and the quadrotor drone flies forward with a warning sign;

[0013] S3: The laser ranging radar of the patrol car measures the distance M of the warning sign. When the distance M meets the preset operation point B, the quadcopter drone descends and lights up the warning sign to start the warning operation;

[0014] S4: Send the command for the quadrotor drone to return to its original position, and the quadrotor drone returns;

[0015] S5: The quadrotor drone is accurately docked at the fixed interface, the docking is completed, and the mission is completed.

[0016] Further optimization, in step S1, includes the following steps:

[0017] S11: Obtain lane marking images based on the patrol car’s monocular camera;

[0018] S12: preprocessing the lane marking image, segmenting the region of interest, and obtaining an image within the region of interest;

[0019] S13: graying the image within the region of interest based on the mean method, and then performing edge detection on the grayed image to identify lane markings;

[0020] S14: Calculating the number of pixels between the left and right lane markings and the center line of the image based on the image data of the identified lane markings;

[0021] S15: Compare the difference between the pixel quantity data obtained above and the preset pixel quantity data of the picture, and according to the preset threshold value T1, if the threshold value T1 is exceeded, the deviation direction data is fed back to the vehicle controller, and the vehicle controller adjusts the patrol car;

[0022] S16: Based on the obtained difference being less than the preset threshold value T1, the vehicle maintains the current direction of travel.

[0023] Further optimization, in step S2, includes the following steps:

[0024] S21: A monocular camera based on the UAV’s visual sensor to obtain road marking images;

[0025] S22: Based on the road marking image, focus on identifying pixel contents of the left and right thirds of the field of view of the image by image partitioning;

[0026] S23: extracting the grayscale of the end point area by using the mean method, and identifying the road markings by using edge detection;

[0027] S24: The autopilot controller of the quadcopter drone adjusts the quadcopter drone to fly in the middle of the lane through a posture algorithm based on the pixel information fed back by the visual sensor.

[0028] Further optimized, in step S5, the quad-rotor drone is accurately parked at a fixed interface under the guidance of a monocular camera and a laser ranging radar, comprising the following steps:

[0029] S51: The monocular camera of the quad-rotor drone takes a picture of the patrol car, grayscales the image using a mean value method, and performs edge recognition and feature value extraction on the patrol car;

[0030] S52: comparing the extracted feature values ​​with the pre-stored photos, and according to a preset threshold, if the threshold is not met, feeding back to the posture algorithm module of the quad-rotor drone to calculate the transformation angle of the quad-rotor drone, and the quad-rotor drone performs posture transformation according to the algorithm parameters until the threshold is met;

[0031] S53: Based on meeting the above threshold, the quadrotor drone descends 1 meter, the laser ranging radar detects the bottom of the warning sign and calculates the dynamic distance N, and the quadrotor drone executes the terminal precise alignment algorithm to compare the Acuro code pixel information with the pre-stored photos. According to the preset threshold T2, if the threshold T2 is not met, the posture algorithm module of the quadrotor drone is fed back to calculate the transformation angle of the quadrotor drone. The quadrotor drone performs posture transformation actions according to the algorithm parameters until the posture meets the threshold T2 and the dynamic distance N is within the standard distance value, so that the bottom of the drone warning sign lands accurately to the fixed interface of the patrol car.

[0032] Further optimized, in step S3, after the quad-rotor drone starts the warning operation, it also includes: S31: the quad-rotor drone takes off regularly to display a warning; S32: hovers for a period of time at a height of 3-5 meters above the ground; S33: then descends and waits; and repeats step S31.

[0033] The present invention has the following positive effects:

[0034] 1. The present invention realizes the unmanned operation of the entire process of road maintenance and sealing operations and the isolation of people and vehicles, thus ensuring the safety of operations.

[0035] 2. The UAV and patrol car of the present invention only need to realize the functions of road marking recognition and fixed distance straight driving. Both use a monocular camera to acquire images and a long-distance single-point laser ranging radar to measure distances. The automation solution has low cost.

[0036] 3. The present invention realizes the functions of warning, guiding and diverting vehicles. The patrol car and the drone have two-level warning sign protection function, which improves the safety factor. In particular, during the operation and recovery process of the present invention, the drone can automatically take off at the operation point to give a timed warning, so it has the advantages of a wide range of influence and strong reminder function. After the drone and the patrol car form a system, they can take advantage of the strengths of both and improve the level of intelligent road management. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 Schematic diagram of the operation process of the present invention;

[0038] Figure 2 A schematic diagram of the process of a patrol car driving along a marked line according to the present invention;

[0039] Figure 3 This is a schematic diagram of the process of the quad-rotor drone of the present invention arriving at the operating point;

[0040] Figure 4 This is a schematic diagram of the terminal alignment guidance process of the quad-rotor drone of the present invention;

[0041] Figure 5 It is a schematic diagram of the structure of the present invention;

[0042] Figure 6 This is a schematic diagram of the structure of the drone of the present invention;

[0043] Figure 7 This is a schematic diagram of the patrol vehicle structure of the present invention;

[0044] Figure 8 It is a schematic diagram of the operation process of the present invention;

[0045] Fig. 9 It is a schematic diagram of the clamping device of the quad-rotor UAV of the present invention.

[0046] Explanation of the numbers in the figure: 1-quad-rotor UAV, 11-warning device, 12-long-distance laser ranging radar of the UAV, 13-monocular camera of the UAV, 2-patrol car, 21-warning device, 22-long-distance laser ranging radar of the patrol car, 23-monocular camera of the patrol car, 24-Aruco code positioning mark, 25-quad-rotor UAV parking and fixing interface, 26-pressure plate, 27-telescopic rod, 28-electric push rod. DETAILED DESCRIPTION

[0047] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be interpreted as limiting the present invention.

[0048] Example: Figure 5 , 6 As shown in , 7 and 9, a road maintenance unmanned patrol car system based on drones, the system includes a remote control, a patrol car with an image feedback function, the working patrol car is equipped with a vehicle-mounted controller based on a PID control module, the system also includes a quad-rotor drone, the patrol car and the quad-rotor drone are both equipped with a monocular camera, the quad-rotor drone and the patrol car communicate data and provide status feedback through the Mavlink protocol, a warning sign is fixed at the bottom of the quad-rotor drone, and triangular legs are provided at both ends of the bottom of the warning sign, a fixing frame is provided at the rear of the patrol car, an interface slot for drone parking is provided at the top of the fixing frame, a display screen is fixed at the front of the patrol car, the bottom of the sign is inserted into the interface slot, a pressure plate is provided in the interface slot, an electric push rod is fixed outside the interface slot, the telescopic rod of the electric push rod passes through the side of the interface slot and is connected to the pressure plate, the pressure plate orders the side of the sign to be locked or released under the action of the electric push rod, and a long-distance single-point laser ranging radar is also provided on the top of the patrol car display screen.

[0049] Further, such as Figure 7 As shown, the patrol car has a flat chassis with integrated patrol, obstacle removal, fire extinguishing and cone barrel deployment functions.

[0050] Furthermore, the display screen of the patrol car and the sign of the quad-rotor drone are at the same height from the ground and are both within the viewing angle of the laser ranging radar.

[0051] Furthermore, the remote controller has the function of remote controlling the quad-rotor UAV and the patrol car, and adopts one-key switching to realize the remote control of the quad-rotor UAV and the patrol car respectively.

[0052] Furthermore, the quad-rotor drone has a one-key return function.

[0053] Further, such as Figure 6 As shown, the laser ranging module of the quad-rotor UAV is equipped with two long-distance single-point laser ranging radars, which are respectively arranged in the horizontal direction and the vertical direction of the quad-rotor UAV; the laser ranging module of the patrol car is equipped with a long-distance single-point laser ranging radar, and the model is the same.

[0054] Further, such as Figure 6 As shown in 7, the warning device of the patrol car and the warning device of the quad-rotor drone are at the same height from the ground and are both within the viewing angle of the laser ranging radar.

[0055] In order to achieve the above purposes and other related purposes, such as Figure 1 As shown, the present invention also provides a method for operating an unmanned patrol vehicle for road maintenance based on a drone, comprising the following steps:

[0056] S1: The operator controls the patrol car through the remote control, drives along the right side of the lane marking on the highway, reaches the preset position A operation point and lights up the display screen;

[0057] S2: Start the quadrotor drone, and the quadrotor drone flies forward with a warning sign;

[0058] S3: The laser ranging radar of the patrol car measures the distance M of the warning sign. When the distance M meets the preset operation point B, the quadcopter drone descends and lights up the warning sign to start the warning operation;

[0059] S4: Send the command for the quadrotor drone to return to its original position, and the quadrotor drone returns;

[0060] S5: The quadrotor drone is accurately docked at the fixed interface, the docking is completed, and the mission is completed.

[0061] Further, such as Figure 2 As shown, in step S1, the following steps are included:

[0062] S11: Obtain lane marking images based on the patrol car’s monocular camera;

[0063] S12: preprocessing the lane marking image, segmenting the region of interest, and obtaining an image within the region of interest;

[0064] S13: graying the image within the region of interest based on the mean method, and then performing edge detection on the grayed image to identify lane markings;

[0065] S14: Calculating the number of pixels between the left and right lane markings and the center line of the image based on the image data of the identified lane markings;

[0066] S15: Compare the difference between the pixel quantity data obtained above and the preset pixel quantity data of the picture, and according to the preset threshold value T1, if the threshold value T1 is exceeded, the deviation direction data is fed back to the vehicle controller, and the vehicle controller adjusts the patrol car;

[0067] S16: Based on the obtained difference being less than the preset threshold value T1, the vehicle maintains the current direction of travel.

[0068] Further, such as Figure 3 As shown, in step S2, the following steps are included:

[0069] S21: A monocular camera based on the UAV’s visual sensor to obtain road marking images;

[0070] S22: Based on the road marking image, focus on identifying pixel contents of the left and right thirds of the field of view of the image by image partitioning;

[0071] S23: extracting the grayscale of the end point area by using the mean method, and identifying the road markings by using edge detection;

[0072] S24: The autopilot controller of the quadcopter drone adjusts the quadcopter drone to fly in the middle of the lane through a posture algorithm based on the pixel information fed back by the visual sensor.

[0073] Further, such as Figure 4 As shown, in step S5, the quad-rotor drone is accurately parked at a fixed interface under the guidance of a monocular camera and a laser ranging radar, including the following steps:

[0074] S51: The monocular camera of the quad-rotor drone takes a picture of the patrol car, grayscales the image using a mean value method, and performs edge recognition and feature value extraction on the patrol car;

[0075] S52: comparing the extracted feature values ​​with the pre-stored photos, and according to a preset threshold, if the threshold is not met, feeding back to the posture algorithm module of the quad-rotor drone to calculate the transformation angle of the quad-rotor drone, and the quad-rotor drone performs posture transformation according to the algorithm parameters until the threshold is met;

[0076] S53: Based on meeting the above threshold, the quadrotor drone descends 1 meter, the laser ranging radar detects the bottom of the warning sign and calculates the dynamic distance N, and the quadrotor drone executes the terminal precise alignment algorithm to compare the Acuro code pixel information with the pre-stored photos. According to the preset threshold T2, if the threshold T2 is not met, the posture algorithm module of the quadrotor drone is fed back to calculate the transformation angle of the quadrotor drone. The quadrotor drone performs posture transformation actions according to the algorithm parameters until the posture meets the threshold T2 and the dynamic distance N is within the standard distance value, so that the bottom of the drone warning sign lands accurately to the fixed interface of the patrol car.

[0077] Further optimized, in step S3, after the quad-rotor drone starts the warning operation, it also includes: S31: the quad-rotor drone takes off regularly to display a warning; S32: hovers 3-5 meters above the ground for a period of time; S33: then descends and waits; and repeats step S31.

[0078] Specifically, the quad-rotor drone arrives at the early warning operation point and takes off every ten minutes to perform early warning operations. The warning sign light flashes to remind passing vehicles to avoid it. It lands after hovering for three minutes and then repeats the above operations, achieving different visual angles to remind passing vehicles of different models to pay attention to safety, thereby improving safety during the operation.

[0079] Specifically, the outside of the Aruco code is composed of a square frame. Therefore, the candidate frame must be detected first. Due to the influence of perspective transformation, the projection of the square on the image can be any quadrilateral. The only known property is that this quadrilateral must be convex, and the code also uses this property for screening. The corner points detected in the previous steps are all pixel-level, which may have errors when used for subsequent quadcopter posture estimation. Therefore, after the corner points are detected, they need to be refined to obtain sub-pixel corner points. There are two methods of refinement: corner point refinement and fitting line refinement. Then the quadcopter flies to the target point, sends the expected target point to the control module of the quadcopter, makes adjustments, and finally reaches the fixed interface of the unmanned patrol car.

[0080] In summary, the present invention realizes the unmanned operation of the entire process of road maintenance and sealing operations and the isolation of people and vehicles, ensuring the safety of operations. At the same time, not only do the drones and patrol cars only need to realize the functions of road marking recognition and fixed-distance straight-line driving, so only a monocular camera is needed for image acquisition and a laser ranging radar. The automation solution has low cost. Moreover, during the operation and recovery process, the drones warn, guide, and divert incoming vehicles before the patrol cars. The operation support vehicles and operators are always protected by two-level warning signs of drones and unmanned vehicles, and the safety factor is improved.

[0081] The above descriptions are only some embodiments of the present invention. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A working method of an unmanned patrol vehicle for road maintenance based on a drone, characterized in that: The following steps are involved: S1. The operator controls the patrol car through the remote control, drives along the right side of the lane marking on the highway, reaches the preset position A operation point and lights up the display screen; S2. Start the quadrotor drone, and the quadrotor drone flies forward with a warning sign; S3. The laser ranging radar of the patrol car measures the distance M of the warning sign. When the distance M meets the preset operation point B, the quadcopter drone descends and lights up the warning sign to start the warning operation; S4. Send a homing command to the quadrotor drone, and the quadrotor drone returns; S5. The quadrotor drone is accurately parked at the fixed interface and docking is completed; In step S1, the following steps are included: S11. Obtaining a lane marking image based on a monocular camera of a patrol car; S12. Preprocessing the lane marking image, segmenting the region of interest, and obtaining an image within the region of interest; S13. grayscale the image within the region of interest based on the mean method, and then perform edge detection on the grayscale image to identify lane markings; S14. Based on the image data of the identified lane markings, the number of pixels between the left and right lane markings and the center line of the image is calculated; S15. The difference between the pixel quantity data obtained above and the preset pixel quantity data of the screen is compared, and according to the preset threshold value T1, if the threshold value T1 is exceeded, the deviation direction data is fed back to the vehicle controller, and the vehicle controller adjusts the patrol car; S16. According to the obtained difference being less than the preset threshold value T1, the vehicle maintains the current direction of travel; In step S2, the following steps are included: S21. Obtaining a road marking image based on a monocular camera of a visual sensor of a UAV; S22. Based on the road marking image, focus on identifying the pixel contents of the left and right thirds of the field of view of the image by image partitioning; S23. Extract the grayscale image of the end point area by the mean method and identify the road markings by edge detection; S24. The autopilot controller of the quadcopter drone adjusts the quadcopter drone to fly in the middle of the lane through a posture algorithm based on the pixel information fed back by the visual sensor; In step S5, the quad-rotor drone is accurately parked at a fixed interface under the guidance of a monocular camera and a laser ranging radar, including the following steps: S51. The monocular camera of the quadcopter drone takes a picture of the patrol car, grays the image using the mean method, and performs edge recognition and feature extraction on the patrol car; S52. The extracted feature values ​​are compared with the pre-stored photos. According to the preset threshold, if the threshold is not met, the feature values ​​are fed back to the posture algorithm module of the quad-rotor drone to calculate the transformation angle of the quad-rotor drone. The quad-rotor drone performs the posture transformation action according to the algorithm parameters until the threshold is met. S53. Based on meeting the above threshold, the quadrotor drone descends 1 meter, the laser ranging radar detects the bottom of the warning sign and calculates the dynamic distance N, and the quadrotor drone executes the terminal precise alignment algorithm to compare the Acuro code pixel information with the pre-stored photos. According to the preset threshold T2, if the threshold T2 is not met, the posture algorithm module of the quadrotor drone is fed back to calculate the transformation angle of the quadrotor drone. The quadrotor drone performs posture transformation actions according to the algorithm parameters until the posture meets the threshold T2 and the dynamic distance N is within the standard distance value, so that the bottom of the drone warning sign lands accurately to the fixed interface of the patrol car.

2. The working method of the unmanned patrol vehicle for road maintenance based on a drone according to claim 1 is characterized in that: In step S3, after the quad-rotor drone starts the early warning operation, it also includes: S31. Quad-rotor drone scheduled take-off warning display; S32. Hover for a period of time at a height of 3-5 meters above the ground; S33. Then descend and wait, and repeat step S31.

3. A system for implementing the working method of the unmanned patrol vehicle for road maintenance based on a drone as described in any one of claims 1-2, characterized in that: The system includes a remote control and a patrol car with an image feedback function. The operating patrol car is equipped with a vehicle-mounted controller based on a PID control module. The system is characterized in that: the system also includes a four-rotor drone, the patrol car and the four-rotor drone are both equipped with a monocular camera, the four-rotor drone and the patrol car communicate data and provide status feedback through the Mavlink protocol, a warning sign is fixed at the bottom of the four-rotor drone, and triangular legs are provided at both ends of the bottom of the warning sign. A fixing frame is provided at the rear of the patrol car, and an interface slot for parking the drone is provided at the top of the fixing frame. A display screen is fixed at the front of the patrol car, the bottom of the sign is inserted into the interface slot, a pressure plate is provided in the interface slot, and an electric push rod is fixed outside the interface slot, the telescopic rod of the electric push rod passes through the side of the interface slot and is connected to the pressure plate, and the pressure plate orders the side of the warning sign to be locked or released under the action of the electric push rod, and a long-distance single-point laser ranging radar is also provided on the top of the patrol car display screen.

4. The system according to claim 3, characterized in that: The display screen of the patrol car and the sign of the quad-rotor drone are at the same height from the ground and are both within the viewing angle of the laser ranging radar.

5. The system according to claim 3, characterized in that: The remote controller has the functions of remote controlling the quad-rotor UAV and the patrol car, and adopts one-key switching to realize the remote controlling of the quad-rotor UAV and the patrol car respectively.

6. The system according to claim 3, characterized in that: The four-rotor drone has a one-key return function.

Images