System and method for using drones to assist in autonomous driving of vehicles

Through the collaboration between the drone and the vehicle autonomous driving system, the problem of high cost and easy damage to the autonomous driving system on small vehicles is solved, and more efficient autonomous driving applications and convenient maintenance of sensing devices is achieved.

CN110780667BActive Publication Date: 2024-11-05HYUNDAI MOTOR CO LTD +1
View PDF 11 Cites 0 Cited by

Patent Information

Application Number
CN201811323717.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2018-07-26
Filing Date
2018-11-08
Publication Date
2024-11-05
Estimated Expiration
2038-11-08

AI Technical Summary

Technical Problem

The high cost of existing autonomous driving systems and the problem of easy damage to sensing devices lead to low efficiency in application on small vehicles and may lead to accidents.

Method used

By renting a drone equipped with cameras and radar, it cooperates with the vehicle's autonomous driving system to provide peripheral information to assist autonomous driving, and manages the rental and recycling of drones under the control of the server.

Benefits of technology

It reduces the unit cost of vehicles, improves the application efficiency of autonomous driving systems, reduces the risk of accidents caused by damage to sensing devices, and provides convenience through the maintenance of drone rental companies.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN110780667B_ABST
    Figure CN110780667B_ABST
Patent Text Reader

Abstract

The present invention relates to a system and method for assisting vehicle autonomous driving using a drone. A system for assisting autonomous driving using a drone includes: a drone that obtains peripheral information through at least one or more detectors; a vehicle that performs autonomous driving by using the peripheral information; and a server that leases the drone in response to a drone leasing request from the vehicle and assists the autonomous driving of the vehicle.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is based upon and claims the benefit of priority from Korean Patent Application No. 10-2018-0087348 filed on July 26, 2018 in the Korean Intellectual Property Office, the disclosure of which is incorporated herein in its entirety by reference. Technical Field

[0003] The present invention relates to a system and method for assisting autonomous driving using a drone, which assists autonomous driving by drone rental when desired. Background Art

[0004] With the development of information technology (IT), vehicles equipped with autonomous driving functions, services operated in conjunction with autonomous driving systems, and infrastructure operated in conjunction with autonomous driving systems are increasing. However, due to the cost burden of cameras, radars, controllers, etc., which are basic elements of autonomous driving systems, it is difficult to apply autonomous driving systems to various vehicles such as small cars, trucks, etc. Since controllers, cameras, radars, etc. are always installed even if users do not frequently use the autonomous driving system, vehicles to which the autonomous driving system is applied are not efficient.

[0005] When the camera, radar, and controller installed in the vehicle malfunction, it is difficult to continue using the vehicle because a visit to a service center is required to inspect the vehicle. In addition, during autonomous driving, an accident may occur when a sensing error of the camera or radar occurs due to dust and foreign matter or due to scratches during car washing. Summary of the invention

[0006] The present invention is used to solve the above-mentioned problems in the prior art, while maintaining the advantages achieved by the prior art intactly.

[0007] One aspect of the present invention provides a system and method for using drones to assist autonomous driving, which rents drones equipped with sensing devices such as cameras, radars, etc. and assists autonomous driving through collaboration between the drones and autonomous driving systems in vehicles.

[0008] The technical problems to be solved by the present inventive concept are not limited to the above-mentioned problems, and any other technical problems not mentioned herein should be clearly understood by those skilled in the art to which the present invention pertains from the following description.

[0009] According to an embodiment of the present invention, a system for using a drone to assist autonomous driving may include: a drone that obtains peripheral information through at least one or more detectors; a vehicle that performs autonomous driving by using the peripheral information; and a server that leases the drone in response to a drone rental request from the vehicle and assists the vehicle's autonomous driving.

[0010] Each of the at least one or more detectors may be composed of at least one or more of a camera and a radar.

[0011] When receiving a drone rental request from a vehicle, the server can determine whether it can communicate with the vehicle's drone and whether the autonomous driving function is installed to determine the permission for drone service use.

[0012] When renting a drone, the server can send the vehicle's location information to the drone.

[0013] The drone can move to the location of the vehicle under the control of the server.

[0014] The drone may include a communication device for wireless communication between a vehicle and a server, a global positioning system (GPS) receiver for detecting position information of the drone, a gyro sensor for detecting angle information of the detector, and a processor for correcting peripheral information detected by the detector based on the position information of the drone and the angle information of the detector.

[0015] The vehicle can determine whether the detector installed in the drone is abnormal, and request the server to re-rent another drone according to the determination result.

[0016] When autonomous driving is terminated, the vehicle may determine, in response to user input, whether to terminate drone service usage.

[0017] When it is determined that the use of the drone service has been terminated, the vehicle can issue a drone recovery request to the server.

[0018] In response to the vehicle's drone recovery request, the server can send the return location to the drone.

[0019] According to another embodiment of the present invention, a vehicle may include: a communication device for wirelessly communicating between a server and a drone; a detector for detecting vehicle status information, and a processor configured to issue a drone rental request to the server to rent a drone, obtain peripheral information through at least one or more detectors installed in the drone, and perform autonomous driving based on the obtained peripheral information and vehicle status information.

[0020] According to another embodiment of the present invention, a method for using a drone to assist autonomous driving may include: a vehicle issuing a request for the use of a drone service to a server; the server determining whether the vehicle can use the drone service; the vehicle issuing a drone rental request to the server based on the server's determination result; the server renting a drone in response to the vehicle's request; moving the drone to the location of the vehicle under the control of the server; and the vehicle performing autonomous driving based on peripheral information obtained by at least one or more detectors installed in the drone.

[0021] The detector may be composed of at least one or more of a camera and a radar.

[0022] Determining whether the vehicle can use drone services may include determining whether the vehicle is capable of communicating with the drone and whether the vehicle is equipped with autonomous driving capabilities.

[0023] When performing autonomous driving, the drone may correct information obtained by the at least one or more detectors based on the position information of the drone and the angle information of the at least one or more detectors.

[0024] While performing autonomous driving, the vehicle may determine whether at least one or more detectors installed in the drone are abnormal.

[0025] While performing autonomous driving, when at least one or more of the at least one or more detectors are abnormal, the vehicle may request the server to rent the drone again.

[0026] The method may also include determining, by the vehicle, whether to terminate autonomous driving, determining, by the vehicle, whether to terminate the use of the drone service based on whether the autonomous driving is terminated, issuing a drone recovery request to the server based on whether the autonomous driving is terminated, and controlling the return of the drone by the server after executing autonomous driving.

[0027] When controlling the return of the drone, when the server causes the drone to return, the server can send the return position to the drone.

[0028] The vehicle can communicate wirelessly between the drone and the server. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The above and other objects, features and advantages of the present invention will become more apparent from the following detailed description in conjunction with the accompanying drawings:

[0030] Figure 1 A block diagram of a system for using a drone to assist autonomous driving according to an embodiment of the present invention is shown;

[0031] Figure 2 for Figure 1 A block diagram of the drone shown in;

[0032] Figure 3 for Figure 1 A block diagram of the vehicle shown in ;

[0033] Figure 4 A flowchart showing a method for using a drone to assist autonomous driving according to an embodiment of the present invention; and

[0034] Figure 5 Detailed description is a flow chart showing a process of terminating drone service according to an embodiment of the present invention. DETAILED DESCRIPTION

[0035] Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals will be used throughout to represent the same or equivalent elements. In addition, detailed descriptions of well-known features and functions will be excluded to avoid unnecessarily obscuring the subject matter of the present invention.

[0036] When describing the elements of the exemplary embodiments of the present invention, the terms first, second, "A", "B", (a), (b), etc. may be used herein. These terms are only used to distinguish one element from another element, but do not limit the corresponding elements, which has nothing to do with the order or priority of the corresponding elements. In addition, unless otherwise defined, all terms used herein, including technical and scientific terms, should be interpreted as the conventions of the field to which the present invention belongs. It should be understood that the terms used herein should be interpreted as having a meaning consistent with their meaning in the context of the present invention and related fields, and unless explicitly defined as such in this article, they will not be interpreted in an idealized or overly formal sense.

[0037] Figure 1 A block diagram of a system for using a drone to assist autonomous driving according to an embodiment of the present invention is shown.

[0038] like Figure 1 As shown in , according to an embodiment of the present invention, a system using a drone to assist autonomous driving may include a drone 100 , a vehicle 200 , and a server 300 .

[0039] The drone 100 may have at least one or more detectors (sensors). Under the control of the server 300, the drone 100 may move to the location of the vehicle 200 that rents the drone 100. The drone 100 obtains peripheral information of the corresponding vehicle 200 through at least one or more detectors. The drone 100 provides the obtained peripheral information to the vehicle 200.

[0040] The vehicle 200 rents the drone 100 through the server 300. The vehicle 200 performs autonomous driving by using peripheral information from the rented drone 100. When the autonomous driving is terminated, the vehicle 200 requests recovery of the drone 100.

[0041] The server 300 can manage at least one or more drones 100. The server 300 stores and manages identification information, status information (e.g., whether the drone 100 fails), rental information of each drone 100 (e.g., whether the drone 100 is rented, rental time, lender, vehicle renting the drone 100, rental fee, etc.), status information of each detector (e.g., whether the detector is contaminated, whether the detector fails, etc.), and other information in the form of a database (DB).

[0042] In response to the drone rental request of the vehicle 200, the server 300 rents one drone among at least one or more drones 100 mastered by the server 300. At this time, the server 300 may rent two or more drones 100 at the request of the vehicle 200.

[0043] In response to the drone recovery request (request to return the drone) of the vehicle 200 , the server 300 may move the rented drone 100 to the recovery location (return location).

[0044] Such a server 300 may include a processor, a memory, and a communication device. The processor may include at least one or more of an application specific integrated circuit (ASIC), a digital signal processor (DSP), a programmable logic device (PLD), a field programmable gate array (FPGA), a central processing unit (CPU), a microcontroller, and a microprocessor. The memory may be implemented with one or more storage media such as a random access memory (RAM), a read-only memory (ROM), a register, etc. The communication device may allow the server 300 to communicate wirelessly with the drone 100 and the vehicle 200. The communication device utilizes communication technologies such as wireless Internet, short-range communication, and / or mobile communication. In this article, wireless LAN (WLAN) (Wi-Fi), wireless broadband (Wibro), global microwave access interoperability (Wimax), etc. may be used as wireless Internet technologies. Bluetooth, near field communication (NFC), radio frequency identification (RFID), infrared data association (IrDA), ultra-wideband (UWB), ZigBee, etc. may be used as short-range communication technologies. Code Division Multiple Access (CDMA), Global System for Mobile Communications (GSM), Long Term Evolution (LTE), LTE-Advanced (LTE-A), etc. may be used as mobile communication technology.

[0045] Figure 2 Shows Figure 1 A block diagram of the drone 100 is shown in FIG.

[0046] refer to Figure 2 , the drone 100 includes a camera 110, a radio detection and ranging (radar) 120, a communication device 130, a global positioning system (GPS) receiver 140, a gyro sensor 150, a memory 160, a power supply 170, and a processor 180. In this document, the communication device 130, the GPS receiver 140, the gyro sensor 150, the memory 160, the power supply 170, and the processor 180 are collectively referred to as a drone control unit (DCU).

[0047] The camera 110 may be installed in the drone 100 to obtain image information. A plurality of cameras 110 may be provided. For example, the camera 110 may be installed on the rear surface of the drone 100, and / or the camera 110 may be installed on the front side, the rear side, the right side, and the left side of the drone 100, respectively. The camera 110 may be implemented using at least one or more image sensors such as a charge coupled device (CCD) image sensor, a complementary metal oxide semiconductor (CMOS) image sensor, a charge induced device (CPD) image sensor, a charge injection device (CID) image sensor, and the like. The camera 110 may include at least one or more lenses such as a standard lens, an ultra-wide-angle lens, a wide-angle lens, a zoom lens, a macro lens, a telephoto lens, a fisheye lens, and a quasi-fisheye lens. The camera 110 may include a device capable of adjusting the angle (lens angle) of each camera 110.

[0048] The camera 110 may be connected to the processor 180 via a low voltage differential signal (LVDS) cable or an Ethernet cable. The camera 110 transmits acquired image raw data to the processor 180.

[0049] The radar 120 may transmit electromagnetic waves to surrounding objects, may receive electromagnetic waves reflected from surrounding objects, and may verify the distance to the surrounding objects, the direction to the surrounding objects, and the height of the surrounding objects. At least one or more radars 120 may be installed on the front side of the drone 100, may be installed on the rear side of the drone 100, may be installed on the right side of the drone 100, and / or may be installed on the left side of the drone 100.

[0050] The radar 120 may communicate with the processor 180 by using a communication technology such as a controller area network (CAN), Flexray, etc. The radar 120 transmits sensed (measured) radar raw data to the processor 180 .

[0051] The communication device 130 is a hardware device and wirelessly communicates with the vehicle 200 and the server 300 by using wireless Internet, short-range communication and / or mobile communication. Herein, WLAN (Wi-Fi), Wibro, Wimax, etc. can be used as wireless Internet technology. Bluetooth, NFC, RFID, IrDA, UWB, ZigBee, etc. can be used as short-range communication technology. CDMA, GSM, LTE, LTE-A, etc. can be used as mobile communication technology.

[0052] The GPS receiver 140 measures the current position of the drone 100. The GPS receiver 140 is a device capable of receiving information from a GPS satellite and calculating the current position of the drone 100 by using a signal from the GPS satellite.

[0053] The gyro sensor 150 measures the azimuth change of the drone 100. That is, the gyro sensor 150 measures the angular velocity of the drone 100. The gyro sensor 150 may be replaced by a gyroscope.

[0054] The memory 160 may store software for controlling the overall operation of the drone 100, image processing algorithms, radar data processing algorithms, algorithms for processing and calculating image and radar data, correction algorithms, etc. The memory 160 may temporarily store input data and / or output data of the processor 180. In addition, the memory 160 may store images obtained by the camera 110 and radar data measured by the radar 120.

[0055] The memory 160 can be implemented using at least one or more storage media (recording media) including flash memory, hard disk, secure digital (SD) card, RAM, ROM, electrically erasable and programmable ROM (EEPROM), erasable programmable ROM (EPROM), registers, removable disks, network storage devices, etc.

[0056] The power supply 170 provides power required to operate each component installed on the drone 100. The power supply 170 receives power from a battery, a fuel cell, etc. installed in the drone 100 and then provides the power to each component. In the present invention, the power supply 170 is an electrical device that provides power to an electrical load.

[0057] The processor 180 controls the overall operation of the drone 100. For example, the processor 180 may control the flight of the drone 100 according to a control command from a remote controller. The remote controller may be implemented in the server 300, or may be implemented independently of the server 300. Such a processor 180 may include at least one or more of an ASIC, a DSP, a PLD, an FPGA, a CPU, a microcontroller, and a microprocessor.

[0058] The processor 180 receives the location of the vehicle (rental vehicle location) where the drone 100 will be rented from the server 300 through the communication device 130. The processor 180 may move the drone 100 to the received rental vehicle location. At this time, the processor 180 controls the flight controller (not shown) to move the drone 100 to the rental vehicle location.

[0059] When the current position of the drone 100 matches the rental vehicle position, the processor 180 lands the drone 100 and installs it on the vehicle 200. After the drone 100 is installed on the vehicle 200, the processor 180 is connected to the vehicle 200 for communication.

[0060] When the communication connection with the vehicle 200 is successful, the processor 180 obtains peripheral information of the vehicle 200 through the camera 110 and the radar 120. That is, the processor 180 obtains peripheral image information of the vehicle 200 through the camera 110 and obtains peripheral object information of the vehicle 200 through the radar 120.

[0061] Such a processor 180 includes an image processor 181 , a data processor 182 , and a complex processor 183 .

[0062] The image processor 181 performs image processing on the image obtained by the camera 110 , such as noise removal, distortion correction, color reproduction, adjustment of image quality and saturation, file compression, etc. The image processor 181 may also be implemented with a separate image processing processor.

[0063] The image processor 181 corrects the image obtained by the camera 110 based on the drone position information and the camera angle information obtained through the GPS receiver 140 and the gyro sensor 150. The image processor 181 obtains a front view image, a rear view image, and a side view image through the front view camera 110, the rear view camera 110, and the side view camera 110, and corrects the obtained images respectively.

[0064] The data processor 182 performs data processing such as noise removal on the radar data (radar raw data) sensed by the radar 120. The data processor 182 corrects the acquired radar data based on the drone position information and radar angle information obtained through the GPS receiver 140 and the gyro sensor 150. The data processor 182 obtains a plurality of multi-directional radar data (forward-looking data, rear-looking data, and side-looking data) through a plurality of radars 120 and corrects each of the acquired radar data.

[0065] The composite processor 183 may combine the plurality of pieces of image information corrected by the image processor 181 to generate one piece of image information. For example, the composite processor 183 may generate one image by using a corrected front view image, a corrected rear view image, and a corrected side view image.

[0066] The composite processor 183 may combine the corrected forward-looking radar data, the corrected rearward-looking radar data, and the corrected side-looking radar data transmitted from the data processor 182 to generate one piece of radar information.

[0067] The composite processor 183 generates final image information and final radar information by using the combined image information and the combined radar information. The composite processor 183 generates a packet (transmission data) according to the communication protocol so as to transmit the final image information and the final radar information to the vehicle 200 through the communication device 130. The composite processor 183 transmits the final image information and the final radar information to the vehicle 200 through the communication device 130.

[0068] In the present invention, the image processor 181, the data processor 182, and the composite processor 183 may be embedded in the processor 180 as one device.

[0069] One embodiment is illustrated as controlling the flight of the drone 100 by a remote controller. However, embodiments of the present invention are not limited thereto. For example, the drone 100 may be implemented to perform autonomous flight.

[0070] Figure 3 Shows Figure 1 A block diagram of a vehicle 200 is shown in FIG.

[0071] The vehicle 200 includes a communication device 210 , a detector 220 , a memory 230 , a steering controller 240 , a brake controller 250 , a steering controller 260 , a transmission controller 270 , and a processor 280 .

[0072] The communication device 210 may wirelessly communicate with the drone 100 and the server 300. Wireless Internet, short-range communication, and / or mobile communication may be used as a wireless communication technology. The communication device 210 receives peripheral information of the vehicle 200 from the drone 100. The communication device 210 may send a message for requesting (applying) drone rental and drone return (recovery) to the server 300.

[0073] The detector 220 obtains vehicle status information from various sensors and / or ECUs installed in the vehicle 200. The various sensors may include a steering angle sensor, a vibration sensor, a speed sensor, an acceleration sensor, etc. The detector 220 obtains vehicle information (e.g., vehicle position, braking status, etc.) from ECUs such as an airbag system, a door system, an electronic stability control (ESC), a traction control system (TCS), an anti-lock braking system (ABS), etc. through a vehicle network. The vehicle network may be implemented using communication technologies such as a controller area network (CAN), a media oriented systems transport (MOST) network, a local interconnect network (LIN), and / or Flexray.

[0074] The memory 230 stores software for programming the processor 280 to perform specified operations, navigation programs, autonomous driving control algorithms (programs), etc. The memory 230 may store input data and / or output data of the processor 280. In addition, the memory 230 may store data (information) received through the communication device 210, vehicle status information obtained through the detector 220, map data, etc.

[0075] The memory 230 may be implemented using at least one or more storage media (recording media) such as a flash memory, a hard disk, an SD card, a RAM, a ROM, an EEPROM, an EPROM, a register, a removable disk, a network storage device, and the like.

[0076] The steering controller 240 may be implemented with a motor driven power steering (MDPS) as an actuator for controlling the steering of the vehicle.

[0077] The brake controller 250 may be implemented with an electronic stability control (ESC) as an actuator to control deceleration of the vehicle 200. The brake controller 250 adjusts the brake pressure to follow the target speed requested by the processor 280 during autonomous driving.

[0078] The driving controller 260 controls the acceleration of the vehicle 200 as an actuator for controlling the engine of the vehicle 200. The driving controller 260 may be implemented by an engine management system (EMS). The driving controller 260 controls the driving torque of the engine according to the accelerator pedal position information output from the accelerator pedal position sensor. The driving controller 260 controls the output of the engine so as to follow the driving speed of the vehicle 200 requested by the processor 280 during autonomous driving (smart cruise control (SCC) system is turned on).

[0079] The transmission controller 270 may be implemented with a shift-by-wire (SBW) as an actuator for controlling a transmission (shifting) of the vehicle 200. The transmission controller 270 controls the shifting of the vehicle 200 according to a gear position and a gear state range.

[0080] Additionally, the vehicle 200 may include a body controller to control windows, air conditioning and heating systems, an anti-theft system, and the like.

[0081] In the present invention, each of the steering controller 240 , the brake controller 250 , the steering controller 260 , and the transmission controller 270 may be an electronic control unit (ECU) or may be embedded in one ECU for controlling various electrical systems as described above.

[0082] The processor 280 may request the server 300 for drone service use through the communication device 210. The processor 280 may request the server 300 for drone rental according to whether the server 300 accepts the drone service use. In other words, when the server 300 accepts the drone service use, the processor 280 may request drone rental. The server 300 determines whether the vehicle 200 is able to communicate with the drone 100 and whether the vehicle 200 is equipped with an autonomous driving function, and then determines approval of the drone service use according to the determination result.

[0083] After the request for renting a drone is made, when the rented drone 100 lands (arrives) on the vehicle 200 and is then mounted on the vehicle 200, the processor 280 sets a communication connection with the communication device 130 of the drone 100 through the communication device 210. After setting the communication connection, the processor 280 checks whether the camera 110 and the radar 120 of the drone 100 are abnormal.

[0084] When the camera 110 and the radar 120 of the drone 100 are normal, the processor 280 waits for autonomous driving. Meanwhile, when the camera 110 and the radar 120 of the drone 100 are abnormal, the processor 280 requests the server 300 to rent a new drone 100. An embodiment is illustrated as follows: after the rented drone 100 is normally installed in the vehicle 200, the vehicle 200 determines whether the camera 110 and the radar 120 installed in the drone 100 are abnormal. However, embodiments of the present invention are not limited thereto. For example, the server 300 may be implemented to rent the drone 100 after the server 300 determines that the camera 110 and the radar 120 installed in the drone 100 are normal.

[0085] When the vehicle 200 starts to travel, the processor 280 operates the autonomous driving function and then enters the autonomous driving mode. The processor 280 performs autonomous driving of the vehicle 200 based on the peripheral information obtained by the camera 110 and the radar 120 of the drone 100 and the vehicle state information obtained by the detector 220. In other words, the processor 280 controls at least one or more of the steering controller 240, the brake controller 250, the driving controller 260, and the transmission controller 270 based on the obtained peripheral information and the obtained vehicle information.

[0086] When autonomous driving is terminated, the processor 280 notifies the driver whether to terminate the use of the drone. The driver manipulates a user interface (not shown) to input data associated with whether the use of the drone is terminated, and the processor 280 may determine whether the use of the drone is terminated based on the data input from the user interface (not shown). When it is determined that the use of the drone is terminated, the processor 280 issues a drone recovery request to the server 300. When the driver wishes to use the drone 100 continuously, the processor 280 may issue a request to the server 300 to extend the use of the drone.

[0087] Figure 4 The flowchart is a method for using a drone to assist autonomous driving according to an embodiment of the present invention.

[0088] refer to Figure 4 In operation S110, the vehicle 200 issues a request for drone service use to the server 300. When the request for drone service use is issued, the vehicle 200 sends vehicle information to the server 300. The vehicle information includes a vehicle identification number, a vehicle number, a vehicle type, communications that can be used by the vehicle, whether an autonomous driving function is installed, and the like.

[0089] In operation S120, the server 300 checks whether the vehicle 200 can use the drone service at the request of the vehicle 200. The server 300 determines whether the vehicle 200 can communicate with the drone 100 and whether the vehicle 200 is installed with an autonomous driving function to determine approval (permission) for the use of the drone service. In other words, when the vehicle 200 is able to communicate with the drone 100 and the autonomous driving function is installed, the server 300 may allow the use of the drone service. When the vehicle 200 cannot communicate with the drone 100 or the autonomous driving function is not installed, or when the vehicle 200 cannot communicate with the drone 100 and the autonomous driving function is not installed, the server 300 may not allow the use of the drone service.

[0090] In operation S130, the server 300 transmits a check result obtained by checking whether the drone service use is possible to the vehicle 200. In other words, the server 300 transmits a notification of allowing (approving) the drone service use or a notification of not allowing the drone service use.

[0091] In operations S140 and S150, when the received inspection result indicates that the drone service can be used, the vehicle 200 sends a request for drone rental to the server 300. The vehicle 200 sends a drone rental request message including information such as the desired duration of drone use and the type of drone.

[0092] In operation S160, in response to the request of the vehicle 200 for drone rental, the server 300 rents one of the owned drones 100. At this time, the server 300 may send information about the rented drone 100 (e.g., drone identification information, the number of installed cameras, the number of installed radars, etc.) to the vehicle 200.

[0093] In operation S170 , the server 300 transmits the location of the vehicle renting the drone 100 (rented vehicle location) to the rented drone 100 .

[0094] In operation S180, the drone 100 moves to the rental vehicle location and is installed in the corresponding vehicle 200. Since the drone 100 forms or releases a vacuum state between the body surface of the vehicle 200 and a suction plate installed on the drone 100, the drone 100 may include a vacuum device detachable to the body surface of the vehicle 200.

[0095] In operation S190, when the drone 100 arrives, the vehicle 200 sets up a connection for communication with the drone 100. In other words, the vehicle 200 and the drone 100 perform a connection procedure for communication between each other.

[0096] After connecting the communication with the drone 100, the vehicle 200 determines whether the camera 110 and the radar 120 installed in the drone 100 are abnormal in operation S200. The vehicle 200 determines whether the camera 110 and the radar 120 are abnormal by using a test algorithm stored in the memory 230.

[0097] In operation S210, when the camera 110 and the radar 120 installed in the drone 100 are normal, the vehicle 200 performs autonomous driving. The vehicle 200 controls autonomous driving by using peripheral information obtained by the camera 110 and the radar 120 installed in the drone 100 and vehicle state information obtained from various sensors and ECUs installed in the vehicle 200.

[0098] When the camera 110 and the radar 120 installed in the drone 100 are abnormal, the vehicle 200 returns to operation S150, and then issues a new drone rental request to the server 300 to rent a new drone 100. When the vehicle 200 requests the server 300 to rent the drone 100 again due to the abnormality of the camera 110 and / or the radar 120 of the rented drone 100, the vehicle 200 sends the server 300 the reason why the vehicle 200 requests the server 300 to rent the drone 100 again.

[0099] Figure 5 Detailed description is a flow chart showing a process of terminating drone service according to an embodiment of the present invention.

[0100] In operation S310 and operation S320, while performing autonomous driving, the vehicle 200 determines whether to terminate the autonomous driving. The vehicle 200 determines to terminate the autonomous driving when the vehicle 200 arrives at a destination set in the navigation terminal or when the driver inputs a command to terminate the autonomous driving.

[0101] In operation S330, when the autonomous driving is terminated, the vehicle 200 determines whether to terminate the use of the drone. The vehicle 200 outputs a notification of the termination of the autonomous driving, and then when the autonomous driving is terminated, determines whether to terminate the use of the drone in response to a user input.

[0102] When the drone use termination is verified, the vehicle 200 disconnects the communication connection with the drone 100. In operation S350, the vehicle 200 requests the server 300 to collect the drone 100.

[0103] In operation S360, when receiving the drone recovery request of the vehicle 200, the server 300 directs the rented drone 100 to return. When directing the rented drone 100 to return, the server 300 may provide information about the return location (recovery location) to the drone 100. Meanwhile, the information about the return location may be stored in the memory 160 of the drone 100.

[0104] In operation S370, the drone 100 moves to the return position according to the return instruction of the server 300. When the drone 100 reaches the return position, the drone 100 may send a notification that the drone 100 moves to the return position to the server 300. When verifying that the drone 100 moves to the return position, the server 300 may send a notification that the drone 100 moves to the return position to the vehicle 200.

[0105] The present invention can rent drones equipped with sensing devices such as cameras, radars, etc., and can assist autonomous driving through collaboration between drones and autonomous driving systems in vehicles, and thus can assist autonomous driving of various vehicles without being limited by vehicle types and vehicle categories.

[0106] In addition, the present invention uses the sensing device installed in the drone only when the vehicle is in autonomous driving. Therefore, there is no need to install sensing devices such as cameras, radars, etc. in the vehicle, thereby reducing the unit cost of the vehicle.

[0107] In addition, drone rental companies can maintain sensing devices such as cameras, radars, etc. installed in drones, thereby providing convenience for pilots.

[0108] In the above, although the present invention has been described with reference to exemplary embodiments and the accompanying drawings, the present invention is not limited thereto, but can be variously modified and changed by a person skilled in the art of the present invention without departing from the spirit and scope of the attached claims. Therefore, the embodiments of the present invention are not intended to limit the technical spirit of the present invention, but are provided only for illustrative purposes. The protection scope of the present invention should be interpreted by the attached claims, and all equivalents thereof should be interpreted as being included within the scope of the present invention.

Claims

1. A system for assisting autonomous driving using a drone, the system include: A drone, wherein the drone is configured to obtain peripheral information through at least one or more detectors; a vehicle configured to perform autonomous driving by using the peripheral information; as well as A server, wherein the server is configured to: In response to a drone service usage request by the vehicle, determining whether drone service usage by the vehicle is possible, and leasing the drone to assist autonomous driving of the vehicle in response to a drone leasing request from the vehicle; wherein, according to the determination result of the server, the vehicle sends a drone rental request to the server to rent the drone; When receiving a drone service usage request from the vehicle, the server determines whether it is possible to communicate with the drone of the vehicle and whether an autonomous driving function is available to determine a drone service usage permission.

2. The system according to claim 1, in, Each of the at least one or more detectors includes at least one or more of a camera and a radar.

3. The system according to claim 1, in, When the drone is rented, the server sends the location information of the vehicle to the drone.

4. The system according to claim 3, in, The drone moves to the location of the vehicle under the control of the server.

5. The system according to claim 1, in, The drone comprises: a communication device for wirelessly communicating between the vehicle and the server; a global positioning system (GPS) receiver, wherein the global positioning system (GPS) receiver is configured to detect the location information of the drone; a gyro sensor configured to detect angle information of the at least one or more detectors; and A processor is configured to correct peripheral information detected by the at least one or more detectors based on the position information of the drone and the angle information of the at least one or more detectors.

6. The system according to claim 1, in, The vehicle determines whether one of the at least one or more detectors installed in the drone is abnormal, and requests the server to rent another drone according to the determination result of the vehicle.

7. The system according to claim 1, in, When autonomous driving is terminated, the vehicle determines whether to terminate drone service usage in response to user input.

8. The system according to claim 7, in, When it is determined that the use of the drone service is terminated, the vehicle sends a drone recovery request to the server.

9. The system according to claim 8, in, In response to the vehicle's drone recovery request, the server sends a return location to the drone.

10. A vehicle, include: a communication device configured to communicate wirelessly with a server and a drone; a detector configured to detect vehicle status information; as well as a processor, the processor being in communication with the communication device and the detector, the processor being configured to: issuing a drone service usage request to the server, wherein the server is configured to determine whether the vehicle can use the drone service, According to the determination result of the server, a drone rental request is sent to the server to rent a drone; obtaining peripheral information by at least one or more detectors installed in the drone; as well as Performing autonomous driving based on the acquired peripheral information and the vehicle state information, wherein, in response to the drone service use request of the vehicle, the server determines whether drone service use of the vehicle is possible; When receiving a drone service usage request from the vehicle, the server determines whether it is possible to communicate with the drone of the vehicle and whether an autonomous driving function is available to determine a drone service usage permission.

11. A method for using a drone to assist autonomous driving, the method The following steps are involved: The vehicle sends a request to the server for the use of drone services; determining, by the server, whether drone service use of the vehicle is possible; The vehicle sends a drone rental request to the server according to the determination result of the server; leasing the drone by the server in response to a request by the vehicle; moving the drone to the location of the vehicle under the control of the server; and performing autonomous driving by the vehicle based on peripheral information obtained by at least one or more detectors installed in the drone; When receiving a drone service usage request from the vehicle, the server determines whether it is possible to communicate with the drone of the vehicle and whether an autonomous driving function is available to determine a drone service usage permission.

12. The method according to claim 11, in, The at least one or more detectors of the drone include at least one or more of a camera and a radar.

13. The method according to claim 11, in, In the step of performing autonomous driving, the drone corrects information obtained by the at least one or more detectors based on position information of the drone and angle information of the at least one or more detectors.

14. The method according to claim 11, in, In the step of performing autonomous driving, the vehicle determines whether the at least one or more detectors installed in the drone are abnormal.

15. The method according to claim 14, in, In the step of performing autonomous driving, when at least one or more of the at least one or more detectors are abnormal, the vehicle sends a request to the server to re-rent another drone.

16. The method according to claim 11, further comprising: include: After executing the steps of autonomous driving, Determining, by the vehicle, whether to terminate autonomous driving; The vehicle determines whether the use of the drone service is terminated according to whether the autonomous driving is terminated; Sending a drone recovery request to the server according to whether the autonomous driving is terminated; and The return of the drone is controlled by the server.

17. The method according to claim 16, in, In the step of controlling the return of the drone, when the server causes the drone to return, the server sends the return position to the drone.

18. The method according to claim 11, in, The vehicle performs wireless communication between the drone and the server.

Citation Information

Patent Citations

  • Method for compressing point cloud

    KR1020180087348A

  • Vehicular-mounted unmanned aerial vehicle auxiliary driving system and operation mode

    CN105825713A

  • Monitoring system and device based on unmanned aerial vehicle

    CN106325290A

  • Unmanned aerial vehicle automobile system and management method thereof

    CN106843279A

  • UAV(unmanned aerial vehicle) rental system and method based on mobile terminal

    CN107330773A