Method and device for guiding truck to run by unmanned aerial vehicle, unmanned aerial vehicle, equipment and medium
The truck is guided to drive by taking bird's-eye images by drones, which solves the problem that there are many obstacles in the yard, which makes the truck unable to detect surrounding obstacles, improves driving safety and detection accuracy, and reduces cost and applicability.
Patent Information
- Application Number
- CN202411156749.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-21
- Publication Date
- 2024-11-29
AI Technical Summary
In the yard, due to the large number of obstacles, the sensor is easily blocked, causing the truck to be unable to detect all obstacles around, reducing driving safety.
Use a drone to follow the truck, take a bird's eye image and send it to a computing device, guide the truck through the bird's eye image, cover more obstacles, and avoid obstacle occlusion issues.
Improves truck driving safety and reduces costs. It is suitable for different types of open-air yards and trucks, without the need to renovate the yard, and enhances the accuracy of detecting obstacles and guiding the truck.
Smart Images

Figure CN119043278A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of vehicle technology, and in particular to a method, device, drone, equipment and medium for guiding truck driving by drone. Background Art
[0002] There are yards in airports, ports, mines, and logistics parks. The yard is one of the driving scenes for trucks, where trucks load and unload goods. The environment in the yard is complex, with various obstacles, such as moving vehicles, walking pedestrians, and roadside railings, which can easily cause collisions. Therefore, how to ensure the driving safety of trucks in the yard is a problem that people are concerned about.
[0003] In the prior art, sensors such as lidar and cameras are usually installed on trucks, so that trucks can realize functions such as forward collision warning (FCW), active emergency braking (AEB), blind spot detection (BSD), and rear cross traffic alert (RCTA). By detecting obstacles around the vehicle through sensors, the probability of a truck collision is reduced.
[0004] However, when there are a large number of obstacles in the yard, the sensors may be blocked by some obstacles and the truck may not be able to detect other obstacles, which may easily cause collisions and reduce the safety of truck driving. Summary of the invention
[0005] The embodiments of the present application provide a method, device, drone, equipment and medium for guiding truck driving by drone, which are used to solve the problem in the prior art that when there are a large number of obstacles in the yard, the truck may not be able to detect all obstacles around the truck, resulting in low truck driving safety.
[0006] In a first aspect, an embodiment of the present application provides a method for a drone to guide a truck to travel, which is applied to a drone, and the method includes:
[0007] Upon receiving a take-off instruction, flying to a preset identification area to capture a truck image, wherein the take-off instruction includes a target truck identification of the target truck;
[0008] Determining whether the target truck is in the preset recognition area according to the target truck identifier and the truck image;
[0009] If it is determined that the target truck is in the preset identification area, flying along the target truck corresponding to the target truck identifier and taking a bird's-eye view image of the target truck in real time;
[0010] The bird's-eye view image of the target truck captured in real time is sent to a computing device.
[0011] In a specific implementation, determining whether the target truck is in the preset recognition area according to the target truck identifier and the truck image includes:
[0012] Performing image recognition on the truck image to obtain a recognition result, wherein the recognition result includes a result identifier indicating a recognition failure, or includes at least one recognized truck identifier to be determined;
[0013] If the recognition result includes a result mark indicating recognition failure, it is determined that the target truck is not in the preset recognition area;
[0014] If the recognition result includes at least one truck identifier to be determined, it is determined whether the target truck is in the preset recognition area according to the at least one truck identifier to be determined and the target truck identifier.
[0015] In a specific implementation, the determining whether the target truck is in the preset identification area according to the at least one to-be-determined truck identifier and the target truck identifier includes:
[0016] Determining whether the at least one to-be-determined truck identifier includes the target truck identifier;
[0017] If the at least one to-be-determined truck identifier includes the target truck identifier, determining that the target truck is in the preset identification area;
[0018] If the at least one truck identification to be determined does not include the target truck identification, it is determined that the target truck is not in the preset recognition area.
[0019] In a specific embodiment, the method further comprises:
[0020] When receiving the drone release command, it will fly to the preset parking area and stop.
[0021] In a specific embodiment, the bird's-eye view image is processed by the computing device to guide the driving of the truck.
[0022] In a second aspect, an embodiment of the present application provides a method for a drone to guide a truck to travel, which is applied to a computing device, and the method includes:
[0023] Receive a real-time bird's-eye view image of a target truck sent by a drone;
[0024] The truck is guided to travel according to the bird's-eye view image of the target truck captured in real time.
[0025] In a specific implementation, guiding the truck to travel according to the bird's-eye view image of the target truck captured in real time includes:
[0026] Predicting whether the target truck will collide based on the bird's-eye view image of the target truck captured in real time;
[0027] If it is predicted that the target truck will collide, a collision warning message is output.
[0028] In a specific implementation, predicting whether the target truck will collide based on the bird's-eye view image of the target truck captured in real time includes:
[0029] Calculating the turning angle, traveling direction and speed of each obstacle in the bird's-eye view image, as well as the turning angle, traveling direction and speed of the target truck, based on the bird's-eye view image of the target truck captured in real time;
[0030] Whether the target truck will collide is predicted based on the turning angle, traveling direction and speed of each obstacle in the bird's-eye view image and the turning angle, traveling direction and speed of the target truck.
[0031] In a specific embodiment, the method further comprises:
[0032] When receiving a reversing signal sent by the target truck, generating a reversing trajectory according to the turning angle and the traveling direction of the target truck;
[0033] Superimposing the reversing track onto the bird's-eye view image of the target truck captured in real time to obtain a reversing prompt image;
[0034] The reversing prompt image is outputted.
[0035] In a third aspect, an embodiment of the present application provides a drone-guided truck driving device, comprising:
[0036] A flight control module, configured to control the UAV to fly to a preset identification area to capture a truck image upon receiving a take-off instruction, wherein the take-off instruction includes a target truck identifier of the target truck;
[0037] Processing modules for:
[0038] Determining whether the target truck is in the preset recognition area according to the target truck identifier and the truck image;
[0039] If it is determined that the target truck is in the preset identification area, the flight control module follows the target truck corresponding to the target truck identifier to fly;
[0040] A shooting module, used for shooting a bird's-eye view image of the target truck in real time;
[0041] The sending module is used to send the bird's-eye view image of the target truck taken in real time to a computing device.
[0042] In a fourth aspect, an embodiment of the present application provides a drone-guided truck driving device, comprising:
[0043] A receiving module, used for receiving a bird's-eye view image of a target truck shot in real time and sent by a UAV;
[0044] The guiding module is used to guide the truck to travel according to the bird's-eye view image of the target truck taken in real time.
[0045] In a fifth aspect, an embodiment of the present application provides a drone, including:
[0046] Processor, memory, communication interface, camera equipment;
[0047] The memory is used to store executable instructions of the processor;
[0048] Wherein, the processor is configured to execute the drone-guided truck driving method described in any one of the first aspects by executing the executable instructions.
[0049] In a sixth aspect, an embodiment of the present application provides a computing device, including:
[0050] Processor, memory, communication interface;
[0051] The memory is used to store executable instructions of the processor;
[0052] Wherein, the processor is configured to execute the drone-guided truck driving method described in any one of the second aspects by executing the executable instructions.
[0053] In the seventh aspect, an embodiment of the present application provides a readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method for guiding a truck to travel by a drone as described in any one of the first aspect or any one of the second aspect is implemented.
[0054] In an eighth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, is used to implement the drone-guided truck driving method described in any one of the first aspect or any one of the second aspect.
[0055] The embodiments of the present application provide a method, device, drone, equipment and medium for guiding truck driving by drone. When the drone receives a take-off command, it flies to a preset identification area to capture an image of the truck. When it determines that the target truck is in the preset identification area, it follows the target truck and sends a real-time bird's-eye view image of the target truck to a computing device. The computing device guides the truck to drive based on the bird's-eye view image. Since the bird's-eye view image has a wider viewing angle and can cover more obstacles around the truck, and there will be no obstacles blocking the camera on the drone, the present solution guides the truck to drive by taking a bird's-eye view image of the truck by the drone, which effectively improves the driving safety of the truck. BRIEF DESCRIPTION OF THE DRAWINGS
[0056] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0057] Figure 1 Schematic diagram of the scene of loading and unloading goods for trucks;
[0058] Figure 2 This is a schematic diagram of the storage yard scene;
[0059] Figure 3 A schematic diagram of an application scenario of the method for guiding truck driving by a drone provided in this application;
[0060] Figure 4 A schematic diagram of a process flow of a first embodiment of a method for guiding a truck to travel by a drone provided in the present application;
[0061] Figure 5 A schematic diagram of the flow chart of Embodiment 2 of the method for guiding truck driving by a drone provided in this application;
[0062] Figure 6 A schematic diagram of a flow chart of a fourth embodiment of a method for guiding a truck to travel by a drone provided in the present application;
[0063] Figure 7 A schematic diagram of a reversing reminder image provided for this application;
[0064] Figure 8 This is a structural schematic diagram of Embodiment 1 of the UAV-guided truck driving device provided in this application;
[0065] Fig. 9 A schematic diagram of the structure of the second embodiment of the UAV-guided truck driving device provided in this application;
[0066] Fig.10A schematic diagram of the structure of a drone provided for this application;
[0067] Fig.11 A schematic diagram of the structure of a computing device provided in this application. DETAILED DESCRIPTION
[0068] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments made by ordinary technicians in this field under the enlightenment of the embodiments belong to the scope of protection of the present application.
[0069] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0070] Trucks are indispensable for transporting goods in airports, ports, mines, logistics parks and other scenarios. In these scenarios, there are yards where trucks load and unload goods. Due to the complex environment of the yard, there are many obstacles, such as stones, railings, moving vehicles, and pedestrians on the roadside, which will affect the driving safety of trucks.
[0071] For example, Figure 1 A schematic diagram of a truck loading and unloading scenario, such as Figure 1 As shown, the truck drives from outside the yard to inside the yard, completes loading and unloading of goods in the yard, and then drives out of the yard.
[0072] In the prior art, in order to reduce the impact of obstacles on the driving safety of trucks, sensors such as laser radars and cameras are usually installed on trucks to detect obstacles. However, when there are multiple obstacles around the truck, some obstacles may block the sensors, making it impossible for other sensors to detect, which may easily cause collisions and lead to lower driving safety of the truck.
[0073] For example, Figure 2This is a schematic diagram of the storage yard scene. Figure 2 As shown, a laser radar is installed on the side of the truck 21. The black diamond block in the figure is an obstacle 22, and the black circle is an obstacle 23. The truck 21 is reversing, the obstacle 22 is a stationary obstacle, and the obstacle 23 is an obstacle moving toward the truck 21. The laser radar can scan the obstacle 22, but the laser radar cannot scan the obstacle 23 because the obstacle 23 is located behind the obstacle 22. When the truck 21 is reversing, the obstacle 23 moves toward the truck 21, which may cause a collision.
[0074] In view of the problems existing in the prior art, the inventor discovered during the process of studying the method of using drones to guide truck driving that the reason for the collision in the prior art is that the sensor cannot detect many obstacles around the truck, while the bird's-eye view has a larger viewing angle and is a top-down view, which can cover more obstacles around the truck. Therefore, a drone can be used to follow the truck and take a bird's-eye view of the truck, and then the bird's-eye view can be used to guide the truck to drive, thereby improving the safety of truck driving. Based on the above inventive concept, the drone-guided truck driving scheme in this application is designed.
[0075] For example, Figure 3 A schematic diagram of an application scenario of the method for guiding truck driving by a drone provided in this application, such as Figure 3 As shown, the application scenario includes: a truck 31, a drone 32 and a computing device 33.
[0076] exist Figure 3 In the application scenario shown, a preset identification area is set at the entrance of the yard. Truck 31 drives to the preset identification area, and the gate in the yard identifies the license plate number of truck 31. Then, based on the correspondence between the preset license plate number and the truck identification, the target truck identification corresponding to the license plate number of truck 31 is determined, and then a take-off command is generated and sent to drone 32, and the take-off command includes the target truck identification of truck 31.
[0077] After receiving the take-off command, the drone 32 flies to the preset identification area to capture the truck image, and then determines whether the truck 31 is in the preset identification area based on the target truck logo and the truck image.
[0078] When the drone 32 determines that the truck 31 is in the preset identification area, it follows the truck 31 and takes a bird's-eye view image of the truck 31 in real time, and then sends the bird's-eye view image of the truck 31 taken in real time to the computing device 33 .
[0079] The computing device 33 guides the truck to travel according to the bird's-eye view image of the truck 31 taken in real time. For example, after generating a driving trajectory according to the turning angle and the direction of travel of the vehicle, the driving trajectory is superimposed on the bird's-eye view image of the truck 31 taken in real time and sent to the truck 31. The truck 31 travels according to the image to prevent a collision. Alternatively, the computing device 33 determines the position of the obstacle according to the bird's-eye view image of the truck 31 taken in real time, and then controls the driving of the truck 31 to avoid a collision of the truck 31. Alternatively, the computing device 33 predicts whether the truck 31 will collide according to the bird's-eye view image of the truck 31 taken in real time. If it is predicted that the truck 31 will collide, a collision warning message is output. The output method may be to send the collision warning message to the vehicle computer of the truck 31, and the vehicle computer of the truck 31 displays and plays the collision warning message, so that the driver of the truck 31 can be informed that a collision is about to occur, thereby driving safely.
[0080] It should be noted that Figure 3 This is only a schematic diagram of an application scenario provided by the embodiment of the present application. Figure 3 The actual form of the various devices included in the Figure 3 The interaction mode between devices is limited, and in the specific application of the solution, it can be set according to actual needs.
[0081] The technical solution of the present application is described in detail below through specific embodiments. It should be noted that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.
[0082] Figure 4 This is a flow chart of the first embodiment of the method for guiding a truck to travel by a drone provided in this application. This embodiment of the application describes the situation where a drone takes a bird's-eye view image of a truck, and a computing device guides the truck to travel based on the bird's-eye view image. The method in this embodiment can be implemented by software, hardware, or a combination of software and hardware. Figure 4 As shown, the method for guiding a truck to travel by a drone specifically includes the following steps:
[0083] S401: When receiving the take-off command, the UAV flies to the preset identification area to capture the image of the truck.
[0084] Exemplarily, a preset identification area is set at the entrance of the yard. After the target truck drives into the preset identification area, the yard's gate can identify the license plate number of the target truck, and then determine the target truck identification corresponding to the license plate number based on the preset correspondence between the license plate number and the truck identification, that is, the target truck identification of the target truck, and then generate a take-off command and send it to the drone, the take-off command including the target truck identification of the target truck.
[0085] Exemplarily, the preset identification area is set at the entrance of the yard, the target truck is equipped with a radio frequency identification (RFID) tag, the RFID tag contains the target truck identification of the target truck, and the gate of the yard is equipped with an RFID reader. After the target truck drives to the preset identification area, the RFID reader in the gate can read the content in the RFID tag in the target truck and obtain the target truck identification of the target truck. Then the gate generates a take-off command and sends it to the drone, and the take-off command includes the target truck identification of the target truck.
[0086] Exemplarily, the preset identification area is set in a designated area within the yard. After the target truck drives into the preset identification area, the yard manager can use the terminal device to send a take-off instruction to the drone, and the take-off instruction includes the target truck identifier of the target truck.
[0087] It should be noted that the embodiment of the present application does not limit the method of sending take-off instructions to the drone, which can be determined according to actual conditions.
[0088] In this step, upon receiving the take-off command, the drone flies to the preset identification area to capture images of the truck.
[0089] It should be noted that the preset identification area can be set in the yard or outside the yard, and the preset identification area is an area that the truck passes through. The embodiment of the present application does not limit the location of the preset identification area, which can be determined according to actual conditions.
[0090] S402: The UAV determines whether the target truck is in a preset identification area based on the target truck identification and the truck image.
[0091] In this step, after the drone captures the truck image, it can perform image recognition on it to determine whether the target truck logo can be identified, and then determine whether the target truck is in the preset identification area to determine whether to continue the process of guiding the truck to drive.
[0092] S403: If the UAV determines that the target truck is in the preset identification area, it will fly along the target truck corresponding to the target truck identifier and take a bird's-eye view image of the target truck in real time.
[0093] In this step, if the UAV determines that the target truck is in the preset identification area, it means that it is necessary to continue the process of guiding the truck to drive. The UAV follows the target truck corresponding to the target truck identification and takes a bird's-eye view image of the target truck in real time. The bird's-eye view image is processed by a computing device and used to guide the truck to drive. Guiding the truck to drive includes truck collision warning.
[0094] It should be noted that if the drone determines that the target truck is not in the preset identification area, it means that there is no need to continue to guide the truck to drive, and the drone returns to the preset parking area to stop so that it can subsequently guide the truck to drive again or charge.
[0095] It should be noted that the preset parking area can be set inside the yard or outside the yard. The embodiment of the present application does not limit the location of the preset parking area, which can be determined according to actual conditions.
[0096] S404: The drone sends the bird's-eye view image of the target truck taken in real time to the computing device.
[0097] In this step, after the drone captures the bird's-eye view image of the target truck in real time, the bird's-eye view image of the target truck is sent to the computing device so that the computing device can subsequently guide the truck to travel.
[0098] It should be noted that the drone can also send bird's-eye view images to the vehicle computer or the driver's terminal device for display, so that the driver can observe the obstacles around the truck and improve driving safety.
[0099] S405: After receiving the bird's-eye view image of the target truck taken in real time and sent by the drone, the computing device guides the truck to travel according to the bird's-eye view image of the target truck taken in real time.
[0100] In this step, after the drone sends the bird's-eye view image to the computing device, the computing device can receive the bird's-eye view image and then guide the truck to travel according to the bird's-eye view image of the target truck taken in real time.
[0101] The method of guiding the truck to travel may be: according to the real-time bird's-eye view image of the target truck, the turning angle and travel direction of each obstacle in the bird's-eye view image, as well as the turning angle and travel direction of the target truck are calculated, and then the driving track of the target truck and the driving track of the obstacle are generated and superimposed on the real-time bird's-eye view image of the target truck, and then the image is sent to the target truck. The target truck travels according to the image to prevent collision.
[0102] Another method of guiding the driving of a truck is to calculate the turning angle and direction of each obstacle in the bird's-eye view image of the target truck taken in real time, determine the driving trajectory of the obstacle, and then control the driving of the target truck to avoid collision with the target truck.
[0103] Another method of guiding the driving of the truck is to predict whether the target truck will collide based on a bird's-eye view image of the target truck taken in real time, and output collision warning information if it is predicted that the target truck will collide.
[0104] It should be noted that the embodiment of the present application does not limit the method of guiding the truck to travel, which can be determined according to actual conditions.
[0105] For predicting whether the target truck will collide based on the bird's-eye view image of the target truck taken in real time, if the target truck is predicted to collide, the implementation method of outputting the collision warning information may be:
[0106] Based on the bird's-eye view image of the target truck captured in real time, the turning angle, moving direction and speed of each obstacle in the bird's-eye view image, as well as the turning angle, moving direction and speed of the target truck are calculated.
[0107] Since the bird's-eye view images are taken in real time, the computing device can sort the bird's-eye view images in order from the earliest to the latest time of shooting to obtain an image sequence. Then, the last bird's-eye view image in the image sequence is selected for image recognition to identify obstacles, obstacle types, and target trucks.
[0108] If the obstacle is a stationary type, such as a railing, a stone, a cone barrel, etc., it can be determined that the turning angle of the obstacle is 0, the direction of travel is a null value, and the speed is 0.
[0109] If the obstacle is a moving semi-trailer tractor, the direction of the front of the vehicle is taken as the direction of travel of the obstacle. The result of image recognition also includes the main suspension folding angle of the obstacle, that is, the angle between the tractor and the trailer of the semi-trailer tractor, and then the turning angle is calculated based on the main suspension folding angle. Then, the speed is calculated based on the turning angle, the position of the vehicle in the last bird's-eye view image and the second to last bird's-eye view image, and the frequency of the drone taking bird's-eye views.
[0110] If the obstacle is a moving non-trailer tractor, the direction of the vehicle head is taken as the direction of travel of the obstacle. Then the turning angle is calculated based on the direction of travel and the direction of the vehicle head in the second-to-last bird's-eye view image in the image sequence. Then the speed is calculated based on the turning angle, the position of the vehicle in the last bird's-eye view image and the second-to-last bird's-eye view image, and the frequency of the drone taking bird's-eye view images.
[0111] If the obstacle is another moving object, such as a pedestrian, the obstacle's direction and speed are determined based on the obstacle's position in the second-to-last bird's-eye view image in the image sequence, the obstacle's position in the last bird's-eye view image, and the frequency of the drone taking bird's-eye view images, and the turning angle of the obstacle is set to 0.
[0112] For the target truck, if the target truck is a semi-trailer tractor, the direction of the front of the truck is taken as the direction of the obstacle. The image recognition also includes the main trailer folding angle of the truck, that is, the angle between the tractor and the trailer of the semi-trailer tractor, and then the turning angle is calculated based on the main trailer folding angle. Then, the speed is calculated based on the turning angle, the position of the vehicle in the last bird's-eye view image and the second to last bird's-eye view image, and the frequency of the drone taking bird's-eye views.
[0113] If the target truck is not a semi-trailer tractor, the direction of the head of the truck is taken as the direction of travel of the target truck. Then the turning angle is calculated based on the direction of travel and the direction of the head of the truck in the second-to-last bird's-eye view image in the image sequence. Then the speed is calculated based on the turning angle, the position of the target truck in the last bird's-eye view image and the second-to-last bird's-eye view image, and the frequency of the drone taking bird's-eye view images.
[0114] Then, based on the turning angle, direction and speed of each obstacle in the bird's-eye view image, as well as the turning angle, direction and speed of the target truck, it is predicted whether the target truck will collide.
[0115] For each obstacle, the motion trajectory of the obstacle in a preset time can be generated according to the turning angle, travel direction and speed of the obstacle. The motion trajectory of the target truck in a preset time can be generated according to the turning angle, travel direction and speed of the target truck.
[0116] Then, for each obstacle, determine whether the obstacle's motion trajectory intersects with the target truck's motion trajectory; if the obstacle's motion trajectory intersects with the target truck's motion trajectory, determine whether the obstacle's arrival time at the intersection is the same as the target truck's arrival time based on the obstacle's turning angle, travel direction and speed, and the target truck's turning angle, travel direction and speed. If the obstacle's arrival time at the intersection is the same as the target truck's arrival time at the intersection, mark the obstacle as a collision obstacle.
[0117] Finally, it is determined whether there is a collision obstacle in the obstacles. If there is a collision obstacle in the obstacles, it is predicted that the target truck will collide. If there is no collision obstacle in the obstacles, it is predicted that the target truck will not collide.
[0118] It should be noted that the preset length can be 5 seconds, 10 seconds, 30 seconds, etc. The embodiment of the present application does not limit the preset length and can be set according to actual conditions.
[0119] It should be noted that the computing device may be a vehicle computer of the target truck, a server, a drone for taking a bird's-eye view image of the target truck, etc. The embodiment of the present application does not limit the computing device and may be determined according to actual conditions.
[0120] If the computing device predicts that the target truck will collide, it will output collision warning information.
[0121] It should be noted that the method of outputting the collision warning information may be: sending the collision warning information to the vehicle computer of the target truck, and then the vehicle computer displays the collision warning information and plays a warning prompt tone. It may also be: sending the collision warning information to the terminal device of the target truck driver, and then the terminal device displays the collision warning information and plays a warning prompt tone. It may also be: sending the collision warning information to the terminal device of the collision management center so that the staff of the collision management center can view it immediately. The embodiment of the present application does not limit the method of outputting the collision warning information, which can be determined according to actual conditions.
[0122] It should be noted that the warning information may include the location of the collision obstacle, the movement trajectory of the collision obstacle, a warning icon, etc. The embodiment of the present application does not limit the warning information and can be determined according to actual conditions.
[0123] The method for guiding truck driving by drone provided in this embodiment is that when the drone receives the take-off command, it flies to the preset identification area to shoot the truck image, and then determines that the target truck is in the preset identification area, follows the target truck to fly, and sends the real-time bird's-eye view image of the target truck to the computing device. The computing device guides the truck to drive according to the bird's-eye view image. Compared with the prior art in which the sensor cannot detect other obstacles due to obstacles blocking the sensor, this solution guides the truck to drive by taking the bird's-eye view image of the truck by drone. Since the bird's-eye view image has a wider viewing angle, it can cover more obstacles around the truck, and there will be no obstacles to block the camera equipment on the drone, which effectively improves the driving safety of the truck. In addition, in the prior art, it is necessary to install sensors on each truck, especially when the truck is a semi-trailer tractor, it is also necessary to install a special connection interface so that the vehicle machine in different tractors can be connected to the sensors installed on different trailers. This method is costly. This solution does not need to equip different trucks with different drones, and the number of drones used is small, which is low in cost. This solution is suitable for different types of open-air yards and trucks, and there is no need to transform the yard, which improves the applicability of the solution. When the sensor installed in the prior art is a camera, due to the harsh working environment of the truck, the camera is easily covered by dirt such as dust and sand, causing damage, thereby affecting the accuracy of detecting obstacles. This solution uses bird's-eye views taken by drones to detect obstacles and guide truck driving. Since the drone is in the air, the camera of the drone is not easily covered by dirt, which effectively improves the accuracy of detecting obstacles and guiding truck driving.
[0124] Figure 5This is a flow chart of the second embodiment of the method for guiding truck driving by drone provided in this application. Based on the above embodiment, this embodiment of the application describes the situation where the drone determines whether the target truck is in the preset identification area based on the target truck logo and truck image. Figure 5 As shown, the method for guiding a truck to travel by a drone specifically includes the following steps:
[0125] S501: Perform image recognition on the truck image to obtain a recognition result.
[0126] In this step, after the drone captures the truck image, it performs image recognition on the truck image to obtain a recognition result. The recognition result includes a result mark indicating a recognition failure, or includes at least one recognized truck mark to be determined.
[0127] Since a truck logo is printed on the top of the truck, if there is a truck in the preset recognition area, image recognition can be performed on the truck image to identify at least one truck logo to be determined, and a recognition result including the identified at least one truck logo to be determined is generated.
[0128] If there is no truck in the preset recognition area, the truck identification cannot be recognized, and a recognition result including a result identification indicating recognition failure is generated.
[0129] It should be noted that the truck identification to be determined and the target truck identification can be numbers, letters, QR codes, bar codes, etc. The embodiment of the present application does not limit the truck identification to be determined and the target truck identification, and can be determined according to actual conditions.
[0130] S502: Determine whether the recognition result includes a result identifier indicating recognition failure; if the recognition result includes a result identifier indicating recognition failure, execute step S505; if the recognition result includes at least one truck identifier to be determined, execute step S503.
[0131] In this step, after obtaining the recognition result, the UAV can determine whether the recognition result includes a result mark indicating recognition failure, so as to determine whether the target truck is in the preset recognition area.
[0132] S503: Determine whether the at least one truck identification to be determined includes the target truck identification; if the at least one truck identification to be determined includes the target truck identification, execute step S504; if the at least one truck identification to be determined does not include the target truck identification, execute step S505.
[0133] In this step, if the drone determines that the recognition result includes at least one truck identifier to be determined, it means that there is a truck in the preset recognition area. Based on the at least one truck identifier to be determined and the target truck identifier, it is determined whether the target truck is in the preset recognition area.
[0134] It is necessary to continue to determine whether there is a target truck among these trucks, that is, to determine whether the at least one to-be-determined truck identifier includes a target truck identifier.
[0135] It should be noted that when the truck identification to be determined and the target truck identification are QR codes or bar codes, the drone determines whether the target truck identification is included in at least one truck identification to be determined by: scanning each truck identification to be determined to obtain the scan result to be determined corresponding to the truck identification to be determined; scanning the target truck identification to obtain the target scan result, and then determining whether the target scan result is included in all the scan results to be determined.
[0136] S504: Determine that the target truck is in the preset recognition area.
[0137] In this step, if the drone determines that the at least one to-be-determined truck identification includes the target truck identification, it means that the trucks in the preset identification area include the target truck, that is, it is determined that the target truck is in the preset identification area.
[0138] S505: Determine that the target truck is not in the preset recognition area.
[0139] In this step, if the drone determines that the recognition result includes a result mark indicating recognition failure, it means that there is no truck in the preset recognition area and the target truck is not in the preset recognition area.
[0140] If the drone determines that the at least one to-be-determined truck identification does not include the target truck identification, it means that the trucks in the preset identification area do not include the target truck, that is, it is determined that the target truck is not in the preset identification area.
[0141] It should be noted that the drone can transmit the target truck identification and truck image to the computing device, and the computing device determines whether the target truck is in the preset identification area, and then the computing device sends the determination result to the drone.
[0142] The method for guiding truck driving by a drone provided in this embodiment determines whether the target truck is in a preset recognition area by combining the recognition result obtained by truck image recognition with the target truck identification, thereby effectively improving the judgment accuracy.
[0143] The following describes the situation of releasing the drone through Example 3 of the method for guiding a truck to travel by a drone provided in the application.
[0144] When the drone receives the drone release command, it flies to the preset parking area and completes the parking process so that it can subsequently execute the process of guiding the truck to drive or charge.
[0145] It should be noted that the way in which the drone receives the drone release command can be: when the target truck leaves the yard, the gate in the yard determines the target truck identification based on the license plate of the target truck, and then sends the drone release command to the drone that receives the flight command including the target truck identification, and the drone can receive the drone release command.
[0146] Alternatively, when the target truck is powered off, it sends a drone release command to the drone, and the drone receives the drone release command.
[0147] Alternatively, the yard staff may send a drone release command to the drone through the terminal device, and the drone may receive the drone release command.
[0148] The embodiment of the present application does not limit the manner in which the drone receives the drone release instruction, which can be determined based on actual conditions.
[0149] It should be noted that the preset parking area can be set inside the yard or outside the yard. The embodiment of the present application does not limit the location of the preset parking area, which can be determined according to actual conditions.
[0150] The drone-guided truck driving method provided in this embodiment saves drone energy consumption by flying to a preset parking area and stopping when the drone receives a drone release command.
[0151] Figure 6 This is a flow chart of the fourth embodiment of the method for guiding truck driving by drone provided in this application. This embodiment of the application describes the situation in which the computing device generates and outputs a reversing prompt image when the target truck is reversing. Figure 6 As shown, the method for guiding a truck to travel by a drone specifically includes the following steps:
[0152] S601: When a reversing signal sent by a target truck is received, a reversing trajectory is generated according to the turning angle and the traveling direction of the target truck.
[0153] When the target truck is driving in the yard, reversing is a process that is prone to collision, so it is necessary to remind the driver when the target truck is reversing to improve driving safety, and provide reversing assistance to the driver to improve the convenience and efficiency of reversing.
[0154] In this step, when the target truck is reversing, it sends a reversing signal to the computing device. When the computing device receives the reversing signal, it generates a reversing trajectory according to the turning angle and travel direction of the target truck.
[0155] In one implementation, the target truck may send its own vehicle operation data, such as turning angle, vehicle speed, travel direction, main suspension folding angle, steering wheel angle, tire steering angle, etc., to a computing device so that the computing device can generate a reversing trajectory.
[0156] S602: Superimposing the reversing track onto the bird's-eye view image of the target truck captured in real time to obtain a reversing prompt image.
[0157] In this step, after the computing device generates the reversing trajectory, in order to allow the driver to view the reversing trajectory more intuitively, the reversing trajectory needs to be superimposed on the bird's-eye view image of the target truck taken in real time to obtain a reversing prompt image.
[0158] S603: Outputting a reversing prompt image.
[0159] In this step, after the computing device obtains the reversing prompt image, it outputs the reversing prompt image. The driver can view the reversing prompt image and adjust the vehicle posture or vehicle speed in time to improve driving safety.
[0160] For example, Figure 7 A schematic diagram of a reversing reminder image provided for this application, such as Figure 7 As shown in the figure, there are multiple parking spaces, and the target truck is going to reverse into the parking space. The dotted line at the rear of the target truck represents the reversing trajectory. In addition, the computing device can also determine the coverage area of the target truck during the reversing process based on the reversing trajectory and the current posture of the target truck. The area between the two solid lines extending from the front of the vehicle to the parking space in the figure is the coverage area. The computing device can also generate a blind spot area around the target truck for the driver to view. The black area in the figure represents the blind spot area.
[0161] It should be noted that the method of outputting the reversing prompt image can be: sending the reversing prompt image to the vehicle computer of the target truck, and then the vehicle computer displays the reversing prompt image. It can also be: sending the reversing prompt image to the terminal device of the target truck driver, and then the terminal device displays the reversing prompt image. It can also be: sending the reversing prompt image to the head-up display device of the target truck, and the head-up device projects the reversing prompt image onto the truck glass. The embodiment of the present application does not limit the method of outputting the reversing prompt image, which can be determined according to actual conditions.
[0162] It should be noted that the driver can send the target position to the computing device through the vehicle computer or terminal device. The computing device can determine the current position of the target truck and the position of each obstacle based on the bird's-eye view image, and then perform path planning to obtain the planned path from the current position to the target position, and then send a control signal to the target truck according to the planned path, so that the target truck travels to the target position according to the planned path.
[0163] It should be noted that the computing device can send the position of each obstacle to the controller of the target truck. The controller performs path planning based on the position of each obstacle, the target position and the current position of the target truck, obtains the planned path from the current position to the target position, and then controls the target truck to travel to the target position along the planned path.
[0164] The method for guiding truck driving by a drone provided in this embodiment improves driving safety by generating and outputting a reversing prompt image by a computing device when the target truck is reversing.
[0165] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.
[0166] Figure 8 This is a schematic diagram of the structure of the first embodiment of the UAV-guided truck driving device provided in this application; the device can be integrated into the UAV in the above method embodiment, or can be implemented by the UAV in the above method embodiment. Figure 8 As shown, the drone-guided truck driving device 80 includes:
[0167] The flight control module 81 is used to control the UAV to fly to a preset identification area to capture a truck image upon receiving a take-off instruction, wherein the take-off instruction includes a target truck identifier of the target truck;
[0168] The processing module 82 is used to:
[0169] Determining whether the target truck is in the preset recognition area according to the target truck identifier and the truck image;
[0170] If it is determined that the target truck is in the preset identification area, the flight control module 81 follows the target truck corresponding to the target truck identifier to fly;
[0171] A shooting module 83, used for shooting a bird's-eye view image of the target truck in real time;
[0172] The sending module 84 is used to send the bird's-eye view image of the target truck taken in real time to a computing device.
[0173] Furthermore, the processing module 82 is specifically used for:
[0174] Performing image recognition on the truck image to obtain a recognition result, wherein the recognition result includes a result identifier indicating a recognition failure, or includes at least one recognized truck identifier to be determined;
[0175] If the recognition result includes a result mark indicating recognition failure, it is determined that the target truck is not in the preset recognition area;
[0176] If the recognition result includes at least one truck identifier to be determined, it is determined whether the target truck is in the preset recognition area according to the at least one truck identifier to be determined and the target truck identifier.
[0177] Furthermore, the processing module 82 is further configured to:
[0178] Determining whether the at least one to-be-determined truck identifier includes the target truck identifier;
[0179] If the at least one to-be-determined truck identifier includes the target truck identifier, determining that the target truck is in the preset identification area;
[0180] If the at least one truck identification to be determined does not include the target truck identification, it is determined that the target truck is not in the preset recognition area.
[0181] Furthermore, the flight control module 81 is also used to fly to a preset parking area and park upon receiving a drone release command.
[0182] Furthermore, the bird's-eye view image is processed by the computing device to guide the driving of the truck.
[0183] The drone-guided truck driving device provided in this embodiment is used to execute the technical solution of the drone in any of the aforementioned method embodiments. Its implementation principle and technical effects are similar and will not be repeated here.
[0184] Fig. 9 This is a schematic diagram of the structure of the second embodiment of the UAV-guided truck driving device provided in this application; the device can be integrated into the computing device in the above method embodiment, or can be implemented by the computing device in the above method embodiment. Fig. 9 As shown, the drone-guided truck driving device 90 includes:
[0185] A receiving module 91 is used to receive a bird's-eye view image of a target truck shot in real time and sent by a drone;
[0186] The guiding module 92 is used to guide the truck to travel according to the bird's-eye view image of the target truck taken in real time.
[0187] Furthermore, the guiding module is specifically used for:
[0188] Predicting whether the target truck will collide based on the bird's-eye view image of the target truck captured in real time;
[0189] If it is predicted that the target truck will collide, a collision warning message is output.
[0190] Furthermore, the guiding module 92 is specifically used for:
[0191] Calculating the turning angle, traveling direction and speed of each obstacle in the bird's-eye view image, as well as the turning angle, traveling direction and speed of the target truck, based on the bird's-eye view image of the target truck captured in real time;
[0192] Whether the target truck will collide is predicted based on the turning angle, traveling direction and speed of each obstacle in the bird's-eye view image and the turning angle, traveling direction and speed of the target truck.
[0193] The processing module 93 is used for:
[0194] When a reversing signal sent by the target truck is received through the receiving module 91, a reversing trajectory is generated according to the turning angle and the traveling direction of the target truck;
[0195] Superimposing the reversing track onto the bird's-eye view image of the target truck captured in real time to obtain a reversing prompt image;
[0196] Furthermore, the guiding module 92 is also used to output the reversing prompt image.
[0197] The drone-guided truck driving device provided in this embodiment is used to execute the technical solution of the computing device in any of the aforementioned method embodiments. Its implementation principle and technical effects are similar and will not be repeated here.
[0198] Fig.10 This is a schematic diagram of the structure of a drone provided in this application. Fig.10 As shown, the drone 1000 includes:
[0199] Processor 1001, memory 1002, communication interface 1003, and camera 1004;
[0200] The memory 1002 is used to store executable instructions of the processor 1001;
[0201] The processor 1001 is configured to execute the technical solution of the drone in any of the aforementioned method embodiments by executing the executable instructions.
[0202] Optionally, the memory 1002 may be independent or integrated with the processor 1001 .
[0203] Optionally, when the memory 1002 is a device independent of the processor 1001, the drone 1000 may further include:
[0204] The bus 1005 , the camera device 1004 , the memory 1002 and the communication interface 1003 are connected to the processor 1001 via the bus 1005 and communicate with each other. The communication interface 1003 is used to communicate with other devices.
[0205] Optionally, the communication interface 1003 can be implemented by a transceiver. The communication interface is used to implement communication between the database access device and other devices (such as a client, a read-write library, and a read-only library). The memory may include a random access memory (RAM) and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage.
[0206] The bus 1005 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.
[0207] The above-mentioned processor can be a general-purpose processor, including a central processing unit CPU, a network processor (NP), etc.; it can also be a digital signal processor DSP, an application-specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
[0208] The drone is used to execute the technical solution of the drone in any of the aforementioned method embodiments, and its implementation principle and technical effect are similar, which will not be repeated here.
[0209] Fig.11 This is a schematic diagram of the structure of a computing device provided in this application. Fig.11 As shown, the computing device 1100 includes:
[0210] Processor 1101, memory 1102, and communication interface 1103;
[0211] The memory 1102 is used to store executable instructions of the processor 1101;
[0212] The processor 1101 is configured to execute the technical solution of the computing device in any of the aforementioned method embodiments by executing the executable instructions.
[0213] Optionally, the memory 1102 may be independent or integrated with the processor 1101 .
[0214] Optionally, when the memory 1102 is a device independent of the processor 1101, the computing device 1100 may further include:
[0215] The bus 1104 , the memory 1102 and the communication interface 1103 are connected to the processor 1101 via the bus 1104 and communicate with each other. The communication interface 1103 is used to communicate with other devices.
[0216] Optionally, the communication interface 1103 may be implemented by a transceiver. The communication interface is used to implement communication between the database access device and other devices (such as a client, a read-write library, and a read-only library). The memory may include a random access memory (RAM) and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage.
[0217] The bus 1104 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.
[0218] The above-mentioned processor can be a general-purpose processor, including a central processing unit CPU, a network processor (NP), etc.; it can also be a digital signal processor DSP, an application-specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
[0219] The computing device is used to execute the technical solution of the computing device in any of the aforementioned method embodiments, and its implementation principle and technical effect are similar and will not be repeated here.
[0220] An embodiment of the present application also provides a readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the technical solution provided in any of the aforementioned embodiments is implemented.
[0221] An embodiment of the present application also provides a computer program product, including a computer program, which is used to implement the technical solution provided by any of the aforementioned method embodiments when executed by a processor.
[0222] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.
[0223] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.
Claims
1. A method for guiding a truck to travel using an unmanned aerial vehicle, characterized in that: Applied to a drone, the method comprises: Upon receiving a take-off instruction, flying to a preset identification area to capture a truck image, wherein the take-off instruction includes a target truck identification of the target truck; Determining whether the target truck is in the preset recognition area according to the target truck identifier and the truck image; If it is determined that the target truck is in the preset identification area, flying along the target truck corresponding to the target truck identifier and taking a bird's-eye view image of the target truck in real time; The bird's-eye view image of the target truck captured in real time is sent to a computing device.
2. The method according to claim 1, characterized in that The determining, according to the target truck identifier and the truck image, whether the target truck is in the preset recognition area comprises: Performing image recognition on the truck image to obtain a recognition result, wherein the recognition result includes a result identifier indicating a recognition failure, or includes at least one identified truck identifier to be determined; If the recognition result includes a result mark indicating recognition failure, it is determined that the target truck is not in the preset recognition area; If the recognition result includes at least one truck identifier to be determined, it is determined whether the target truck is in the preset recognition area according to the at least one truck identifier to be determined and the target truck identifier.
3. The method according to claim 2, characterized in that The determining, based on the at least one to-be-determined truck identifier and the target truck identifier, whether the target truck is in the preset identification area comprises: Determining whether the at least one to-be-determined truck identifier includes the target truck identifier; If the at least one to-be-determined truck identifier includes the target truck identifier, determining that the target truck is in the preset identification area; If the at least one truck identification to be determined does not include the target truck identification, it is determined that the target truck is not in the preset recognition area.
4. The method according to claim 1, characterized in that: The method further comprises: When receiving the drone release command, it will fly to the preset parking area and stop.
5. The method according to claim 1, characterized in that The bird's-eye view image is processed by the computing device and used to guide the truck.
6. A method for guiding a truck to travel using a drone, characterized in that: Applied to a computing device, the method comprises: Receive a real-time bird's-eye view image of a target truck sent by a drone; The truck is guided to travel according to the bird's-eye view image of the target truck captured in real time.
7. The method according to claim 6, characterized in that The step of guiding the truck to travel according to the bird's-eye view image of the target truck captured in real time includes: Predicting whether the target truck will collide based on the bird's-eye view image of the target truck captured in real time; If it is predicted that the target truck will collide, a collision warning message is output.
8. The method according to claim 7, characterized in that The predicting whether the target truck will collide based on the bird's-eye view image of the target truck taken in real time includes: Calculating the turning angle, traveling direction and speed of each obstacle in the bird's-eye view image, as well as the turning angle, traveling direction and speed of the target truck, based on the bird's-eye view image of the target truck captured in real time; Whether the target truck will collide is predicted based on the turning angle, traveling direction and speed of each obstacle in the bird's-eye view image and the turning angle, traveling direction and speed of the target truck.
9. The method according to claim 8, characterized in that The method further comprises: When receiving a reversing signal sent by the target truck, generating a reversing trajectory according to the turning angle and the traveling direction of the target truck; Superimposing the reversing track onto the bird's-eye view image of the target truck captured in real time to obtain a reversing prompt image; The reversing prompt image is outputted.
10. A drone-guided truck driving device, characterized in that: include: A flight control module, configured to control the UAV to fly to a preset identification area to capture a truck image upon receiving a take-off instruction, wherein the take-off instruction includes a target truck identifier of the target truck; Processing modules for: Determining whether the target truck is in the preset recognition area according to the target truck identifier and the truck image; If it is determined that the target truck is in the preset identification area, the flight control module follows the target truck corresponding to the target truck identifier to fly; A shooting module, used for shooting a bird's-eye view image of the target truck in real time; The sending module is used to send the bird's-eye view image of the target truck taken in real time to a computing device.
11. A drone-guided truck driving device, characterized in that: include: A receiving module, used for receiving a bird's-eye view image of a target truck shot in real time and sent by a UAV; The guiding module is used to guide the truck to travel according to the bird's-eye view image of the target truck taken in real time.
12. A drone, characterized in that: include: Processor, memory, communication interface, camera equipment; The memory is used to store executable instructions of the processor; Wherein, the processor is configured to execute the drone-guided truck driving method described in any one of claims 1 to 5 by executing the executable instructions.
13. A computing device, characterized in that: include: Processor, memory, communication interface; The memory is used to store executable instructions of the processor; Wherein, the processor is configured to execute the drone-guided truck driving method described in any one of claims 6 to 9 by executing the executable instructions.
14. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the drone-guided truck driving method described in any one of claims 1 to 5 or any one of claims 6 to 9 is implemented.
15. A computer program product, characterized in that It includes a computer program, which, when executed by a processor, is used to implement the drone-guided truck driving method described in any one of claims 1 to 5, or any one of claims 6 to 9.
