Unmanned aerial vehicle dynamic trigger control method and system for unmanned logistics vehicle

Abstract

The invention provides an unmanned aerial vehicle dynamic trigger control method and system for an unmanned logistics vehicle, and relates to the technical field of automatic driving, and the method comprises the steps that a data processing module obtains and monitors the visual confidence degree of the unmanned logistics vehicle in real time; when it is monitored that the visual confidence coefficient is lower than a preset visual confidence coefficient threshold value for a continuous preset duration, the safety boundary shrinkage rate and the high-dynamic-range image overexposure index of the unmanned logistics vehicle at the current moment are obtained, and according to the visual confidence coefficient, the safety boundary shrinkage rate and the high-dynamic-range image overexposure index, the safety boundary shrinkage rate and the high-dynamic-range image overexposure index are obtained; judging whether the current condition meets a preset unmanned aerial vehicle starting condition or not; and if yes, the data processing module starts an unmanned aerial vehicle deployed in an unmanned aerial vehicle module through the unmanned aerial vehicle module of the unmanned logistics vehicle, and activates a communication link between the unmanned aerial vehicle and the unmanned logistics vehicle through a communication module of the unmanned logistics vehicle. According to the invention, the energy consumption of the unmanned aerial vehicle can be reduced, and the operation safety of the unmanned logistics vehicle in a long-tail scene is also considered.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a method and system for dynamically triggering and controlling an unmanned aerial vehicle (UAV) for an unmanned logistics vehicle. Background Art

[0002] At present, the visual confidence of unmanned logistics vehicles based on pure vision solutions will drop significantly (the actual measurement shows that the drop exceeds 40%) when encountering long-tail scenarios such as strong light, bad weather and severe occlusion. Visual confidence is one of the important factors affecting the operational safety of unmanned logistics vehicles.

[0003] To address this shortcoming, existing technologies use drones for continuous flight. Drones have a wide field of view, and the image data they collect can provide supplementary perception for purely visual unmanned logistics vehicles. However, the continuous flight of drones results in high energy consumption.

[0004] Based on this, how to reduce the energy consumption of drones while taking into account the operational safety of unmanned logistics vehicles in long-tail scenarios has become a technical problem that needs to be solved urgently. Summary of the Invention

[0005] In view of this, in order to solve the above technical problems, the present invention provides a method and system for dynamically triggering and controlling a drone for an unmanned logistics vehicle.

[0006] The present invention adopts the following technical solutions: In a first aspect, the present invention provides a method for dynamically triggering and controlling a drone for an unmanned logistics vehicle, comprising: The data processing module acquires and monitors the visual confidence of the unmanned logistics vehicle in real time; When the data processing module detects that the visual confidence level is lower than a preset visual confidence threshold for a continuous preset period of time, the data processing module obtains the safety margin contraction rate and high dynamic range image overexposure index of the unmanned logistics vehicle at the current moment; The data processing module determines whether the current conditions meet the preset drone startup conditions based on the visual confidence, the safety margin shrinkage rate, and the high dynamic range image overexposure index, and obtains a judgment result; If the judgment result is yes, the data processing module starts the drone deployed in the drone module through the drone module of the unmanned logistics vehicle, and activates the communication link between the drone and the unmanned logistics vehicle through the communication module of the unmanned logistics vehicle.

[0007] Optionally, judging whether current conditions meet preset drone startup conditions based on the visual confidence level, the safety margin shrinkage rate, and the high dynamic range image overexposure index includes: Determining whether the safety margin shrinkage rate is greater than a preset shrinkage rate threshold; If the safety margin shrinkage rate is greater than a preset shrinkage rate threshold, determining whether the high dynamic range image overexposure index is greater than a preset index threshold; If the high dynamic range image overexposure index is greater than a preset index threshold, calculating a comprehensive risk index based on the visual confidence, the safety margin shrinkage rate, and the high dynamic range image overexposure index; Determining whether the comprehensive risk indicator is greater than a preset indicator threshold; If the comprehensive risk index is greater than the preset index threshold, it is determined that the current conditions meet the preset drone startup conditions.

[0008] Optionally, the calculation formula of the comprehensive risk index is:

[0009] in, is the comprehensive risk indicator; 、 、 is the weight coefficient; is the visual confidence; is the safety margin shrinkage rate; is the overexposure index of the high dynamic range image.

[0010] Optionally, the drone module includes: an ejection control unit and a drone storage compartment; The data processing module activates the drone deployed in the drone module through the drone module of the unmanned logistics vehicle, specifically including: The data processing module sends an ejection instruction to the ejection control unit; The ejection control unit controls the UAV to be ejected from the UAV storage compartment in response to the ejection instruction.

[0011] Optionally, the communication link includes a millimeter wave communication link and a cellular network communication link.

[0012] Optionally, the cellular network communication link is a default communication link; The data processing module is further configured to: When it is detected that the electromagnetic interference intensity is greater than a preset interference intensity threshold, the communication link is switched to the millimeter wave communication link.

[0013] In a second aspect, the present invention also provides a drone dynamic trigger control system for an unmanned logistics vehicle, comprising: a data processing module, a drone module, and a communication module; The data processing module, the drone module and the communication module are used to execute the drone dynamic triggering control method for an unmanned logistics vehicle as described above.

[0014] Optionally, the drone module includes: an ejection control unit and a drone storage compartment.

[0015] Optionally, the drone storage cabin includes: a cabin body, a cabin cover, a wireless charging device and an ejection mechanism; The wireless charging device and the ejection mechanism are arranged in a space enclosed by the cabin body and the cabin cover; The wireless charging device is used to charge the drone; The ejection mechanism is used to control the drone to be ejected from the drone storage compartment when the drone is started.

[0016] The present invention adopts the above technical solution. By considering the three indicators closely related to long-tail scenarios, namely the visual confidence, safety boundary shrinkage rate and high dynamic range image overexposure index of the unmanned logistics vehicle, when judging whether the current conditions meet the preset drone start-up conditions, the present invention can improve the coverage rate of long-tail scenarios, and thus ensure the operation safety of the unmanned logistics vehicle in long-tail scenarios to a certain extent; and, by starting the drone only when it is judged that the current conditions meet the preset drone start-up conditions, and using the perception data obtained by the drone to assist the unmanned logistics vehicle in making driving decisions, the present invention can reduce the energy consumption of the drone. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 This is a structural diagram of a UAV dynamic trigger control system for an unmanned logistics vehicle provided by an embodiment of the present invention; Figure 2 This is a structural diagram of a drone storage compartment provided by an embodiment of the present invention; Figure 3 This is a flow chart of a method for dynamically triggering and controlling a drone for an unmanned logistics vehicle provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0019] To make the objectives, technical solutions, and advantages of the present invention more apparent, the technical solutions of the present invention will be described in detail below. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other implementations obtained by those of ordinary skill in the art without inventive effort are within the scope of protection of the present invention.

[0020] To make this plan easier to understand, some professional terms that may be involved in this plan are explained below: Visual confidence: This quantifies the degree of certainty of the vehicle's perception system in its detection and recognition results (e.g., vehicles, pedestrians, lane markings, etc.). Visual confidence ranges from 0 to 1. Under normal lighting conditions, visual confidence is typically stable above 0.95.

[0021] Safety Margin Contraction Rate: The safety margin is the area within which the vehicle can safely navigate over the next few seconds, planned based on its current environmental perception and motion status. When the perception system's performance degrades and it cannot accurately identify distant or lateral obstacles, the vehicle adopts a conservative strategy for safety reasons, causing the planned safety margin to shrink rapidly. This rate of contraction is the safety margin contraction rate. The safety margin contraction rate is measured as the percentage of area contracted per second. For example, if an unrecognizable area suddenly appears in front of the vehicle, the safety margin may rapidly contract at a rate of 14.3% per second.

[0022] High dynamic range image overexposure index: To cope with scenes with drastic changes in lighting, automotive cameras typically enable high dynamic range imaging mode. However, even in this mode, instantaneous extremely high lighting (such as the ambient illumination suddenly jumping from 50 lux in a tunnel to 100,000 lux at the exit) can still cause a large number of pixels in the image sensor to reach saturation (i.e., the pixel value reaches the maximum value, such as 255). The high dynamic range image overexposure index is the percentage of saturated pixels in the image. A high overexposure index (such as over 30%) directly indicates that the information obtained by the visual sensor has been severely distorted.

[0023] Figure 1 This is a structural diagram of a UAV dynamic trigger control system for an unmanned logistics vehicle provided by an embodiment of the present invention. The UAV dynamic trigger control system for an unmanned logistics vehicle is integrated on the unmanned logistics vehicle. Figure 1 As shown, the system includes: a data processing module 11, a drone module 12, a communication module 13 and a vehicle-mounted vision module 14.

[0024] The data processing module 11, the drone module 12, the communication module 13 and the vehicle-mounted vision module 14 work together to implement a drone dynamic triggering control method for an unmanned logistics vehicle provided by the present invention.

[0025] The onboard vision module 14 can consist of eight industrial-grade cameras deployed around the vehicle. These cameras include two forward-facing cameras (one wide-angle, one telephoto), two rear-facing cameras, and two side-view cameras on each side of the vehicle. These cameras provide a 360-degree, unobstructed view of the vehicle's surroundings and continuously capture high-definition image data at a high frame rate (e.g., 30 frames per second). The image data stream captured by the onboard vision module 14 is transmitted in real time to the data processing module 11.

[0026] The data processing module 11 is the decision-making core of the system. In terms of hardware implementation, the functions of the data processing module 11 can be realized by the onboard computing platform of the unmanned logistics vehicle.

[0027] The drone module 12 includes an ejection control unit 121 and a drone storage compartment 122 .

[0028] Figure 2 This is a schematic diagram of the structure of a drone storage compartment provided by an embodiment of the present invention. Figure 2 As shown, the drone storage cabin 122 includes: a cabin body 21 , a cabin cover 22 , a wireless charging device 23 and an ejection mechanism 24 .

[0029] The wireless charging device 23 and the ejection mechanism 24 are disposed in a space enclosed by the cabin 21 and the cabin cover 22 .

[0030] The material of the hatch cover 22 can be carbon fiber, so that the hatch cover 22 can reach the IP67 protection level.

[0031] The wireless charging device 23 is used to charge the drone. The charging voltage can be 48V DC, enabling rapid charging (recharging 10% of the energy in 90 seconds). During the charging process, the drone contacts the wireless charging contacts, and the wireless charging device 23 draws power from the unmanned logistics vehicle's power supply to charge the drone.

[0032] The ejection mechanism 24 may be a spring ejection mechanism, which is used to eject the drone from the drone storage compartment when the drone is started. The ejection stroke may be 0.5 m.

[0033] The ejection control unit 121 is internally provided with an electromagnetic lock release mechanism for controlling the working state of the ejection mechanism 24 in response to the control instruction of the data processing module 11 to eject the drone from the drone storage compartment 122 .

[0034] It should be noted that the above-mentioned structures (such as the data processing module 11, the communication module 13, the vehicle-mounted vision module 14, the ejection control unit 121, the wireless charging device 23 and the ejection mechanism 24) are all existing technologies, so their specific structural composition will not be repeated here.

[0035] When the drone is in the cabin 21, its folding rotors are in a folded state. After starting the drone, its folding rotors can be unfolded within 1.2 seconds. This helps to improve the response speed of the drone of this application.

[0036] Figure 3 FIG is a flow chart of a method for dynamically triggering and controlling a drone for an unmanned logistics vehicle provided by an embodiment of the present invention. Figure 3 As shown, this process includes: Step 301: The data processing module obtains and monitors the visual confidence of the unmanned logistics vehicle in real time.

[0037] Step 302: When the data processing module detects that the visual confidence level is lower than the preset visual confidence level threshold for a continuous preset period of time, the data processing module obtains the safety margin contraction rate and high dynamic range image overexposure index of the unmanned logistics vehicle at the current moment.

[0038] The preset duration may be 3 seconds, and the preset visual confidence threshold may be 0.85. For example, when a vehicle exits a tunnel, the camera is momentarily exposed to strong light, causing the image to be severely overexposed. This causes the visual confidence to remain below 0.85 for 3.2 seconds. In this case, it is determined that the visual confidence has remained below the preset visual confidence threshold for the preset duration.

[0039] It should be noted that the data processing module calculates the visual confidence, safety boundary shrinkage rate and high dynamic range image overexposure index based on the image data stream collected by the on-board vision module. The calculation method of these three indicators is existing technology, so it will not be elaborated here.

[0040] It is understandable that if the data processing module does not detect that the visual confidence level is lower than the preset visual confidence level threshold for a continuous preset period of time, the unmanned logistics vehicle maintains the pure vision mode.

[0041] Step 303: The data processing module determines whether the current conditions meet the preset drone launch conditions based on the visual confidence level, the safety margin shrinkage rate, and the high dynamic range image overexposure index, and obtains a judgment result. If the judgment result is yes, step 304 is executed.

[0042] In addition, if the judgment result is yes, the preset alarm action can also be executed, such as sending an alarm prompt information to the preset terminal, so that the staff can be informed in time according to the alarm prompt information that there is an operation risk of the unmanned logistics vehicle and it is necessary to start the drone to assist the unmanned logistics vehicle in making driving decisions.

[0043] Step 304: The data processing module activates the drone deployed within the drone module through the unmanned logistics vehicle's drone module and activates the communication link between the drone and the unmanned logistics vehicle through the unmanned logistics vehicle's communication module. The drone transmits its generated image data to the data processing module via this communication link. Specifically, the image data generated by the drone can be a halo-free bird's-eye view.

[0044] Furthermore, to reduce hardware costs, drones can reuse the onboard RTK (Real-Time Kinematic) positioning module of unmanned logistics vehicles and share base stations with them. The drones can connect to the onboard RTK positioning module of the unmanned logistics vehicle via the aforementioned communication link.

[0045] The present invention adopts the above technical solution. By considering the three indicators closely related to long-tail scenarios, namely the visual confidence, safety boundary shrinkage rate and high dynamic range image overexposure index of the unmanned logistics vehicle, when judging whether the current conditions meet the preset drone start-up conditions, the present invention can improve the coverage rate of long-tail scenarios, and thus ensure the operation safety of the unmanned logistics vehicle in long-tail scenarios to a certain extent; and, by starting the drone only when it is judged that the current conditions meet the preset drone start-up conditions, and using the perception data obtained by the drone to assist the unmanned logistics vehicle in making driving decisions, the present invention can reduce the energy consumption of the drone.

[0046] In an embodiment of the present invention, judging whether the current conditions meet the preset drone startup conditions based on the visual confidence, the safety margin shrinkage rate, and the high dynamic range image overexposure index may specifically include: (1) Determine whether the safety margin shrinkage rate is greater than the preset shrinkage rate threshold.

[0047] The preset shrinkage rate threshold may be equal to 10% / S.

[0048] (2) If the safety boundary shrinkage rate is greater than the preset shrinkage rate threshold, it is determined whether the high dynamic range image overexposure index is greater than the preset index threshold.

[0049] The preset index threshold may be equal to 120 dB (decibel).

[0050] It is understandable that if the safety boundary shrinkage rate is not greater than the preset shrinkage rate threshold, the unmanned logistics vehicle maintains the pure vision mode.

[0051] (3) If the high dynamic range image overexposure index is greater than the preset index threshold, the comprehensive risk index is calculated based on the visual confidence, the safety margin shrinkage rate and the high dynamic range image overexposure index.

[0052] The value of the comprehensive risk index can reflect the size of the vehicle operation risk. The larger the value, the greater the risk.

[0053] (4) Determine whether the comprehensive risk index is greater than the preset index threshold.

[0054] The preset indicator threshold may be equal to 0.75.

[0055] (5) If the comprehensive risk index is greater than the preset index threshold, it is determined that the current conditions meet the preset drone startup conditions.

[0056] In the embodiment of the present invention, the calculation formula of the comprehensive risk index is:

[0057] in, is a comprehensive risk indicator; 、 、 is the weight coefficient; is the visual confidence; is the safety margin shrinkage rate; It is the overexposure index of high dynamic range image.

[0058] Considering that visual confidence determines the reliability of risk judgment, the safety margin shrinkage rate reflects whether the risk occurs, and the high dynamic range image overexposure index reflects the local situation, the visual confidence is given the highest weight, the safety margin shrinkage rate is second, and the high dynamic range image overexposure index is the smallest, thus improving the overall reliability of the comprehensive risk indicator. In a specific example, The value of can be 0.6, The value of can be 0.3, The value can be 0.1.

[0059] In an embodiment of the present invention, the drone module includes: an ejection control unit and a drone storage compartment.

[0060] The data processing module activates the drone deployed in the drone module through the drone module of the unmanned logistics vehicle, which may include: (1) The data processing module sends the ejection command to the ejection control unit.

[0061] (2) The ejection control unit responds to the ejection command and controls the UAV to be ejected from the UAV storage compartment.

[0062] By launching into the air through a catapult, the response speed of the drone can be improved.

[0063] In an embodiment of the present invention, the communication link includes a millimeter wave communication link and a cellular network communication link.

[0064] Specifically, the millimeter wave communication link can operate at 60 GHz. This ensures latency of less than 5 ms when electromagnetic interference intensity exceeds 30 dBm, enabling high-speed and stable data transmission and meeting the stringent real-time requirements of unmanned logistics vehicles.

[0065] In the embodiment of the present invention, the cellular network communication link is the default communication link.

[0066] The data processing module can also be used to: When it is detected that the electromagnetic interference intensity is greater than a preset interference intensity threshold, the communication link is switched to a millimeter wave communication link.

[0067] It can be understood that the same or similar parts of the above embodiments can be referenced to each other, and the contents not described in detail in some embodiments can refer to the same or similar contents in other embodiments.

[0068] It should be noted that, in the description of the present invention, the terms "first", "second", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance. In addition, in the description of the present invention, unless otherwise specified, the meaning of "plurality" is at least two.

[0069] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention pertain.

[0070] It should be understood that various components of the present invention may be implemented using hardware, software, firmware, or a combination thereof. In the aforementioned embodiments, multiple steps or methods may be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one or a combination of the following technologies known in the art may be used: a discrete logic circuit having logic gate circuits for implementing logic functions on data signals, an application-specific integrated circuit having suitable combinational logic gate circuits, a programmable gate array (PGA), a field-programmable gate array (FPGA), etc.

[0071] Those skilled in the art will appreciate that all or part of the steps in the method for implementing the above-mentioned embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0072] Furthermore, the functional units in the various embodiments of the present invention may be integrated into a single processing module, each unit may exist physically separately, or two or more units may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or software functional modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium.

[0073] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0074] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0075] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A method for dynamic triggering control of drones for unmanned logistics vehicles, characterized in that: include: The data processing module acquires and monitors the visual confidence of the unmanned logistics vehicle in real time; When the data processing module detects that the visual confidence level is lower than a preset visual confidence threshold for a continuous preset period of time, the data processing module obtains the safety margin contraction rate and high dynamic range image overexposure index of the unmanned logistics vehicle at the current moment; The data processing module determines whether the current conditions meet the preset drone startup conditions based on the visual confidence, the safety margin shrinkage rate, and the high dynamic range image overexposure index, and obtains a judgment result; If the judgment result is yes, the data processing module starts the drone deployed in the drone module through the drone module of the unmanned logistics vehicle, and activates the communication link between the drone and the unmanned logistics vehicle through the communication module of the unmanned logistics vehicle.

2. The method for dynamic triggering control of a drone for an unmanned logistics vehicle according to claim 1, characterized in that: Determining whether current conditions meet preset drone startup conditions based on the visual confidence level, the safety margin shrinkage rate, and the high dynamic range image overexposure index includes: Determining whether the safety margin shrinkage rate is greater than a preset shrinkage rate threshold; If the safety margin shrinkage rate is greater than a preset shrinkage rate threshold, determining whether the high dynamic range image overexposure index is greater than a preset index threshold; If the high dynamic range image overexposure index is greater than a preset index threshold, calculating a comprehensive risk index based on the visual confidence, the safety margin shrinkage rate, and the high dynamic range image overexposure index; Determining whether the comprehensive risk indicator is greater than a preset indicator threshold; If the comprehensive risk index is greater than the preset index threshold, it is determined that the current conditions meet the preset drone startup conditions.

3. The method for dynamic triggering control of a drone for an unmanned logistics vehicle according to claim 2, characterized in that: The calculation formula of the comprehensive risk index is: in, is the comprehensive risk indicator; 、 、 is the weight coefficient; is the visual confidence; is the safety margin shrinkage rate; is the overexposure index of the high dynamic range image.

4. The method for dynamic triggering control of a drone for an unmanned logistics vehicle according to claim 1, characterized in that: The drone module includes: an ejection control unit and a drone storage compartment; The data processing module activates the drone deployed in the drone module through the drone module of the unmanned logistics vehicle, specifically including: The data processing module sends an ejection instruction to the ejection control unit; The ejection control unit controls the UAV to be ejected from the UAV storage compartment in response to the ejection instruction.

5. The method for dynamic triggering control of a drone for an unmanned logistics vehicle according to claim 1, characterized in that: The communication link includes a millimeter wave communication link and a cellular network communication link.

6. The method for dynamic triggering control of a drone for an unmanned logistics vehicle according to claim 5, characterized in that: The cellular network communication link is a default communication link; The data processing module is further configured to: When it is detected that the electromagnetic interference intensity is greater than a preset interference intensity threshold, the communication link is switched to the millimeter wave communication link.

7. A UAV dynamic trigger control system for an unmanned logistics vehicle, characterized in that: include: Data processing module, drone module and communication module; The data processing module, the drone module and the communication module are used to execute the drone dynamic triggering control method for an unmanned logistics vehicle according to any one of claims 1 to 6.

8. The UAV dynamic trigger control system for unmanned logistics vehicles according to claim 7, characterized in that: The drone module includes: an ejection control unit and a drone storage compartment.

9. The UAV dynamic trigger control system for an unmanned logistics vehicle according to claim 8, characterized in that: The UAV storage cabin includes: a cabin body, a cabin cover, a wireless charging device and an ejection mechanism; The wireless charging device and the ejection mechanism are arranged in a space enclosed by the cabin body and the cabin cover; The wireless charging device is used to charge the drone; The ejection mechanism is used to control the drone to be ejected from the drone storage compartment when the drone is started.

Images