Vehicle-mounted unmanned aerial vehicle resource sharing method, electronic device, storage medium, and computer program product

Abstract

Disclosed in the present invention are a vehicle-mounted unmanned aerial vehicle resource sharing method, an electronic device, and a storage medium. The vehicle-mounted unmanned aerial vehicle resource sharing method is applied to a vehicle-mounted unmanned aerial vehicle. The vehicle-mounted unmanned aerial vehicle has a corresponding direct control terminal. The method comprises: sending a resource sharing request to a non-direct control terminal via the direct control terminal, wherein control terminals for which resource sharing has been successfully established form a resource sharing team; and sharing perception information detected by the vehicle-mounted unmanned aerial vehicle and / or relevant control authority of the vehicle-mounted unmanned aerial vehicle to the resource sharing team.

Description

Vehicle-mounted unmanned aerial vehicle (UAV) resource sharing methods, electronic devices and storage media, and computer program products Technical Field

[0001] This invention relates to vehicle-mounted drones, and more particularly to a method for sharing resources of vehicle-mounted drones, electronic devices and storage media, and computer program products. Background Technology

[0002] Vehicle-mounted drones are products that combine a car and a drone. The car serves as the operating platform for the drone, enabling autonomous takeoff and landing by mounting a mobile airport on the vehicle.

[0003] In related technologies, the integration of vehicles and vehicle-mounted drones is currently limited to using the vehicle as a mobile airport for the drone or using the drone as an aerial camera for the vehicle. Many more functional integrations between vehicles and vehicle-mounted drones are yet to be developed. Summary of the Invention

[0004] This invention provides a method for resource sharing of vehicle-mounted unmanned aerial vehicles, an electronic device and a storage medium, and a computer program product, which are used to solve at least one of the above-mentioned technical problems.

[0005] In a first aspect, embodiments of the present invention provide a method for resource sharing of a vehicle-mounted unmanned aerial vehicle (UAV), wherein the vehicle-mounted UAV has a corresponding direct control terminal, including:

[0006] A resource-sharing link is established between the direct control terminal and the indirect control terminal, wherein each control terminal that successfully establishes the resource-sharing link forms a resource-sharing team.

[0007] The vehicle-mounted drone shares the perception information detected by the drone with the resource-sharing team, and / or receives and responds to the control information of the drone sent by the resource-sharing team.

[0008] Secondly, embodiments of the present invention provide a method for resource sharing of a vehicle-mounted drone, for a non-direct control terminal of a vehicle-mounted drone. The method includes: receiving a resource sharing request from the direct control terminal, wherein each control terminal that successfully establishes resource sharing forms a resource sharing team; in response to the non-direct control terminal joining the resource sharing team, the non-direct control terminal receives and displays the perception information detected by the vehicle-mounted drone and / or the relevant control permissions of the vehicle-mounted drone.

[0009] Thirdly, embodiments of the present invention provide a method for resource sharing of a vehicle-mounted unmanned aerial vehicle (UAV), for use as a direct control terminal of a vehicle-mounted UAV. The method includes: sending a resource sharing request to a non-direct control terminal, wherein each control terminal that successfully establishes resource sharing forms a resource sharing team; in response to the non-direct control terminal joining the resource sharing team, sharing the perception information detected by the vehicle-mounted UAV and / or the relevant control permissions of the vehicle-mounted UAV with the non-direct control terminal.

[0010] Fourthly, embodiments of the present invention provide an electronic device comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform any of the above-described auxiliary identification methods of the present invention.

[0011] Fifthly, embodiments of the present invention provide a storage medium storing one or more programs including execution instructions, the execution instructions being readable and executable by electronic devices (including but not limited to computers, servers, or network devices, etc.) to perform any of the auxiliary identification methods described above.

[0012] Sixthly, embodiments of the present invention also provide a computer program product, the computer program product including a computer program stored on a storage medium, the computer program including program instructions, which, when executed by a computer, cause the computer to perform any of the above-mentioned auxiliary identification methods.

[0013] In a seventh aspect, embodiments of the present invention also provide a mobile platform, characterized in that it includes: a fuselage, a power system, and electronic devices as described above; the power system and electronic devices are mounted on the fuselage, and the power system is used to provide power.

[0014] The method of this application involves a direct control terminal associated with a vehicle-mounted drone sending a resource sharing request to other indirect control terminals. Upon accepting the invitation from the direct control terminal, the indirect control terminals form a resource sharing team with the direct control terminal. After the direct control terminal and the indirect control terminals successfully establish the resource sharing team, the perception information and / or the relevant control permissions of the vehicle-mounted drone are shared to the resource sharing team. This allows all members of the resource sharing team to share the perception information and / or the relevant permissions acquired by the vehicle-mounted drone, thereby effectively improving the information perception capability of the entire fleet and increasing the utilization rate of the vehicle-mounted drone. Attached Figure Description

[0015] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 is a flowchart of a vehicle-mounted unmanned aerial vehicle resource sharing method according to an embodiment of the present invention;

[0017] Figure 2 is a flowchart of another vehicle-mounted unmanned aerial vehicle resource sharing method provided in an embodiment of the present invention;

[0018] Figure 3 is a flowchart of another vehicle-mounted unmanned aerial vehicle resource sharing method provided in an embodiment of the present invention;

[0019] Figure 4 is a flowchart of another vehicle-mounted unmanned aerial vehicle resource sharing method provided in an embodiment of the present invention;

[0020] Figure 5 is a connection diagram with a cellular network for a vehicle-mounted unmanned aerial vehicle resource sharing method according to an embodiment of the present invention.

[0021] Figure 6 is a connection diagram of a non-cellular network for a vehicle-mounted unmanned aerial vehicle resource sharing method according to an embodiment of the present invention.

[0022] Figure 7 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] The vehicle used with the vehicle-mounted drone can be a passenger car, pickup truck, or van, and this application does not limit the specific type of vehicle. The vehicle and the drone communicate with each other, allowing the user to send control commands to the drone from the vehicle. The drone can respond to these commands by performing flight missions (e.g., photography missions, route exploration missions, etc.) and sending the mission data obtained during the missions (e.g., captured image data, sensor perception data, etc.) back to the vehicle.

[0025] The vehicle-mounted UAV includes a flight controller, a positioning module, and a payload module. The positioning module may include at least one visual sensor, an inertial measurement module, a visual-inertial navigation module, a GNSS module, a multi-sensor fusion positioning module, a lidar, millimeter-wave radar, and an ultrasonic sensor, etc., without limitation herein. The payload module may include gimbal control, at least one main camera mounted on the gimbal, gimbal pose estimation, and a wireless data transmission module, etc., without limitation herein. The flight controller may include a perception module and a decision-making, planning, and control module. The perception module is used to construct a local map based on data collected by the positioning module. For example, the local map can be obtained by processing images (e.g., grayscale images) collected by the visual sensor in conjunction with motion data collected by the inertial measurement module to obtain a depth map and semantic recognition results, and then obtaining a local map based on the combination of the depth map and semantic recognition results. The decision-making, planning, and control module is used for task decision-making, planning the flight trajectory of the vehicle-mounted UAV, and performing flight control of the vehicle-mounted UAV. It should be noted that the above content is only one implementation of the vehicle-mounted UAV for implementing the resource sharing method for vehicle-mounted UAVs; other forms may also be used, without limitation herein.

[0026] Vehicle-mounted drones can receive control commands from the control terminal via a wireless data transmission module. Based on these commands, they can make mission decisions to generate specific instructions for different types of flight missions, and then control the gimbal and / or the vehicle-mounted drone based on these specific instructions. For example, for gimbal control, the flight controller can control the gimbal's movement based on specific instructions and gimbal pose estimation data. Similarly, for vehicle-mounted drone flight control, the flight controller can plan a flight trajectory based on specific instructions and a local map generated by the vehicle-mounted drone. During the flight mission, the flight controller can then control the drone based on the planned flight trajectory and the drone's pose determined by data collected from various sensors in the sensing system. For example, the drone's pose can be obtained through multi-sensor fusion positioning, combining images from a visual sensor, motion data from an inertial measurement module, and positioning data from a GNSS positioning module. Furthermore, the vehicle-mounted drone can transmit mission data collected during flight missions to the control terminal via the data transmission module. Mission data includes, but is not limited to, drone pose data, gimbal pose data, image data from the main camera, and environmental perception data from various sensors. It is understandable that the task data fed back to the control terminal is only an example, and the task data corresponding to different types of flight missions are different. For example, in a traffic information collection mission, the task data sent back to the control terminal may include traffic environment information collected by other sensors, local maps obtained by the perception module, etc., in addition to UAV pose data, gimbal pose data and image data collected by the main camera. For example, in an aerial photography mission, the task data sent back to the control terminal may include image data collected by the main camera.

[0027] The inventors discovered that existing vehicle-mounted drones are mostly one drone per vehicle, supporting only single-vehicle operation and unable to share information from the drone of a current vehicle with other vehicles. However, equipping every vehicle with a drone for field operations and / or tourism is prohibitively costly. Furthermore, even if every vehicle were equipped with a drone, for a convoy, since the vehicles follow the lead vehicle, if the lead vehicle could share information collected by its drone with the vehicles, the vehicles would not need to activate their drones, significantly saving energy consumption.

[0028] It should be noted that although the above embodiments describe a scenario in which a vehicle-mounted drone is deeply integrated with a vehicle, this application is not limited to this scenario. The vehicle-mounted drone can also be a regular drone with a control handle, which can be integrated with a vehicle in some way. This application does not impose any restrictions here.

[0029] Please refer to Figure 1, which shows a flowchart of a vehicle-mounted drone resource sharing method according to an embodiment of the present invention. This vehicle-mounted drone resource sharing method is used for vehicle-mounted drones and can exist in the form of an algorithm within the vehicle-mounted drone, such as a piece of program code or software; this application does not impose any limitations. Each vehicle-mounted drone has a directly bound control terminal, which can be a mobile terminal, a vehicle-mounted terminal, or a remote control, or other devices. The terminal can be software, such as an app; this application does not impose any limitations.

[0030] As shown in Figure 1, in step 101, a resource sharing link is established between the direct control terminal and the indirect control terminal, wherein each control terminal that successfully establishes a resource sharing link forms a resource sharing team.

[0031] In step 102, the perception information detected by the vehicle-mounted drone is shared with the resource-sharing team, and / or the control information of the vehicle-mounted drone sent by the resource-sharing team is received and responded to.

[0032] In this embodiment, for step 101, a resource-sharing link is established between the direct control terminal and the indirect control terminal. Each control terminal that successfully establishes a resource-sharing link forms a resource-sharing team. For example, the direct control terminal connected to the vehicle-mounted drone sends a link establishment request to the indirect control terminal. After the indirect control terminal receives and agrees to the request, a resource-sharing link is established, thereby forming a resource-sharing team. Alternatively, the indirect control terminal may request the direct control terminal to establish a link. This application does not limit this. The indirect control terminal refers to a terminal that is not bound to the vehicle-mounted drone. The indirect control terminal may also be a mobile phone, vehicle-mounted device, or other device. This application does not limit this.

[0033] For step 102, the vehicle-mounted drone shares the sensing information detected by the drone with the resource-sharing team, and / or receives control information from the resource-sharing team and responds. For example, after establishing a resource-sharing team, the vehicle-mounted drone will share the detected sensing information with the team and can receive instructions from the resource-sharing team and perform corresponding operations. Alternatively, the vehicle-mounted drone can share the detected sensing information only with the resource-sharing team without receiving instructions. This application does not limit this. The sensing information includes road information, member information, environmental information, etc., which are also not limited here.

[0034] In some specific examples, the direct control terminal has a corresponding resource-sharing operation interface. Users can send resource-sharing requests to non-direct control terminals by entering their unique identifiers on this interface, or they can send resource-sharing requests via social media. When the vehicle-mounted drone detects that another non-direct control terminal has accepted the request, the vehicle-mounted drone will share its sensing information with the non-direct control terminal and / or receive other operational instructions from the non-direct control terminal. Alternatively, other more secure methods can be used, which are not limited in this application. The resource-sharing team can be a group of direct and non-direct control terminals associated with the current vehicle-mounted drone and possessing relevant sensing and / or control permissions. This team can be temporary, such as for three days or a week, with the specific duration defined by the direct control terminal, or it can be permanent, which is not limited in this application. Resource sharing can involve temporarily binding the current vehicle-mounted drone to the non-direct control terminals in the resource-sharing team, allowing the non-direct control terminals to display the current vehicle-mounted drone's sensing and control interfaces. For example, corresponding permissions can be customized for each non-directly controlled terminal, such as uniformly customizing it to be able to view the perception interface and / or control interface of the vehicle-mounted drone, or selectively sharing certain permissions or functions when sending resource sharing requests. This application does not impose any restrictions here.

[0035] In this embodiment, the method involves a direct control terminal associated with the vehicle-mounted drone sending a resource-sharing request to other indirect control terminals. Upon accepting the invitation from the direct control terminal, the indirect control terminals form a resource-sharing team with the direct control terminal. After the direct control terminal and the indirect control terminals successfully establish the resource-sharing team, the perception information and / or the relevant control permissions of the vehicle-mounted drone are shared with the resource-sharing team. This allows all members of the resource-sharing team to share the perception information and / or the relevant permissions acquired by the vehicle-mounted drone, thereby effectively improving the environmental perception capability of the entire fleet and increasing the utilization rate of the vehicle-mounted drone.

[0036] Please further refer to Figure 2, which shows a flowchart of another vehicle-mounted unmanned aerial vehicle (UAV) resource sharing method provided by an embodiment of the present invention. This flowchart mainly illustrates the limitations of the steps for establishing a communication connection.

[0037] As shown in Figure 2, in step 201, a communication connection is established with the direct control terminal of the vehicle-mounted UAV through local link communication;

[0038] In step 202, if the current area is covered by a cellular network, a communication connection is established between the direct control terminal and the indirect control terminal via the cellular network.

[0039] In step 203, if there is no cellular network coverage in the current area, a communication connection is established with the indirect control terminal through the local link communication.

[0040] In this embodiment, for step 201, the vehicle-mounted drone establishes a communication connection with the direct control terminal of the vehicle-mounted drone via local link communication. Local link communication refers to a wireless communication link that supports short-range communication for the drone, such as Wi-Fi, SDR (Software-Defined Radio), etc., or a combination of one or more short-range communication links; this application is not limited thereto. SDR is a radio broadcast communication technology that is based on a software-defined wireless communication protocol rather than hardwired implementation. Frequency bands, air interface protocols, and functions can be upgraded through software downloads and updates without requiring a complete hardware replacement.

[0041] For step 202, if the current area has cellular network coverage, a communication connection is established between the direct control terminal and the indirect control terminal via the cellular network. A cellular network, also known as a mobile network, is a mobile communication hardware architecture, divided into analog cellular networks and digital cellular networks. It is named for the hexagonal shape of the signal coverage of the communication base stations constituting the network coverage, making the entire network resemble a honeycomb. Cellular networks can include 3G, 4G, 5G, etc. For example, by detecting the current network environment, if a cellular network environment is detected, the indirect control terminal connected to the vehicle-mounted drone connects to the indirect control terminal via the cellular network, and then shares resources, thereby reducing the energy consumption of the vehicle-mounted drone.

[0042] For step 203, if there is no cellular network coverage in the current area, a communication connection is established with the non-direct control terminal through the local link communication. For example, the current network environment is detected. If no cellular network is detected in the current environment, the local link communication of the vehicle-mounted drone is used to connect the direct control terminal and the non-direct control terminal, thereby ensuring resource sharing in areas without cellular networks. Specifically, the communication address of the non-direct control terminal can be sent from the direct control terminal to the vehicle-mounted drone, or vice versa, so that the vehicle-mounted drone can establish a communication connection with the non-direct control terminal using local link communication.

[0043] In this embodiment, the method involves a vehicle-mounted drone directly communicating with a direct control terminal via a local link. After connection, the network status of the current area is detected. If the area is covered by a cellular network, the direct control terminal connected to the vehicle-mounted drone connects to a non-direct control terminal via the cellular network for resource sharing. If the area lacks a cellular network, the vehicle-mounted drone connects to both the direct and non-direct control terminals via its local link. This ensures resource sharing for the vehicle-mounted drone in different network environments. Furthermore, in the presence of a cellular network, using the cellular network to achieve resource sharing between the direct and non-direct control terminals reduces the energy consumption of the vehicle-mounted drone.

[0044] In some optional embodiments, the method further includes: the vehicle-mounted drone identifying the location of each vehicle corresponding to each control terminal in the resource-sharing team, and sharing the location of each vehicle with each control terminal in the resource-sharing team. Identifying the location of each vehicle corresponding to each control terminal in the resource-sharing team can be achieved by the vehicle-mounted drone ascending to a certain height, identifying each vehicle in the convoy, and then identifying each vehicle. As for how to associate each vehicle with its corresponding control terminal, it can be done using location information sent by the control terminal, such as GPS signals or other location signals; this application does not limit this. For example, after the vehicle-mounted drone connects to both the direct and indirect control terminals, it identifies the location of vehicles in the resource-sharing team using the GPS signals from both terminals, and then shares the vehicle locations through resource sharing, thereby enabling members of the resource-sharing team to clearly know the location of each member. Optionally, after marking the location of each vehicle, the vehicle-mounted drone can also track the location of each vehicle in real time using its own target tracking algorithm, and share the tracking information with the resource-sharing team. This allows members of the resource-sharing team to know the location of each member's vehicle in real time, reducing the risk of getting lost.

[0045] Furthermore, the method further includes: detecting whether each control terminal in the resource-sharing team is within a preset range; if any control terminal is detected to be outside the preset range, alerting the other control terminals in the resource-sharing team, and marking the location of the most recently detected lost connection of any control terminal and the vehicle information corresponding to that control terminal on a map. Further, in response to receiving an instruction to search for any control terminal, returning to the lost connection location to search, and feeding back the search results to the other control terminals. The preset range can be the maximum range detectable by the vehicle-mounted drone, or a pre-set maximum distance from the lead vehicle or any other vehicle; this application does not limit this. In a specific example, after the vehicle-mounted drone detects a member has fallen behind, it alerts the resource-sharing team, marks the location of the lost connection on a map, and records the vehicle information, thus facilitating the convoy's immediate identification of the lost vehicle upon returning to search. In addition, after receiving instructions to find missing members, the vehicle-mounted drone can return to the recorded location of the lost vehicle for shooting and identification. The vehicle-mounted drone stores information about the missing vehicle (such as license plate number or vehicle characteristics), which can identify the missing vehicle in a timely manner. It can also provide real-time feedback of shooting information and identified missing vehicle information to the resource-sharing team.

[0046] Furthermore, the method also includes: identifying the personnel and / or vehicle conditions in the resource-sharing team based on images captured by the vehicle-mounted drone; if any abnormality is found in the personnel or vehicle conditions, the corresponding abnormality information is fed back to one or more control terminals in the resource-sharing team. In a specific example, the vehicle-mounted drone can also identify whether personnel in connected vehicles are injured and whether vehicles are damaged through captured images, and share the identification information with all vehicles in the team or the nearest vehicle, thereby facilitating timely rescue by other vehicles. This application is not limited thereto.

[0047] In some optional embodiments, the method further includes: in response to a member editing operation by the direct control terminal or the non-direct control terminal, performing editing processing on the control terminals in the resource-sharing team. The editing processing may include adding, deleting, or adjusting permissions of members in the resource-sharing team by the direct control terminal or the non-direct control terminal. For example, the direct control terminal may adjust the permissions of the non-direct control terminal so that the non-direct control terminal can only view the footage captured by the vehicle-mounted drone; or the non-direct control terminal may be able to invite other control terminals to join the resource-sharing team, just like the direct control terminal. This application does not impose any limitations on this. Furthermore, the operable content of the direct control terminal may be different from or the same as that of the non-direct control terminal; this application does not impose any limitations on this.

[0048] In some optional embodiments, the control priority of the direct control terminal is higher than that of the indirect control terminal. If the operation instructions issued by the direct control terminal and the indirect control terminal are different, the vehicle-mounted drone will execute the instructions issued by the direct control terminal first. If the instructions conflict with each other, the instructions of the indirect control terminal will be ignored. This application does not limit this.

[0049] In some optional embodiments, the method further includes: determining whether the detected surrounding environment information contains danger information; if the surrounding environment information contains danger information, feeding back the surrounding environment information and the warning about the danger information to each control terminal in the resource sharing team.

[0050] In this embodiment, after the vehicle-mounted drone takes off, it acquires and detects the surrounding environment to determine whether there are any dangerous areas (such as traffic accident areas, road collapse areas, etc.). If dangerous areas exist, hazard information can be generated (e.g., hazard area identification information). Detection can be based on existing drone target detection algorithms or future target detection algorithms; this application is not limited in this regard. Then, when the vehicle-mounted drone detects a dangerous area in the surrounding environment, it alerts both the direct and indirect control terminals of the resource-sharing team and shares the detected environmental information with all members of the resource-sharing team, ensuring that all team members are aware of the surrounding environment.

[0051] In some optional embodiments, the method further includes: receiving a coordinate sharing instruction from a control terminal in the resource sharing team, wherein the coordinate sharing instruction includes coordinates to be shared and a sharing object; and sharing the coordinates to be shared to the sharing object.

[0052] In this embodiment, when a control terminal of the resource-sharing team sends a coordinate-sharing instruction, the vehicle-mounted drone will receive the instruction. The coordinate-sharing instruction includes the coordinates to be shared and the sharing target. The sharing target can be the entire resource-sharing team, a specific member, or a few members within the team; this application does not impose any limitations. Upon receiving the coordinate-sharing instruction, the vehicle-mounted drone will detect the sharing target and then share the coordinates with that target. This allows the coordinates to be shared with the team or individual members, enabling the resource-sharing team to share its desired location with other members through coordinate sharing. For example, coordinate sharing can be used to reach a specific meeting point.

[0053] Please refer to Figure 3, which shows a flowchart of another vehicle-mounted drone resource sharing method provided by an embodiment of the present invention. This method is used for non-direct control terminals, where "non-direct control terminal" is a term relative to "direct control terminal," referring to a terminal not bound to the vehicle-mounted drone. The non-direct control terminal can also be a mobile phone, vehicle-mounted system, or other device; this application does not limit the scope of the application.

[0054] As shown in Figure 3, in step 301, the resource sharing request of the direct control terminal is received, wherein each control terminal that successfully establishes resource sharing forms a resource sharing team.

[0055] In step 302, in response to the indirect control terminal joining the resource sharing team, the indirect control terminal receives and displays the perception information detected by the vehicle-mounted drone and / or the relevant control permissions of the vehicle-mounted drone.

[0056] In this embodiment, regarding step 301, after the indirect control terminal accepts the resource sharing request from the direct control terminal, the direct control terminal and the indirect control terminal will form a resource sharing team. Then, regarding step 302, after the indirect control terminal joins the resource sharing team, the indirect control terminal will share the perception information and / or related control permissions of the vehicle-mounted drone, or only share the perception information of the vehicle-mounted drone; this application does not impose any limitations here. This ensures that the indirect control terminal can at least obtain the perception information and / or related control permissions of the vehicle-mounted drone.

[0057] In some optional embodiments, the method further includes: in response to receiving a coordinate sharing operation from a user on the non-direct control terminal, converting the point selected by the user on the control interface of the vehicle-mounted drone into coordinates and sharing them with other control terminals of the vehicle-mounted drone. For example, when a user performs a coordinate sharing operation on the operation interface of the non-direct control terminal, the user can select a location on the map by pointing to an image on the vehicle-mounted drone operation interface of the non-direct control terminal. Then, the non-direct control terminal converts the location into coordinates and shares the coordinates with other control terminals connected to the vehicle-mounted drone, thereby achieving coordinate sharing. The technology for converting coordinates by pointing to an image can refer to relevant industry applications or similar technologies developed in the future; this application will not elaborate on it here. Sharing the coordinates with other control terminals connected to the vehicle-mounted drone can be achieved through data transmission via the communication link established in the above embodiments.

[0058] In some optional embodiments, the method further includes: receiving surrounding environment information detected by the vehicle-mounted drone, determining whether the surrounding environment information contains dangerous information; if the surrounding environment information contains dangerous information, issuing a warning; or, receiving dangerous information fed back by the vehicle-mounted drone, issuing a warning, and / or, sharing the dangerous information with the resource-sharing team.

[0059] In this embodiment, after receiving real-time environmental information captured by the vehicle-mounted drone, the indirect control terminal can use a target detection algorithm to detect whether the environmental information contains hazardous information. If the indirect control terminal detects hazardous information, it alerts the user and marks the hazardous information. Alternatively, it can receive the information detected by the vehicle-mounted drone directly, alert the user, and share the hazardous information with members of the team, thereby alerting the resource-sharing team to potential hazards on the roads surrounding them. This detection, alerting, and marking of hazardous information can be performed autonomously by each control terminal, eliminating the need for sharing, or the vehicle-mounted drone can directly perform the detection and alerting, with each terminal receiving the hazardous information and alert. This application does not impose any limitations on this.

[0060] In some optional embodiments, the method further includes: in response to receiving a control command from a user on the indirect control terminal to the vehicle-mounted drone, sending the control command to the vehicle-mounted drone. For example, a user can control the vehicle-mounted drone to fly in other directions on the control interface of the indirect control terminal, and the generated flight command will be sent to the vehicle-mounted drone through the direct control terminal or local link communication, thereby enabling the indirect control terminal to operate the vehicle-mounted drone.

[0061] In some optional embodiments, the method further includes: receiving an alert and the location of any lost control terminal, and displaying the location of the lost control terminal and the corresponding vehicle information on a map. This ensures that when a lost control terminal is detected, a timely alert is received, and the lost control terminal and its corresponding vehicle information are displayed on the map interface of a non-directly controlled terminal. If the vehicle corresponding to the current control terminal is close to the lost control terminal, the current vehicle can proceed to confirm the specific situation of the lost control terminal.

[0062] Optionally, the method further includes: in response to receiving a search command from a user for any of the lost control terminals, sending the search command to the vehicle-mounted drone. Thus, when a lost control terminal is detected, the user can input a search command on a non-direct control terminal to instruct the vehicle-mounted drone to search for it.

[0063] In some optional embodiments, the control priority of the direct control terminal is higher than that of the indirect control terminal. If the operation instructions issued by the direct control terminal and the indirect control terminal are different, the vehicle-mounted drone will execute the instructions issued by the direct control terminal first. If the instructions conflict with each other, the instructions of the indirect control terminal will be ignored. This application does not limit this.

[0064] In some embodiments, the non-direct control terminal can edit member information, such as adding, deleting, and modifying permissions of control terminals in the resource-sharing team.

[0065] Please refer to Figure 4, which shows a flowchart of another vehicle-mounted drone resource sharing method provided by an embodiment of the present invention. The term "direct control terminal" is used in contrast to "non-direct control terminal," referring to a terminal bound to the vehicle-mounted drone. This direct control terminal can also be a mobile phone, vehicle-mounted system, or other device; this application does not impose any limitations on this.

[0066] As shown in Figure 4, in step 401, a resource sharing request is sent to the non-direct control terminal, wherein each control terminal that successfully establishes resource sharing forms a resource sharing team.

[0067] In step 402, in response to the indirect control terminal joining the resource sharing team, the perception information detected by the vehicle-mounted drone and / or the relevant control permissions of the vehicle-mounted drone are shared with the indirect control terminal.

[0068] In this embodiment, for step 401, a resource sharing request is sent to the non-direct control terminal, wherein each control terminal that successfully establishes resource sharing forms a resource sharing team. For example, a direct control terminal sends a resource sharing request to a non-direct control terminal, and when the non-direct control terminal accepts the request, it will establish a resource sharing team with the direct control terminal.

[0069] In step 402, in response to the indirect control terminal joining the resource-sharing team, the perception information detected by the vehicle-mounted drone and / or the relevant control permissions of the vehicle-mounted drone are shared with the indirect control terminal. For example, after establishing a resource-sharing team with the indirect control terminal, the direct control terminal will share the perception information and relevant control permissions of the vehicle-mounted drone, or only the perception information of the vehicle-mounted drone, or only the relevant control permissions, with the indirect control terminal; this application does not impose any limitations here. This enables the indirect control terminal to obtain the perception information and / or the relevant control permissions of the vehicle-mounted drone.

[0070] In some optional embodiments, the method further includes: in response to receiving a coordinate sharing operation from a user on the direct control terminal, converting the point selected by the user on the control interface of the vehicle-mounted drone into coordinates and sharing them with other control terminals of the vehicle-mounted drone. For example, when a user performs a coordinate sharing operation on the operation interface of the direct control terminal, the user can click on a location on a map on the vehicle-mounted drone operation interface of the control terminal, then convert that location into coordinates, and then share those coordinates with other control terminals connected to the vehicle-mounted drone, thereby achieving coordinate sharing. The direct control terminal can directly share coordinates with non-direct control terminals via a cellular network, or it can share coordinates with non-direct control terminals via the vehicle-mounted drone.

[0071] In some optional embodiments, the method further includes: receiving surrounding environment information detected by the vehicle-mounted drone, determining whether the surrounding environment information contains danger information; if the surrounding environment information contains danger information, issuing a warning; or, receiving danger information fed back by the vehicle-mounted drone, issuing a warning, and / or, sharing the danger information with the resource-sharing team. The direct control terminal can be either the receiver of danger information and the generator of the warning, and the tasks it needs to perform differ depending on its role; this application does not impose any limitations on this.

[0072] In this embodiment, the direct control terminal receives environmental information detected by the vehicle-mounted drone, determines whether the environmental information contains hazardous information, and issues a warning if the surrounding environmental information contains hazardous information. For example, if the direct control terminal detects hazardous information in the environmental information, it alerts the user and marks the hazardous information; or, after the vehicle-mounted drone directly detects the environmental information, the terminal directly receives the information detected by the vehicle-mounted drone, then alerts the user and shares the hazardous information with members of the team, thereby alerting the resource-sharing team to dangerous road conditions around them.

[0073] In some optional embodiments, the method further includes: in response to receiving a control command from a user on the direct control terminal to the vehicle-mounted drone, sending the control command to the vehicle-mounted drone. For example, when a user operates the vehicle-mounted drone on the direct control terminal, the command can be sent to the vehicle-mounted drone via local link communication, thereby enabling the vehicle-mounted drone to perform corresponding operations according to the command. Many functions of the direct control terminal are consistent with existing vehicle-mounted drone control terminals and will not be elaborated upon here. Other functions involving sharing with non-direct control terminals mainly concern related operations, and their implementation can utilize existing technologies or new technologies developed in the future; this application does not impose any limitations on these methods.

[0074] In some optional embodiments, the method further includes: receiving an alert and the location of any lost control terminal, and displaying the location of the lost control terminal and the corresponding vehicle information on a map. When the direct control terminal receives an alert and the corresponding location of a lost control terminal, it can display the lost control terminal and the corresponding vehicle information on the map interface of the direct control terminal to determine whether to proceed with the search.

[0075] Optionally, the method further includes: in response to receiving a search command from a user for any of the lost control terminals, sending the search command to the vehicle-mounted drone. Thus, when a search command for a lost control terminal is received from a user on the relevant interface of the direct control terminal, the search command can be sent to the vehicle-mounted drone to initiate the search.

[0076] Furthermore, the control priority of the direct control terminal is higher than that of the indirect control terminal. If the operation instructions issued by the direct control terminal and the indirect control terminal are different, the vehicle-mounted drone will prioritize the execution of the instructions issued by the direct control terminal. If the instructions conflict with each other, the instructions of the indirect control terminal will be ignored. This application does not impose any restrictions here.

[0077] In some embodiments, the direct control terminal can edit member information, including adding, deleting, and modifying permissions of control terminals within the resource-sharing team. For example, through the resource-sharing team information editing interface in the drone control application on the vehicle's large screen, users can add or delete team members and set permissions for team members (such as drone sharing and control permissions).

[0078] Please refer to Figure 5, which shows a cellular network connection diagram of a vehicle-mounted drone resource sharing method according to an embodiment of the present invention. The main vehicle is a vehicle bound to the vehicle-mounted drone, and the main vehicle is equipped with a vehicle-mounted drone hangar and an SDR module for communicating with the drone.

[0079] As shown in Figure 5, in an environment with cellular network coverage, the vehicle-mounted drone connects to the host vehicle (direct control terminal) via local link communication (e.g., SDR). The host vehicle then shares the information of the vehicle-mounted drone with other vehicles in the fleet (non-direct control terminals, such as mobile phones / vehicle-mounted systems with cellular communication capabilities) via a cellular network (e.g., 4 / 5G module), thereby achieving resource sharing. This allows other vehicles in the fleet to also control the flight and detection operations of the vehicle-mounted drone. This application does not impose any limitations on this.

[0080] Please refer to Figure 6, which shows a non-cellular network connection diagram of a vehicle-mounted unmanned aerial vehicle resource sharing method according to an embodiment of the present invention.

[0081] As shown in Figure 6, if there is no cellular network coverage in the current area, the vehicle-mounted drone will connect to the SDR modules of all vehicles in the fleet through local link communication, and then connect to the control terminal (such as mobile phone / vehicle system) on the vehicle through the SDR module. In this way, information is transmitted to the control terminal through the SDR module, realizing resource sharing of the vehicle-mounted drone and enabling all vehicles in the fleet to operate the vehicle-mounted drone.

[0082] In other embodiments, the present invention also provides a non-volatile computer storage medium storing computer-executable instructions that can execute the vehicle-mounted UAV resource sharing method in any of the above method embodiments.

[0083] Non-volatile computer-readable storage media may include a stored program area and a stored data area, wherein the stored program area may store an operating system and an application program required for at least one function; the stored data area may store data created based on the use of the vehicle-mounted drone resource-sharing device, etc. Furthermore, the non-volatile computer-readable storage medium may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the non-volatile computer-readable storage medium may optionally include memory remotely configured relative to a processor, and these remote memories may be connected to the vehicle-mounted drone resource-sharing device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0084] This invention also provides a computer program product, which includes a computer program stored on a non-volatile computer-readable storage medium. The computer program includes program instructions, which, when executed by a computer, cause the computer to perform any of the above-described vehicle-mounted unmanned aerial vehicle resource sharing methods.

[0085] Figure 7 is a schematic diagram of the electronic device provided in an embodiment of the present invention. As shown in Figure 7, the device includes one or more processors 710 and a memory 720. Figure 7 shows an example of one processor 710. The device for the vehicle-mounted drone resource sharing method may further include an input device 730 and an output device 740. The processor 710, memory 720, input device 730, and output device 740 can be connected via a bus or other means. Figure 7 shows an example of connection via a bus. The memory 720 is the aforementioned non-volatile computer-readable storage medium. The processor 710 executes various server functions and data processing by running non-volatile software programs, instructions, and modules stored in the memory 720, thereby implementing the vehicle-mounted drone resource sharing method of the above embodiment. The input device 730 can receive input digital or character information and generate key signal inputs related to user settings and function control of the vehicle-mounted drone resource sharing device of the embodiment. The output device 740 may include a display screen or other display device.

[0086] The above-described product can execute the method provided in the embodiments of the present invention, and has the corresponding functional modules and beneficial effects for executing the method. Technical details not described in detail in this embodiment can be found in the method provided in the embodiments of the present invention.

[0087] In one embodiment, the above-described electronic device is applied in a vehicle-mounted drone resource sharing device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to implement the vehicle-mounted drone resource sharing method provided in any of the above embodiments.

[0088] The electronic devices described in this application exist in various forms, including but not limited to:

[0089] (1) Mobile communication devices: These devices are characterized by their mobile communication capabilities and primarily aim to provide voice and data communication. These terminals include: smartphones (e.g., iPhones), multimedia phones, feature phones, and low-end phones, etc.

[0090] (2) Ultra-mobile personal computer devices: These devices fall under the category of personal computers, possessing computing and processing capabilities, and generally also have mobile internet access features. These terminals include PDAs, MIDs, and UMPCs, such as the iPad.

[0091] (3) Portable entertainment devices: These devices can display and play multimedia content. This category includes: audio and video players (such as iPods), handheld game consoles, e-books, as well as smart toys and portable car navigation devices.

[0092] (4) Server: A device that provides computing services. The components of a server include a processor, hard disk, memory, system bus, etc. Servers are similar to general computer architectures, but because they need to provide highly reliable services, they have higher requirements in terms of processing power, stability, reliability, security, scalability, and manageability.

[0093] (5) Other electronic devices with data interaction functions, such as vehicle domain controllers, drone remote controllers, etc.

[0094] This invention provides a mobile platform, comprising: a fuselage, a power system, and electronic equipment as described in the above embodiments. The power system is installed in the fuselage and provides power; the principle and implementation of the electronic equipment are consistent with those described in the above embodiments and will not be repeated here. Optionally, the mobile platform may include at least one of the following: a vehicle, a mobile robot, or a drone. Optionally, the mobile platform may further include a camera device installed in the fuselage for capturing images of the interior or surroundings of the mobile platform.

[0095] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0096] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of embodiments.

[0097] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for resource sharing of a vehicle-mounted unmanned aerial vehicle (UAV), used for a vehicle-mounted UAV, wherein the vehicle-mounted UAV has a corresponding direct control terminal, comprising: A resource-sharing link is established between the direct control terminal and the indirect control terminal, wherein each control terminal that successfully establishes the resource-sharing link forms a resource-sharing team. The vehicle-mounted drone shares the perception information detected by the drone with the resource-sharing team, and / or receives and responds to the control information of the drone sent by the resource-sharing team.

2. The method according to claim 1, characterized in that, Establishing a resource-sharing link between the direct control terminal and the indirect control terminal includes: A communication connection is established with the direct control terminal of the vehicle-mounted drone through local link communication; If the current area is covered by a cellular network, a communication connection is established between the direct control terminal and the indirect control terminal via the cellular network. If there is no cellular network coverage in the current area, a communication connection is established with the indirect control terminal through the local link communication.

3. The method according to claim 1, characterized in that, The method further includes: Identify the location of each vehicle corresponding to each control terminal in the resource-sharing team, and share the location of each vehicle with each control terminal in the resource-sharing team; And / or, Receive a coordinate sharing instruction from a control terminal in the resource sharing team, wherein the coordinate sharing instruction includes the coordinates to be shared and the sharing object; Share the coordinates to be shared to the sharing object.

4. The method according to claim 1, characterized in that, The method further includes: Detect whether each control terminal in the resource-sharing team is within a preset range; If any control terminal is detected to be outside the preset range, other control terminals in the resource sharing team will be alerted, and the location of the most recently detected lost connection of any control terminal and the vehicle information corresponding to any control terminal will be marked on the map.

5. The method according to claim 4, characterized in that, The method further includes: In response to receiving an instruction to search for any of the control terminals, the system returns to the location where it lost contact to perform the search and feeds back the search results to the other control terminals.

6. The method according to claim 1, characterized in that, The method further includes: In response to a member's edit operation on either the direct control terminal or the indirect control terminal, the control terminal in the resource-sharing team is edited.

7. The method according to claim 1, characterized in that, The method further includes: Determine whether the detected surrounding environmental information contains hazardous information; If the surrounding environment information contains danger information, the surrounding environment information and the warning about the danger information are fed back to each control terminal in the resource sharing team.

8. The method according to any one of claims 1-7, characterized in that, The method further includes: Identify the personnel and / or vehicle information of the resource-sharing team based on images captured by the vehicle-mounted drone; If any abnormality occurs in the personnel or / or vehicle situation, the corresponding abnormality information will be fed back to one or more control terminals in the resource-sharing team.

9. The method according to claim 1, characterized in that, The direct control terminal has a higher control priority than the indirect control terminal.

10. A method for resource sharing of a vehicle-mounted unmanned aerial vehicle (UAV), used in a non-direct control terminal of a vehicle-mounted UAV, the method comprising: Receive resource sharing requests from directly controlled terminals, where each control terminal that successfully establishes resource sharing forms a resource sharing team; In response to the indirect control terminal joining the resource-sharing team, the indirect control terminal receives and displays the perception information detected by the vehicle-mounted drone and / or the relevant control permissions of the vehicle-mounted drone.

11. The method according to claim 10, characterized in that, The method further includes: In response to receiving a coordinate sharing operation from a user on the non-direct control terminal, the point selected by the user on the control interface of the vehicle-mounted drone is converted into coordinates and shared with other control terminals of the vehicle-mounted drone. And / or, In response to receiving a control command from a user on the indirect control terminal for the vehicle-mounted drone, the control command is sent to the vehicle-mounted drone.

12. The method according to claim 10, characterized in that, The method further includes: Receive alerts and the location of any lost control terminal, and display the location of the lost control terminal and the corresponding vehicle information on a map.

13. The method according to claim 12, characterized in that, The method further includes: In response to receiving a search command from the user for any of the lost control terminals, the search command is sent to the vehicle-mounted drone.

14. The method according to claim 10, characterized in that, The method further includes: Receive the surrounding environment information detected by the vehicle-mounted drone and determine whether the surrounding environment information contains dangerous information; If the surrounding environment information contains dangerous information, a warning will be issued; Alternatively, it can receive hazard information from the vehicle-mounted drone, issue warnings, and / or share the hazard information with the resource-sharing team.

15. A method for resource sharing of a vehicle-mounted unmanned aerial vehicle (UAV), used as a direct control terminal for the UAV, the method comprising: Send resource sharing requests to non-directly controlled terminals, where each control terminal that successfully establishes resource sharing forms a resource sharing team; In response to the indirect control terminal joining the resource sharing team, the perception information detected by the vehicle-mounted drone and / or the relevant control permissions of the vehicle-mounted drone are shared with the indirect control terminal.

16. The method according to claim 15, characterized in that, The method further includes: In response to receiving a coordinate sharing operation from a user on the direct control terminal, the point selected by the user on the control interface of the vehicle-mounted drone is converted into coordinates and shared with other control terminals of the vehicle-mounted drone. And / or, In response to receiving a control command from a user on the direct control terminal for the vehicle-mounted drone, the control command is sent to the vehicle-mounted drone.

17. The method according to claim 15, characterized in that, The method further includes: Receive alerts and the location of any lost control terminal, and display the location of the lost control terminal and the corresponding vehicle information on a map.

18. The method according to claim 17, characterized in that, The method further includes: In response to receiving a search command from the user for any of the lost control terminals, the search command is sent to the vehicle-mounted drone.

19. The method according to claim 15, characterized in that, The method further includes: Receive the surrounding environment information detected by the vehicle-mounted drone and determine whether the surrounding environment information contains dangerous information; If the surrounding environment information contains dangerous information, a warning will be issued; Alternatively, it can receive hazard information from the vehicle-mounted drone, issue warnings, and / or share the hazard information with the resource-sharing team.

20. An electronic device comprising: At least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the steps of the method according to any one of claims 1-19.

21. A storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, it implements the steps of the method according to any one of claims 1-19.

22. A computer program product, comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method described in any one of claims 1-19.

23. A mobile platform, characterized in that, include: Airframe, power system, and electronic equipment as described in claim 20; The power system and electronic equipment are mounted on the fuselage, and the power system is used to provide power.

Images