Unmanned aerial vehicle control method and system for live broadcast interaction, and aircraft

Abstract

The invention provides an unmanned aerial vehicle control method and system for live broadcast interaction and an aircraft, and relates to the field of unmanned aerial vehicle control, and the method comprises the steps: building a live broadcast room through employing an unmanned aerial vehicle in a scenic region, and achieving the automatic control of the unmanned aerial vehicle in the live broadcast room through employing a scenic region virtual scene. Even if the user does not have an unmanned aerial vehicle operation skill, the unmanned aerial vehicle can be controlled to fly to the specified target object in the live broadcasting room, so that the problems of high unmanned aerial vehicle control threshold and single service scene in the prior art are solved.

Description

Technical Field

[0001] The present invention relates to the field of unmanned aerial vehicle control, and in particular to a live interactive unmanned aerial vehicle control method, system and aircraft. Background Art

[0002] The application scenarios of drones in the cultural and tourism industry are mainly based on individual operation, which requires individuals to carry drone equipment to the scenic spot for on-site flight tours, or third parties provide drone rental services in the scenic spot for individuals to fly and tour. However, no matter which method is used, individual users need to have drone operation skills, the technical threshold is high, and the service scenarios are limited.

[0003] It can be seen that in the prior art, when using drones for sightseeing in scenic spots, the technical requirements for drone control are high and there is a problem of a single service scenario. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a live interactive drone control method, system and aircraft. The method uses drones in a scenic area to build a live broadcast room, and uses the scenic area virtual scene in the live broadcast room to realize automatic control of the drone. Even if the user does not have drone operation skills, the drone can be controlled to fly to a designated target object in the live broadcast room, thereby solving the problems of high drone control threshold and single service scenario in the prior art.

[0005] In a first aspect, an embodiment of the present invention provides a method for controlling a drone for live interactive broadcasting, the method comprising: Obtain the drones contained in the target area, use the video streams collected by the drones to build a live broadcast room corresponding to the target area, and build a virtual scene corresponding to the target area based on the map data of the target area; Obtaining a list of objects selected by a first user in the live broadcast room through a virtual scene, and using the object list to determine a request instruction corresponding to the first user; Obtaining a target object in an object list obtained by a second user in the live broadcast room after responding to a request instruction; wherein the first user is different from the second user; Determine the control parameters of the drone according to the flight path between the target object and the drone; after controlling the drone to reach the shooting point corresponding to the target object based on the control parameters, control the drone to shoot the target object to obtain image or video data containing the target object; The drone is controlled to transmit the image or video data back to the first user and the second user in the live broadcast room.

[0006] In one embodiment, the step of obtaining a list of objects selected by a first user in a live broadcast room through a virtual scene, and determining a request instruction corresponding to the first user using the list of objects includes: Acquire first interaction data of the live broadcast room corresponding to the first user, and determine the control authority of the first user based on the first interaction data; Acquire a third user whose control authority satisfies a preset authority relationship from the first user, and acquire a target object selected by the third user through a virtual scene; An object list is determined according to the target object, and a request instruction corresponding to the first user is determined according to the object list.

[0007] In one embodiment, the step of obtaining the target object in the object list obtained by the second user in the live broadcast room after responding to the request instruction includes: Get the account level information corresponding to the first user in the object list; generating a priority policy corresponding to the first user based on the account level information; Obtaining a response result after the second user responds to the request instruction according to the priority policy; The target object corresponding to the first user in the object list is obtained using the response result.

[0008] In one embodiment, determining the control parameters of the drone according to the flight path between the target object and the drone includes: Obtaining the viewing angle parameters corresponding to the target object in the virtual scene, and using the viewing angle parameters to determine the shooting point corresponding to the target object; Set the shooting point as the end point of the flight route, and set the parking position of the drone or the real-time position of the drone during flight as the starting point of the flight route; Obstacle information contained in the target area at the start and end points of the flight route is obtained, and the control parameters of the drone are determined using the obstacle information.

[0009] In one embodiment, after controlling the drone to reach a shooting point corresponding to the target object based on the control parameters, the drone is controlled to shoot the target object to obtain image or video data containing the target object, including: Determine shooting parameters of the drone when shooting the target object at the shooting point according to the viewing angle parameters; Use control parameters to control the UAV to perform flight in real time; When it is detected that the drone reaches the shooting point corresponding to the target object, the drone is controlled to shoot the target object using the shooting parameters to obtain image or video data containing the target object.

[0010] In one embodiment, determining the control parameters of the drone according to the flight path between the target object and the drone includes: Obtaining the viewing angle parameters corresponding to the target object in the virtual scene, and using the viewing angle parameters to determine the shooting point corresponding to the target object; Set the shooting point as the end point of the flight route, and set the parking position of the drone or the real-time position of the drone during flight as the starting point of the flight route; Obtaining the first user who has control authority over the drone after the second user responds to the request instruction; A first real-time control parameter when a first user controls the unmanned aerial vehicle to fly between a starting point and an end point of a flight route is determined, and a control parameter of the unmanned aerial vehicle is determined according to the first real-time control parameter.

[0011] In one embodiment, controlling the drone to reach a shooting point corresponding to the target object based on the control parameters includes: Obtain the real-time location information and real-time status information of the drone based on the control parameters; Determine the risk level of the drone based on real-time location information and real-time status information; When the risk level is higher than the preset risk threshold, the drone is controlled to enter a hovering state and the control authority of the drone is set to the second user; A second real-time control parameter is determined when the second user controls the drone to fly between the starting point and the end point of the flight route, and the second real-time control parameter is used to control the drone to reach a shooting point corresponding to the target object.

[0012] In one embodiment, the step of controlling the drone to transmit image or video data back to the first user and the second user in the live broadcast room includes: Obtaining size parameters of image or video data, and determining the image transmission strategy of the UAV according to the size parameters; Determine compression parameters of image or video data using the image transmission strategy, and compress the image or video data using the compression parameters to obtain a thumbnail corresponding to the image or video data; The drone is controlled to transmit the thumbnail back to the first user and the second user in the live broadcast room.

[0013] In a second aspect, an embodiment of the present invention provides a live interactive drone control system, the system comprising: An initialization module is used to obtain the drones contained in the target area, build a live broadcast room corresponding to the target area using the video stream collected by the drones, and build a virtual scene corresponding to the target area based on the map data of the target area; A request instruction determination module is used to obtain a list of objects selected by a first user in the live broadcast room through a virtual scene, and determine a request instruction corresponding to the first user using the object list; A target object acquisition module, used to acquire a target object in an object list obtained by a second user in the live broadcast room after responding to a request instruction; wherein the first user is different from the second user; The UAV flight control module is used to determine the control parameters of the UAV according to the flight route between the target object and the UAV; after controlling the UAV to reach the shooting point corresponding to the target object based on the control parameters, the UAV is controlled to shoot the target object to obtain image or video data containing the target object; The drone image transmission control module is used to control the drone to transmit image or video data back to the first user and the second user in the live broadcast room.

[0014] In a third aspect, an embodiment of the present invention further provides an aircraft, which adopts the steps of the live interactive drone control method provided in the first aspect during the flight positioning control process.

[0015] In a fourth aspect, an embodiment of the present invention further provides an electronic device, including a processor and a memory, wherein the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the steps of the live interactive drone control method provided in the first aspect.

[0016] In a fifth aspect, an embodiment of the present invention further provides a storage medium, which stores computer executable instructions. When the computer executable instructions are called and executed by a processor, the computer executable instructions prompt the processor to implement the steps of the live interactive drone control method provided in the first aspect.

[0017] The embodiment of the present invention provides a method, system and aircraft for controlling a live interactive drone. In the process of controlling a drone to perform scenic sightseeing, the drones contained in a target area are first obtained, a live broadcast room corresponding to the target area is constructed using the video stream collected by the drone, and a virtual scene corresponding to the target area is constructed according to the map data of the target area; then a list of objects selected by a first user in the live broadcast room through the virtual scene is obtained, and a request instruction corresponding to the first user is determined using the object list; then a target object in the object list obtained by a second user in the live broadcast room after responding to the request instruction is obtained; wherein the first user is different from the second user, the first user is an audience user, and the second user is an anchor user; then the control parameters of the drone are determined according to the flight route between the target object and the drone; after controlling the drone to reach the shooting point corresponding to the target object based on the control parameters, the drone is controlled to shoot the target object to obtain image or video data containing the target object; finally, the drone is controlled to transmit the image or video data back to the first user and the second user in the live broadcast room. This method uses drones in a scenic area to build a live broadcast room, and uses the scenic area's virtual scene in the live broadcast room to achieve automatic control of the drone. Even if the user does not have drone operation skills, the drone can be controlled to fly to the designated target object in the live broadcast room, thereby solving the problems of high drone control threshold and single service scenario in the existing technology.

[0018] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0019] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 A flowchart of a live interactive drone control method provided by an embodiment of the present invention; Figure 2 A flowchart of step S102 of a live interactive drone control method provided by an embodiment of the present invention; Figure 3 A flowchart of step S103 of a live interactive drone control method provided by an embodiment of the present invention; Figure 4 A flowchart of determining the control parameters of the drone according to the flight path between the target object and the drone in step S104 of a live interactive drone control method provided by an embodiment of the present invention; Figure 5 In step S104 of a live interactive drone control method provided by an embodiment of the present invention, after controlling the drone to reach a shooting point corresponding to a target object based on control parameters, the drone is controlled to shoot the target object to obtain an image or video data containing the target object; Figure 6 A flowchart of determining the control parameters of the drone according to the flight path between the target object and the drone in step S104 of another live interactive drone control method provided by an embodiment of the present invention; Figure 7 A flowchart of controlling the drone to reach a shooting point corresponding to a target object based on control parameters in step S104 of another live interactive drone control method provided by an embodiment of the present invention; Figure 8 A flowchart of step S105 in a live interactive drone control method provided by an embodiment of the present invention; Fig. 9 A pointing flight control flow chart corresponding to a live interactive drone control method provided by an embodiment of the present invention; Fig.10 A remote control flow chart corresponding to a live interactive drone control method provided by an embodiment of the present invention; Fig.11 A schematic diagram of the structure of a live interactive drone control system provided by an embodiment of the present invention; Fig.12 A schematic diagram of the structure of an aircraft provided by an embodiment of the present invention; Fig.13 A schematic structural diagram of an electronic device provided by an embodiment of the present invention.

[0022] icon: 1110-initialization module; 1120-request instruction determination module; 1130-target object acquisition module; 1140-UAV flight control module; 1150-UAV image transmission control module; 101 - processor; 102 - memory; 103 - bus; 104 - communication interface. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described in combination with the embodiments below. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] The application scenarios of drones in the cultural and tourism industry are mainly based on personal operation. Individuals need to carry drone equipment to the scenic spot for on-site flight tours, or third parties provide drone rental services in the scenic spot for individuals to fly and tour. But no matter which method is used, individual users need to have drone operation skills, the technical threshold is high, and the service scenarios are limited. It can be seen that in the prior art, when using drones for scenic spot sightseeing, the technical requirements for drone control are high, and there is a problem of a single service scenario. Based on this, the present invention implements a live interactive drone control method, system, and aircraft. The method uses drones in the scenic area to build a live broadcast room, and uses the scenic spot virtual scene in the live broadcast room to realize automatic control of the drone. Even if the user does not have drone operation skills, the drone can be controlled in the live broadcast room to fly to the specified target object, thereby solving the problems of high drone control threshold and single service scenario in the prior art.

[0025] To facilitate understanding of this embodiment, a method for controlling a drone for live interactive broadcasting disclosed in an embodiment of the present invention is first described in detail. Figure 1 As shown, including: Step S101, obtain the drones contained in the target area, use the video streams collected by the drones to build a live broadcast room corresponding to the target area, and build a virtual scene corresponding to the target area according to the map data of the target area.

[0026] In this embodiment, an airport with drones deployed in advance in the scenic area is used as a carrier for flying tours of the scenic area. Unlike the prior art, the drones in this solution are not directly controlled by users, but are indirectly controlled by building a live broadcast room based on the video stream collected by the drone. Specifically, the content displayed in the live broadcast room is the video screen collected by the drone in real time, and the audience user can control the drone to fly to the destination corresponding to the request instruction by sending a request instruction to the live broadcast room and obtaining the consent of the anchor user. For the convenience of description, the audience user is described as the first user and the anchor user is described as the second user.

[0027] The request instruction sent by the first user to the live broadcast room is realized through a pre-built virtual scene, which is a three-dimensional model of the scenic spot contained in the target area, and the virtual scene is constructed based on the map data of the target area. In layman's terms, the virtual scene is a virtualized model of the target area, and the first user can freely select the target object in the virtual scene as the destination for the drone flight tour.

[0028] Step S102, obtaining a list of objects selected by a first user in the live broadcast room through a virtual scene, and using the object list to determine a request instruction corresponding to the first user.

[0029] The live broadcast room includes a first user and a second user, the second user is the controller of the second room, and the first user is the requester. After the first user selects one or more target objects in the virtual scene, he obtains an object list of these target objects, generates a corresponding request instruction based on the object list, and sends the request instruction to the second user. Based on the above process, after the first user selects the object list through the virtual scene, the list can be obtained and the request instruction of the first user can be determined.

[0030] Step S103, obtaining the target object in the object list obtained by the second user in the live broadcast room after responding to the request instruction; wherein the first user is different from the second user.

[0031] After receiving the request instruction from the first user, the second user will respond to the request instruction and obtain the target object in the object list. In actual scenarios, the second user, as the anchor user, will review the request instruction of the first user. If the request instruction contains the level information of the first user, the level information can be considered in the subsequent review. When there are multiple first users in the object list, the level information can be used to adjust the priority of the first users.

[0032] Step S104, determining the control parameters of the drone according to the flight path between the target object and the drone; after controlling the drone to reach the shooting point corresponding to the target object based on the control parameters, controlling the drone to shoot the target object to obtain image or video data containing the target object.

[0033] After acquiring the target object through image or video data, the location of the target object can be determined, and this location can be used as the destination of the drone. After combining the current location of the drone, the control parameters of the drone can be determined, and this parameter can control the drone to fly from the current location to the location of the target object. It is worth mentioning that after the drone reaches the location of the target object, it needs to fly to the shooting point corresponding to the target object. Different types of target objects have different corresponding shooting points and shooting parameters; and the shooting points and shooting parameters are obtained through control parameters.

[0034] After the control parameters are used to control the drone to fly to the shooting point of the target object, the drone is controlled to shoot the target object, thereby obtaining image or video data of the target object, which ultimately needs to be transmitted to the first user as a response result of the first user.

[0035] Step S105, controlling the drone to transmit the image or video data back to the first user and the second user in the live broadcast room.

[0036] The process of back-transmission needs to be realized in combination with the image transmission function of the drone itself, which will not be described in detail. It is worth mentioning that the image or video data is not only transmitted back to the first user, the viewer user, but also to the second user, the anchor user, in the live broadcast room, so as to be provided to the second user for use in the live broadcast room.

[0037] In one embodiment, a step S102 of obtaining a list of objects selected by a first user in a live broadcast room through a virtual scene and determining a request instruction corresponding to the first user using the list of objects is performed, such as Figure 2 As shown, including: Step S201, obtaining first interaction data of a live broadcast room corresponding to a first user, and determining the control authority of the first user based on the first interaction data; Step S202, obtaining a third user whose control authority satisfies a preset authority relationship from the first user, and obtaining a target object selected by the third user through a virtual scene; Step S203: determine an object list according to the target object, and determine a request instruction corresponding to the first user according to the object list.

[0038] Specifically, the first interaction data may be the payment data of the first user's corresponding live broadcast room, and the control authority of the first user can be determined based on the payment data after executing the relevant payment process. Then, a paying user whose control authority meets the preset authority relationship is obtained from the first user as the third user, and then the target object selected by the paying user through the virtual scene is obtained, and finally the request instruction corresponding to the first user is determined according to the object list determined by the target object.

[0039] In this implementation, when the first user makes a drone control request to the second user, the relevant payment process needs to be executed. Only the first user who has successfully paid can make a request to the second user, while other users can only watch the images transmitted by the drone of the first user who has paid in the live broadcast room. In the specific implementation process, the control authority of the first user is determined by obtaining the payment data of the first user, and then the paid users whose control authority meets the preset authority relationship are obtained from all the first users, and the target objects selected by the paid users through the virtual scene are obtained; finally, an object list is generated according to the target objects selected by the paid users, and the request instruction is determined according to the object list.

[0040] In one embodiment, step S103 of obtaining the target object in the object list obtained by the second user in the live broadcast room after responding to the request instruction is as follows: Figure 3 As shown, including: Step S301, obtaining the account level information corresponding to the first user in the object list; Step S302, generating a priority policy corresponding to the first user based on the account level information; Step S303, obtaining a response result after the second user responds to the request instruction according to the priority policy; Step S304: using the response result, obtain the target object corresponding to the first user in the object list.

[0041] There may be multiple first users in the object list, and there may also be a situation where a first user selects multiple target objects. Therefore, it is necessary to sort the multiple first users accordingly, so as to guide the second user to respond according to the sorting of the target objects. Specifically, first obtain the account level information corresponding to the first user in the object list, and then obtain the priority policy of the first user based on the account level information. The priority policy will guide the second user to select the first user. After the second user responds to the request instruction according to the priority policy, the target object corresponding to the first user is finally obtained using its response result.

[0042] In one embodiment, the control parameters of the drone are determined based on the flight path between the target object and the drone, such as Figure 4 As shown, including: Step S401, obtaining the viewing angle parameters corresponding to the target object in the virtual scene, and determining the shooting point corresponding to the target object using the viewing angle parameters; Step S402, setting the shooting point as the end point of the flight route, and setting the parking position of the UAV or the real-time position of the UAV during flight as the starting point of the flight route; Step S403, obtaining obstacle information contained in the target area at the start point and the end point of the flight route, and determining the control parameters of the UAV using the obstacle information.

[0043] Once the target object is determined, the flight control parameters of the drone can be determined. The control parameters need to take into account the obstacle information between the flight routes to ensure that the drone can avoid these obstacles during the flight in the real scenic area. At the same time, the attitude coordination of the drone during the flight is also required.

[0044] Specifically, firstly, the viewing angle parameters corresponding to the target object in the virtual scene are obtained, and the shooting point corresponding to the target object is determined using the viewing angle parameters. Different viewing angle parameters correspond to shooting points under different viewing angles.

[0045] After the shooting point is acquired, it is set as the end point of the flight route, and the parking position of the drone or the real-time position of the drone during flight is set as the starting point of the flight route, so as to obtain a specific flight route. After the flight route is acquired, the obstacle information contained in the target area at the starting point and the end point is obtained to generate the flight control parameters of the drone. The control parameters include various flight control parameters such as the speed parameter, steering parameter, and gimbal control parameter of the drone, so that the drone can smoothly execute when turning and switching the video angle, thereby ensuring the stability of the drone's flight process, which can bring a smooth and comfortable viewing experience to the first user.

[0046] In one embodiment, after the drone is controlled to reach the shooting point corresponding to the target object based on the control parameters, the drone is controlled to shoot the target object to obtain image or video data containing the target object, such as Figure 5 As shown, including: Step S501, determining shooting parameters when the drone shoots the target object at the shooting point according to the viewing angle parameters; Step S502, using the control parameters to control the UAV to perform flight in real time; Step S503, when it is detected that the drone reaches the shooting point corresponding to the target object, the drone is controlled to shoot the target object using the shooting parameters to obtain image or video data containing the target object.

[0047] The viewing angle parameters of the target object can not only determine its corresponding shooting point, but also determine the shooting parameters after the drone reaches the shooting point of the target object. Therefore, after using the control parameters to control the drone to fly to the shooting point, the corresponding shooting parameters are used to shoot the target object.

[0048] In actual scenarios, the control of the drone will be delegated to the first user. At this time, after the second user in the live broadcast room selects a target object, the target object will obtain the control authority of the drone, so that the target object can control the drone to fly freely, and the transmitted image will be viewed by other first users and second users in the live broadcast room. In this case, in one embodiment, the control parameters of the drone are determined according to the flight path between the target object and the drone, such as Figure 6 As shown, including: Step S601, obtaining the viewing angle parameters corresponding to the target object in the virtual scene, and determining the shooting point corresponding to the target object using the viewing angle parameters; Step S602, setting the shooting point as the end point of the flight route, and setting the parking position of the UAV or the real-time position of the UAV during flight as the starting point of the flight route; Step S603, obtaining the first user who has control authority over the drone after the second user responds to the request instruction; Step S604, determining a first real-time control parameter when the first user controls the drone to fly between the start point and the end point of the flight route, and determining a control parameter of the drone according to the first real-time control parameter.

[0049] The control parameters obtained at this time are based on the first user controlling the drone in real time. The first user is the first user with drone control authority obtained after the second response request instruction. In the actual scenario, there are multiple first users with drone control authority in the live broadcast room that send request instructions to the second user. The second user responds to these request instructions, selects a first user and issues the actual control of the drone to the first user; the first user controls the drone to fly within a certain range between the starting point and the end point of the flight route, thereby determining the first real-time control parameters of the current drone as the control parameters of the drone in the current scenario.

[0050] Since the first user controls the drone by himself, the risk of the drone needs to be assessed in real time. Once improper operation by the user is found, the drone needs to be controlled to a safe state, such as hovering, automatically returning to the starting point, etc. In one embodiment, the drone is controlled to reach the shooting point corresponding to the target object based on the control parameters, such as Figure 7 As shown, including: Step S701, obtaining the real-time position information and real-time status information of the UAV based on the control parameters; Step S702, determining the risk level of the drone based on the real-time location information and the real-time status information; Step S703: When the risk level is higher than the preset risk threshold, the drone is controlled to enter a hovering state, and the control authority of the drone is set to the second user; Step S704, determining a second real-time control parameter when the second user controls the drone to fly between the starting point and the end point of the flight route, and using the second real-time control parameter to control the drone to reach the shooting point corresponding to the target object.

[0051] Specifically, firstly, the real-time location information and real-time status information of the drone are obtained based on the control parameters, and whether the drone is in a dangerous state is obtained from the two dimensions of location and status. For example, the real-time location information can be used to determine whether the drone has flown out of the target area; the real-time status information can also be used to determine whether the drone has a risk of crashing, or whether the current power can ensure that the user continues to fly, etc. Therefore, the risk level of the drone can be determined by the real-time location information and real-time status information. If the current risk level is higher than the preset risk threshold, it means that the drone is in an abnormal state, then the drone is controlled to enter a hovering state, and the control authority of the drone is set to the second user. At this time, the first user loses the operation authority of the drone, thereby avoiding various safety conditions caused by improper operation when the first user controls the drone by himself.

[0052] In one embodiment, step S105 of controlling the drone to transmit image or video data back to the first user and the second user in the live broadcast room is as follows: Figure 8 As shown, including: Step S801, obtaining size parameters of image or video data, and determining an image transmission strategy of the drone according to the size parameters; Step S802, determining compression parameters of the image or video data using the image transmission strategy, and compressing the image or video data using the compression parameters to obtain a thumbnail corresponding to the image or video data; Step S803: Control the drone to transmit the thumbnail back to the first user and the second user in the live broadcast room.

[0053] Since the live broadcast scene has high network requirements, and the network of the drone is not stable enough when flying to the target area, the image or video data taken by the drone needs to be compressed to ensure the live broadcast effect. The compression process needs to be determined in combination with the size parameters of the captured image or video data, and the image transmission strategy of the drone is determined by the size parameters, and then the compression parameters of the image or video data are obtained, and the image or video data is compressed using the compression parameters and the thumbnail is transmitted to the live broadcast room for the first user and the second user to watch.

[0054] It can be seen that the live interactive drone control method in this embodiment uses the drone in the scenic area to build a live broadcast room, thereby using online live broadcast as a medium to expand the drone tour service objects to Internet users. Even if the user does not have drone operation skills, he can control the drone to fly to the specified target object in the live broadcast room. In the specific implementation process, the method can be implemented through two control modes, namely, pointing flight control and remote control.

[0055] like Fig. 9 The drone pointing flight control flow chart shown in the figure corresponds to the multi-person interactive pointing flight, which specifically includes the following processes: Step S901, the audience user selects the target object in the 3D model and sends an application to the anchor user. By deploying drones and airports in the scenic area and building a live broadcast room based on the real-time video stream of the drone, the audience users in the live broadcast room can switch to view the 3D model of the scenic area and capture points of interest at any time during the live broadcast on the APP. After paying, the audience user can initiate a flying request to the anchor user; in a specific scenario, the audience user needs to apply to the anchor user, and the anchor user responds to the application and sends a task to the drone, so that the drone flies to the corresponding target point in the real world and takes photos at the same position and angle; Step S902: The host user receives and reviews the task request. The host user receives and views the payment application of the audience user in real time on the APP, and conducts content review based on the model world photos sent by the audience user, including the audience user's corresponding: applicant, applicant account level, application time, and target object; Step S903: After reviewing, the anchor user selects one or more target objects and issues a pointing flight or continuous pointing flight mission to the drone; Step S904, according to the target object selected by the viewer user in the model world, analyzing the position and shooting parameters of the target object and automatically planning the route; Step S905: The drone performs the mission and takes photos of the target object, and sends the photos to the anchor user and the corresponding audience user. The drone is automatically controlled to perform the route mission based on the planned route, and hovers to take photos after reaching the target point. After the shooting is completed, the photos are automatically sent to the anchor user and the audience user, and can be viewed and downloaded at any time.

[0056] In actual scenarios, the route can be intelligently planned based on the parameter information of obstacles in the model world to ensure that the drone can avoid static obstacles in time during flight in the real world. The route smoothing algorithm is used to coordinate the position parameters of each waypoint and the drone speed parameters, steering parameters, and gimbal control parameters to achieve a silky-smooth effect of drone steering and real-time video angle switching during flight, allowing viewers to have a smoother and more comfortable viewing experience.

[0057] The anchor user can select a single target object and issue a single-route mission to the drone, or select multiple target objects at one time to issue continuous flight instructions for multiple-route missions to the drone. After receiving the multi-route continuous flight mission initiated by the anchor user, the drone can be dynamically evaluated to see whether it can complete the entire flight mission in combination with the drone's battery life and on-site meteorological information. If it is detected that the drone's current battery power is insufficient to support its round trip to multiple target points, a prompt message will be immediately sent to the anchor user through the APP. The anchor user can reselect the target object or return to the airport to replace the battery first, or ignore the prompt and issue the task directly. On this basis, the drone's breakpoint continuation strategy can be automatically triggered, that is, when the drone's real-time battery power drops to the low-battery return threshold, it automatically records the current location and returns to the airport to replace the battery. After the battery replacement is completed, the drone automatically returns to the marked point before the battery replacement and continues to perform the remaining tasks.

[0058] To ensure data transmission efficiency and high-definition live video without lag, two image sending strategies can be set. First, after shooting, the drone will directly send the high-definition original image to the audience user; second, after shooting, the drone will first send a thumbnail to the audience user for them to check the shooting effect in time, and when the drone returns to the airport, it will automatically read the high-definition image in the memory card and update the thumbnail to the high-definition original image.

[0059] like Fig.10 The remote flight control flow chart of the UAV shown in the figure corresponds to the remote flight of multi-person interaction, which specifically includes the following processes: Step S1001: After paying, the audience user applies to the anchor user in the live broadcast room through the APP for remote control of the drone and enters the queue sequence; Step S1002, the anchor user receives and views the payment application of the viewer user in real time in the APP, including the applicant corresponding to the viewer user, the account level of the applicant, and the application time; Step S1003, the anchor user selects a certain viewer user and transfers the remote control right of the drone, and monitors the operation process of the viewer user from the live broadcast room; Step S1004: The selected audience user obtains remote control rights for the drone within a certain range, and can control the drone to move forward, backward, and turn through the virtual joystick in the APP. The audience user can also adjust the gimbal control parameters to make the high-definition camera mounted on the drone turn to the direction of the point of interest, and take photos or record videos near the point of interest. Step S1005: After the flight experience countdown ends, the control of the drone is transferred to the host user. The audience user can also return the control in advance. In case of an emergency, the host user can directly take over the drone.

[0060] To ensure the safety of drone flight, an administrator role can be set up, and the administrator will monitor the operation process of the anchor user and the audience user in real time. For example, if the audience user encounters an emergency during the operation of the drone, but the anchor user fails to identify the risk in time, the administrator can directly take over the drone across levels and make subsequent disposal.

[0061] Audience users can remotely control drones within a certain spatial range, a certain time range, and a certain authority range. For example, audience users can control the flight of drones within the set airworthy area. If the drone is about to reach the boundary of the airworthy area, the audience user will be reminded in real time and slow down to a hovering state; audience users can control the flight of drones within the purchased time range. If the flight countdown is about to end, the audience user will be reminded in real time whether to renew the experience. After the user renews, the flight experience countdown will be recalculated. If the audience user chooses not to renew, the control right will be automatically returned after the countdown ends. Audience users can control drones within a certain authority range, specifically, they can control the drone's flight attitude, speed, gimbal angle, and take photos and videos. They can preview the photos taken during this experience, but they cannot directly control the flight return or transfer the control of the drone.

[0062] To sum up, the live interactive drone control method in the embodiment of the present invention utilizes the drones in the scenic area to build a live broadcast room, and uses the virtual scene of the scenic area in the live broadcast room to realize automatic control of the drone. Even if the user does not have drone operation skills, the drone can be controlled to fly to the specified target object in the live broadcast room, thereby solving the problems of high drone control threshold and single service scenario in the prior art.

[0063] Regarding the live interactive drone control method provided in the above embodiment, the embodiment of the present invention provides a live interactive drone control system, the system is as follows Fig.11 As shown, including: Initialization module 1110, used to obtain drones contained in the target area, build a live broadcast room corresponding to the target area using the video streams collected by the drones, and build a virtual scene corresponding to the target area according to the map data of the target area; The request instruction determination module 1120 is used to obtain a list of objects selected by a first user in the live broadcast room through a virtual scene, and determine a request instruction corresponding to the first user using the object list; The target object acquisition module 1130 is used to acquire the target object in the object list obtained by the second user in the live broadcast room after responding to the request instruction; wherein the first user is different from the second user; The UAV flight control module 1140 is used to determine the control parameters of the UAV according to the flight path between the target object and the UAV; after controlling the UAV to reach the shooting point corresponding to the target object based on the control parameters, the UAV is controlled to shoot the target object to obtain image or video data containing the target object; The drone image transmission control module 1150 is used to control the drone to transmit image or video data back to the first user and the second user in the live broadcast room.

[0064] The live interactive drone control system provided in the embodiment of the present invention has the same implementation principle and technical effects as those in the aforementioned live interactive drone control method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned live interactive drone control method embodiment.

[0065] The live interactive drone control system provided in the embodiment of the present invention can use the drones in the scenic area to build a live broadcast room, and use the virtual scene of the scenic area in the live broadcast room to realize automatic control of the drone. Even if the user does not have the skills to operate the drone, the drone can be controlled to fly to the specified target object in the live broadcast room, thereby solving the problems of high drone control threshold and single service scenario in the prior art.

[0066] This embodiment also provides an aircraft, such as Fig.12 As shown, the aircraft adopts the steps of the live interactive drone control method provided in the above embodiment during the flight positioning control process.

[0067] This embodiment also provides an electronic device. The structural diagram of the electronic device is as follows: Fig.13As shown, the system includes a processor 101 and a memory 102; wherein the memory 102 is used to store one or more computer instructions, and the one or more computer instructions are executed by the processor to implement the steps of the above-mentioned live interactive drone control method.

[0068] Fig.13 The electronic device shown further includes a bus 103 and a communication interface 104 , and the processor 101 , the communication interface 104 and the memory 102 are connected via the bus 103 .

[0069] The memory 102 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk storage. The bus 103 may be an ISA bus, a PCI bus, or an EISA bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.13 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0070] The communication interface 104 is used to connect to at least one user terminal and other network units through a network interface, and send the encapsulated IPv4 message or IPv4 message to the user terminal through the network interface.

[0071] The processor 101 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 101. The above processor 101 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The disclosed methods, steps and logic block diagrams in the embodiments of the present disclosure can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present disclosure can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module may be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 102, and the processor 101 reads the information in the memory 102 and completes the steps of the method of the above embodiment in combination with its hardware.

[0072] An embodiment of the present invention further provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the live interactive drone control method in the aforementioned embodiment are executed.

[0073] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0074] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0075] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0076] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that can be executed by a processor. Based on this understanding, the technical solution of the present invention can essentially or in other words, the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0077] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for controlling a drone with live interactive broadcasting, characterized in that: The method comprises: Acquire the drones contained in the target area, use the video streams collected by the drones to build a live broadcast room corresponding to the target area, and build a virtual scene corresponding to the target area according to the map data of the target area; Acquire a list of objects selected by a first user in the live broadcast room through the virtual scene, and use the object list to determine a request instruction corresponding to the first user; Obtaining a target object in the object list obtained by a second user in the live broadcast room after responding to the request instruction; wherein the first user is different from the second user; Determining control parameters of the drone according to a flight path between the target object and the drone; after controlling the drone to reach a shooting point corresponding to the target object based on the control parameters, controlling the drone to shoot the target object to obtain image or video data containing the target object; The drone is controlled to transmit the image or the video data back to the first user and the second user in the live broadcast room.

2. The method for controlling a drone for live interactive broadcasting according to claim 1, characterized in that: The step of obtaining a list of objects selected by a first user in the live broadcast room through the virtual scene, and using the list of objects to determine a request instruction corresponding to the first user, comprises: Acquire first interaction data of the first user corresponding to the live broadcast room, and determine the control authority of the first user based on the first interaction data; Acquire a third user whose control authority satisfies a preset authority relationship from the first user, and acquire a target object selected by the third user through the virtual scene; The object list is determined according to the target object, and the request instruction corresponding to the first user is determined according to the object list.

3. The method for controlling a drone for live interactive broadcasting according to claim 1, characterized in that: The step of obtaining the target object in the object list obtained by the second user in the live broadcast room after responding to the request instruction includes: Obtain account level information corresponding to the first user in the object list; generating a priority policy corresponding to the first user based on the account level information; Obtaining a response result after the second user responds to the request instruction according to the priority policy; The target object corresponding to the first user in the object list is obtained by using the response result.

4. The method for controlling a drone for live interactive broadcasting according to claim 1, characterized in that: Determining the control parameters of the drone according to the flight path between the target object and the drone includes: Obtaining a viewing angle parameter corresponding to the target object in the virtual scene, and determining the shooting point corresponding to the target object using the viewing angle parameter, The shooting point is set as the end point of the flight route, and the parking position of the UAV or the real-time position of the UAV during flight is set as the starting point of the flight route; Obstacle information contained in the target area at the starting point and the end point of the flight route is obtained, and the control parameters of the UAV are determined using the obstacle information.

5. The method for controlling a drone for live interactive broadcasting according to claim 4, characterized in that: After controlling the drone to reach a shooting point corresponding to the target object based on the control parameters, controlling the drone to shoot the target object to obtain image or video data containing the target object, including: Determining, according to the viewing angle parameters, shooting parameters when the drone shoots the target object at the shooting point; Using the control parameters to control the UAV to perform flight in real time; When it is detected that the drone reaches the shooting point corresponding to the target object, the drone is controlled to shoot the target object using the shooting parameters to obtain the image or video data containing the target object.

6. The method for controlling a drone for live interactive broadcasting according to claim 1, characterized in that: Determining the control parameters of the drone according to the flight path between the target object and the drone includes: Acquire a viewing angle parameter corresponding to the target object in the virtual scene, and determine the shooting point corresponding to the target object using the viewing angle parameter; The shooting point is set as the end point of the flight route, and the parking position of the UAV or the real-time position of the UAV during flight is set as the starting point of the flight route; Obtaining the first user who has control authority over the drone obtained after the second user responds to the request instruction; Determine a first real-time control parameter when the first user controls the drone to fly between the starting point and the end point of the flight route, and determine the control parameter of the drone according to the first real-time control parameter.

7. The method for controlling a drone for live interactive broadcasting according to claim 6, characterized in that: Controlling the drone to reach a shooting point corresponding to the target object based on the control parameter includes: Acquire real-time position information and real-time status information of the UAV based on the control parameters; Determine the risk level of the drone according to the real-time location information and the real-time status information; When the risk level is higher than a preset risk threshold, the drone is controlled to enter a hovering state, and the control authority of the drone is set to the second user; Determine a second real-time control parameter when the second user controls the drone to fly between the starting point and the end point of the flight route, and use the second real-time control parameter to control the drone to reach the shooting point corresponding to the target object.

8. The method for controlling a drone for live interactive broadcasting according to claim 1, characterized in that: The step of controlling the drone to transmit the image or video data back to the first user and the second user in the live broadcast room includes: Acquire a size parameter of the image or the video data, and determine an image transmission strategy of the drone according to the size parameter; Determining compression parameters of the image or the video data using the image transmission strategy, and compressing the image or the video data using the compression parameters to obtain a thumbnail corresponding to the image or the video data; The drone is controlled to transmit the thumbnail back to the first user and the second user in the live broadcast room.

9. A live interactive drone control system, characterized in that: The system comprises: An initialization module, used to obtain drones contained in a target area, construct a live broadcast room corresponding to the target area using the video streams collected by the drones, and construct a virtual scene corresponding to the target area according to the map data of the target area; A request instruction determination module, used to obtain a list of objects selected by a first user in the live broadcast room through the virtual scene, and determine a request instruction corresponding to the first user using the object list; A target object acquisition module, used to acquire the target object in the object list obtained by the second user in the live broadcast room after responding to the request instruction; wherein the first user is different from the second user; A UAV flight control module is used to determine the control parameters of the UAV according to the flight route between the target object and the UAV; after controlling the UAV to reach the shooting point corresponding to the target object based on the control parameters, the UAV is controlled to shoot the target object to obtain image or video data containing the target object; The drone image transmission control module is used to control the drone to transmit the image or the video data back to the first user and the second user in the live broadcast room.

10. An aircraft, characterized in that: The aircraft adopts the steps of the live interactive drone control method described in any one of claims 1 to 8 during the flight positioning control process.

Images