Unmanned aerial vehicle video collecting and returning method under multiple time scales

Abstract

The invention provides an unmanned aerial vehicle video acquisition and return method under multiple time scales. The method comprises the following steps: establishing a first target optimization model and a target constraint condition; the first target optimization model is used for maximizing the quality of the video acquired by the unmanned aerial vehicle; the target constraint condition is used for constraining the non-returned video amount of the unmanned aerial vehicle; and based on the target constraint condition, solving the first target optimization model to obtain an unmanned aerial vehicle video acquisition resolution under the first time scale and bandwidth resources allocated to the unmanned aerial vehicle for video return under the second time scale. According to the method provided by the embodiment of the invention, the video stream data queuing time delay can be reduced through small-time-scale resource allocation, so that the video acquired by the unmanned aerial vehicle can be returned in time, and video jitter caused by frequent video resolution switching can be effectively reduced through large-time-scale video resolution adjustment, so that the video quality is improved. The quality and stability of unmanned aerial vehicle cluster video data acquisition and return are efficiently improved under the limited bandwidth.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicles, and in particular to a method for collecting and transmitting unmanned aerial vehicle videos at multiple time scales. Background Art

[0002] With the increasing use of drones, especially in monitoring, disaster relief, agricultural inspection and other fields, the demand for real-time collection and transmission of drone video data is growing. The video data collection and transmission capabilities of a single drone in certain tasks can no longer meet the complex and diverse application requirements. Therefore, more and more application scenarios require the deployment of drone clusters for video data collection and transmission.

[0003] In practical applications, the main technical challenges faced by drone swarm video data transmission include limited bandwidth resources, high-quality video data acquisition, and frequent switching of video resolution. Therefore, how to accurately and efficiently collect and transmit drone videos is a technical problem that technicians in this field need to solve urgently. Summary of the invention

[0004] The present invention provides a method for collecting and transmitting UAV videos under multiple time scales, which can not only reduce the queuing delay of video stream data by allocating resources on a small time scale, so that the video collected by the UAV can be transmitted back in time, but also effectively reduce the video jitter caused by frequent switching of video resolution by adjusting the video resolution on a large time scale, so as to achieve the effect of optimizing the quality of the video stream and realize the efficient improvement of the quality and stability of the video data collection and transmission of the UAV cluster under limited bandwidth.

[0005] The present invention provides a method for collecting and transmitting unmanned aerial vehicle videos at multiple time scales, comprising the following steps.

[0006] Establishing a first target optimization model and target constraint conditions; the first target optimization model is used to maximize the quality of the video collected by the drone; the target constraint conditions are used to constrain the amount of video that is not returned by the drone; Based on the target constraint conditions, the first target optimization model is solved to obtain the UAV video acquisition resolution at a first time scale and the bandwidth resources allocated to the UAV for video backhaul at a second time scale; the first time scale is greater than the second time scale.

[0007] According to a method for collecting and transmitting drone videos at multiple time scales provided by the present invention, the first target optimization model includes: ； Wherein, k and n are integers greater than 0; T represents time; Indicates the drone video acquisition rate; The target constraints include: ； in, Indicates the amount of video that the drone has not sent back.

[0008] According to a method for collecting and transmitting drone videos at multiple time scales provided by the present invention, the method determines the drone video collection resolution at a first time scale and the bandwidth resources allocated to the drone for video transmission at a second time scale according to the first target optimization model and the target constraint condition, including: According to the first objective optimization model, the objective constraint condition and the Lyapunov optimization algorithm, a second objective optimization model and a third objective optimization model are obtained; wherein the second objective optimization model is used to determine the UAV video acquisition resolution at the first time scale; and the third objective optimization model is used to determine the bandwidth resources allocated to the UAV for video backhaul at the second time scale; According to the second objective optimization model and the third objective optimization model, the UAV video acquisition resolution at the first time scale and the bandwidth resources allocated to the UAV for video backhaul at the second time scale are determined.

[0009] According to a method for collecting and transmitting drone videos at multiple time scales provided by the present invention, the second objective optimization model includes:

[0010] Wherein, k and n are integers greater than 0; T represents the first time scale; V represents the first coefficient; Indicates the amount of video that the drone has not sent back; Indicates the drone video capture resolution.

[0011] According to a method for collecting and transmitting drone videos at multiple time scales provided by the present invention, the third objective optimization model includes: ; in, It is represented as the bandwidth allocated to the UAV at the second time scale t; Indicates the amount of video that the drone has not sent back; Indicates the transmission power of the aerial base station; represents a constant; represents the path loss exponent; Indicates drone Distance to the aerial base station; represents the noise power spectral density.

[0012] According to a method for collecting and transmitting drone videos at multiple time scales provided by the present invention, the drone video collection resolution includes at least one of the following: SD, HD and UHD.

[0013] The present invention also provides a drone video acquisition and return device under multiple time scales, comprising the following modules: The first determination module is used to establish a first target optimization model and target constraint conditions; the first target optimization model is used to maximize the quality of the video collected by the drone; the target constraint conditions are used to constrain the amount of video that is not returned by the drone; The second determination module is used to solve the first target optimization model based on the target constraint condition to obtain the UAV video acquisition resolution at a first time scale and the bandwidth resources allocated to the UAV for video backhaul at a second time scale; the first time scale is greater than the second time scale.

[0014] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the method for collecting and transmitting drone videos at multiple time scales as described in any one of the above is implemented.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for collecting and transmitting drone videos at multiple time scales as described in any one of the above is implemented.

[0016] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned methods for collecting and transmitting drone videos at multiple time scales.

[0017] The drone video acquisition and return method under multiple time scales provided by the present invention, on the one hand, by determining the first target optimization model and the target constraint condition, it is also possible to maximize the acquisition quality when the video data collected by the drone is returned in time, and at the same time meet the needs of high-quality video acquisition and timely return of the collected video. On the other hand, by analyzing and processing the first target optimization model and the target constraint condition, the drone video acquisition resolution of the first time scale and the bandwidth resources allocated to the drone for video return of the second time scale that meet the target optimization model and the target constraint condition are obtained, wherein the first time scale is greater than the second time scale, so that the bandwidth can be allocated on a small time scale, and the video resolution is adjusted on a larger time scale, that is, the video resolution is switched only in the initial time slot of each large time window. Through this multi-time scale joint decision control method, not only can the video stream data queuing delay be reduced by resource allocation on a small time scale, so that the video collected by the drone is returned in time, but also the video resolution adjustment on a large time scale can effectively reduce the video jitter caused by frequent switching of the video resolution, so as to achieve the effect of optimizing the quality of the video stream, and realize the efficient improvement of the quality and stability of the drone cluster video data acquisition and return under limited bandwidth. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 This is one of the flow charts of the method for collecting and transmitting drone videos at multiple time scales provided by the present invention.

[0020] Figure 2 It is a schematic diagram of the drone swarm system model provided by the present invention.

[0021] Figure 3 It is a schematic diagram of coverage path planning under large time scale UAV video resolution provided by the present invention.

[0022] Figure 4 This is the second flow chart of the method for collecting and transmitting drone videos at multiple time scales provided by the present invention.

[0023] Figure 5 This is the third flow chart of the method for collecting and transmitting drone videos at multiple time scales provided by the present invention.

[0024] Figure 6It is a structural schematic diagram of the UAV video acquisition and return transmission device under multiple time scales provided by the present invention.

[0025] Figure 7 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] Combine the following Figure 1-Figure 7 The present invention describes the drone video acquisition and return method under multiple time scales.

[0028] In order to facilitate a clearer understanding of the technical solutions of the embodiments of the present application, some technical contents related to the embodiments of the present application are first introduced.

[0029] With the increasing use of drones, especially in monitoring, disaster relief, agricultural inspection and other fields, the demand for real-time collection and transmission of drone video data is growing. The video data collection and transmission capabilities of a single drone in certain tasks can no longer meet the complex and diverse application requirements. Therefore, more and more application scenarios require the deployment of drone clusters for video data collection and transmission. However, in actual applications, drone cluster video data transmission faces the following major technical challenges: Limited bandwidth resources: Each drone in a drone cluster needs to transmit video data back. When the number of drones increases, the total bandwidth demand increases significantly. In the case of limited bandwidth resources, how to efficiently allocate bandwidth to ensure stable transmission of video data from each drone is a key issue.

[0030] High-quality video data collection requirements: Mission requirements often require drones to collect high-quality video data, but high-resolution video data is large in volume and occupies more bandwidth. How to maximize the quality of video data when bandwidth is limited is an important technical problem.

[0031] Frequent switching or fixed video resolution: Frequent switching of video resolution may cause video jitter, which in turn affects the viewing experience of the drone. Fixed resolution wastes resources when bandwidth is sufficient and causes delays and packet loss when bandwidth is insufficient.

[0032] Through the study of existing methods, it is not difficult to find that the fixed bandwidth allocation method allocates a fixed bandwidth to each drone, but in practical applications, due to the dynamic changes in mission requirements and network conditions, the fixed allocation method often cannot adapt to actual needs, resulting in resource waste or insufficient bandwidth. In addition, some existing methods use a single time scale to operate on bandwidth allocation, such as evenly allocating bandwidth in each time slot. However, this method cannot effectively deal with the contradiction between video resolution adjustment and changes in bandwidth requirements, and often leads to large fluctuations in video quality. In existing solutions, the choice of video resolution is mostly static, lacking a dynamic adjustment mechanism, and it is difficult to respond flexibly when bandwidth resources change. This leads to a sharp drop in video quality when bandwidth is tight, or a failure to fully utilize resources to improve video quality when bandwidth is sufficient.

[0033] Figure 1 This is one of the flow charts of the method for collecting and transmitting drone videos at multiple time scales provided by the present invention, such as Figure 1 As shown, the method includes the following: Step 101, establish a first target optimization model and target constraint conditions; the first target optimization model is used to maximize the quality of the video collected by the drone; the target constraint conditions are used to constrain the amount of video that the drone does not send back. The first determination module is used to establish a first target optimization model and target constraint conditions; the first target optimization model is used to maximize the quality of the video collected by the drone; the target constraint conditions are used to constrain the amount of video that the drone does not send back; Specifically, in the existing technology, it is difficult to simultaneously ensure high-quality video data acquisition and stable and efficient backhaul when backhaul bandwidth resources are limited. Existing solutions based on frequent switching of video resolutions often lead to severe video jitter, which seriously affects the quality of drone experience. In addition, existing methods lack effective dynamic adaptability when facing changes in backhaul bandwidth capacity and video quality requirements.

[0034] In order to solve the above problems, the first target optimization model and target constraint conditions are first determined in the embodiment of the present application; wherein, the first target optimization model is used to maximize the quality of the video collected by the drone; and the target constraint conditions are used to constrain the amount of video that the drone has not transmitted back. That is, in the process of determining the first target optimization model and the target constraint conditions in the present application, not only the quality of the video collected by the drone is considered to be guaranteed, but also the timely transmission of the video collected by the drone is required. Optionally, the video quality is related to the resolution of the video, including videos in formats such as standard definition, high definition and ultra-high definition. Optionally, the storage device in the drone can be used to store the collected video. If the efficiency of the transmission is too low and the collected video continues to grow, it will lead to insufficient storage space in the drone and the newly collected video of the drone cannot be stored and transmitted back in time. Therefore, the target constraint conditions need to constrain that during the operation of the drone, the amount of video that the drone has not transmitted back is less than the preset value or the amount of video that the drone has not transmitted back cannot grow indefinitely, so as to avoid the problem that the newly collected video of the drone cannot be stored and transmitted back in time due to insufficient storage space of the drone. In the embodiment of the present application, by determining the first target optimization model and the target constraint conditions, it is possible to maximize the quality of the video data collected by the drone when the video data is transmitted back in a timely manner, while meeting the needs of high-quality video collection and timely transmission of the collected video.

[0035] Step 102: Solve the first target optimization model based on the target constraint conditions to obtain the UAV video acquisition resolution at the first time scale and the bandwidth resources allocated to the UAV for video backhaul at the second time scale; the first time scale is greater than the second time scale.

[0036] Specifically, after determining the first target optimization model and target constraint conditions, the present application can obtain the drone video acquisition resolution and the bandwidth resources allocated to the drone for video backhaul that meet the target optimization model and target constraint conditions through analysis and processing of the first target optimization model and target constraint conditions.

[0037] Optionally, in the process of determining the drone video acquisition resolution and the bandwidth resources allocated to the drone for video backhaul according to the first objective optimization model and objective constraints, the present application takes into account that if a single time scale is used for the selection of video acquisition resolution and the allocation of video backhaul bandwidth, there is a problem of being unable to effectively deal with the contradiction between video resolution adjustment and changes in bandwidth requirements, which will cause large fluctuations in video quality; and frequent switching of video resolution may cause video jitter, thereby affecting the viewing experience of the drone.

[0038] In order to solve the above problems, in the embodiment of the present application, the bandwidth is allocated on a small time scale, while the video resolution is adjusted on a larger time scale, and the video resolution is switched only in the initial time slot of each large time window. Through this multi-time scale joint decision-making and control method, the jitter of the video stream can be effectively reduced, ensuring that the queuing delay of the video stream data is limited, thereby optimizing the quality of the video stream, reducing the effect of video jitter caused by frequent switching of video resolution, and achieving efficient improvement of the quality and stability of drone cluster video data acquisition and return under limited bandwidth. That is, in the embodiment of the present application, the video resolution selection and bandwidth allocation of the drone cluster are combined by the multi-time scale drone cluster, and the bandwidth allocation and resolution selection are dynamically and adaptively jointly optimized, which effectively solves the video data acquisition and return of the drone cluster under limited bandwidth conditions, realizes the stability of data return and improves the video acquisition quality at the same time.

[0039] The method of the above embodiment, on the one hand, by determining the first target optimization model and the target constraint condition, can also achieve the maximum acquisition quality when the video data collected by the drone is returned in time, and at the same time meet the needs of high-quality video acquisition and timely return of the collected video. On the other hand, by analyzing and processing the first target optimization model and the target constraint condition, the drone video acquisition resolution and the bandwidth resources allocated to the drone for video return that meet the target optimization model and the target constraint condition are obtained, and the bandwidth can be allocated on a small time scale, while the video resolution is adjusted on a larger time scale, and the video resolution is switched only in the initial time slot of each large time window. Through this multi-time scale joint decision-making control method, the jitter of the video stream can be effectively reduced, and the queuing delay of the video stream data is ensured to be limited, so as to achieve the effect of optimizing the quality of the video stream and reducing the video jitter caused by frequent switching of the video resolution, so as to achieve the efficient improvement of the quality and stability of the drone cluster video data acquisition and return under limited bandwidth.

[0040] In one embodiment, the first target model includes: ; Wherein, k and n are integers greater than 0; T represents time; Indicates the drone video acquisition rate; The target constraints include: ; in, Indicates the amount of video that the drone has not sent back.

[0041] Specifically, the application scenarios in the embodiments of the present application are as follows: Figure 2 As shown, it is assumed that there is drones, an aerial base station and a ground control station. The task of the drone swarm is to collect video information in the mission area. Each drone is responsible for a specific unit in the mission area, and the flight path follows the coverage path planning algorithm to fully cover the mission area. During the flight, the drone collects video data of the area it is responsible for and transmits it back to the air base station, which then transmits the data back to the ground control center via wired communication. The system's operating timeline is divided into equal lengths of The time slot index is .make Indicates that the drone swarm is in the time slot The distance from the aerial drone base station is affected by the path planning strategy and the choice of video acquisition resolution.

[0042] Optionally, each time slots form a large time window, and the A large time window is defined as a set of In addition, each drone flies at a different constant altitude according to the video resolution. at a constant speed Flight. Optionally, time slot The second time scale is The time window composed of time slots is the first time scale. Optionally, the first time scale and the second time scale may also be defined according to actual conditions, and no specific limitation is made in the embodiments of the present application.

[0043] Optionally, the mission backhaul bandwidth of the entire drone cluster is provided by the aerial drone base station, with a total bandwidth of B in MHz. All drones share the total bandwidth. In each time slot, the total bandwidth is allocated to each drone

[0044] (1)

[0045] Among them, the variable For each drone in the time slot The allocated bandwidth is determined by each time slot by considering the communication channel model and the data backhaul. The video data backhaul transmission is air-to-air communication (from the sensor drone to the air base station), so it is considered as a free space line-of-sight model. The upper limit of the data rate on the drone video data backhaul channel link is defined as

[0046] (2)

[0047] in, Indicates time slot The drone at the beginning Video data transmission rate to the aerial base station, is the transmitting power of the aerial base station in watts. is the path loss exponent, For the time slot Drones The distance to the aerial base station, is the noise power spectral density, in W / Hz.

[0048] Optionally, in order to perform video acquisition through a drone swarm, a given task area needs to be divided into N drones. The first step of region decomposition is to decompose a given complex polygonal task area of ​​any shape into multiple simple polygonal units, thereby allocating a large task area to multiple drones. The typical sampling-based centroid Voronoi diagram method is used because it is scalable to a large number of units and a simple method is used to obtain decomposition units of roughly equal size, which can also pay more attention to the impact of communication bandwidth on the overall cluster task planning performance. Within the given task area boundary, evenly distributed points are sampled and clustered using the k-means clustering algorithm. Clusters are clustered. Then, based on the centroids, the Voronoi partitioning method is applied to decompose the given task area into Each decomposed cell is assigned to each drone in a one-to-one manner (the nth cell is assigned to drone n). Figure 3 and Figure 4 In , an example of polygonal region decomposition is shown, and the loop overlay path planning method is used to plan the mission path for each UAV.

[0049] Furthermore, during the video acquisition process, the size of telemetry and other data packets is negligible compared to the size of the video data transmitted by the drone backhaul. Therefore, it can be considered that the channel capacity of the bandwidth link allocated to the drone is fully used for the mission video backhaul transmission, that is, , that is, the link capacity is approximated as the maximum available bit rate for video transmission. When the encoding method and frame rate are fixed, the video bit rate (Mbps) is proportional to the total number of pixels in the video frame (height pixels × width pixels). Fixed video acquisition frame rate and video frame aspect ratio, each time slot The number of acquired frames is fixed.

[0050] Optionally, drone exist The time slot resolution is

[0051] (3)

[0052] in is the maximum positive constant of the resolution, Indicates the video resolutions as standard definition, high definition, and ultra high definition respectively.

[0053] Furthermore, the drone video data acquisition rate is

[0054] (4)

[0055] in, is a constant determined by the video capture frame rate and fixed coding efficiency.

[0056] Optionally, each drone performing a mission has a cache data queue, whose length indicates the amount of video data that has not been sent back. is the queue length vector, where For drones In time slot The queue length. All queues are initially empty.

[0057] (5)

[0058] The queue length varies based on the data arrival rate of the drone video collection and the video data return transmission rate: (6)

[0059] The maximum value between its value and 0 is taken here because the number of video data packets actually served cannot exceed the current queue size. The goal of this application is to design an online video acquisition quality selection and resource allocation algorithm to maximize the video acquisition quality.

[0060] Furthermore, based on the above drone swarm system model, this application determines the first objective optimization model and objective constraints as follows: (7)

[0061] in, It's a drone The average queue length over time, all Constraints The stability of the network is guaranteed, thus achieving bounded traffic delay. According to Little's law, With drones The target constraint condition can avoid the problem that the newly collected videos of the drone cannot be stored and transmitted back in time due to insufficient storage space of the drone. In this application, based on the drone cluster system model, the first target optimization model and target constraint conditions are established, which can maximize the quality of the collected video data when the drone is transmitted back in time, while meeting the needs of high-quality video collection and timely transmission of collected videos.

[0062] The method of the above embodiment, by establishing a drone cluster system model, and then determining the first target optimization model and target constraint conditions based on the drone cluster system model, can maximize the quality of collection when the video data collected by the drone is transmitted back in a timely manner, while meeting the needs of high-quality video collection and timely transmission of collected videos.

[0063] In one embodiment, according to the first objective optimization model and the objective constraint condition, determining the drone video acquisition resolution at the first time scale and the bandwidth resources allocated to the drone for video backhaul at the second time scale includes: According to the first objective optimization model, the objective constraint conditions and the Lyapunov optimization algorithm, the second objective optimization model and the third objective optimization model are obtained; wherein the second objective optimization model is used to determine the UAV video acquisition resolution at the first time scale; the third objective optimization model is used to determine the bandwidth resources allocated to the UAV for video backhaul at the second time scale; According to the second objective optimization model and the third objective optimization model, the UAV video acquisition resolution at the first time scale and the bandwidth resources allocated to the UAV for video backhaul at the second time scale are determined.

[0064] Specifically, for the first-goal optimization model and the target constraints, if the average backlog of the message queue in the drone is finite, then the queue is strongly stable. The optimization problem (7) corresponding to the first-goal optimization model can be solved by using the Lyapunov optimization method, which is reduced to solving the following determination problem in each time slot: (9)

[0065] It is obvious that and They are decoupled from each other. The optimization problem (9) can be decomposed into the following problems: Video capture resolution decision: For each drone , determine the UAV video acquisition resolution by solving (10) .

[0066] (10)

[0067] UAV base station bandwidth allocation decision: For each UAV , the bandwidth allocated to the UAV is determined by solving (11) That is, the third objective optimization model can be expressed as follows: (11)

[0068] For example, if drone video collection and transmission are performed at a single time scale, the drone video collection and transmission can be performed based on the following process: For each time slot, the aerial drone base station and the mission drone perform the following steps: Step 1: Initialization ,

[0069] Step 2: Resolution selection and bandwidth allocation. For each time slot , the aerial drone base station obtains the amount of video that each drone has not sent back And data transmission is performed according to the bandwidth allocated by (11). , get its own parameters and queue backlog ,According to (10), the video resolution of the UAV video acquisition is determined, and the coverage path of the UAV is replanned.

[0070] Step 3: Update the data cache queue according to (6) .

[0071] Step 4: Set the time slot Return to step 2.

[0072] For example, if drone video collection and transmission at multiple time scales are used, drone video collection and transmission can be performed based on the following process: The multi-scale in the embodiment of the present application is reflected in the operation of the drone system on multiple time scales. Since frequent switching between different resolutions can cause jitter between video frames, affecting the drone service experience and possibly causing non-negligible costs, the present application performs resolution selection of the drone on a large time scale, while capture selection and rate adaptation are performed on a small time scale. The drone only needs to switch resolution on a large time scale to ensure the smoothness of the video.

[0073] Figure 4 The figure shows a multi-time scale dynamic operation, which is divided into two time scales. The UAV determines the resolution of video acquisition at the beginning of each large time scale interval (large time scale); the aerial base station determines the bandwidth allocation of each UAV at the beginning of each time slot (small time scale).

[0074] By evaluating (9) at different time scales, the optimal solution requires explicit a priori knowledge of the future backhaul rate state, which is not feasible in practice. The future queue backlog is approximated as the time slot The current backlog at each time slot , , and thus the solution is derived. Each drone passes through the time window Maximize the upper and lower formulas to decide whether to switch the drone video capture resolution , that is, the second objective optimization model can be expressed as follows: (12)

[0075] After that, drones exist Time window usage Resolution, in each time slot , each drone uses Perform video capture.

[0076] Figure 5 The process of multi-time scale dynamic adaptive resolution selection and bandwidth allocation method is as follows: Based on each time slot as a small scale and T composed of multiple time slots as a large time scale, the steps of multi-time scale dynamic adaptive resolution selection and bandwidth allocation method are as follows: Step 1: Initialization ,

[0077] Step 2: On a large time scale, the initial time slot of each large time window , each drone passes through the time window Maximize (12) to decide whether to switch the video resolution , and then plan the coverage path for the UAV. On a small time scale, each time slot , the aerial drone base station obtains the amount of video that each drone has not sent back in each time slot And data transmission is carried out according to the bandwidth allocated by (11).

[0078] Step 3: Update the data queue according to (6) .

[0079] Step 4: Update time slots , return to step 2.

[0080] The method of the above embodiment obtains the second objective optimization model and the third objective optimization model according to the first objective optimization model, the objective constraint condition and the Lyapunov optimization algorithm; wherein the second objective optimization model is used to characterize the video acquisition resolution decision of the drone and the bandwidth allocation decision of the drone base station; and then according to the second objective optimization model and the third objective optimization model, the drone video acquisition resolution at the first time scale and the bandwidth resources allocated to the drone for video return at the second time scale are determined, which effectively realizes the decomposition of the multi-objective optimization problem of drone video resolution selection and return bandwidth allocation, reduces the difficulty of solving the multi-objective optimization problem, and improves the efficiency of determining the drone video acquisition resolution at multiple time scales and the bandwidth resources allocated to the drone for video return, thereby effectively reducing the jitter of the video stream and ensuring that the queuing delay of the video stream data is limited, thereby optimizing the video stream quality and reducing the video jitter caused by frequent switching of video resolution, and realizing the efficient improvement of the quality and stability of drone cluster video data acquisition and return under limited bandwidth.

[0081] In summary, it should be noted that the multi-scale UAV cluster acquisition resolution selection and bandwidth allocation method in the embodiment of the present application decomposes a given task area into multiple task areas through a centroid-based Voronoi diagram method, and each UAV is responsible for video data acquisition in a specific area, realizing dynamic adaptive resource scheduling, allocating limited bandwidth to each UAV in each time slot to ensure the return transmission of video data, and dynamically selecting the resolution of video acquisition on a large time scale, maximizing the quality of UAV cluster video data acquisition while ensuring effective information return. In the embodiment of the present application, a method of combining small-time-scale UAV bandwidth allocation with large-time-scale video acquisition resolution is adopted to realize dynamic adaptation of video acquisition resolution selection and bandwidth allocation, and achieve a balance between video acquisition quality and data return stability.

[0082] Compared with the prior art, the embodiment of the present application provides a multi-time scale drone cluster video resolution selection and bandwidth allocation method. By dynamically and adaptively optimizing bandwidth allocation and resolution selection, bandwidth is dynamically allocated to each drone in each time slot, and the video resolution is adjusted on a larger time scale, that is, the video resolution is switched only in the initial time slot of each large time window. This method can not only ensure the stability of video data return, but also optimize the video acquisition quality and improve the drone experience under the condition of limited bandwidth resources. The method in the embodiment of the present application, (1) comprehensively considers the limited bandwidth allocation return and video resolution selection, so that the drone swarm can pursue higher quality video resolution while ensuring the effective return of the collected video data. Compared with the prior art solution that only pursues maximizing the minimum return bandwidth of the drone, the rationality of the video acquisition resolution of the drone swarm and the rationality of the limited bandwidth allocation for video return are greatly improved. (2) The allocation control of the system is reflected in two time scales. The drone determines the resolution of video acquisition at the beginning of each large time scale interval, and the base station determines the bandwidth allocation of each drone at the beginning of each time slot. The multi-time-scale resolution selection and bandwidth allocation method proposed in this application can ensure the stable and effective return of the video data of the entire drone swarm. At the same time, the large-time-scale video resolution selection can also ensure the stability of video acquisition. (3) It achieves a balance between video acquisition quality and data return performance, realizes dynamic adaptive cross-layer bandwidth resource allocation scheduling, and can be expanded to large groups. It can make choices based on different needs, effectively solving the problems of video resolution selection and return bandwidth resource allocation in the process of drone cluster video data acquisition and return.

[0083] The following is a description of the drone video acquisition and return device at multiple time scales provided by the present invention. The drone video acquisition and return device at multiple time scales described below and the drone video acquisition and return method at multiple time scales described above can be referred to each other. Figure 6 As shown, including: The first determination module 610 is used to determine a first target optimization model and a target constraint condition; the first target optimization model is used to maximize the quality of the video collected by the drone; the target constraint condition is used to constrain the amount of video that is not returned by the drone; The second determination module 620 is used to determine the UAV video acquisition resolution at the first time scale and the bandwidth resources allocated to the UAV for video backhaul at the second time scale according to the first target optimization model and the target constraint conditions.

[0084] Optionally, the first objective optimization model includes: ； Wherein, k and n are integers greater than 0; T represents time; Indicates the drone video acquisition rate; The target constraints include: ； in, Indicates the amount of video that the drone has not sent back.

[0085] Optionally, the second determining module 620 is specifically configured to: According to the first objective optimization model, the objective constraint conditions and the Lyapunov optimization algorithm, the second objective optimization model and the third objective optimization model are obtained; wherein the second objective optimization model is used to determine the UAV video acquisition resolution at the first time scale; the third objective optimization model is used to determine the bandwidth resources allocated to the UAV for video backhaul at the second time scale; According to the second objective optimization model and the third objective optimization model, the UAV video acquisition resolution at the first time scale and the bandwidth resources allocated to the UAV for video backhaul at the second time scale are determined.

[0086] Optionally, the second objective optimization model includes:

[0087] Wherein, k and n are integers greater than 0; T represents the first time scale; V represents the first coefficient; Indicates the amount of video that the drone has not sent back; Indicates the drone video capture resolution.

[0088] Optionally, the third objective optimization model includes: ; in, It is represented as the bandwidth allocated to the UAV at the second time scale t; Indicates the amount of video that the drone has not sent back; Indicates the transmission power of the aerial base station; represents a constant; represents the path loss exponent; Indicates drone Distance to the aerial base station; represents the noise power spectral density.

[0089] Optionally, the drone video capture resolution includes at least one of the following: SD, HD and UHD.

[0090] Figure 7The schematic diagram of the physical structure of an electronic device is illustrated, and the electronic device may include: a processor 710, a communication interface 720, a memory 730 and a communication bus 740, wherein the processor 710, the communication interface 720 and the memory 730 communicate with each other through the communication bus 740. The processor 710 may call the logic instructions in the memory 730 to execute the drone video acquisition and return method under multiple time scales, and the method includes: determining a first target optimization model and a target constraint condition; the first target optimization model is used to maximize the quality of the video collected by the drone; the target constraint condition is used to constrain the amount of video that is not returned by the drone; according to the first target optimization model and the target constraint condition, determining the drone video acquisition resolution under the first time scale and the bandwidth resources allocated to the drone for video return under the second time scale.

[0091] In addition, the logic instructions in the above-mentioned memory 730 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0092] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the drone video acquisition and return method at multiple time scales provided by the above methods, and the method includes: determining a first target optimization model and target constraints; the first target optimization model is used to maximize the quality of the video collected by the drone; the target constraints are used to constrain the amount of video not returned by the drone; based on the first target optimization model and the target constraints, determine the drone video acquisition resolution at the first time scale and the bandwidth resources allocated to the drone for video return at the second time scale.

[0093] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the UAV video acquisition and feedback methods at multiple time scales provided by the above-mentioned methods, the method comprising: determining a first target optimization model and target constraints; the first target optimization model is used to maximize the quality of the video collected by the UAV; the target constraints are used to constrain the amount of video not transmitted back by the UAV; based on the first target optimization model and the target constraints, determining the UAV video acquisition resolution at the first time scale and the bandwidth resources allocated to the UAV for video feedback at the second time scale.

[0094] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0095] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for collecting and transmitting drone videos at multiple time scales, characterized in that: include: Establish the first objective optimization model and objective constraints; The first objective optimization model is used to maximize the quality of the video collected by the drone; The target constraint condition is used to constrain the amount of video that is not transmitted back by the drone; Based on the target constraint conditions, the first target optimization model is solved to obtain the UAV video acquisition resolution at a first time scale and the bandwidth resources allocated to the UAV for video backhaul at a second time scale; the first time scale is greater than the second time scale.

2. The method for collecting and transmitting drone videos at multiple time scales according to claim 1 is characterized in that: The first objective optimization model includes: ； Wherein, k and n are integers greater than 0; T represents time; Indicates the drone video acquisition rate; The target constraints include: ； in, Indicates the amount of video that the drone has not sent back.

3. The method for collecting and transmitting drone videos at multiple time scales according to claim 2 is characterized in that: Determining the UAV video acquisition resolution at the first time scale and the bandwidth resources allocated to the UAV for video backhaul at the second time scale according to the first target optimization model and the target constraint condition includes: According to the first objective optimization model, the objective constraint condition and the Lyapunov optimization algorithm, a second objective optimization model and a third objective optimization model are obtained; wherein the second objective optimization model is used to determine the UAV video acquisition resolution at the first time scale; and the third objective optimization model is used to determine the bandwidth resources allocated to the UAV for video backhaul at the second time scale; According to the second objective optimization model and the third objective optimization model, the UAV video acquisition resolution at the first time scale and the bandwidth resources allocated to the UAV for video backhaul at the second time scale are determined.

4. The method for collecting and transmitting drone videos at multiple time scales according to claim 3 is characterized in that: The second objective optimization model includes: Wherein, k and n are integers greater than 0; T represents the first time scale; V represents the first coefficient; Indicates the amount of video that the drone has not sent back; Indicates the drone video capture resolution.

5. The method for collecting and transmitting drone videos at multiple time scales according to claim 4 is characterized in that: The third objective optimization model includes: ； in, It is represented as the bandwidth allocated to the UAV at the second time scale t; Indicates the amount of video that the drone has not sent back; Indicates the transmission power of the aerial base station; represents a constant; represents the path loss exponent; Indicates drone Distance to the aerial base station; represents the noise power spectral density.

6. A drone video acquisition and return device at multiple time scales, characterized in that: include: A first determination module is used to establish a first target optimization model and target constraint conditions; The first objective optimization model is used to maximize the quality of the video collected by the drone; The target constraint condition is used to constrain the amount of video that is not transmitted back by the drone; The second determination module is used to solve the first target optimization model based on the target constraint condition to obtain the UAV video acquisition resolution at a first time scale and the bandwidth resources allocated to the UAV for video backhaul at a second time scale; the first time scale is greater than the second time scale.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method for collecting and transmitting drone videos at multiple time scales as described in any one of claims 1 to 5 is implemented.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for collecting and transmitting drone videos at multiple time scales as described in any one of claims 1 to 5 is implemented.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for collecting and transmitting drone videos at multiple time scales as described in any one of claims 1 to 5 is implemented.

Images