Real-time feedback method for application data of unmanned aerial vehicle

Abstract

The invention relates to the technical field of data transmission, and particularly discloses an unmanned aerial vehicle application data real-time feedback method which comprises the following steps: performing clock synchronization on a ground base station and an unmanned aerial vehicle based on standard time information provided by a global positioning system as a reference, and collecting network delay data in a preset time period at the same time; and a deep learning technology is introduced to carry out forward and backward time sequence fluctuation analysis on network delay data so as to capture time sequence dynamic characteristics of network delay, so that corresponding time compensation parameters are intelligently estimated, and the sending time of unmanned aerial vehicle application data is adjusted. Therefore, the sending and receiving time of the data is adjusted at the ground station and the two ends of the unmanned aerial vehicle according to the estimated network delay, the network delay can be effectively compensated, and the real-time performance and the accuracy of unmanned aerial vehicle application data feedback are improved.

Description

Technical Field

[0001] The present application relates to the technical field of data transmission, and more specifically, to a method for real-time feedback of drone application data. Background Art

[0002] With the rapid development of UAV (Unmanned Aerial Vehicle) technology and its wide application in various fields, such as agricultural monitoring, environmental monitoring, logistics distribution, film and television shooting, disaster relief, etc., the real-time and accuracy requirements for UAV application data are getting higher and higher. UAVs usually communicate with ground base stations through wireless networks and transmit collected data such as video streams and sensor information back to the ground station for analysis and processing.

[0003] However, in actual applications, due to the physical distance between the ground station and the drone, the efficiency of the network protocol used, hardware performance, the instability of the wireless communication channel, multipath effects, changing weather conditions and other factors, the data transmission between the drone and the ground base station often experiences unpredictable network delays. This network delay will directly affect the real-time feedback of the drone data, and may further affect the quality of decisions made based on the drone data. For example, in situations where precise control is required, the existence of network delays may cause the control instructions to fail to reach the drone in time, resulting in control lags or errors.

[0004] In order to solve the above problems, the existing technology has proposed a variety of clock synchronization methods to ensure the time consistency between the UAV and the ground base station, such as using the high-precision timestamp provided by the Global Positioning System (GPS). However, relying solely on GPS clock synchronization can only ensure that the devices at both ends start timing at the same time, and cannot solve the additional delay generated during data transmission and eliminate the impact of network delay.

[0005] Therefore, an optimized real-time feedback method for UAV application data is expected. Summary of the invention

[0006] In order to solve the above technical problems, the present application is proposed. An embodiment of the present application provides a method for real-time feedback of drone application data, which synchronizes the clocks of ground base stations and drones based on the standard time information provided by the global positioning system as a reference, collects network delay data for a predetermined time period, and introduces deep learning technology to perform forward and backward timing fluctuation analysis on network delay data to capture the timing dynamic characteristics of network delay, thereby intelligently estimating the corresponding time compensation parameters to adjust the sending time of drone application data. In this way, by adjusting the sending and receiving time of data according to the estimated network delay at both the ground station and the drone, the network delay can be effectively compensated, and the real-time and accuracy of drone application data feedback can be improved.

[0007] According to one aspect of the present application, a method for real-time feedback of drone application data is provided, which includes: Defining communication parameters between the UAV and the ground base station, wherein the communication parameters include communication frequency, protocol selection and safety measures; Performing clock synchronization between the ground base station and the drone based on standard time information provided by the global positioning system as a reference; Collect network delay data for a predetermined time period; Calculating a time compensation parameter based on the network delay data of the predetermined time period; Based on the time compensation parameter, the sending time of the application data is adjusted.

[0008] Compared with the prior art, the real-time feedback method for drone application data provided by this application uses the standard time information provided by the global positioning system as a reference to synchronize the clocks of the ground base station and the drone, collects network delay data for a predetermined time period, and introduces deep learning technology to perform forward and backward timing fluctuation analysis on the network delay data to capture the timing dynamic characteristics of the network delay, thereby intelligently estimating the corresponding time compensation parameters to adjust the sending time of the drone application data. In this way, by adjusting the sending and receiving time of the data according to the estimated network delay at both the ground station and the drone, the network delay can be effectively compensated, and the real-time and accuracy of the drone application data feedback can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0010] Figure 1Flow chart of a method for real-time feedback of drone application data according to an embodiment of the present application.

[0011] Figure 2 This is a flowchart of sub-step S4 of the method for real-time feedback of drone application data according to an embodiment of the present application.

[0012] Figure 3 Schematic diagram of data flow of sub-step S4 of the method for real-time feedback of drone application data according to an embodiment of the present application.

[0013] Figure 4 This is a flowchart of sub-step S41 of the real-time feedback method for drone application data according to an embodiment of the present application.

[0014] Figure 5 This is a flowchart of sub-step S42 of the method for real-time feedback of drone application data according to an embodiment of the present application.

[0015] Figure 6 This is a flowchart of sub-step S422 of the real-time feedback method for drone application data according to an embodiment of the present application. DETAILED DESCRIPTION

[0016] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0017] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0018] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. Meanwhile, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.

[0019] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0020] It is worth noting that in this application, all actions to obtain data are carried out in compliance with the relevant data protection laws and policies of the country where the data is located, and with the authorization given by the owner of the corresponding device.

[0021] In response to the technical problems described in the above background technology, this application proposes an optimized real-time feedback method for drone application data, which uses the standard time information provided by the global positioning system as a reference to synchronize the clocks of the ground base station and the drone, and collects network delay data for a predetermined time period. It also introduces deep learning technology to perform forward and backward timing fluctuation analysis on the network delay data to capture the timing dynamic characteristics of the network delay, thereby intelligently estimating the corresponding time compensation parameters to adjust the sending time of the drone application data. In this way, by adjusting the sending and receiving time of the data at both the ground station and the drone according to the estimated network delay, the network delay can be effectively compensated, and the real-time and accuracy of the drone application data feedback can be improved.

[0022] Figure 1 Flow chart of the method for real-time feedback of drone application data according to an embodiment of the present application. Figure 1 As shown, the real-time feedback method for UAV application data includes the following steps: S1, defining the communication parameters between the UAV and the ground base station, wherein the communication parameters include communication frequency, protocol selection and safety measures; S2, synchronizing the clocks of the ground base station and the UAV based on the standard time information provided by the global positioning system as a reference; S3, collecting network delay data for a predetermined time period; S4, calculating time compensation parameters based on the network delay data for the predetermined time period; S5, adjusting the sending time of the application data based on the time compensation parameters.

[0023] In the above-mentioned method for real-time feedback of drone application data, the step S1 defines the communication parameters between the drone and the ground base station, and the communication parameters include communication frequency, protocol selection and security measures. It should be understood that the setting of communication parameters is the basis for ensuring stable, efficient and secure communication between the drone and the ground base station. The communication frequency determines the frequency band range of signal transmission. The appropriate frequency can reduce interference and improve the penetration and transmission distance of the signal. The protocol selection standardizes the format, sequence and control method of data transmission between the two to ensure that the data can be correctly interpreted and processed. Security measures are used to prevent data from being stolen and tampered with, and to ensure the reliability and privacy of communication. By setting appropriate communication parameters, it helps to ensure efficient and secure data transmission between the drone and the ground base station.

[0024] Specifically, first, when selecting a communication frequency band, coverage, interference, regulatory restrictions, and application requirements need to be considered. Lower frequencies (such as 700MHz) provide greater coverage and stronger penetration capabilities but lower data rates; higher frequencies (such as 5GHz) support higher data rates, but their coverage and obstacle penetration capabilities are poor. High frequency bands are susceptible to interference from other wireless devices, especially in urban environments. When selecting frequency bands, try to avoid known high-interference areas or use technologies with strong anti-interference capabilities, and comply with relevant standards set by the International Telecommunication Union (ITU). Based on specific application scenarios, for example, for high-definition video streaming, it may be necessary to select a frequency band that supports higher bandwidth; for simple control command transmission, a lower frequency can be selected to ensure a wider coverage range. Common drone communication frequency bands include the 900MHz band, which is suitable for long-distance low-rate transmission and has good building penetration performance, the 2.4GHz and 5.8GHz bands, which are widely used in Wi-Fi and other short-range wireless networks and can quickly exchange data, and the Sub-6GHz and millimeter wave bands that can provide extremely high data transmission rates and low latency. The latter is very suitable for applications with strict real-time requirements.

[0025] After determining the communication frequency band, the next step is to select a suitable communication protocol. Some protocols provide better error detection and correction mechanisms to ensure the integrity of data transmission, but may introduce additional delays; while other protocols are faster but do not automatically retransmit in the event of packet loss. Modern drones often need to process large amounts of sensor data and high-definition image / video streams, so efficient compression algorithms and transmission formats are essential. With the continuous expansion of drone application scenarios and technological advancement, the selected protocol should have certain scalability and adaptability so that there is no need to massively reconstruct the existing architecture when upgrading the system in the future. According to the different uses of drones, several communication protocols worth considering include: MQTT, a lightweight message queue protocol, is designed for resource-constrained devices and is very suitable for drone data transmission in Internet of Things (IoT) applications; CoAP, which is optimized for resource-constrained environments, is based on the design concept of HTTP and adopts a more compact data representation form, and can be transmitted through UDP; RTSP is suitable for live video or real-time image transmission, which can effectively manage multimedia data streams and support playback control functions; for applications that require ultra-high bandwidth and ultra-low latency, such as unmanned vehicle collaboration or drone cluster operations in smart cities, 5GNR is undoubtedly one of the best choices, which can provide data transmission rates of up to several Gb / s and a latency of only a few milliseconds.

[0026] In order to protect the communication between the drone and the ground base station from being eavesdropped or tampered with, appropriate security measures must be taken. This includes encrypting all transmitted data using symmetric or asymmetric encryption algorithms to ensure that only authorized parties can decrypt and read the content. For example, AES (Advanced Encryption Standard) is a commonly used symmetric encryption method, and RSA is a representative of asymmetric encryption; using public key infrastructure (PKI), adding digital signatures to each message to verify the identity of the sender and the authenticity and integrity of the message. In addition to encryption, a strict access control system needs to be established to limit which entities have the right to communicate with the drone. There are many ways to achieve user authentication, from simple username and password combinations to complex biometrics. It can be used as an option, usually combined with hardware feature codes or other unique identifiers for identity confirmation. Once the user is authenticated, the next step is to assign the corresponding permission level to clarify what operations the user can perform. Finally, considering the continuous evolution of cybersecurity threats, the system should be regularly checked for potential security vulnerabilities and patches should be applied in a timely manner. At the same time, a complete firmware update process should be developed to ensure that all drones and ground stations can obtain the latest security enhancements.

[0027] In the above-mentioned method for real-time feedback of drone application data, the step S2 synchronizes the clocks of the ground base station and the drone based on the standard time information provided by the global positioning system as a reference. It should be understood that clock synchronization is the key to achieving accurate data transmission and real-time feedback. Since the drone and the ground base station may be in different geographical locations and operating environments, their internal clocks may have deviations, which will lead to inconsistencies in data sending and receiving times, thereby affecting data processing and analysis. The Global Positioning System (GPS) can provide high-precision standard time information. By receiving GPS signals, drones and ground base stations can obtain a unified time reference to calibrate their respective clocks so that the two are consistent in time.

[0028] Specifically, GPS satellites carry high-precision atomic clocks and provide extremely accurate timestamps to receivers on Earth through broadcast signals. Each GPS satellite sends a data packet containing its location and current time every second. These timestamps are based on Coordinated Universal Time (UTC), which is a globally unified standard time. The core of clock synchronization using GPS is that both ground base stations and drones can receive timestamps from multiple satellites at the same time, so that they can adjust their internal clocks to match this common time reference.

[0029] For ground base stations, achieving synchronization with GPS time usually involves installing a dedicated GPS receiver module or antenna. These devices can capture signals from multiple GPS satellites and extract the most accurate time information from them. To ensure the best reception, the GPS antenna should be placed in an open, unobstructed location, away from electronic devices that may cause interference. Once the GPS signal is successfully received, the receiver module will interpret the time information and adjust the local clock to be consistent with the GPS time. In addition, some high-end GPS receivers can also provide a pulse per second (PPS) output, which is a physical signal used to indicate the beginning of each second. The PPS signal can be used to further improve the accuracy of clock synchronization, which can provide a more direct time reference than software-parsed timestamps.

[0030] Considering the mobility and aerial operation characteristics of drones, higher requirements are placed on their GPS reception capabilities. Drones need to be equipped with lightweight but high-performance GPS receiving modules to ensure stable reception of GPS signals even in fast flight or complex terrain conditions. Most modern commercial drones have built-in GPS modules, which can be used not only for navigation and positioning, but also as a source of time synchronization. However, in order to achieve the highest level of clock synchronization accuracy, some professional applications may choose to add specially designed high-precision GPS receivers. Such receivers are able to operate over a wider frequency range, enhance anti-interference capabilities, and support signal reception from more satellite systems, such as GLONASS, Galileo, etc., to increase the number of available satellites and improve the reliability of positioning and time synchronization.

[0031] When both the ground base station and the drone have obtained standard time information through their respective GPS receivers, measures need to be taken to ensure that the clocks of the two remain synchronized. This involves two key aspects: one is initial synchronization, that is, adjusting the clocks of both parties to the same time point for the first time; the other is continuous synchronization, which ensures that the clocks do not drift for various reasons during operation. For initial synchronization, adjustments can be made by comparing the latest GPS timestamps received by each. If a difference is found, one of them is used as the master clock source and the other is corrected accordingly. As for continuous synchronization, considering that GPS signals are not always available (for example, indoors or when the signal is blocked), a common practice is to set a fixed time interval after the initial synchronization, during which the local clock is regularly checked and fine-tuned to keep it consistent with the GPS time.

[0032] In order to further improve the accuracy of clock synchronization, the Network Time Protocol (NTP) or other similar technologies can be used to assist GPS synchronization. NTP is a widely used Internet standard protocol that allows computers to exchange time information with each other on the network and automatically adjust their own clocks to maintain consistency. Although NTP itself cannot achieve microsecond or even lower accuracy like GPS, it can be used as a supplementary means to help maintain a more accurate time synchronization state in places where there is no direct GPS signal coverage. Especially for ground base stations deployed inside buildings or other areas where it is difficult to receive GPS signals, the combination of NTP and other time sources can help ground base stations maintain time consistency with external drones.

[0033] In the above-mentioned method for real-time feedback of drone application data, the step S3 collects network delay data for a predetermined time period. It should be understood that the present application takes into account that during the data transmission process, due to the complexity of the network, including factors such as signal transmission distance and network congestion, there will be a certain time delay when data is sent from the drone to the ground base station or vice versa, which will affect the real-time and accuracy of the data. Therefore, the present application continuously monitors and records the network delay data for a predetermined time period in order to timely understand the network delay situation and make targeted optimizations.

[0034] Specifically, it is necessary to first determine the length of the time period to be monitored and the sampling frequency. The choice of time period depends on the specific needs of the application; for example, when performing real-time video streaming or emergency response tasks, high-frequency sampling in a shorter time interval may be required to capture instantaneous changes; while in relatively static application scenarios such as agricultural monitoring, longer time intervals and lower sampling rates are acceptable. In addition, it is necessary to define the type of data of interest, including but not limited to one-way delay (from ground base station to drone or vice versa), round-trip delay (Round Trip Time, RTT), and possible jitter (i.e., delay changes).

[0035] In order to accurately measure latency indicators, specially designed software or hardware tools are usually deployed at both ends of the drone and the ground base station. These tools are responsible for sending probe packets and calculating the delay after receiving the echo reply. The probe packet should be designed to be as small and lightweight as possible to reduce the impact on normal business traffic. At the same time, in order to ensure the validity of the measurement results, the probe packet should contain a unique identifier, a sending timestamp, and other necessary metadata information so that the receiving end can correctly parse and calculate the delay. In some cases, specific protocols such as NTP (for time synchronization) or ICMP (Internet Control Message Protocol, usually used for simple connectivity tests) can also be used to send probe packets.

[0036] When it comes to complex environmental conditions, such as multipath effects, changing weather conditions, and other factors that may cause wireless channels to be unstable, traditional methods based on a single probe package may not be sufficient to provide sufficient accuracy. Therefore, a statistical sampling method can be introduced to send probe packages repeatedly, record the results of each time, and finally take the average value or other statistics as the delay estimate within the time period. This method not only increases the number of samples, thereby improving the reliability of the measurement, but also effectively alleviates the impact of occasional extreme values ​​on the overall evaluation. For particularly sensitive applications, you can even consider using advanced analysis algorithms, such as Kalman filters, to dynamically adjust the prediction model parameters to further improve the accuracy of delay estimation.

[0037] In order to ensure that the collected data is representative, representative test locations and time periods should be selected for sampling. This means not only covering different lighting conditions during the day and at night, but also taking into account the impact of seasonal climate differences. Especially in outdoor working environments, factors such as temperature, humidity, and wind speed will have a significant effect on wireless communications. Therefore, when formulating a test plan, these variables should be fully considered, and multiple experiments should be arranged under different meteorological conditions as much as possible to obtain a more comprehensive data set. In addition, since the nature of the drone's mission determines that it may be in a state of constant movement, it must also be taken into consideration when planning the test path to ensure that a sufficient number of delay samples can be obtained at different geographic coordinates.

[0038] In the above-mentioned method for real-time feedback of drone application data, the step S4 calculates the time compensation parameters based on the network delay data of the predetermined time period. It should be understood that in order to compensate for the impact of network delay on data transmission, the present application further introduces a deep learning algorithm to calculate the time compensation parameters based on the network delay data of the predetermined time period, so as to perform corresponding time compensation for the network delay, thereby effectively reducing the impact of network delay on data real-time performance. Among them, Figure 2 This is a flowchart of sub-step S4 of the method for real-time feedback of drone application data according to an embodiment of the present application. Figure 3 FIG. 4 is a data flow diagram of sub-step S4 of the method for real-time feedback of drone application data according to an embodiment of the present application. Figure 2 and Figure 3As shown, the step S4 includes the steps of: S41, performing bidirectional time series coding on the network delay data of the predetermined time period to obtain a network delay forward time series fluctuation coding vector and a network delay backward time series fluctuation coding vector; S42, performing bidirectional attention interaction response analysis on the network delay forward time series fluctuation coding vector and the network delay backward time series fluctuation coding vector to obtain a network delay time series bidirectional joint coding vector; S43, determining the time compensation parameter based on the network delay time series bidirectional joint coding vector.

[0039] Figure 4 FIG. 4 is a flowchart of sub-step S41 of the method for real-time feedback of drone application data according to an embodiment of the present application. Figure 4 As shown, the step S41 includes the steps of: S411, arranging the network delay data of the predetermined time period in the order of timestamps to obtain a network delay timing fluctuation input vector; S412, inputting the network delay timing fluctuation input vector into a network delay forward timing encoder based on a forward LSTM model to obtain the network delay forward timing fluctuation encoding vector; S413, inputting the network delay timing fluctuation input vector into a network delay backward timing encoder based on a backward LSTM model to obtain the network delay backward timing fluctuation encoding vector.

[0040] More specifically, in step S411, the network delay data of the predetermined time period is arranged in the order of timestamps to obtain a network delay timing fluctuation input vector. It should be understood that since the network delay changes over time, the present application can clearly reflect the changing trend of the network delay by arranging the network delay data of the predetermined time period in the order of timestamps, forming a network delay timing fluctuation input vector, thereby providing a basic structure for subsequent time compensation analysis.

[0041] More specifically, in step S412, the network delay time series fluctuation input vector is input into the network delay forward time series encoder based on the forward LSTM model to obtain the network delay forward time series fluctuation encoding vector. Specifically, in order to achieve effective learning of the time series change law of network delay data, the present application adopts a forward LSTM model that performs well in the field of time series analysis to perform time series encoding on the network delay time series fluctuation input vector, so as to extract the change trend characteristics of the network delay data from the past to the present in time, and generate a network delay forward time series fluctuation encoding vector. Those of ordinary skill in the art should know that the LSTM model (Long Short-Term Memory Network) is a special recurrent neural network that can effectively handle long-term dependency problems in time series data. In the forward encoding process, the memory unit of the LSTM model can remember the important information of the previous time step, and update the memory state according to the current input, and through the iteration of multiple time steps, the dynamic change law of the network delay data on the forward time axis is captured, which provides a key basis for subsequent time compensation.

[0042] More specifically, in step S413, the network delay time series fluctuation input vector is input into the network delay backward time series encoder based on the backward LSTM model to obtain the network delay backward time series fluctuation encoding vector. Specifically, in order to more comprehensively understand the time series change characteristics of the network delay, the present application also uses a backward LSTM model to reverse time series encode the network delay time series fluctuation input vector to extract the change trend characteristics of the network delay data from now to the past in time, and generate a network delay backward time series fluctuation encoding vector. Here, the backward LSTM model also uses the memory unit of the LSTM model to iterate in the reverse order of the time axis, that is, to trace back from the current time step to the past time step to capture the dynamic change law of the network delay data on the reverse time axis. Based on this, through the processing of the forward LSTM model and the backward LSTM model, the network delay change information in different directions can be obtained. The two complement each other and can more comprehensively reflect the dynamic change characteristics of the network delay.

[0043] Specifically, the step S42 performs a bidirectional attention interaction response analysis on the network delay forward timing fluctuation coding vector and the network delay backward timing fluctuation coding vector to obtain a network delay timing bidirectional joint coding vector. It should be understood that since the network delay forward timing fluctuation coding vector and the network delay backward timing fluctuation coding vector respectively contain network delay characteristics in different directions, in order to effectively fuse the two to form a more comprehensive and accurate network delay feature representation, the present application proposes a bidirectional attention interaction response analysis mechanism, which performs bidirectional attention allocation on the two based on the correlation between the network delay forward timing fluctuation coding vector and the network delay backward timing fluctuation coding vector, adaptively focusing on the most important network delay change characteristics in two directions, and realizing enhanced interactive fusion of bidirectional network delay characteristics. Among them, Figure 5 FIG. 4 is a flowchart of sub-step S42 of the method for real-time feedback of drone application data according to an embodiment of the present application. Figure 5 As shown, the step S42 includes the steps of: S421, performing homography projection transformation on the network delay forward timing fluctuation coding vector and the network delay backward timing fluctuation coding vector to obtain the network delay forward timing fluctuation characteristic homography projection coding vector and the network delay backward timing fluctuation characteristic homography projection coding vector; S422, constructing a positive and negative bidirectional attention balance field between the network delay forward timing fluctuation characteristic homography projection coding vector and the network delay backward timing fluctuation characteristic homography projection coding vector to obtain the network delay timing fluctuation positive and negative bidirectional attention balance field; S423, based on the network delay timing fluctuation positive and negative bidirectional attention balance field, performing field mapping interactive analysis on the network delay forward timing fluctuation coding vector and the network delay backward timing fluctuation coding vector to obtain the network delay timing bidirectional joint coding vector.

[0044] More specifically, the step S421 is expressed by the formula: in, represents the network delay forward timing fluctuation encoding vector, represents the network delay backward timing fluctuation encoding vector, and are different homography projection matrices, and They respectively represent the homography projection coding vector of the forward timing fluctuation characteristics of the network delay and the homography projection coding vector of the backward timing fluctuation characteristics of the network delay.

[0045] That is, by performing homography projection transformation on the network delay forward timing fluctuation encoding vector and the network delay backward timing fluctuation encoding vector, the two can be mapped to a shared feature space. In this way, the relative positions and relationships of the two in the public feature space can be more intuitive and easy to compare, thereby providing a more effective basis for interactive analysis.

[0046] Figure 6 FIG. 4 is a flowchart of sub-step S422 of the method for real-time feedback of drone application data according to an embodiment of the present application. Figure 6 As shown, the step S422 includes the steps of: S4221, calculating the forward attention score field of the network delay forward timing fluctuation characteristic homography projection coding vector relative to the network delay backward timing fluctuation characteristic homography projection coding vector to obtain the network delay timing fluctuation forward attention score field; S4222, calculating the reverse attention score field of the network delay backward timing fluctuation characteristic homography projection coding vector relative to the network delay forward timing fluctuation characteristic homography projection coding vector to obtain the network delay timing fluctuation reverse attention score field; S4223, constructing the network delay timing fluctuation positive and negative bidirectional attention balance field based on the network delay timing fluctuation forward attention score field and the network delay timing fluctuation reverse attention score field.

[0047] In a specific example of the present application, the step S4221 includes: calculating the network delay forward timing fluctuation feature homography projection coding vector multiplied by the transposed vector of the network delay backward timing fluctuation feature homography projection coding vector and then dividing by the attention score scaling factor to obtain the network delay timing fluctuation forward attention score field, which is expressed by the formula: in, represents the transpose of a vector, represents the matrix multiplication operation, is the attention score scaling factor, It indicates the positive attention score field of network delay timing fluctuation.

[0048] That is, by calculating the forward attention score field to analyze the network delay forward timing fluctuation encoding vector and the network delay backward timing fluctuation encoding vector after the homography projection transformation, the importance of each part of the network delay forward timing fluctuation feature when interacting with the backward timing fluctuation feature can be evaluated. In this way, the most relevant and critical information fragments of the forward timing fluctuation feature can be identified and mined from the backward timing fluctuation feature, thereby enhancing the response sensitivity of the forward timing fluctuation feature to the backward timing fluctuation feature, making the relationship between the two clearer and helping to extract more valuable interaction information.

[0049] In a specific example of the present application, step S4222 is expressed by the formula: in, Represents the reverse attention score field of network delay timing fluctuation.

[0050] That is, similar to the calculation of the above-mentioned forward attention score field, the present application further performs an analysis of the reverse attention score field to identify the characteristic parts closely related to the forward timing fluctuation of the network delay from the characteristics of the backward timing fluctuation of the network delay.

[0051] In a specific example of the present application, step S4223 is expressed by the formula: in, Indicates cascade, Represents the network delay timing fluctuation positive and negative bidirectional attention balance field Representation 3 3 convolution operations.

[0052] That is, in order to comprehensively consider the forward and reverse attention information, the present application further constructs a positive and negative bidirectional attention balance field of the network delay timing fluctuation to ensure that in the final interactive response encoding process, it will not over-emphasize the characteristics of one party, while not ignoring the contribution of the other party. In this way, the interaction between the forward and backward timing characteristics of the network delay can be handled more balanced, thereby achieving a more harmonious and accurate interactive relationship between the two.

[0053] More specifically, the step S423 includes: first, mapping the network delay forward timing fluctuation feature homography projection coding vector and the network delay backward timing fluctuation feature homography projection coding vector to the network delay timing fluctuation positive and negative bidirectional attention balance field respectively to obtain the network delay forward timing fluctuation homography projection attention modulation coding vector and the network delay backward timing fluctuation homography projection attention modulation coding vector, which are expressed as follows: in, and They respectively represent the network delay forward timing fluctuation homography projection attention modulation coding vector and the network delay backward timing fluctuation homography projection attention modulation coding vector.

[0054] Here, the network delay forward timing fluctuation coding vector and the network delay backward timing fluctuation coding vector that have undergone homography projection transformation are respectively mapped to the network delay timing fluctuation positive and negative bidirectional attention balance field. Through the attention weight distribution of the positive and negative bidirectional attention balance field, the feature representations of the two are adjusted to enhance the feature parts related to important information, while suppressing the feature parts related to minor information, thereby obtaining the network delay forward timing fluctuation homography projection attention modulation coding vector and the network delay backward timing fluctuation homography projection attention modulation coding vector.

[0055] Then, the network delay forward timing fluctuation homography projection attention modulation coding vector and the network delay backward timing fluctuation homography projection attention modulation coding vector are calculated and divided by the position point to obtain the network delay timing bidirectional joint coding vector, which is expressed as: in, represents the network delay timing bidirectional joint coding vector; It should be understood that in the above-mentioned attention balance field projection process, the expression of one party is essentially adjusted based on the reaction of the other party, so that the generated network delay forward time series fluctuation homography projection attention modulation coding vector and the network delay backward time series fluctuation homography projection attention modulation coding vector reflect the synergistic relationship between the forward and backward time series characteristics of the network delay to a certain extent. Finally, by further performing position point division operations, the network delay forward time series fluctuation homography projection attention modulation coding vector and the network delay backward time series fluctuation homography projection attention modulation coding vector are interacted element by element, so as to integrate the characteristics of the two and form a more comprehensive and accurate network delay feature representation, that is, the network delay time series bidirectional joint coding vector. In this way, not only the changing trend of network delay in different directions is extracted, but also the most important time series feature part in the interaction process is effectively highlighted, providing a more reliable basis for subsequent time compensation.

[0056] Specifically, the step S43 determines the time compensation parameter based on the network delay timing bidirectional joint coding vector. In a specific example of the present application, the step S43 includes: inputting the network delay timing bidirectional joint coding vector into a compensation parameter estimation module based on a decoder to obtain the time compensation parameter. It should be understood that a decoder is a model that converts coding features into raw data or target values ​​related to raw data. In the present application, the compensation parameter estimation module uses the structure of the decoder and the trained network parameters to map the network delay timing feature information contained in the network delay timing bidirectional joint coding vector to the time compensation parameter space, thereby obtaining a specific numerical value that can compensate for the network delay, and providing a specific numerical basis for the subsequent adjustment of the drone application data sending time.

[0057] In a specific example of the present application, the network delay timing bidirectional joint coding vector is passed through a compensation parameter estimation module based on a decoder to obtain a time compensation parameter, including: First, the eigenvalues ​​of the network delay time sequence bidirectional joint coding vector are arranged in ascending order to form a network delay time sequence bidirectional joint sequential coding vector; Secondly, in response to the network delay timing, the first bidirectional joint sequence encoding vector The eigenvalue and The absolute value of the difference between the eigenvalues ​​is less than or equal to the distance difference hyperparameter , calculate the The eigenvalues ​​are related to the The weighted sum of the eigenvalues ​​is the optimized eigenvalues, and calculate the square root of the sum of the squares of all eigenvalues ​​of the bidirectional joint coding vector of the network delay time series, which is expressed as follows: in, represents the first bidirectional joint coding vector of the network delay timing eigenvalues, represents the length of the bidirectional joint coding vector of the network delay timing, The square root of the sum of squares of all eigenvalues ​​of the bidirectional joint coding vector representing the network delay timing; Then, Multiply by 2 and then divide by the square of the length of the network delay timing bidirectional joint coding vector to obtain the network delay timing bidirectional joint spatial primitive value, which is expressed as: in, Represents the network delay timing bidirectional joint space primitive value; Next, in response to the network delay timing, the first bidirectional joint sequence encoding vector The eigenvalue and The absolute value of the difference between the eigenvalues ​​is greater than the distance difference hyperparameter , multiply the network delay timing bidirectional joint space primitive value by the first After the eigenvalues, calculate the product and the The weighted reduction between the eigenvalues ​​is the optimized eigenvalues; Then, on the basis of keeping the first eigenvalue of the bidirectional joint sequential coding vector of the network delay time sequence unchanged, the optimized eigenvalues ​​to obtain the optimized network delay timing bidirectional joint coding vector; Finally, the optimized network delay timing bidirectional joint coding vector is passed through a decoder-based compensation parameter estimation module to obtain time compensation parameters.

[0058] Here, when the network delay forward timing fluctuation coding vector and the network delay backward timing fluctuation coding vector represent the timing correlation characteristics of the network delay data along the forward time dimension and the reverse time dimension respectively, when performing interactive responses based on feature positive and negative attention fields, the complexity of the positive and negative attention field mechanisms encoded in different time dimensions will lead to insufficient representation of the long-distance interactive response of the network delay timing bidirectional joint coding vector, thereby reducing the expression effect of the network delay timing bidirectional joint coding vector and affecting the accuracy of the time compensation parameters obtained through the decoder-based compensation parameter estimation module.

[0059] Therefore, in order to address the problem that the feature set of the network delay time series bidirectional joint coding vector has insufficient global interactive response representation capability due to the long distance exceeding the predetermined local distribution interval threshold under the predetermined eigenvalue sequential distribution, the high-dimensional feature space primitive representation of the network delay time series bidirectional joint coding vector based on self-inner product fusion is used to capture the complex structure of the global network interaction of its eigenvalues, thereby reconstructing the interactive response relationship between the eigenvalues ​​of the network delay time series bidirectional joint coding vector by simulating the scale-based high-dimensional feature space potential primitives, so as to realize the coding reconstruction of the real sequence distribution behavior of the network delay time series bidirectional joint coding vector under long distance, improve the interactive response expression effect of the network delay time series bidirectional joint coding vector, and improve the accuracy of the time compensation parameters obtained by the decoder-based compensation parameter estimation module.

[0060] In the above-mentioned real-time feedback method for drone application data, the step S5 adjusts the sending time of the application data based on the time compensation parameter. For example, when the time compensation parameter is 500 milliseconds, the data is sent 500 milliseconds in advance based on the original planned time for sending the data. At the same time, at the receiving end of the ground base station, the time window for receiving data is also adjusted according to the corresponding time compensation parameter to ensure that the adjusted data can be accurately received. In this way, network delays can be effectively compensated, the real-time and accuracy of drone application data can be improved, and the ground base station can obtain the latest data collected by the drone in a timely manner, providing strong support for subsequent decision-making and analysis.

[0061] In summary, the real-time feedback method of drone application data based on the embodiment of the present application is explained, which uses the standard time information provided by the global positioning system as a reference to synchronize the clocks of the ground base station and the drone, collects network delay data for a predetermined time period, and introduces deep learning technology to perform forward and backward timing fluctuation analysis on the network delay data to capture the timing dynamic characteristics of the network delay, thereby intelligently estimating the corresponding time compensation parameters to adjust the sending time of the drone application data. In this way, by adjusting the sending and receiving time of the data according to the estimated network delay at both the ground station and the drone, the network delay can be effectively compensated, and the real-time and accuracy of the drone application data feedback can be improved.

[0062] The basic principle of the present invention is described above in conjunction with specific embodiments. However, it should be pointed out that the advantages, strengths, effects, etc. mentioned in the present invention are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. must be possessed by each embodiment of the present invention. In addition, the specific details of the above embodiments are only for the purpose of illustration and facilitation of understanding, rather than limitation, and the above details do not limit the present invention to being implemented by adopting the above specific details.

[0063] In the above embodiments, the description of each embodiment has its own emphasis. For the parts that are not described or recorded in detail in a certain embodiment, please refer to the relevant description of other embodiments. In the several embodiments provided by the present invention, it should be understood that the disclosed system and method can be implemented in other ways. For example, the system embodiment described above is only schematic. For example, the unit division is only a logical function division, and there may be other division methods in actual implementation. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0064] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference to a figure in a claim should not be considered as limiting the claim to which it relates.

[0065] In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. The multiple units stated in the system claims can also be implemented by one unit through software or hardware.

[0066] Finally, it should be noted that the above description has been given for the purpose of illustration and description. In addition, the above embodiments are only used to illustrate the technical solution of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for real-time feedback of drone application data, characterized in that: include: Defining communication parameters between the UAV and the ground base station, wherein the communication parameters include communication frequency, protocol selection and safety measures; Performing clock synchronization between the ground base station and the drone based on standard time information provided by the global positioning system as a reference; Collect network delay data for a predetermined period of time; Calculating a time compensation parameter based on the network delay data of the predetermined time period; Based on the time compensation parameter, the sending time of the application data is adjusted.

2. The method for real-time feedback of drone application data according to claim 1, characterized in that: Calculating a time compensation parameter based on the network delay data of the predetermined time period includes: Performing bidirectional time series coding on the network delay data of the predetermined time period to obtain a network delay forward time series fluctuation coding vector and a network delay backward time series fluctuation coding vector; Performing a bidirectional attention interaction response analysis on the network delay forward timing fluctuation coding vector and the network delay backward timing fluctuation coding vector to obtain a network delay timing bidirectional joint coding vector; The time compensation parameter is determined based on the network delay timing bidirectional joint coding vector.

3. The method for real-time feedback of drone application data according to claim 2, characterized in that: Bidirectional time series coding is performed on the network delay data of the predetermined time period to obtain a network delay forward time series fluctuation coding vector and a network delay backward time series fluctuation coding vector, including: Arranging the network delay data of the predetermined time period in the order of timestamps to obtain a network delay timing fluctuation input vector; Inputting the network delay timing fluctuation input vector into a network delay forward timing encoder based on a forward LSTM model to obtain the network delay forward timing fluctuation encoding vector; The network delay timing fluctuation input vector is input into the network delay backward timing encoder based on the backward LSTM model to obtain the network delay backward timing fluctuation encoding vector.

4. The method for real-time feedback of drone application data according to claim 3, characterized in that: Performing a bidirectional attention interaction response analysis on the network delay forward timing fluctuation coding vector and the network delay backward timing fluctuation coding vector to obtain a network delay timing bidirectional joint coding vector, including: Performing homography projection transformation on the network delay forward timing fluctuation coding vector and the network delay backward timing fluctuation coding vector to obtain a network delay forward timing fluctuation feature homography projection coding vector and a network delay backward timing fluctuation feature homography projection coding vector; Constructing a positive and negative bidirectional attention balance field between the homography projection coding vector of the network delay forward time series fluctuation feature and the homography projection coding vector of the network delay backward time series fluctuation feature to obtain a positive and negative bidirectional attention balance field of the network delay time series fluctuation; Based on the positive and negative bidirectional attention balance field of the network delay timing fluctuation, the network delay forward timing fluctuation encoding vector and the network delay backward timing fluctuation encoding vector are interactively analyzed by field mapping to obtain the network delay timing bidirectional joint encoding vector.

5. The method for real-time feedback of drone application data according to claim 4, characterized in that: Constructing a positive and negative bidirectional attention balance field between the homography projection coding vector of the network delay forward time series fluctuation feature and the homography projection coding vector of the network delay backward time series fluctuation feature to obtain a positive and negative bidirectional attention balance field of the network delay time series fluctuation, including: Calculate the forward attention score field of the homography projection coding vector of the network delay forward timing fluctuation feature relative to the homography projection coding vector of the network delay backward timing fluctuation feature to obtain the network delay timing fluctuation forward attention score field; Calculate the reverse attention score field of the homography projection coding vector of the network delay backward timing fluctuation feature relative to the homography projection coding vector of the network delay forward timing fluctuation feature to obtain the network delay timing fluctuation reverse attention score field; Based on the network delay timing fluctuation positive attention score field and the network delay timing fluctuation reverse attention score field, the network delay timing fluctuation positive and negative bidirectional attention balance field is constructed.

6. The method for real-time feedback of drone application data according to claim 5, characterized in that: Calculating the forward attention score field of the homography projection coding vector of the network delay forward timing fluctuation feature relative to the homography projection coding vector of the network delay backward timing fluctuation feature to obtain the network delay timing fluctuation forward attention score field, including: Calculate the network delay forward timing fluctuation feature homography projection coding vector multiplied by the transposed vector of the network delay backward timing fluctuation feature homography projection coding vector and then divide by the attention score scaling factor to obtain the network delay timing fluctuation forward attention score field.

7. The method for real-time feedback of drone application data according to claim 6, characterized in that: Based on the network delay timing fluctuation positive and negative bidirectional attention balance field, the network delay forward timing fluctuation coding vector and the network delay backward timing fluctuation coding vector are subjected to field mapping interactive analysis to obtain the network delay timing bidirectional joint coding vector, including: Mapping the network delay forward timing fluctuation feature homography projection coding vector and the network delay backward timing fluctuation feature homography projection coding vector to the network delay timing fluctuation positive and negative bidirectional attention balance field respectively to obtain the network delay forward timing fluctuation homography projection attention modulation coding vector and the network delay backward timing fluctuation homography projection attention modulation coding vector; The network delay timing bidirectional joint coding vector is obtained by calculating the network delay forward timing fluctuation homography projection attention modulation coding vector and the network delay backward timing fluctuation homography projection attention modulation coding vector by dividing them by the position points.

8. The method for real-time feedback of drone application data according to claim 7, characterized in that: Determining the time compensation parameter based on the network delay timing bidirectional joint coding vector includes: The network delay timing bidirectional joint coding vector is input into a decoder-based compensation parameter estimation module to obtain the time compensation parameter.

Images