A method for real-time feedback of UAV application data

Abstract

The present application relates to the technical field of data transmission, and specifically discloses a method for real-time feedback of UAV application data. It synchronizes the clocks of the ground base station and the UAV based on the standard time information provided by the Global Positioning System as a reference. At the same time, it collects network delay data for a predetermined period of time, and introduces deep learning technology to perform forward and backward time series fluctuation analysis on the network delay data to capture the time series dynamic characteristics of the network delay, so as to intelligently estimate the corresponding time compensation parameters and adjust the transmission time of UAV application data accordingly. In this way, by adjusting the data transmission and reception times at both ends of the ground station and the UAV according to the estimated network delay, the network delay can be effectively compensated, and the real-time performance and accuracy of UAV application data feedback can be improved.

Description

Technical Field

[0001] This application relates to the technical field of data transmission, and more specifically, to a method for real-time feedback of UAV application data. Background Art

[0002] With the rapid development of Unmanned Aerial Vehicle (UAV) technology and its wide application in various fields, such as agricultural monitoring, environmental monitoring, logistics distribution, film shooting, disaster relief, etc., the requirements for the real-time and accuracy of UAV application data are getting higher and higher. UAVs usually communicate with ground base stations through wireless networks, and transmit the collected data such as video streams, sensor information, etc. back to the ground station for analysis and processing.

[0003] However, in practical applications, due to factors such as the physical distance between the ground station and the UAV, the efficiency of the network protocol used, hardware performance, instability of the wireless communication channel, multipath effect, and weather condition changes, the data transmission between the UAV and the ground base station often experiences unpredictable network delays. Such network delays will directly affect the real-time feedback of UAV data, and may further affect the quality of decisions made based on UAV data. For example, in situations that require precise control, the existence of network delays may cause control commands to fail to reach the UAV in time, resulting in lag or error in control.

[0004] To solve the above problems, various clock synchronization methods have been proposed in the prior art to ensure time consistency between the UAV and the ground base station, such as using the high-precision timestamps provided by the Global Positioning System (GPS). However, relying solely on GPS clock synchronization can only ensure that the two devices start timing at the same moment, and cannot solve the additional delays generated during data transmission and eliminate the impact of network delays.

[0005] Therefore, an optimized method for real-time feedback of UAV application data is expected. Summary of the Invention

[0006] To solve the above technical problems, the present application is proposed. Embodiments of the present application provide a method for real-time feedback of UAV application data, which synchronizes the clocks of the ground base station and the UAV based on the standard time information provided by the Global Positioning System as a reference, collects network delay data for a predetermined period of time, and introduces deep learning technology to perform forward and backward time series fluctuation analysis on the network delay data to capture the time series dynamic characteristics of the network delay, so as to intelligently estimate the corresponding time compensation parameters and adjust the transmission time of UAV application data accordingly. In this way, by adjusting the data transmission and reception times at both ends of the ground station and the UAV according to the estimated network delay, the network delay can be effectively compensated, and the real-time performance and accuracy of UAV application data feedback can be improved.

[0007] According to one aspect of the present application, there is provided a method for real-time feedback of UAV application data, which includes:

[0008] Define the communication parameters between the UAV and the ground base station, where the communication parameters include communication frequency, protocol selection, and security measures;

[0009] Synchronize the clocks of the ground base station and the UAV based on the standard time information provided by the Global Positioning System as a reference;

[0010] Collect network delay data for a predetermined period of time;

[0011] Calculate the time compensation parameter based on the network delay data for the predetermined period of time;

[0012] Adjust the transmission time of the application data based on the time compensation parameter.

[0013] Compared with the prior art, the method for real-time feedback of UAV application data provided by the present application synchronizes the clocks of the ground base station and the UAV based on the standard time information provided by the Global Positioning System as a reference, collects network delay data for a predetermined period of time, and introduces deep learning technology to perform forward and backward time series fluctuation analysis on the network delay data to capture the time series dynamic characteristics of the network delay, so as to intelligently estimate the corresponding time compensation parameters and adjust the transmission time of UAV application data accordingly. In this way, by adjusting the data transmission and reception times at both ends of the ground station and the UAV according to the estimated network delay, the network delay can be effectively compensated, and the real-time performance and accuracy of UAV application data feedback can be improved. Description of the Drawings

[0014] The above and other objects, features, and advantages of the present application will become more apparent by describing the embodiments of the present application in more detail with reference to the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation to the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0015] Figure 1 It is a flowchart of a method for real-time feedback of drone application data according to an embodiment of the present application.

[0016] Figure 2 It is a flowchart of sub-step S4 of the method for real-time feedback of drone application data according to an embodiment of the present application.

[0017] Figure 3 It is a schematic diagram of data flow of sub-step S4 of the method for real-time feedback of drone application data according to an embodiment of the present application.

[0018] Figure 4 It is a flowchart of sub-step S41 of the method for real-time feedback of drone application data according to an embodiment of the present application.

[0019] Figure 5 It is a flowchart of sub-step S42 of the method for real-time feedback of drone application data according to an embodiment of the present application.

[0020] Figure 6 It is a flowchart of sub-step S422 of the method for real-time feedback of drone application data according to an embodiment of the present application. Detailed implementation

[0021] As shown in the present application and the claims, unless the context clearly indicates otherwise, words such as "a", "an", "one", and / or "the" are not specifically singular and may also include plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of the clearly identified steps and elements, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0022] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, however, any number of different modules can be used and run on the user terminal and / or the server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0023] In this application, flowcharts are used to illustrate the operations performed by the system according to the embodiments of this application. It should be understood that the operations before or below do not necessarily need to be executed precisely in order. Instead, various steps can be processed in reverse order or simultaneously as needed. At the same time, other operations can also be added to these processes, or one or several operations can be removed from these processes.

[0024] Next, exemplary embodiments according to this application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all the embodiments of this application. It should be understood that this application is not limited by the exemplary embodiments described herein.

[0025] It is worth noting that in this application, all actions of obtaining data are carried out on the premise of complying with the corresponding data protection regulations and policies of the country where it is located and obtaining authorization from the corresponding device owner.

[0026] In response to the technical problems described in the above background art, this application proposes an optimized method for real-time feedback of drone application data. It synchronizes the clocks of the ground base station and the drone based on the standard time information provided by the Global Positioning System as a reference. At the same time, it collects network delay data for a predetermined period, and introduces deep learning technology to perform forward and backward time series fluctuation analysis on the network delay data to capture the time series dynamic characteristics of the network delay, so as to intelligently estimate the corresponding time compensation parameters and adjust the transmission time of drone application data. In this way, by adjusting the data transmission and reception times at both the ground station and the drone according to the estimated network delay, the network delay can be effectively compensated, and the real-time performance and accuracy of the drone application data feedback can be improved.

[0027] Figure 1 It is a flowchart of the method for real-time feedback of drone application data according to the embodiments of this application. As Figure 1 shown, the method for real-time feedback of drone application data includes the steps: S1, defining communication parameters between the drone and the ground base station, where the communication parameters include communication frequency, protocol selection, and security measures; S2, synchronizing the clocks of the ground base station and the drone based on the standard time information provided by the Global Positioning System as a reference; S3, collecting network delay data for a predetermined period; S4, calculating time compensation parameters based on the network delay data for the predetermined period; S5, adjusting the transmission time of application data based on the time compensation parameters.

[0028] In the above-mentioned method for real-time feedback of drone application data, the step S1 defines the communication parameters between the drone and the ground base station, and the communication parameters include communication frequency, protocol selection and security measures. It should be understood that the setting of communication parameters is the basis for ensuring stable, efficient and secure communication between the drone and the ground base station. The communication frequency determines the frequency band range of signal transmission. The appropriate frequency can reduce interference and improve the penetration and transmission distance of the signal. The protocol selection standardizes the format, sequence and control method of data transmission between the two to ensure that the data can be correctly interpreted and processed. Security measures are used to prevent data from being stolen and tampered with, and to ensure the reliability and privacy of communication. By setting appropriate communication parameters, it helps to ensure efficient and secure data transmission between the drone and the ground base station.

[0029] Specifically, first, when selecting a communication frequency band, coverage, interference, regulatory restrictions, and application requirements need to be considered. Lower frequencies (such as 700MHz) provide greater coverage and stronger penetration capabilities but lower data rates; higher frequencies (such as 5GHz) support higher data rates, but their coverage and obstacle penetration capabilities are poor. High frequency bands are susceptible to interference from other wireless devices, especially in urban environments. When selecting frequency bands, try to avoid known high-interference areas or use technologies with strong anti-interference capabilities, and comply with relevant standards set by the International Telecommunication Union (ITU). Based on specific application scenarios, for example, for high-definition video streaming, it may be necessary to select a frequency band that supports higher bandwidth; for simple control command transmission, a lower frequency can be selected to ensure a wider coverage range. Common drone communication frequency bands include the 900MHz band, which is suitable for long-distance low-rate transmission and has good building penetration performance, the 2.4GHz and 5.8GHz bands, which are widely used in Wi-Fi and other short-range wireless networks and can quickly exchange data, and the Sub-6GHz and millimeter wave bands that can provide extremely high data transmission rates and low latency. The latter is very suitable for applications with strict real-time requirements.

[0030] After determining the communication frequency band, the next step is to select an appropriate communication protocol. Some protocols provide better error detection and correction mechanisms, ensuring the integrity of data transmission, but may introduce additional latency; while other protocols are faster, but do not automatically retransmit in case of lost packets. Modern drones often need to process a large amount of sensor data and high-definition image / video streams, so efficient compression algorithms and transmission formats are crucial. With the continuous expansion of drone application scenarios and the progress of technology, the selected protocol should have a certain degree of scalability and adaptability, so that large-scale reconstruction of the existing architecture is not required when upgrading the system in the future. For different uses of drones, several communication protocols worth considering include: the lightweight message queue protocol MQTT, designed for resource-constrained devices, which is very suitable for drone data transmission in Internet of Things (IoT) applications; CoAP, optimized for resource-constrained environments, based on the design concept of HTTP, using a more compact data representation form, and can be transmitted via UDP; RTSP is suitable for video live streaming or real-time video transmission, can effectively manage multimedia data streams, and supports play control functions; for applications that require ultra-high bandwidth and ultra-low latency, such as collaborative work of driverless vehicles or drone cluster operations in smart cities, 5GNR is undoubtedly one of the best choices, which can provide data transmission rates of up to several Gb / s, and the latency is only in the order of a few milliseconds.

[0031] To protect the communication between the drone and the ground base station from being eavesdropped or tampered with, appropriate security measures must be taken. This includes encrypting all transmitted data using symmetric or asymmetric encryption algorithms to ensure that only authorized parties can decrypt and read the content. For example, AES (Advanced Encryption Standard) is a commonly used symmetric encryption method, and RSA is a representative of asymmetric encryption; using public key infrastructure (PKI) to add digital signatures to each message to verify the identity of the sender and the authenticity and integrity of the message. In addition to encryption, a strict access control system needs to be established to limit which entities are authorized to communicate with the drone. There are many methods to implement user authentication, from simple username-password combinations to complex biometric recognition can be options, and usually, hardware feature codes or other unique identifiers are combined for identity confirmation. Once the user passes the authentication, the corresponding permission level needs to be assigned to clarify what operations the user can perform. Finally, considering the continuous evolution of network security threats, the system should be regularly checked for potential security vulnerabilities, and patch programs should be applied in a timely manner. At the same time, a complete firmware update process should be developed to ensure that all drones and ground stations can obtain the latest security enhancement features.

[0032] In the above method for real-time feedback of UAV application data, in step S2, the ground base station and the UAV are clock-synchronized based on the standard time information provided by the Global Positioning System as a reference. It should be understood that clock synchronization is the key to achieving accurate data transmission and real-time feedback. Since the UAV and the ground base station may be in different geographical locations and operating environments, there may be deviations in their internal clocks, which will lead to inconsistencies in data sending and receiving times, and further affect data processing and analysis. The Global Positioning System (GPS) can provide high-precision standard time information. By receiving GPS signals, the UAV and the ground base station can obtain a unified time reference to calibrate their respective clocks and make them consistent in time.

[0033] Specifically, GPS satellites carry high-precision atomic clocks and provide extremely accurate timestamps to receivers on Earth through broadcast signals. Each GPS satellite sends a data packet containing its position and the current time every second. These timestamps are based on Coordinated Universal Time (UTC), which is a globally unified standard time. The core of using GPS for clock synchronization is that both the ground base station and the UAV can receive timestamps from multiple satellites at the same moment, so that they can adjust their internal clocks to match this common time reference.

[0034] For the ground base station, achieving synchronization with GPS time usually involves installing a dedicated GPS receiving module or antenna. These devices can capture signals from multiple GPS satellites and extract the most accurate time information from them. To ensure the best reception effect, the GPS antenna should be placed in an open and unobstructed location, away from electronic devices that may cause interference. Once the GPS signal is successfully received, the receiving module will parse the time information therein and adjust the local clock to be consistent with the GPS time. In addition, some high-end GPS receiving devices can also provide Pulse Per Second (PPS) output, which is a physical signal used to indicate the start of each second. The PPS signal can be used to further improve the accuracy of clock synchronization, and it can provide a more direct time reference than software parsing timestamps.

[0035] Given the mobility of drones and the characteristics of aerial operations, higher requirements are imposed on their GPS reception capabilities. Drones need to be equipped with lightweight yet high-performance GPS receiving modules to ensure stable reception of GPS signals even during fast flight or under complex terrain conditions. Most modern commercial drones are built with GPS modules, which can be used not only for navigation and positioning but also as a source of time synchronization. However, for the highest level of clock synchronization accuracy, some professional applications may choose to additionally install specially designed high-precision GPS receivers. Such receivers can operate within a wider frequency range, enhancing anti-interference capabilities, while supporting signal reception from more satellite systems, such as GLONASS, Galileo, etc., to increase the number of available satellites and improve the reliability of positioning and time synchronization.

[0036] Once the ground base station and the drone have obtained standard time information through their respective GPS receiving devices, measures need to be taken to ensure that their clocks remain synchronized. This involves two key aspects: one is initial synchronization, that is, initially adjusting the clocks of both parties to the same time point; the other is continuous synchronization, ensuring that the clocks do not drift due to various reasons during operation. For initial synchronization, it can be adjusted by comparing the latest GPS timestamps received by each. If a difference is found, one of them is used as the master clock source and the other is corrected accordingly. As for continuous synchronization, considering that GPS signals are not always available (for example, indoors or when the signal is blocked), a common practice is to set a fixed time interval after the initial synchronization. During this period, the local clock is regularly checked and fine-tuned to keep it consistent with the GPS time.

[0037] To further improve the accuracy of clock synchronization, the Network Time Protocol (NTP) or other similar technologies can also be used to assist GPS in synchronization. NTP is a widely used Internet standard protocol that allows computers to exchange time information with each other over the network and automatically adjust their own clocks to be consistent. Although NTP itself cannot achieve the microsecond-level or even lower accuracy like GPS, it can be used as a supplementary means in areas without direct GPS signal coverage to help maintain a relatively accurate time synchronization state. Especially for ground base stations deployed inside buildings or other areas where it is difficult to receive GPS signals, combining NTP with other time sources can help the ground base station maintain time consistency with external drones.

[0038] In the above method for real-time feedback of UAV application data, in step S3, network delay data for a predetermined time period is collected. It should be understood that in this application, considering the complexity of the network during data transmission, including factors such as signal transmission distance and network congestion level, there will be a certain time delay when data is sent from the UAV to the ground base station or vice versa, which will affect the timeliness and accuracy of the data. Therefore, this application continuously monitors and records the network delay data for a predetermined time period to facilitate timely understanding of the network delay situation and make targeted optimizations.

[0039] Specifically, first, it is necessary to determine the length of the time period to be monitored and the sampling frequency. The selection of the time period depends on the specific requirements of the application; for example, in real-time video stream transmission or emergency response tasks, high-frequency sampling at shorter time intervals may be required to capture instantaneous changes; while in relatively static application scenarios such as agricultural monitoring, longer time intervals and lower sampling rates can be accepted. In addition, the types of data to be concerned about also need to be defined, including but not limited to one-way delay (from the ground base station to the UAV or vice versa), round-trip delay (Round Trip Time, RTT), and possible jitter (i.e., the change in delay).

[0040] To accurately measure the delay metrics, specially designed software or hardware tools are usually deployed at both ends of the UAV and the ground base station. These tools are responsible for sending probe packets and calculating the delay after receiving the echo reply. The probe packets should be designed to be as small and lightweight as possible to reduce the impact on normal traffic. At the same time, to ensure the validity of the measurement results, the probe packets should contain unique identifiers, send timestamps, and other necessary metadata information so that the receiving end can correctly parse and calculate the delay. In some cases, specific protocols such as NTP (for time synchronization) or ICMP (Internet Control Message Protocol, usually used for simple connectivity tests) can also be used to send probe packets.

[0041] When it comes to complex environmental conditions, such as multipath effects and weather condition changes that may cause the wireless channel to be unstable, traditional methods based on a single probe packet may not be sufficient to provide sufficient accuracy. Therefore, statistical sampling methods can be introduced. By repeatedly sending probe packets multiple times and recording the results each time, the average value or other statistics are finally taken as the delay estimate for this time period. This method can not only increase the number of samples, thereby improving the reliability of the measurement, but also effectively alleviate the impact of occasional extreme values on the overall evaluation. For particularly sensitive applications, even advanced analysis algorithms such as the Kalman filter can be considered to dynamically adjust the prediction model parameters to further improve the accuracy of the delay estimation.

[0042] To ensure that the collected data is representative, representative test sites and periods should be selected for sampling. This means not only covering different lighting conditions during the day and night, but also considering the impact brought by seasonal climate differences. Especially in the outdoor operation environment, factors such as temperature, humidity, and wind speed will have a significant impact on wireless communication. Therefore, when formulating the test plan, these variables should be fully considered, and multiple experiments should be arranged under different meteorological conditions as much as possible to obtain a more comprehensive data set. In addition, due to the nature of the drone's mission, it may be in a constantly moving state, so the test path planning also needs to be taken into account to ensure that sufficient rich delay samples can be obtained at different geographical coordinate points.

[0043] In the above method for real-time feedback of drone application data, in step S4, based on the network delay data of the predetermined time period, a time compensation parameter is calculated. It should be understood that in order to compensate for the impact of network delay on data transmission, the present application further introduces a deep learning algorithm to calculate the time compensation parameter based on the network delay data of the predetermined time period, so as to perform corresponding time compensation for the network delay situation, thereby effectively reducing the impact of network delay on data real-time performance. Among them, Figure 2 FIG. is a flowchart of sub-step S4 of the method for real-time feedback of drone application data according to an embodiment of the present application. Figure 3 FIG. is a schematic diagram of data flow of sub-step S4 of the method for real-time feedback of drone application data according to an embodiment of the present application. As Figure 2 and Figure 3 shown, step S4 includes the steps of: S41, performing bidirectional time series encoding on the network delay data of the predetermined time period to obtain a network delay forward time series fluctuation encoding vector and a network delay backward time series fluctuation encoding vector; S42, performing bidirectional attention interaction response analysis on the network delay forward time series fluctuation encoding vector and the network delay backward time series fluctuation encoding vector to obtain a network delay time series bidirectional joint encoding vector; S43, based on the network delay time series bidirectional joint encoding vector, determining the time compensation parameter.

[0044] Figure 4 FIG. is a flowchart of sub-step S41 of the method for real-time feedback of drone application data according to an embodiment of the present application. As Figure 4As shown, the step S41 includes steps: S411, arranging the network delay data of the predetermined time period in the order of time stamps to obtain a network delay time series fluctuation input vector; S412, inputting the network delay time series fluctuation input vector into a network delay forward time series encoder based on a forward LSTM model to obtain the network delay forward time series fluctuation coding vector; S413, inputting the network delay time series fluctuation input vector into a network delay backward time series encoder based on a backward LSTM model to obtain the network delay backward time series fluctuation coding vector.

[0045] More specifically, in the step S411, the network delay data of the predetermined time period is arranged in the order of time stamps to obtain a network delay time series fluctuation input vector. It should be understood that since the network delay changes continuously over time, in this application, by arranging the network delay data of the predetermined time period in the order of time stamps, the change trend of the network delay can be clearly reflected, forming a network delay time series fluctuation input vector, thereby providing a basic structure for subsequent time compensation analysis.

[0046] More specifically, in the step S412, the network delay time series fluctuation input vector is input into a network delay forward time series encoder based on a forward LSTM model to obtain the network delay forward time series fluctuation coding vector. Specifically, in order to effectively learn the time series change law of the network delay data, this application uses a forward LSTM model, which performs excellently in the field of time series analysis, to perform time series coding on the network delay time series fluctuation input vector, so as to extract the change trend characteristics of the network delay data from the past to the present in time, and generate a network delay forward time series fluctuation coding vector. Those of ordinary skill in the art should know that the LSTM model (long short-term memory network) is a special recurrent neural network, which can effectively handle the long-term dependence problem in time series data. During the forward coding process, the memory unit of the LSTM model can remember the important information of the previous time step and update the memory state according to the current input. Through iterations of multiple time steps, the dynamic change law of the network delay data on the positive time axis can be captured, providing a key basis for subsequent time compensation.

[0047] More specifically, in step S413, the network delay time series fluctuation input vector is input into a network delay backward time series encoder based on a backward LSTM model to obtain the network delay backward time series fluctuation coding vector. Specifically, in order to more comprehensively understand the time series change characteristics of network delay, the present application also uses a backward LSTM model to perform backward time series coding on the network delay time series fluctuation input vector, so as to extract the change trend characteristics of network delay data from the present to the past in time, and generate a network delay backward time series fluctuation coding vector. Here, the backward LSTM model also uses the memory unit of the LSTM model and iterates in reverse order along the time axis, that is, tracing from the current time step to the past time step, so as to capture the dynamic change law of network delay data on the reverse time axis. Based on this, through the processing of the forward LSTM model and the backward LSTM model, the network delay change information in different directions can be obtained, and the two complement each other, which can more comprehensively reflect the dynamic change characteristics of network delay.

[0048] Specifically, in step S42, a two-way attention interaction response analysis is performed on the network delay forward time series fluctuation coding vector and the network delay backward time series fluctuation coding vector to obtain a network delay time series two-way joint coding vector. It should be understood that since the network delay forward time series fluctuation coding vector and the network delay backward time series fluctuation coding vector respectively contain network delay characteristics in different directions, in order to effectively fuse the two to form a more comprehensive and accurate network delay feature representation, the present application proposes a two-way attention interaction response analysis mechanism. Based on the correlation between the network delay forward time series fluctuation coding vector and the network delay backward time series fluctuation coding vector, two-way attention allocation is performed on the two, adaptively focusing on the most important network delay change characteristics in both directions, and realizing the enhanced interaction fusion of two-way network delay features. Among them, Figure 5 is a flowchart of sub-step S42 of the real-time feedback method for UAV application data according to an embodiment of the present application. As Figure 5As shown, step S42 includes steps: S421, performing a homography projection transformation on the network delay forward temporal fluctuation coding vector and the network delay backward temporal fluctuation coding vector to obtain a network delay forward temporal fluctuation feature homography projection coding vector and a network delay backward temporal fluctuation feature homography projection coding vector; S422, constructing a forward and backward bidirectional attention balance field between the network delay forward temporal fluctuation feature homography projection coding vector and the network delay backward temporal fluctuation feature homography projection coding vector to obtain a network delay temporal fluctuation forward and backward bidirectional attention balance field; S423, based on the network delay temporal fluctuation forward and backward bidirectional attention balance field, performing a field mapping interaction analysis on the network delay forward temporal fluctuation coding vector and the network delay backward temporal fluctuation coding vector to obtain the network delay temporal bidirectional joint coding vector.

[0049] More specifically, step S421 is expressed by the formula:

[0050]

[0051]

[0052] Wherein, represents the network delay forward temporal fluctuation coding vector, represents the network delay backward temporal fluctuation coding vector, and are different homography projection matrices, and respectively represent the network delay forward temporal fluctuation feature homography projection coding vector and the network delay backward temporal fluctuation feature homography projection coding vector.

[0053] That is, by performing a homography projection transformation on the network delay forward temporal fluctuation coding vector and the network delay backward temporal fluctuation coding vector, the two can be mapped to a shared feature space, so that the relative positions and relationships of the two in the common feature space are more intuitive and easier to compare, thereby providing a more effective basis for interaction analysis.

[0054] Figure 6 is a flowchart of sub-step S422 of the real-time feedback method for UAV application data according to an embodiment of the present application. As Figure 6As shown, step S422 includes steps: S4221, calculating the forward attention score field of the homography projection coding vector of the network delay forward temporal fluctuation feature with respect to the homography projection coding vector of the network delay backward temporal fluctuation feature to obtain the network delay temporal fluctuation forward attention score field; S4222, calculating the backward attention score field of the homography projection coding vector of the network delay backward temporal fluctuation feature with respect to the homography projection coding vector of the network delay forward temporal fluctuation feature to obtain the network delay temporal fluctuation backward attention score field; S4223, constructing the network delay temporal fluctuation forward and backward two-way attention balance field based on the network delay temporal fluctuation forward attention score field and the network delay temporal fluctuation backward attention score field.

[0055] In a specific example of the present application, step S4221 includes: calculating the network delay temporal fluctuation forward attention score field by multiplying the homography projection coding vector of the network delay forward temporal fluctuation feature by the transpose vector of the homography projection coding vector of the network delay backward temporal fluctuation feature and then dividing by the attention score scaling factor, which is expressed by the formula:

[0056]

[0057] where represents the transpose of the vector, represents the matrix multiplication operation, is the attention score scaling factor, represents the network delay temporal fluctuation forward attention score field.

[0058] That is, by calculating the forward attention score field, the homography projection transformed network delay forward temporal fluctuation coding vector and the network delay backward temporal fluctuation coding vector are analyzed, so as to evaluate the importance of each part of the forward temporal fluctuation feature of the network delay when interacting with the backward temporal fluctuation feature. In this way, the most relevant and key information segments can be identified from the forward temporal fluctuation feature, so as to enhance the response sensitivity of the forward temporal fluctuation feature to the backward temporal fluctuation feature, make the relationship between the two more clear, and help to extract more valuable interaction information.

[0059] In a specific example of the present application, step S4222 is expressed by the formula:

[0060]

[0061] where represents the network delay temporal fluctuation backward attention score field.

[0062] That is, similar to the calculation of the above-mentioned forward attention score field, the present application further analyzes the reverse attention score field to identify the feature part closely associated with the forward temporal fluctuation of network latency from the reverse temporal fluctuation characteristics of network latency.

[0063] In a specific example of the present application, the step S4223 is expressed by the formula:

[0064]

[0065] Wherein, represents concatenation, represents the forward and reverse attention balance field of network latency temporal fluctuation represents 3 3 convolution operation.

[0066] That is, in order to comprehensively consider the forward and reverse attention information, the present application further constructs the forward and reverse attention balance field of network latency temporal fluctuation to ensure that in the final interactive response encoding process, it will not overly focus on the features of one side and will not ignore the contribution of the other side. In this way, it can more evenly handle the interaction between the forward and reverse temporal features of network latency, thereby achieving a more harmonious and accurate interaction relationship between the two.

[0067] More specifically, the step S423 includes: First, the homography projection encoding vectors of the forward temporal fluctuation features of network latency and the homography projection encoding vectors of the reverse temporal fluctuation features of network latency are respectively mapped to the forward and reverse attention balance field of network latency temporal fluctuation to obtain the homography projection attention modulation encoding vectors of the forward temporal fluctuation of network latency and the homography projection attention modulation encoding vectors of the reverse temporal fluctuation of network latency, which is expressed by the formula:

[0068]

[0069]

[0070] Wherein, and respectively represent the homography projection attention modulation encoding vectors of the forward temporal fluctuation of network latency and the homography projection attention modulation encoding vectors of the reverse temporal fluctuation of network latency.

[0071] Here, the network delay forward temporal fluctuation encoded vector and the network delay backward temporal fluctuation encoded vector after the homography projection transformation are respectively mapped into the network delay temporal fluctuation forward and backward attention balance field. Through the attention weight distribution of the forward and backward attention balance field, the feature representations of the two are adjusted, enhancing the feature parts related to important information and suppressing the feature parts related to secondary information simultaneously, so as to obtain the network delay forward temporal fluctuation homography projection attention modulation encoded vector and the network delay backward temporal fluctuation homography projection attention modulation encoded vector.

[0072] Then, calculate the element-wise division between the network delay forward temporal fluctuation homography projection attention modulation encoded vector and the network delay backward temporal fluctuation homography projection attention modulation encoded vector to obtain the network delay temporal bidirectional joint encoded vector, which is expressed by the formula:

[0073]

[0074] Where, represents the network delay temporal bidirectional joint encoded vector;

[0075] It should be understood that in the above attention balance field projection process, essentially, the expression mode of one party is adjusted relative to the reaction of the other party, so that the generated network delay forward temporal fluctuation homography projection attention modulation encoded vector and the network delay backward temporal fluctuation homography projection attention modulation encoded vector reflect the cooperative relationship between the forward and backward temporal features of the network delay to a certain extent. Finally, through the element-wise division operation, the network delay forward temporal fluctuation homography projection attention modulation encoded vector and the network delay backward temporal fluctuation homography projection attention modulation encoded vector are interacted element by element, so as to integrate the features of the two and form a more comprehensive and accurate network delay feature representation, that is, the network delay temporal bidirectional joint encoded vector. In this way, not only the change trends of the network delay in different directions are extracted, but also the most important temporal feature parts in the interaction process are effectively highlighted, providing a more reliable basis for subsequent time compensation.

[0076] Specifically, in step S43, based on the network delay time series bidirectional joint coding vector, the time compensation parameter is determined. In a specific example of the present application, step S43 includes: inputting the network delay time series bidirectional joint coding vector into a compensation parameter estimation module based on a decoder to obtain the time compensation parameter. It should be understood that a decoder is a model that converts encoded features into the original data or the target value related to the original data. In the present application, the compensation parameter estimation module uses the structure of the decoder and the trained network parameters to map the network delay time series feature information contained in the network delay time series bidirectional joint coding vector to the time compensation parameter space, so as to obtain a specific value that can compensate for the network delay, providing a specific numerical basis for the subsequent adjustment of the UAV application data sending time.

[0077] In a specific example of the present application, obtaining the time compensation parameter by passing the network delay time series bidirectional joint coding vector through a compensation parameter estimation module based on a decoder includes:

[0078] First, arrange each eigenvalue of the network delay time series bidirectional joint coding vector in ascending order to obtain a network delay time series bidirectional joint sequential coding vector;

[0079] Second, in response to the absolute value of the difference between the -th eigenvalue and the -th eigenvalue of the network delay time series bidirectional joint sequential coding vector being less than or equal to the distance difference hyperparameter , calculate the weighted sum of the -th eigenvalue and the -th eigenvalue as the optimized -th eigenvalue, and calculate the square root of the sum of the squares of all eigenvalues of the network delay time series bidirectional joint coding vector, which is expressed by the formula:

[0080]

[0081] where represents the -th eigenvalue of the network delay time series bidirectional joint coding vector, represents the length of the network delay time series bidirectional joint coding vector, represents the square root of the sum of the squares of all eigenvalues of the network delay time series bidirectional joint coding vector;

[0082] Then, multiply by 2 and then divide by the square of the length of the network delay time series bidirectional joint coding vector to obtain the network delay time series bidirectional joint space basis value, which is expressed by the formula:

[0083]

[0084] Among them, represents the network delay time series bidirectional joint space basis element value;

[0085] Next, in response to the absolute value of the difference between the -th eigenvalue and the -th eigenvalue of the network delay time series bidirectional joint sequential coding vector being greater than the distance difference hyperparameter , after multiplying the network delay time series bidirectional joint space basis element value by the -th eigenvalue, calculate the weighted subtraction between the product and the -th eigenvalue as the optimized -th eigenvalue;

[0086] Then, on the basis of keeping the first eigenvalue of the network delay time series bidirectional joint sequential coding vector unchanged, combine the optimized -th eigenvalue to obtain an optimized network delay time series bidirectional joint coding vector;

[0087] Finally, pass the optimized network delay time series bidirectional joint coding vector through a compensation parameter estimation module based on a decoder to obtain a time compensation parameter.

[0088] Here, in the case where the network delay forward time series fluctuation coding vector and the network delay backward time series fluctuation coding vector respectively represent the time series correlation characteristics of network delay data along the forward time dimension and the reverse time dimension, when performing the interactive response based on the positive and negative attention fields of features, the complexity of the positive and negative attention field mechanisms of different time dimension codings will result in insufficient long-distance interactive response representation of the network delay time series bidirectional joint coding vector, thereby reducing the expression effect of the network delay time series bidirectional joint coding vector and affecting the accuracy of the time compensation parameter obtained by it through the compensation parameter estimation module based on a decoder.

[0089] Therefore, for the problem of insufficient global interaction response representation ability caused by long distances exceeding the predetermined local distribution interval threshold in the feature set of the network delay time series bidirectional joint coding vector under the predetermined eigenvalue order distribution, the high-dimensional feature space primitive representation based on self-inner product fusion of the network delay time series bidirectional joint coding vector is used to capture the complex structure of the eigenvalue global network interaction, so as to reconstruct the interaction response relationship between eigenvalues of the network delay time series bidirectional joint coding vector by simulating the potential primitive of the high-dimensional feature space based on scale, so as to realize the coding reconstruction of the true sequence distribution behavior of the network delay time series bidirectional joint coding vector under long distances, improve the interaction response expression effect of the network delay time series bidirectional joint coding vector, and improve the accuracy of the time compensation parameter obtained by its compensation parameter estimation module based on the decoder.

[0090] In the above real-time feedback method for UAV application data, in step S5, based on the time compensation parameter, the transmission time of the application data is adjusted. For example, when the time compensation parameter is 500 milliseconds, the data is sent 500 milliseconds in advance based on the originally planned data transmission time. At the same time, at the receiving end of the ground base station, the time window for receiving data is also adjusted according to the corresponding time compensation parameter to ensure that the adjusted data can be accurately received. In this way, the network delay can be effectively compensated, and the real-time performance and accuracy of UAV application data can be improved, enabling the ground base station to obtain the latest data collected by the UAV in a timely manner, providing strong support for subsequent decision-making and analysis.

[0091] In summary, the real-time feedback method for UAV application data based on the embodiments of the present application is elucidated. It synchronizes the clocks of the ground base station and the UAV based on the standard time information provided by the global positioning system as a reference, collects network delay data for a predetermined period of time, and introduces deep learning technology to perform forward and backward time series fluctuation analysis on the network delay data to capture the time series dynamic characteristics of the network delay, so as to intelligently estimate the corresponding time compensation parameter, and use this to adjust the transmission time of UAV application data. In this way, by adjusting the data transmission and reception times at both ends of the ground station and the UAV according to the estimated network delay, the network delay can be effectively compensated, and the real-time performance and accuracy of UAV application data feedback can be improved.

[0092] The basic principles of the present invention have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, advantages, effects, etc. mentioned in the present invention are only examples and not limitations. It cannot be considered that these advantages, advantages, effects, etc. are essential for each embodiment of the present invention. In addition, the specific details of the above embodiments are only for the purpose of illustration and easy understanding, rather than limitations. The above details do not limit the present invention to necessarily adopt the above specific details to implement.

[0093] In the above embodiments, the descriptions of the various embodiments have their own focuses. For parts not detailed or recorded in a certain embodiment, reference may be made to the relevant descriptions of other embodiments. In the several embodiments provided by the present invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the unit division is only a logical function division, and there may be other division methods in actual implementation. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0094] For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and without departing from the spirit or basic characteristics of the present invention, the present invention can be implemented in other specific forms. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be embraced by the present invention. Any reference signs in the claims should not be regarded as limiting the claimed rights.

[0095] In addition, it is obvious that the word "including" does not exclude other units or steps, and the singular does not exclude the plural. The multiple units stated in the system claims can also be implemented by one unit through software or hardware.

[0096] Finally, it should be noted that the above description has been given for purposes of illustration and description. In addition, the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for real-time feedback of drone application data, characterized in that: include: Defining communication parameters between the UAV and the ground base station, wherein the communication parameters include communication frequency, protocol selection and safety measures; Performing clock synchronization between the ground base station and the drone based on standard time information provided by the global positioning system as a reference; Collect network delay data for a predetermined period of time; Calculating a time compensation parameter based on the network delay data of the predetermined time period; Based on the time compensation parameter, adjusting the sending time of the application data; Calculating a time compensation parameter based on the network delay data of the predetermined time period includes: Performing bidirectional time series coding on the network delay data of the predetermined time period to obtain a network delay forward time series fluctuation coding vector and a network delay backward time series fluctuation coding vector; Performing a bidirectional attention interaction response analysis on the network delay forward timing fluctuation coding vector and the network delay backward timing fluctuation coding vector to obtain a network delay timing bidirectional joint coding vector; The time compensation parameter is determined based on the network delay timing bidirectional joint coding vector.

2. The method for real-time feedback of drone application data according to claim 1, characterized in that: Bidirectional time series coding is performed on the network delay data of the predetermined time period to obtain a network delay forward time series fluctuation coding vector and a network delay backward time series fluctuation coding vector, including: Arranging the network delay data of the predetermined time period in the order of timestamps to obtain a network delay timing fluctuation input vector; Inputting the network delay timing fluctuation input vector into a network delay forward timing encoder based on a forward LSTM model to obtain the network delay forward timing fluctuation encoding vector; The network delay timing fluctuation input vector is input into the network delay backward timing encoder based on the backward LSTM model to obtain the network delay backward timing fluctuation encoding vector.

3. The method for real-time feedback of drone application data according to claim 2, characterized in that: Performing a bidirectional attention interaction response analysis on the network delay forward timing fluctuation coding vector and the network delay backward timing fluctuation coding vector to obtain a network delay timing bidirectional joint coding vector, including: Performing homography projection transformation on the network delay forward timing fluctuation coding vector and the network delay backward timing fluctuation coding vector to obtain a network delay forward timing fluctuation feature homography projection coding vector and a network delay backward timing fluctuation feature homography projection coding vector; Constructing a positive and negative bidirectional attention balance field between the homography projection coding vector of the network delay forward time series fluctuation feature and the homography projection coding vector of the network delay backward time series fluctuation feature to obtain a positive and negative bidirectional attention balance field of the network delay time series fluctuation; Based on the positive and negative bidirectional attention balance field of the network delay timing fluctuation, the network delay forward timing fluctuation encoding vector and the network delay backward timing fluctuation encoding vector are interactively analyzed by field mapping to obtain the network delay timing bidirectional joint encoding vector.

4. The method for real-time feedback of drone application data according to claim 3, characterized in that: Constructing a positive and negative bidirectional attention balance field between the homography projection coding vector of the network delay forward time series fluctuation feature and the homography projection coding vector of the network delay backward time series fluctuation feature to obtain a positive and negative bidirectional attention balance field of the network delay time series fluctuation, including: Calculate the forward attention score field of the homography projection coding vector of the network delay forward timing fluctuation feature relative to the homography projection coding vector of the network delay backward timing fluctuation feature to obtain the network delay timing fluctuation forward attention score field; Calculate the reverse attention score field of the homography projection coding vector of the network delay backward timing fluctuation feature relative to the homography projection coding vector of the network delay forward timing fluctuation feature to obtain the network delay timing fluctuation reverse attention score field; Based on the network delay timing fluctuation positive attention score field and the network delay timing fluctuation reverse attention score field, the network delay timing fluctuation positive and negative bidirectional attention balance field is constructed.

5. The method for real-time feedback of drone application data according to claim 4, characterized in that: Calculating the forward attention score field of the homography projection coding vector of the network delay forward timing fluctuation feature relative to the homography projection coding vector of the network delay backward timing fluctuation feature to obtain the network delay timing fluctuation forward attention score field, including: Calculate the network delay forward timing fluctuation feature homography projection coding vector multiplied by the transposed vector of the network delay backward timing fluctuation feature homography projection coding vector and then divide by the attention score scaling factor to obtain the network delay timing fluctuation forward attention score field.

6. The method for real-time feedback of drone application data according to claim 5, characterized in that: Based on the network delay timing fluctuation positive and negative bidirectional attention balance field, the network delay forward timing fluctuation coding vector and the network delay backward timing fluctuation coding vector are subjected to field mapping interactive analysis to obtain the network delay timing bidirectional joint coding vector, including: Mapping the network delay forward timing fluctuation feature homography projection coding vector and the network delay backward timing fluctuation feature homography projection coding vector to the network delay timing fluctuation positive and negative bidirectional attention balance field respectively to obtain the network delay forward timing fluctuation homography projection attention modulation coding vector and the network delay backward timing fluctuation homography projection attention modulation coding vector; The network delay timing bidirectional joint coding vector is obtained by calculating the network delay forward timing fluctuation homography projection attention modulation coding vector and the network delay backward timing fluctuation homography projection attention modulation coding vector by dividing them by the position points.

7. The method for real-time feedback of drone application data according to claim 6, characterized in that: Determining the time compensation parameter based on the network delay timing bidirectional joint coding vector includes: The network delay timing bidirectional joint coding vector is input into a decoder-based compensation parameter estimation module to obtain the time compensation parameter.

Images