Unmanned aerial vehicle flight process image transmission method and system

Abstract

The invention relates to the technical field of unmanned aerial vehicle image transmission, and particularly provides an unmanned aerial vehicle flight process image transmission method and system, and the method comprises the steps: obtaining electromagnetic environment information, channel state information and unmanned aerial vehicle internal parameter information; generating a comprehensive state index according to the electromagnetic environment information, the channel state information and the internal parameter information of the unmanned aerial vehicle; querying a pre-constructed mapping relation table about state indexes and transmission parameter combinations according to the comprehensive state indexes to obtain a target transmission parameter combination; according to the target transmission parameter combination, coding, packaging, modulating and sending the image acquired by the unmanned aerial vehicle; the method can effectively solve the problems of image jamming, image frame loss, image mosaic or color distortion and the like of the image flow received by the ground station due to interference or enhanced channel attenuation.

Description

Technical Field

[0001] This application relates to the technical field of UAV image transmission. Specifically, it relates to a method and system for transmitting images during the flight of a UAV. Background Art

[0002] As a flexible and efficient platform, UAVs are playing an increasingly important role in tasks such as the inspection of urban infrastructure. In order to achieve real-time monitoring by ground operators, remote precise control, and instant judgment of the target state, UAVs need to transmit high-resolution image data collected during flight to the ground control station in real time and with low latency through a wireless communication link.

[0003] However, the electromagnetic environment and channel state encountered by UAVs during flight in different regions are dynamically changing (the type, intensity, direction, and duration of interference sources may vary), that is, the interference and channel attenuation suffered by UAVs during flight are dynamic. For example, UAVs may face strong industrial electromagnetic interference when approaching industrial areas, a large number of Wi-Fi and Bluetooth signal interferences when passing through dense residential or commercial areas, and strong electromagnetic interference with specific frequencies and intensities when approaching high-voltage power lines.

[0004] Existing UAV image transmission methods usually adopt fixed image transmission strategies (using fixed coding, encapsulation, modulation, and transmission parameters). Such fixed image transmission strategies cannot effectively adapt to the dynamically changing transmission environment, thus facing many limitations. For example, if a fixed strong error correction coding rate is adopted to enhance robustness, it will result in wasting precious bandwidth and the computing resources of the UAV when the channel conditions are good and the interference is weak; if a fixed weak error correction coding rate is adopted, it will result in the inability to resist interference when encountering sudden strong interference or severe fading, and a large amount of image data will be lost. Since the interference and channel attenuation suffered by UAVs during flight are dynamic, and existing UAV image transmission methods adopt fixed image transmission strategies, there are problems in the prior art such as frame freezing, image frame loss, mosaic or color distortion in the image stream received by the ground station due to the enhanced interference or channel attenuation, resulting in a decrease in the quality of the image stream, and further leading to situations where defects cannot be detected in time or misjudgments occur due to the decrease in the quality of the image stream.

[0005] In response to the above problems, there is currently no effective technical solution. It should be noted that the above information disclosed in this part is only used to understand the background of the inventive concept of the present invention, and therefore may include information that does not constitute the prior art. Summary of the Invention

[0006] The purpose of this application is to provide a method and system for transmitting images during the flight of an unmanned aerial vehicle (UAV), which can effectively solve the problems such as frame freezing, image frame loss, mosaic appearance or color distortion in the image stream received by the ground station due to increased interference or channel attenuation.

[0007] In a first aspect, this application provides a method for transmitting images during the flight of a UAV, which is used to transmit the images collected by the UAV, and includes the following steps: S1. Obtain electromagnetic environment information, channel state information, and UAV internal parameter information; S2. Generate a comprehensive state index based on the electromagnetic environment information, channel state information, and UAV internal parameter information; S3. Query a pre-constructed mapping relation table of state indexes and transmission parameter combinations according to the comprehensive state index to obtain a target transmission parameter combination, where the target transmission parameter combination includes modulation order, forward error correction coding rate, transmission power, image frame rate, data packet size, image resolution, and compression ratio; S4. Encode, encapsulate, modulate, and transmit the images collected by the UAV according to the target transmission parameter combination.

[0008] For the method for transmitting images during the flight of a UAV provided by this application, first, a comprehensive state index that can reflect the overall state of the UAV is generated by integrating the electromagnetic environment information, channel state information, and UAV internal parameter information. Then, based on this comprehensive index, a target transmission parameter combination is dynamically synthesized. Finally, the images collected by the UAV are encoded, encapsulated, modulated, and transmitted according to the target transmission parameter combination. That is, this application is equivalent to first understanding the interference situation of the environment where the UAV is located, the current transmission link quality, and the UAV's own state, and then adaptively adjusting the image transmission strategy according to the interference situation, the current transmission link quality, and the UAV's own state. Therefore, even if the interference and channel attenuation during the flight of the UAV are dynamic, this application can select an appropriate image transmission strategy for image transmission according to the dynamically changing interference and channel attenuation, so as to effectively solve the problems such as frame freezing, image frame loss, mosaic appearance or color distortion in the image stream received by the ground station due to increased interference or channel attenuation, thereby effectively improving the quality of the image stream, and further effectively avoiding the situation where defects cannot be detected in time or misjudgment occurs due to the decline in the quality of the image stream.

[0009] In a second aspect, this application also provides a system for transmitting images during the flight of a UAV, which is used to transmit the images collected by the UAV, and includes: An information acquisition module, which is used to obtain electromagnetic environment information, channel state information, and UAV internal parameter information; The comprehensive status indicator acquisition module is used to generate a comprehensive status indicator based on the electromagnetic environment information, channel status information, and the internal parameter information of the UAV; The transmission parameter combination determination module is used to query the pre-constructed mapping relationship table of status indicators and transmission parameter combinations according to the comprehensive status indicator to obtain the target transmission parameter combination, where the target transmission parameter combination includes modulation order, forward error correction coding rate, transmit power, image frame rate, data packet size, image resolution, and compression ratio; The image processing module is used to encode, encapsulate, modulate, and transmit the images collected by the UAV according to the target transmission parameter combination.

[0010] For the image transmission system during the flight of the UAV provided in this application, first, a comprehensive status indicator that can reflect the overall state of the UAV is generated by integrating the electromagnetic environment information, channel status information, and the internal parameter information of the UAV. Then, the target transmission parameter combination of the dynamic comprehensive status indicator is based on this comprehensive indicator. Finally, the images collected by the UAV are encoded, encapsulated, modulated, and transmitted according to the target transmission parameter combination. That is, this application is equivalent to first understanding the interference situation of the environment where the UAV is located, the current transmission link quality, and the state of the UAV itself, and then adaptively adjusting the image transmission strategy according to the interference situation, the current transmission link quality, and the state of the UAV itself. Therefore, even if the interference and channel attenuation during the flight of the UAV are dynamic, this application can select an appropriate image transmission strategy for image transmission according to the dynamically changing interference and channel attenuation, so as to effectively solve the problems such as frame freezing, image frame loss, mosaic or color distortion in the image stream received by the ground station due to increased interference or channel attenuation, thereby effectively improving the quality of the image stream, and further effectively avoiding the situation where it is impossible to detect defects in time or misjudgment occurs due to the decline in the quality of the image stream.

[0011] As can be seen from the above, for a method and system for transmitting images during the flight of a drone provided by this application, first, a comprehensive state index reflecting the overall state of the drone is generated by integrating electromagnetic environment information, channel state information, and internal parameter information of the drone. Then, based on this comprehensive index, a target transmission parameter combination for the dynamic comprehensive state index is determined. Finally, the images collected by the drone are encoded, encapsulated, modulated, and sent according to the target transmission parameter combination. That is, this application is equivalent to first understanding the interference situation in the environment where the drone is located, the current transmission link quality, and the state of the drone itself, and then adaptively adjusting the image transmission strategy according to the interference situation, the current transmission link quality, and the state of the drone itself. Therefore, even if the interference and channel attenuation during the flight of the drone are dynamic, this application can select an appropriate image transmission strategy for image transmission according to the dynamically changing interference and channel attenuation, so as to effectively solve the problem that the image stream received by the ground station has problems such as frame freezing, image frame loss, mosaic or color distortion in the picture due to increased interference or channel attenuation, thereby effectively improving the quality of the image stream and further effectively avoiding the situation where it is impossible to detect defects in time or misjudgment occurs due to the degradation of the image stream quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a flowchart of a method for transmitting images during the flight of a drone provided by an embodiment of this application.

[0013] Figure 2 It is a schematic structural diagram of a system for transmitting images during the flight of a drone provided by an embodiment of this application.

[0014] Reference numerals: 1, information acquisition module; 2, comprehensive state index acquisition module; 3, transmission parameter combination determination module; 4, image processing module. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0015] Next, the technical solutions in the embodiments of this application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all of the embodiments. Usually, the components of the embodiments of this application described and shown here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents the selected embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of this application.

[0016] It should be noted that: Similar reference numerals and letters denote similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of the present application, terms such as "first" and "second" are only used for distinguishing descriptions and cannot be construed as indicating or implying relative importance.

[0017] In traditional existing UAV image transmission methods, since the types, intensities, directions, and durations of interference sources encountered by UAVs are different when flying in different regions, the interference and channel attenuation suffered by UAVs during flight are dynamic. Existing UAV image transmission methods adopt fixed image transmission strategies, that is, fixed coding, encapsulation, modulation, and sending strategies. This fixed image transmission strategy results in a waste of bandwidth and UAV computing resources when the channel conditions are good and the interference is weak, because a strong error correction coding rate is used to enhance robustness; while in the face of sudden strong interference or severe fading, the weak error correction coding rate used is insufficient to resist interference, resulting in a large amount of image data loss, and thus causing a decrease in the quality of the image stream received by the ground station, such as image frame loss, mosaic appearance, or color distortion in the picture.

[0018] For example, assume that a UAV performs an inspection task on urban infrastructure. The UAV takes off from an open area (good channel conditions and less interference), and then flies into the vicinity of an industrial area, where the UAV faces strong industrial electromagnetic interference. Then it flies over a dense residential area, where the UAV encounters a large number of Wi-Fi and Bluetooth signal interferences. Finally, it approaches high-voltage power lines, where the UAV is subjected to strong electromagnetic interference of a specific frequency and intensity. If a fixed modulation order, forward error correction coding rate, transmission power, image frame rate, data packet size, image resolution, and compression ratio (the image transmission strategy consists of these parameters) are used for image transmission, in the open area, high-resolution image transmission may consume too many resources; while in the industrial area, residential area, or near high-voltage power lines, due to the influence of enhanced interference or severe channel attenuation, the fixed image transmission strategy cannot ensure reliable data transmission, the image data packet loss rate rises sharply, and the image stream received by the ground station appears stuck, incomplete, or with abnormal colors.

[0019] If the above problems are not solved, the degradation of image quality directly affects the user's immediate judgment of the target state. For example, picture distortion or loss may lead to misjudgment by the user of the on-site situation, resulting in the inability to timely detect subtle defects in infrastructure or the neglect of potential safety hazards.

[0020] In response to this, on the first aspect, as Figure 1 shown, the present application provides a method for transmitting images during the flight of a UAV, which is used to transmit images collected by the UAV, and it includes the following steps: S1. Obtain electromagnetic environment information, channel state information, and UAV internal parameter information; S2. Generate a comprehensive status index based on the electromagnetic environment information, channel state information, and UAV internal parameter information; S3. Query a pre-constructed mapping relation table of status index and transmission parameter combinations according to the comprehensive status index to obtain a target transmission parameter combination, where the target transmission parameter combination includes modulation order, forward error correction coding rate, transmit power, image frame rate, data packet size, image resolution, and compression ratio; S4. Encode, encapsulate, modulate, and transmit the images collected by the UAV according to the target transmission parameter combination.

[0021] The electromagnetic environment information in step S1 refers to the electromagnetic wave distribution and interference situation in the space where the UAV is located. In step S1, the electromagnetic environment information can be obtained by using devices such as spectrum analyzers and interference source detectors to measure the signal strength in specific frequency bands and identify the sources and characteristics of interference signals. Through this electromagnetic environment information, this embodiment can understand the impact of external electromagnetic interference on wireless communication. The channel state information in step S1 refers to the transmission characteristics and quality status of the wireless communication link between the UAV and the ground station. In step S1, the channel state information can be obtained by measuring the received signal strength, calculating the signal-to-noise ratio, statistically analyzing the data transmission error rate, or analyzing the time difference of the signal arriving through different paths. This channel state information directly reflects the current quality of the wireless communication link (signal attenuation degree, interference level, and multipath effect). Through this channel state information, this embodiment can more accurately evaluate the bearing capacity and stability of the current transmission link. Since the bearing capacity and stability of the current transmission link are related to the transmission quality of wireless communication, this embodiment can understand the impact of the current transmission link on wireless communication by obtaining the channel state information. The internal parameter information of the UAV in this embodiment refers to the resource usage and operating status of the UAV itself. Since existing UAVs usually carry an operating system or a dedicated flight control system, the operating system and flight control system usually provide APIs (application programming interfaces) to allow developers to access various internal parameters of the UAV. Therefore, in this embodiment, the internal parameter information of the UAV can be obtained through the APIs provided by the UAV operating system or flight control system. This embodiment can also use sensors integrated in the UAV to obtain the internal parameter information of the UAV. Since the UAV continuously sends telemetry data to the ground station during flight, this data usually contains various status information of the UAV, and this status information includes the internal parameters of the UAV. Therefore, this embodiment can also obtain the internal parameter information of the UAV by analyzing the telemetry data of the UAV. Through the internal parameter information of the UAV, this embodiment can understand the ability of the UAV to perform image processing and transmission tasks. It should be understood that since the ability of the UAV to perform image processing and transmission tasks affects the transmission quality of wireless communication, this embodiment can understand the impact of the UAV's own characteristics on wireless communication by obtaining the internal parameter information of the UAV.

[0022] The comprehensive status indicator in step S2 is one or a set of values obtained by comprehensively evaluating the electromagnetic environment information, channel status information, and UAV internal parameter information. In this embodiment, the comprehensive status indicator can be generated by directly integrating the electromagnetic environment information, channel status information, and UAV internal parameter information. For example, when the electromagnetic environment information is strong narrowband interference, the signal status information is that the quality of the current transmission link is poor, and the UAV internal parameter information is that the remaining battery power is 25%, the comprehensive status indicator generated in step S2 is "strong narrowband interference, severe channel attenuation, and low remaining battery power". Step S2 can also use methods such as direct summation, weighted summation, fuzzy logic judgment, or machine learning models to generate the comprehensive status indicator according to the electromagnetic environment information, channel status information, and UAV internal parameter information. For example, query the pre-constructed mapping relationship table of electromagnetic environment and transmission quality impact scores according to the electromagnetic environment information to obtain the first score; query the pre-constructed mapping relationship table of channel status and transmission quality impact scores according to the channel status information to obtain the second score; query the pre-constructed mapping relationship table of UAV internal parameters and transmission quality impact scores according to the UAV internal parameter information to obtain the third score; sum the first score, the second score, and the third score to obtain the comprehensive status indicator. It should be understood that step S2 is equivalent to integrating multiple influencing factors (the impact of external electromagnetic interference on wireless communication, the impact of the current transmission link quality on wireless communication, and the impact of the UAV's own characteristics on wireless communication) into a unified metric standard for subsequent transmission parameter decision-making.

[0023] The mapping relationship table of status indicators and transmission parameter combinations in step S3 can be a mapping relationship table established by means of experiments, simulations, or expert experience before system deployment, that is, the data of this mapping relationship table can be experimental data, simulation data, or empirical data. This mapping relationship table stores the transmission parameter combinations corresponding to different status indicators. The target transmission parameter combination in step S3 refers to the transmission parameter combination obtained by querying the mapping relationship table according to the comprehensive status indicator. This target transmission parameter combination is used to guide a series of parameter settings for the current image transmission. Specifically, this target transmission parameter combination includes modulation order, forward error correction coding rate, transmit power, image frame rate, packet size, image resolution, and compression ratio, etc. It should be understood that since the target transmission parameter combination of this embodiment is determined based on the comprehensive status indicator, and the comprehensive status of this embodiment is one or a set of values obtained by comprehensively evaluating the electromagnetic environment information, channel status information, and UAV internal parameter information, this embodiment is equivalent to dynamically determining a suitable image transmission strategy according to reducing the impact of external electromagnetic interference, the current transmission link quality (equivalent to the degree of channel attenuation), and the UAV's own characteristics on wireless communication, so as to eliminate or minimize the impact of external electromagnetic interference, the current transmission link quality, and the UAV's own characteristics on wireless communication.

[0024] Step S4 encodes, encapsulates, modulates, and transmits the images collected by the drone according to the target transmission parameter combination, which is preferably the prior art, and its working principle and process will not be elaborated here.

[0025] A method for transmitting images during the flight of a drone provided by this application first generates a comprehensive state index that can reflect the overall state of the drone by integrating electromagnetic environment information, channel state information, and internal parameter information of the drone, then dynamically synthesizes the target transmission parameter combination based on this comprehensive index, and finally encodes, encapsulates, modulates, and transmits the images collected by the drone according to the target transmission parameter combination. That is, this application is equivalent to first understanding the interference situation in the environment where the drone is located, the current transmission link quality, and the state of the drone itself, and then adaptively adjusting the image transmission strategy according to the interference situation, the current transmission link quality, and the state of the drone itself. Therefore, even if the interference and channel attenuation during the flight of the drone are dynamic, this application can select an appropriate image transmission strategy for image transmission according to the dynamically changing interference and channel attenuation, so as to effectively solve the problems such as frame freezing, image frame loss, mosaic or color distortion in the image stream received by the ground station due to increased interference or channel attenuation, thereby effectively improving the quality of the image stream, and further effectively avoiding the situation of being unable to detect defects in time or making misjudgments due to the decline in the quality of the image stream. It should be understood that since this application can adaptively adjust the image transmission strategy according to the interference situation in the environment where the drone is located, the current transmission link quality, and the state of the drone itself, this application can improve the transmission efficiency when the channel conditions are good to make full use of the bandwidth and resources, and enhance the robustness of the transmission when the channel conditions are poor to effectively resist interference and channel attenuation and reduce data loss.

[0026] In some preferred embodiments, step S3 includes: S31. Query the pre-constructed mapping relationship table between the state index and the transmission parameter combination according to the comprehensive state index to obtain a preliminary transmission parameter combination; S32. Analyze whether there is building occlusion in the images collected by the drone. If so, execute step S33; if not, use the preliminary transmission parameter combination as the target transmission parameter combination; S33. Obtain the current pose information and flight altitude information of the drone, and predict the influence degree of building occlusion on image transmission according to the current pose information, flight altitude information of the drone, and the pre-constructed 3D urban model to obtain the first image transmission influence degree; S34. Query the pre-constructed mapping relation table of the image transmission influence degree and the transmission parameter combination adjustment strategy according to the first image transmission influence degree to obtain the first transmission parameter combination adjustment strategy, and then adjust the preliminary transmission parameter combination according to the first transmission parameter combination adjustment strategy to obtain the target transmission parameter combination.

[0027] Among them, analyzing whether there is building occlusion based on the images collected by the drone in this embodiment means processing the image data transmitted by the drone to determine whether there is a building structure in the image content that may obstruct the wireless signal transmission path. This embodiment can use image recognition technology, computer vision algorithms, or deep learning models to analyze whether there is building occlusion based on the images collected by the drone. The current pose information of the drone in this embodiment refers to the position and attitude information of the drone in three-dimensional space. This embodiment can obtain the current pose information of the drone by fusing sensor data such as Global Positioning System (GPS) data, Inertial Measurement Unit (IMU) data, or visual odometry. The flight altitude information in this embodiment refers to the altitude of the drone relative to a certain reference plane (such as the ground or sea level). This embodiment can use a barometric altimeter, radar altimeter, or GPS altitude data to obtain the flight altitude information. The pre-constructed 3D urban model in this embodiment refers to a digital model containing spatial position, height, shape, etc. information formed by digitizing the buildings, terrain, etc. in a specific urban area. The 3D urban model can be an existing 3D urban model, and the 3D urban model can be constructed using lidar scanning, photogrammetry, or Geographic Information System (GIS) data. Predicting the degree of influence of building occlusion on image transmission in this embodiment means calculating or estimating the possibility, penetration distance, or blocking degree of the signal transmission path passing through the building based on the relative position between the drone and the ground station and the 3D urban model, so as to quantify the influence of building blockage on wireless signal attenuation. Specifically, the process of predicting the degree of influence of building occlusion on image transmission according to the current pose information, flight altitude information of the drone, and the pre-constructed 3D urban model in this embodiment can be: determining the precise position of the drone in the 3D urban model according to the current pose information and flight altitude information of the drone; emitting a virtual ray from the precise position of the drone in the 3D urban model to the precise position of the ground station in the 3D urban model (known value), and this ray represents the main path of wireless signal propagation; obtaining the penetration length and building material corresponding to each building penetrated by the ray; for each penetrated building, querying the pre-constructed mapping relationship table of penetration length and building material, building material, and transmission influence score to obtain the image transmission influence score; directly summing or weighted summing all the image transmission influence scores, and taking the summation result as the degree of influence of building occlusion on image transmission. The first image transmission influence degree in this embodiment refers to the quantitative index of the influence of the predicted building occlusion on image transmission, and the first image transmission influence degree can be expressed as a signal attenuation value, a link quality prediction value, or a discrete occlusion level.The mapping relationship table of the image transmission impact degree and the transmission parameter combination adjustment strategy of this embodiment stores the transmission parameter combination adjustment strategy corresponding to different image transmission impact degrees, and the transmission parameter combination adjustment strategy may include instructions or values for increasing, decreasing or keeping unchanged the parameters such as modulation order, forward error correction coding rate, transmission power, image frame rate, data packet size, image resolution or compression ratio. This embodiment can use parameter superposition, parameter replacement or rule-based parameter modification method to adjust the preliminary transmission parameter combination according to the first transmission parameter combination adjustment strategy.

[0028] Specifically, the method first obtains a preliminary transmission parameter combination based on the comprehensive state index, which reflects the optimal or suboptimal transmission configuration under the current non-obstructed state. Subsequently, the system analyzes the real-time images collected by the drone to determine whether there are signs of building obstruction in the image content. If the image analysis shows that there is building obstruction, a further refined adjustment process is triggered. At this time, the system obtains the precise position and attitude information of the drone and the flight altitude, and analyzes the signal path from the drone to the ground station in combination with the pre-built three-dimensional urban model to predict the specific impact of the building on the signal transmission. Then, the corresponding first transmission parameter adjustment strategy is obtained according to the specific impact. Finally, the preliminary transmission parameter combination obtained before is modified according to the strategy, such as increasing the forward error correction capability, reducing the bit rate or resolution, etc., so as to obtain the target parameter combination for image transmission. If the image analysis does not detect building obstruction, the preliminary transmission parameter combination is directly adopted as the target transmission parameter combination to avoid unnecessary calculations. Since this embodiment can identify building occlusions encountered by drones when flying in urban environments, and make targeted adjustments to image transmission parameters based on the degree of occlusion impact predicted based on the building occlusions, that is, this embodiment is equivalent to adding recognition and targeted response mechanisms for building occlusion problems unique to urban environments on the basis of adaptive capabilities based on comprehensive state indicators. Therefore, this embodiment can make the confirmation of image transmission parameters more precise and robust, thereby effectively improving the accuracy and reliability of the target transmission parameter combination, and further effectively improving the accuracy and reliability of the image transmission strategy adjustment.

[0029] In some preferred embodiments, step S31 includes: S311, querying a pre-built mapping relationship table of status indicators and transmission parameter combinations according to the comprehensive status indicators to obtain an original transmission parameter combination; S312, acquiring real-time meteorological data, and then querying a pre-built mapping relationship table between meteorological data and transmission parameter combination adjustment strategy according to the real-time meteorological data to obtain a second transmission parameter combination adjustment strategy; S313. Adjust the original transmission parameter combination according to the second transmission parameter combination adjustment strategy to obtain a preliminary transmission parameter combination.

[0030] The real-time meteorological data in this embodiment is the meteorological information of the environment where the UAV is located. The real-time meteorological data is preferably meteorological information related to wireless signal propagation (such as rainfall, haze concentration, temperature). In this embodiment, the real-time meteorological data can be obtained through the meteorological data prediction information pre-released by the astronomical observatory, and this embodiment can also obtain the real-time meteorological data by querying the meteorological data released in real time by the astronomical observatory. The mapping relationship table of meteorological data and transmission parameter combination adjustment strategy in this embodiment stores the transmission parameter combination adjustment strategies corresponding to different meteorological data. In this embodiment, the mapping relationship table of meteorological data and transmission parameter combination adjustment strategy can be constructed by analyzing the influence of different meteorological conditions on wireless signal propagation before the UAV actually takes off. The second transmission parameter combination adjustment strategy in this embodiment refers to a specific set of rules or instructions determined according to the real-time meteorological data and used to guide how to modify the original transmission parameter combination. The second transmission parameter combination adjustment strategy may include instructions to increase, decrease, or replace one or more image transmission parameters in the original transmission parameter combination. In this embodiment, methods such as parameter superposition, parameter replacement, or rule-based parameter modification can be used to adjust the original transmission parameter combination according to the second transmission parameter combination adjustment strategy. Since this embodiment can dynamically adjust the image transmission parameters according to the real-time meteorological conditions by first determining the second transmission parameter combination adjustment strategy according to the real-time meteorological data and then adjusting the original transmission parameter combination according to the second transmission parameter combination adjustment strategy, this embodiment can make the preliminary transmission parameter combination better adapt to the signal attenuation and interference caused by complex meteorological environments such as rainfall and haze, so as to reduce the bit error rate and reduce image frame loss and picture distortion, thereby further improving the accuracy and reliability of the target transmission parameter combination, and further improving the accuracy and reliability of the image transmission strategy adjustment.

[0031] In some preferred embodiments, step S313 includes: A1. Obtain the current task type of the UAV, and then query the pre-constructed mapping relationship table of task execution type and transmission parameter combination adjustment strategy according to the current task type of the UAV to obtain the third transmission parameter combination adjustment strategy; A2. Adjust the original transmission parameter combination according to the second transmission parameter combination adjustment strategy and the third transmission parameter combination adjustment strategy to obtain a preliminary transmission parameter combination.

[0032] The type of task currently being executed by the UAV in this embodiment refers to the type of specific flight task that the UAV is currently performing (such as power inspection, urban mapping, security monitoring, or logistics distribution). In this embodiment, the type of task currently being executed by the UAV can be obtained through the UAV's mission planning system or ground station. The mapping relationship table regarding the combination adjustment strategy of the task execution type and transmission parameters in this embodiment stores the combination adjustment strategies of transmission parameters corresponding to different task execution types. The third transmission parameter combination adjustment strategy in this embodiment refers to a set of rules or instructions for adjusting the original transmission parameter combination obtained from the mapping relationship table according to the type of task currently being executed by the UAV. The third transmission parameter combination adjustment strategy can include specific instructions or adjustment amounts for increasing, decreasing, or keeping unchanged parameters such as modulation order, forward error correction coding rate, transmit power, image frame rate, data packet size, image resolution, and compression ratio. The purpose of introducing the third transmission parameter combination adjustment strategy in this embodiment is to enable the initial setting of transmission parameters to better meet the specific requirements of the current task for image transmission performance. For example, for a rescue task that requires high real-time performance, the third transmission parameter combination adjustment strategy instructs to reduce the image resolution and compression ratio to improve the transmission rate; for an inspection task that requires high clarity, the third transmission parameter combination adjustment strategy instructs to increase the resolution and compression ratio. This embodiment can adopt methods such as parameter superposition, parameter replacement, or rule-based parameter modification to adjust the original transmission parameter combination according to the second transmission parameter combination adjustment strategy and the third transmission parameter combination adjustment strategy. Since this embodiment can achieve targeted optimization and adjustment of image transmission parameters according to the type of task currently being executed by the UAV by first determining the third transmission parameter combination adjustment strategy based on the type of task currently being executed by the UAV and then adjusting the original transmission parameter combination according to the second transmission parameter combination adjustment strategy and the third transmission parameter combination adjustment strategy, this embodiment can enable the UAV to better balance performance indicators such as real-time performance, clarity, and robustness of image transmission when performing different types of tasks, thereby improving the efficiency and quality of image transmission to better meet the specific requirements in different application scenarios, and further improving the accuracy and reliability of the target transmission parameter combination and image transmission strategy adjustment.

[0033] In some preferred embodiments, step S2 includes: S21. Obtain the type of task currently being executed by the UAV, and then query the pre-constructed mapping relationship table regarding the task execution type and weighted weight combination according to the type of task currently being executed by the UAV to obtain the weighted weights corresponding to the electromagnetic environment information, channel state information, and UAV internal parameter information respectively; S22. Calculate the comprehensive state index according to the electromagnetic environment information and its corresponding weighted weight, channel state information and its corresponding weighted weight, and UAV internal parameter information and its corresponding weighted weight.

[0034] The mapping relation table regarding the combination of task types and weighted weights in this embodiment stores the weighted weight combinations corresponding to different task types. The weighted weight combination includes the weighted weight corresponding to the electromagnetic environment information, the weighted weight corresponding to the channel state information, and the weighted weight corresponding to the internal parameter information of the UAV. Therefore, in this embodiment, the weighted weights corresponding to the electromagnetic environment information, the channel state information, and the internal parameter information of the UAV can be obtained by querying the mapping relation table regarding the combination of task types and weighted weights according to the current task type of the UAV. It should be understood that the weighted weight combination in this embodiment can reflect the relative importance degrees of the electromagnetic environment information, the channel state information, and the internal parameter information of the UAV under a specific task type. For example, in the power inspection task, electromagnetic environment interference may be more critical, so the weight of the electromagnetic environment information may be higher; while in the long-distance mapping task, channel attenuation and signal-to-noise ratio may be more important, and the weight of the channel state information may be higher. Since when the UAV performs different types of tasks, the electromagnetic environment information, the channel state information, and the internal parameter information of the UAV have different degrees of influence on the UAV image transmission, and this embodiment can realize considering the influence degrees of the electromagnetic environment information, the channel state information, and the internal parameter information of the UAV on the UAV image transmission when performing different types of tasks by dynamically adjusting the weighted weights of the electromagnetic environment information, the channel state information, and the internal parameter information of the UAV when calculating the comprehensive state index. Therefore, this embodiment can make the comprehensive state index more accurately reflect the key factors and overall situation affecting image transmission in the current task scenario, providing a more reliable basis for subsequent selection of a more appropriate image transmission parameter combination, thereby effectively improving the accuracy and reliability of the target transmission parameter combination and image transmission strategy adjustment, and further effectively improving the quality and reliability of the UAV image transmission.

[0035] In some preferred embodiments, the electromagnetic environment information includes the operating frequency band and adjacent frequency bands of the UAV, the spectrum occupancy, the interference signal frequency band, the interference signal bandwidth, and the interference signal intensity. The operating frequency band and adjacent frequency bands in this embodiment refer to the communication frequency band currently used by the UAV and other frequency bands adjacent to this frequency band. This embodiment can be implemented by configuring the scanning range of the spectrum sensing module to obtain the operating frequency band and adjacent frequency bands. For example, if the operating frequency band of the UAV is 2.4 GHz, then the scanning range of the spectrum sensing module can be set to 2.3 GHz to 2.5 GHz to cover the adjacent frequency bands. Then, by analyzing the scanning results of the spectrum sensing module, the signal intensity, frequency occupancy, and potential interference signals within the operating frequency band and adjacent frequency bands can be determined. This embodiment can understand the co-frequency or adjacent-frequency interference that the UAV communication may be subject to based on the operating frequency band and adjacent frequency bands of the UAV. The spectrum occupancy in this embodiment refers to the presence and distribution state of radio signals within a specific frequency band. This embodiment can measure the signal power equation at different frequency points by using existing spectrum analysis techniques to obtain the spectrum occupancy. This embodiment can understand which frequency bands are occupied and which are idle based on the spectrum occupancy. The interference signal frequency band in this embodiment refers to the frequency range where the detected interference signal is located. This embodiment can use existing peak detection or threshold judgment algorithms to obtain the interference signal frequency band. For example, this embodiment can use the frequency range where the signal intensity exceeds a certain threshold as the interference signal frequency band. The interference signal bandwidth in this embodiment refers to the frequency width occupied by the interference signal. This embodiment can use the bandwidth measurement function of existing spectrum analyzers or signal processing algorithms to obtain the interference signal bandwidth. The interference signal intensity in this embodiment refers to the power magnitude of the interference signal. This embodiment can use existing spectrum analyzers to obtain the interference signal intensity.

[0036] In some preferred embodiments, the channel state information includes received signal strength indication, signal-to-noise ratio, bit error rate, and multipath delay spread. The received signal strength indication in this embodiment refers to the magnitude of the signal power measured at the receiving end. In this embodiment, the received signal strength indication can be obtained by using an existing radio frequency front-end power detection circuit. The received signal strength indication reflects the overall strength of the received signal and is an important indicator for judging the signal coverage and signal attenuation degree. The signal-to-noise ratio in this embodiment refers to the ratio of the received signal power to the noise power. In this embodiment, the signal-to-noise ratio can be obtained by the baseband processing unit through analyzing and calculating the received signal. The signal-to-noise ratio reflects the quality of the signal. The higher the signal-to-noise ratio, the less the signal is interfered by noise, and the higher the image transmission quality. The bit error rate in this embodiment refers to the proportion of the number of bits in error during transmission to the total number of transmitted bits. In this embodiment, the bit error rate can be obtained by the receiving end detecting errors in the received data or by estimating through sending a known sequence. The bit error rate reflects the reliability of data transmission. The lower the bit error rate, the fewer data transmission errors, and the higher the image transmission quality. The multipath delay spread in this embodiment refers to the maximum value or root mean square value of the time difference of the signal arriving at the receiving end through different propagation paths. In this embodiment, the multipath delay spread can be obtained by using channel estimation technology to analyze the received pilot signal or training sequence. The multipath delay spread reflects the time difference of the signal arriving at the receiving end through different paths. The larger the multipath delay spread, the more serious the signal interference, and the worse the image transmission quality.

[0037] In some preferred embodiments, the internal parameter information of the unmanned aerial vehicle (UAV) includes the remaining battery power of the UAV, CPU load, and memory usage rate. The remaining battery power of the UAV in this embodiment refers to the current power level of the UAV battery. The remaining battery power of the UAV reflects the ability of the UAV to continue working. In this embodiment, the remaining battery power of the UAV can be obtained by reading the data of the UAV battery management system. The CPU load in this embodiment refers to the workload of the central processing unit of the UAV. The CPU load reflects the ability of the UAV to process tasks. In this embodiment, the CPU load can be obtained by reading the performance data provided by the UAV operating system or monitoring software. The memory usage rate in this embodiment refers to the proportion of the random access memory of the UAV that is occupied. The memory usage rate reflects the data caching and processing ability of the UAV. In this embodiment, the memory usage rate can be obtained by reading the performance data provided by the UAV operating system or monitoring software.

[0038] As can be seen from the above, a method for transmitting images during the flight of a drone provided by this application first generates a comprehensive state index that can reflect the overall state of the drone by integrating electromagnetic environment information, channel state information, and internal parameter information of the drone. Then, based on this comprehensive index, a target transmission parameter combination of the dynamic comprehensive state index is generated. Finally, the images collected by the drone are encoded, encapsulated, modulated, and sent according to the target transmission parameter combination. That is, this application is equivalent to first understanding the interference situation of the environment where the drone is located, the current transmission link quality, and the state of the drone itself, and then adaptively adjusting the image transmission strategy according to the interference situation, the current transmission link quality, and the state of the drone itself. Therefore, even if the interference and channel attenuation during the flight of the drone are dynamic, this application can select an appropriate image transmission strategy according to the dynamically changing interference and channel attenuation for image transmission, so as to effectively solve the problems such as frame freezing, image frame loss, mosaic or color distortion in the image stream received by the ground station due to enhanced interference or channel attenuation, thereby effectively improving the quality of the image stream, and further effectively avoiding the situation where it is impossible to detect defects in time or misjudgment occurs due to the degradation of the image stream quality.

[0039] In a second aspect, as Figure 2 shown, this application also provides a system for transmitting images during the flight of a drone, which is used to transmit the images collected by the drone, and includes: An information acquisition module 1, which is used to acquire electromagnetic environment information, channel state information, and internal parameter information of the drone; A comprehensive state index acquisition module 2, which is used to generate a comprehensive state index according to the electromagnetic environment information, channel state information, and internal parameter information of the drone; A transmission parameter combination determination module 3, which is used to query a pre-constructed mapping relation table of state indexes and transmission parameter combinations according to the comprehensive state index to obtain a target transmission parameter combination. The target transmission parameter combination includes modulation order, forward error correction coding rate, transmission power, image frame rate, data packet size, image resolution, and compression ratio; An image processing module 4, which is used to encode, encapsulate, modulate, and send the images collected by the drone according to the target transmission parameter combination.

[0040] A system for transmitting images during the flight of a drone provided by this application includes an information acquisition module 1, a comprehensive state index acquisition module 2, a transmission parameter combination determination module 3, and an image processing module 4. The system for transmitting images during the flight of a drone provided in this embodiment is used to execute the steps in a method for transmitting images during the flight of a drone provided in the first aspect above. The principle of the system for transmitting images during the flight of a drone provided in this embodiment is the same as that of the method for transmitting images during the flight of a drone provided in the first aspect above, and will not be elaborated in detail here.

[0041] In some preferred embodiments, the process of querying a pre-constructed mapping relationship table of state indicators and transmission parameter combinations according to the comprehensive state indicator to obtain the target transmission parameter combination includes: B1. Query a pre-constructed mapping relationship table of state indicators and transmission parameter combinations according to the comprehensive state indicator to obtain a preliminary transmission parameter combination; B2. Analyze whether there is building occlusion based on the image collected by the UAV. If so, perform step B3; if not, use the preliminary transmission parameter combination as the target transmission parameter combination; B3. Obtain the current pose information and flight altitude information of the UAV, and predict the influence degree of building occlusion on image transmission according to the current pose information, flight altitude information of the UAV and the pre-constructed urban three-dimensional model to obtain the first image transmission influence degree; B4. Query a pre-constructed mapping relationship table of image transmission influence degree and transmission parameter combination adjustment strategy according to the first image transmission influence degree to obtain the first transmission parameter combination adjustment strategy, and then adjust the preliminary transmission parameter combination according to the first transmission parameter combination adjustment strategy to obtain the target transmission parameter combination.

[0042] As can be seen from the above, a method and system for image transmission during the flight of a UAV provided by this application first generates a comprehensive state indicator that can reflect the overall state of the UAV by integrating electromagnetic environment information, channel state information, and UAV internal parameter information, and then dynamically synthesizes the target transmission parameter combination based on this comprehensive indicator. Finally, the image collected by the UAV is encoded, encapsulated, modulated, and sent according to the target transmission parameter combination. That is, this application is equivalent to first understanding the interference situation in the environment where the UAV is located, the current transmission link quality, and the state of the UAV itself, and then adaptively adjusting the image transmission strategy according to the interference situation, current transmission link quality, and the state of the UAV itself. Therefore, even if the interference and channel attenuation suffered by the UAV during flight are dynamic, this application can select an appropriate image transmission strategy for image transmission according to the dynamically changing interference and channel attenuation, so as to effectively solve the problem that the image stream received by the ground station has problems such as picture jamming, image frame loss, picture mosaic, or color distortion due to increased interference or channel attenuation, thereby effectively improving the quality of the image stream, and further effectively avoiding the situation that defects cannot be detected in time or misjudgment occurs due to the decrease in the quality of the image stream.

[0043] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the above division of units is only a logical function division. In actual implementation, there may be other division methods. For another example, multiple units or components can be combined or integrated into another robot, or some features can be ignored or not executed. Another point, the displayed or discussed coupling or direct coupling or communication connection between each other can be through some communication interfaces. The indirect coupling or communication connection of devices or units can be in electrical, mechanical or other forms.

[0044] In addition, in each embodiment of the present application, the various functional modules can be integrated together to form an independent part, or each module can exist alone, or two or more modules can be integrated to form an independent part.

[0045] In this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0046] The above are only the embodiments of the present application and are not used to limit the protection scope of the present application. For those skilled in the art, the present application can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for transmitting images during the flight of an unmanned aerial vehicle, which is used to transmit the images collected by the unmanned aerial vehicle, and is characterized in that, The method for transmitting images during the flight of the unmanned aerial vehicle includes the following steps: S1. Obtain electromagnetic environment information, channel state information, and internal parameter information of the unmanned aerial vehicle; S2. Generate a comprehensive status index according to the electromagnetic environment information, the channel state information, and the internal parameter information of the unmanned aerial vehicle; S3. Query a pre-constructed mapping relation table of status index and transmission parameter combinations according to the comprehensive status index to obtain a target transmission parameter combination, where the target transmission parameter combination includes modulation order, forward error correction coding rate, transmission power, image frame rate, data packet size, image resolution, and compression ratio; S4. Encode, encapsulate, modulate, and transmit the images collected by the unmanned aerial vehicle according to the target transmission parameter combination.

2. The method for transmitting images during the flight of a drone according to claim 1, wherein Step S3 includes: S31. Query a pre-constructed mapping relation table of status index and transmission parameter combinations according to the comprehensive status index to obtain a preliminary transmission parameter combination; S32. Analyze whether there is building occlusion in the images collected by the unmanned aerial vehicle. If so, execute step S33; if not, use the preliminary transmission parameter combination as the target transmission parameter combination; S33. Obtain the current pose information and flight altitude information of the unmanned aerial vehicle, and predict the influence degree of building occlusion on image transmission according to the current pose information, the flight altitude information of the unmanned aerial vehicle, and a pre-constructed three-dimensional urban model to obtain a first image transmission influence degree; S34. Query a pre-constructed mapping relation table of image transmission influence degree and transmission parameter combination adjustment strategy according to the first image transmission influence degree to obtain a first transmission parameter combination adjustment strategy, and then adjust the preliminary transmission parameter combination according to the first transmission parameter combination adjustment strategy to obtain a target transmission parameter combination.

3. The method for transmitting images during the flight of a drone according to claim 2, wherein Step S31 includes: S311. Query a pre-constructed mapping relation table of status index and transmission parameter combinations according to the comprehensive status index to obtain an original transmission parameter combination; S312. Obtain real-time meteorological data, and then query a pre-constructed mapping relation table of meteorological data and transmission parameter combination adjustment strategy according to the real-time meteorological data to obtain a second transmission parameter combination adjustment strategy; S313. Adjust the original transmission parameter combination according to the second transmission parameter combination adjustment strategy to obtain a preliminary transmission parameter combination.

4. The method for transmitting images during the flight of a drone according to claim 3, wherein Step S313 includes: A1. Obtain the current task type executed by the unmanned aerial vehicle, and then query a pre-constructed mapping relation table of execution task type and transmission parameter combination adjustment strategy according to the current task type executed by the unmanned aerial vehicle to obtain a third transmission parameter combination adjustment strategy; A2. Adjust the original transmission parameter combination according to the second transmission parameter combination adjustment strategy and the third transmission parameter combination adjustment strategy to obtain a preliminary transmission parameter combination.

5. The method for transmitting images during the flight of a drone according to claim 1, characterized in that, Step S2 includes: S21. Obtain the current task type being executed by the drone, and then query the pre-constructed mapping relation table of task types and weighted weight combinations according to the current task type being executed by the drone, so as to respectively obtain the weighted weights corresponding to the electromagnetic environment information, the channel state information, and the internal parameters information of the drone. S22. Calculate the comprehensive state index according to the electromagnetic environment information and its corresponding weighted weight, the channel state information and its corresponding weighted weight, and the internal parameters information of the drone and its corresponding weighted weight.

6. The method for transmitting images during the flight of a drone according to claim 1, wherein The electromagnetic environment information includes the working frequency band and adjacent frequency bands of the drone, the spectrum occupancy, the interference signal frequency band, the interference signal bandwidth, and the interference signal strength.

7. The method for transmitting images during the flight of a drone according to claim 1, wherein The channel state information includes received signal strength indication, signal-to-noise ratio, bit error rate, and multipath delay spread.

8. The method for transmitting images during the flight of a drone according to claim 1, characterized in that, The internal parameters information of the drone includes the remaining battery power of the drone, CPU load, and memory usage rate.

9. An image transmission system during the flight of a drone, which is used to transmit the images collected by the drone, and is characterized in that The image transmission system during the flight of the drone includes: An information acquisition module, configured to acquire electromagnetic environment information, channel state information, and internal parameters information of the drone. A comprehensive state index acquisition module, configured to generate a comprehensive state index according to the electromagnetic environment information, the channel state information, and the internal parameters information of the drone. A transmission parameter combination determination module, configured to query the pre-constructed mapping relation table of state indexes and transmission parameter combinations according to the comprehensive state index, so as to obtain a target transmission parameter combination, where the target transmission parameter combination includes modulation order, forward error correction coding rate, transmission power, image frame rate, data packet size, image resolution, and compression ratio. An image processing module, configured to encode, encapsulate, modulate, and transmit the images collected by the drone according to the target transmission parameter combination.

10. The drone flight process image transmission system according to claim 9, characterized in that, The process of querying the pre-constructed mapping relation table of state indexes and transmission parameter combinations according to the comprehensive state index to obtain the target transmission parameter combination includes: B1. Query the pre-constructed mapping relation table of state indexes and transmission parameter combinations according to the comprehensive state index to obtain a preliminary transmission parameter combination. B2. Analyze whether there is building occlusion in the images collected by the drone. If so, execute step B3; if not, use the preliminary transmission parameter combination as the target transmission parameter combination. B3. Obtain the current pose information and flight altitude information of the drone, and predict the influence degree of building occlusion on image transmission according to the current pose information of the drone, the flight altitude information, and the pre-constructed 3D urban model, so as to obtain the first image transmission influence degree. B4. Query the pre-constructed mapping relation table of image transmission influence degree and transmission parameter combination adjustment strategy according to the first image transmission influence degree to obtain a first transmission parameter combination adjustment strategy, and then adjust the preliminary transmission parameter combination according to the first transmission parameter combination adjustment strategy to obtain the target transmission parameter combination.

Images