An airborne image processing method based on long-distance image transmission

Abstract

The application discloses an airborne image processing method based on long-distance image transmission, and relates to the technical field of image processing, and comprises the following steps: S1, collecting original images by using an airborne camera; S2, collecting image transmission link state parameters of the original images, and performing image airborne terminal preprocessing according to the image transmission link state parameters to obtain latest images and performing transmission; and S3, performing dynamic enhancement on the received latest images by using a ground receiving end. The application adopts block processing aiming at the heterogeneous characteristics of long-distance image transmission local interference, multipath fading and bandwidth fluctuation; and the processing is completed in combination with physical parameters of the image transmission link and content features of the images, so that the image processing always conforms to the actual interference state of the link.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and particularly relates to an airborne image processing method based on long-distance image transmission. BACKGROUND

[0002] With the rapid development of unmanned aerial vehicles and airborne investigation technology, the demand for long-distance image transmission is increasingly urgent. However, the transmission from the airborne device to the ground receiving end is easily affected by various factors: (1) channel level: there are problems such as multipath fading, electromagnetic interference (such as the same frequency interference of other wireless devices) and unstable channel gain; (2) transmission level: long distance leads to significant signal power attenuation, and transmission errors, image block loss or distortion are prone to occur. These problems directly lead to defects such as noise concentration, detail blur and texture distortion in the transmitted image, which seriously affects the subsequent analysis and application (such as target recognition and scene judgment) of the image.

[0003] In addition, the existing airborne image processing has poor adaptability: the existing airborne image processing mostly uses a fixed threshold denoising algorithm, which does not combine the real-time noise type of the image transmission link, resulting in excessive denoising (loss of details) or insufficient denoising (residual noise). SUMMARY

[0004] The present application is proposed to solve the above problems, and provides an airborne image processing method based on long-distance image transmission.

[0005] The technical scheme of the present application is: an airborne image processing method based on long-distance image transmission comprises the following steps:

[0006] S1, acquiring an original image by using an airborne camera;

[0007] S2, acquiring a state parameter of an image transmission link for transmitting the original image, and performing image airborne terminal preprocessing according to the state parameter of the image transmission link to obtain a latest image and transmit the latest image;

[0008] S3, performing dynamic enhancement on the received latest image by using a ground receiving end.

[0009] Further, S2 comprises the following sub-steps:

[0010] S21, performing block processing on the original image to obtain a plurality of image blocks;

[0011] S22, acquiring a state parameter of an image transmission link for transmitting the original image, and calculating a signal transmission interference value of the original image based on the plurality of image blocks;

[0012] S23, acquiring a state parameter of an image transmission link for a historical image, calculating a signal transmission interference value of each historical image, and setting a mean value of the signal transmission interference values of all historical images as a transmission threshold value;

[0013] S24, performing Gaussian kernel function weighted fusion processing on the image block when the signal transmission interference value is greater than the transmission threshold value; and performing mean filtering on the image block when the signal transmission interference value is less than or equal to the transmission threshold value.

[0014] The above further scheme has the beneficial effect that: in the present application, when the whole image data volume is extremely large, the transmission delay is high (such as > 200 ms under a 20 km link), which cannot meet the low delay requirements of real-time control of unmanned aerial vehicles and real-time early warning of security, and when the link bandwidth fluctuates (such as from 10 Mbps to 2 Mbps), the whole image compression rate needs to be greatly adjusted, which easily leads to excessive distortion of compression or waste of bandwidth redundancy, and the block transmission is a fine optimization of the whole image transmission. For blocks with serious interference, Gaussian kernel weighted fusion is used (to retain details while suppressing noise), and for blocks with slight interference, mean filtering is used (to efficiently denoise).

[0015] Further, S21 includes the following sub-steps:

[0016] S221, collecting a graph transmission link state parameter of the original image for transmission; wherein the graph transmission link state parameter includes an image block operating frequency point and a modulation phase;

[0017] S222, determining a transmission energy coefficient of the image block in a single block transmission duration based on the graph transmission link state parameter of the image block using a cosine function;

[0018] S223, inputting the inverse of the ratio between the minimum transmission energy coefficient and the maximum transmission energy coefficient into an exponential function as an interference indication weight of the original image;

[0019] S224, calculating a signal transmission interference value of the original image according to the interference indication weight of the original image.

[0020] The above further scheme has the beneficial effect that: in the present application, the image block operating frequency point and the modulation phase are the core of the link physical parameters. Long-distance image transmission often uses frequency hopping technology (dynamic switching of frequency points) and phase modulation (control of signal waveform), and these parameters directly determine the transmission characteristics of the signal in the channel. The transmission energy coefficient is essentially the aggregation of time-frequency energy, and the inverse of the minimum / maximum transmission energy coefficient is input into the exponential function to obtain the interference indication weight. The greater the energy fluctuation (the smaller the minimum / maximum ratio), the more unstable the energy distribution in the time-frequency domain of the link (such as energy sudden drop of some blocks due to frequency point conflict), and the more serious the interference.

[0021] Further, in S222, the transmission energy coefficient is expressed as:

[0022] ;

[0023] wherein, Indicates the first The operating frequency of each image block Indicates the first The modulation phase of each image block, This indicates the transmission power of the image transmission link. This indicates the number of image patches in the original image. Indicates the start time of image block transmission.

[0024] The beneficial effects of the above-mentioned further solutions are as follows: In the present invention, in anti-interference image transmission scenarios, in order to avoid dynamic interference (such as sudden electromagnetic signals and conflicts with devices on the same frequency), a set of available frequency points is usually preset (for example, dividing the 5.8GHz band into 8 clean sub-frequency points). Before transmission, each image block will be transmitted by selecting the frequency point with the least interference in the set through link interference detection (such as real-time monitoring of the interference power of each frequency point). A single image block uses only one frequency point in one transmission process and will not occupy multiple frequency points at the same time.

[0025] Furthermore, in S224, the signal transmission interference value of the original image The expression is:

[0026] ;

[0027] in, The interference indicator weights represent the original image. This indicates the transmission power of the interference signal. This represents the white Gaussian noise at the ground receiver. This represents the channel gain from the airborne terminal to the ground receiver. This indicates the transmission power of the image transmission link. This represents the channel gain of the interference signal.

[0028] The beneficial effects of the above-mentioned further solutions are as follows: In this invention, the transmit power refers to the signal power output by the airborne transmitter (such as a UAV image transmission radio), which directly determines the signal transmission distance and anti-interference capability; when an image uses a uniform transmit power, each image block can still have its own operating frequency and modulation phase. Traditional SINR formulas only consider interference power, neglecting the dynamic fluctuations of link energy. For example, two regions may have the same interference power, but due to different levels of link energy fluctuation, the actual perceived interference differs significantly. This formula can accurately distinguish this difference, making subsequent decisions regarding the comparison of interference values ​​with thresholds more accurate.

[0029] Furthermore, in S24, the Gaussian kernel weighted fusion processing of the image patches includes the following sub-steps:

[0030] S241. Acquire the gray-level co-occurrence matrix of the original image and the local gray-level co-occurrence matrix of each image block;

[0031] S242. Determine the entropy difference of the image blocks based on the gray-level co-occurrence matrix of each local gray-level matrix and the gray-level co-occurrence matrix of the original image;

[0032] S243. Perform clustering on all entropy differences to determine the clusters corresponding to the image patches;

[0033] S244. The ratio between the entropy difference of the image patch and the center value of the cluster to which the image patch belongs is used as the adjustment weight of the Gaussian kernel;

[0034] S245. The variance of the Gaussian kernel is optimized by adjusting the weights of the Gaussian kernel, and the optimized Gaussian kernel is used to complete the weighted fusion processing of the Gaussian kernel function on the image patch.

[0035] The beneficial effect of the above further scheme is that, in this invention, entropy measures the randomness of gray-level co-occurrence in GLCM. The ratio of the entropy difference of an image block to the value of its cluster center is used as the Gaussian kernel adjustment weight. The cluster center value is the average entropy difference of that type of block. The larger the ratio of the entropy difference to the cluster center, the more extreme the deviation of the block within the cluster, requiring a stronger Gaussian kernel fusion (e.g., increasing the variance to expand the fusion range). The Gaussian kernel variance is optimized by adjusting the weights to complete the weighted fusion of image blocks.

[0036] Furthermore, in S242, the absolute value of the difference between the entropy of each local gray-level co-occurrence matrix and the entropy of the gray-level co-occurrence matrix of the original image is calculated as the entropy difference of the image patch.

[0037] Furthermore, in S3, adaptive gamma correction is used to dynamically enhance the received latest image.

[0038] The beneficial effects of this invention are:

[0039] (1) In view of the heterogeneous characteristics of local interference, multipath fading and bandwidth fluctuation in long-distance image transmission, the present invention adopts block processing; the processing is completed by combining the physical parameters of the image transmission link and the content features of the image, so that the image processing always fits the actual interference state of the link.

[0040] (2) This invention is applicable to long-distance image transmission scenarios of airborne platforms such as UAVs, helicopters and fixed-wing aircraft, and realizes collaborative optimization of airborne lightweight preprocessing + link transmission + ground fine post-processing to improve image quality and real-time performance under long-distance image transmission. Attached Figure Description

[0041] Figure 1 This is a flowchart of an airborne image processing method based on long-distance image transmission. Detailed Implementation

[0042] The embodiments of the present application are further illustrated below with reference to the drawings.

[0043] As Figure 1 shown, the present application provides an airborne image processing method based on long-distance image transmission, comprising the following steps:

[0044] S1, collecting original images by using an airborne camera;

[0045] S2, collecting image transmission link state parameters of the original images, and performing image airborne terminal preprocessing according to the image transmission link state parameters to obtain the latest images and perform transmission;

[0046] S3, using a ground receiving end to perform dynamic enhancement on the received latest images.

[0047] In the embodiments of the present application, S2 comprises the following sub-steps:

[0048] S21, performing block processing on the original images to obtain a plurality of image blocks;

[0049] S22, collecting image transmission link state parameters of the original images, and calculating a signal transmission interference value of the original images based on the plurality of image blocks;

[0050] S23, obtaining image transmission link state parameters of historical images, calculating signal transmission interference values of the historical images, and setting a mean value of the signal transmission interference values of all the historical images as a transmission threshold;

[0051] S24, when the signal transmission interference value is greater than the transmission threshold, performing Gaussian kernel function weighted fusion processing on the image blocks; and when the signal transmission interference value is less than or equal to the transmission threshold, performing mean filtering on the image blocks.

[0052] In the present application, when the whole image data volume is extremely large, the transmission delay is high (such as >200ms under a 20km link), which cannot meet the low-latency requirements of real-time control of unmanned aerial vehicles and real-time early warning of security, etc. When the link bandwidth fluctuates (such as from 10Mbps to 2Mbps), the whole image compression rate needs to be greatly adjusted, which is easy to cause excessive distortion or bandwidth redundancy waste due to compression. Block transmission is a fine optimization of whole image transmission. For blocks with serious interference, Gaussian kernel weighted fusion is adopted (details are retained while noise is suppressed), and for blocks with slight interference, mean filtering is adopted (efficient denoising).

[0053] In the embodiments of the present application, S21 comprises the following sub-steps:

[0054] S221, collecting image transmission link state parameters of the original images; wherein the image transmission link state parameters comprise image block operating frequency points and modulation phases;

[0055] S222. Based on the image block image transmission link status parameters, the transmission energy coefficient of the image block during the single block transmission duration is determined using a cosine function.

[0056] S223. Input the negative of the ratio between the minimum transmission energy coefficient and the maximum transmission energy coefficient into the exponential function as the interference indicator weight of the original image.

[0057] S224. Calculate the signal transmission interference value of the original image based on the interference indication weight of the original image.

[0058] In this invention, the image block operating frequency and modulation phase are the core physical parameters of the link. Long-distance image transmission often employs frequency hopping technology (dynamic frequency switching) and phase modulation (control signal waveform), and these parameters directly determine the signal transmission characteristics in the channel. The transmission energy coefficient is essentially an aggregation of time-frequency domain energy. The inverse of the minimum / maximum transmission energy coefficient is input into an exponential function to obtain the interference indication weight. The greater the energy fluctuation (the smaller the minimum / maximum ratio), the more unstable the energy distribution in the time-frequency domain of the link (e.g., frequency conflicts causing a sudden drop in energy in some blocks), and the more severe the interference.

[0059] In this embodiment of the invention, in S222, the transmission energy coefficient The expression is:

[0060] ;

[0061] in, Indicates the first The operating frequency of each image block Indicates the first The modulation phase of each image block, This indicates the transmission power of the image transmission link. This indicates the number of image patches in the original image. Indicates the start time of image block transmission.

[0062] In this invention, in anti-interference image transmission scenarios, to avoid dynamic interference (such as sudden electromagnetic signals and conflicts with devices on the same frequency), a set of available frequency points is usually preset (for example, dividing the 5.8GHz band into 8 clean sub-frequency points). Before transmission, each image block will be transmitted using the frequency point with the least interference in the set through link interference detection (such as real-time monitoring of the interference power of each frequency point). A single image block uses only one frequency point in one transmission process and will not occupy multiple frequency points simultaneously.

[0063] In this embodiment of the invention, in S224, the signal transmission interference value of the original image The expression is:

[0064] ;

[0065] wherein, represents the interference indication weight of the original image, represents the transmission power of the interference signal, represents the white Gaussian noise of the ground receiving end, represents the channel gain from the airborne terminal to the ground receiving end, represents the transmission power of the image transmission link, represents the channel gain of the interference signal.

[0066] In the present application, the transmission power refers to the signal power output by the airborne transmitting end (such as a UAV image transmission radio), which directly determines the transmission distance and anti-interference capability of the signal; when a uniform transmission power is adopted for an image, each image block can still have its own working frequency point and modulation phase. The traditional SINR formula only considers the interference power and ignores the defect of dynamic fluctuation of link energy. For example, two regions can have the same interference power, but due to different degrees of fluctuation of link energy, the actual interference perception difference is significant, and the formula can accurately distinguish such difference, so that the subsequent decision of comparing the interference value with the threshold is more accurate.

[0067] In the embodiment of the present application, in S24, the Gaussian kernel function weighted fusion processing of the image blocks comprises the following sub-steps:

[0068] S241, acquiring the gray level co-occurrence matrix of the original image and the local gray level co-occurrence matrix of each image block;

[0069] S242, determining the entropy difference of the image block according to each local gray level co-occurrence matrix and the gray level co-occurrence matrix of the original image;

[0070] S243, performing clustering processing on all the entropy differences to determine the clustering cluster corresponding to the image block;

[0071] S244, taking the ratio between the entropy difference of the image block and the center value of the clustering cluster to which the image block belongs as the adjustment weight of the Gaussian kernel;

[0072] S245, optimizing the variance of the Gaussian kernel by using the adjustment weight of the Gaussian kernel, and completing the Gaussian kernel function weighted fusion processing of the image block by using the optimized Gaussian kernel.

[0073] In the present application, the entropy measures the randomness of gray level co-occurrence in GLCM. The ratio between the entropy difference of the image block and the center value of the cluster is taken as the adjustment weight of the Gaussian kernel. The cluster center value is the average entropy difference of the blocks in this category, and the greater the ratio between the entropy difference and the cluster center, the more extreme the deviation of the block in the cluster, and a stronger Gaussian kernel fusion (such as increasing the variance and expanding the fusion range) is needed. The adjustment weight is used to optimize the variance of the Gaussian kernel to complete the weighted fusion of the image block.

[0074] In the embodiment of the present application, in S242, the absolute value of the difference between the entropy of each local gray level co-occurrence matrix and the entropy of the gray level co-occurrence matrix of the original image is calculated as the entropy difference of the image block.

[0075] In the embodiment of the present application, in S3, the received latest image is dynamically enhanced by using adaptive gamma correction.

[0076] Those skilled in the art will understand that the embodiments described herein are for the purpose of understanding the principles of the present application and should be understood as not limiting the scope of protection of the present application to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations according to the technical inspiration disclosed in the present application without departing from the essence of the present application, and these modifications and combinations are still within the scope of protection of the present application.

Claims

1. An airborne image processing method based on long-range image transmission, characterized in that, The method comprises the following steps: S1, collecting original images by using an airborne camera; S2, collecting image transmission link state parameters of the original images, and performing image airborne terminal preprocessing according to the image transmission link state parameters to obtain latest images and perform transmission; S3, performing dynamic enhancement on the received latest images by using a ground receiving terminal; The S2 comprises the following sub-steps: S21, performing block processing on the original images to obtain a plurality of image blocks; S22, collecting image transmission link state parameters of the original images, and calculating a signal transmission interference value of the original images based on the plurality of image blocks; S23, acquiring image transmission link state parameters of historical images, calculating signal transmission interference values of the historical images, and setting a mean value of the signal transmission interference values of all the historical images as a transmission threshold value; S24, when the signal transmission interference value is greater than the transmission threshold value, performing Gaussian kernel function weighted fusion processing on the image blocks; and when the signal transmission interference value is less than or equal to the transmission threshold value, performing mean filtering on the image blocks.

2. The method of claim 1, wherein the method further comprises: The S22 comprises the following sub-steps: S221, collecting image transmission link state parameters of the original images; wherein the image transmission link state parameters comprise an image block operating frequency point and a modulation phase; S222, determining a transmission energy coefficient of the image block in a single block transmission duration by using a cosine function based on the image transmission link state parameters of the image block; S223, inputting an inverse number of a ratio between a minimum transmission energy coefficient and a maximum transmission energy coefficient into an exponential function as an interference indication weight of the original image; S224, calculating a signal transmission interference value of the original image according to the interference indication weight of the original image.

3. The method of claim 2, wherein the method further comprises: In the S222, the transmission energy coefficient The expression of the transmission energy coefficient is ； wherein, denotes the working frequency of the image block, denotes the modulation phase of the image block, denotes the transmission power of the image transmission link, denotes the number of image blocks of the original image, denotes the start transmission time of the image block.

4. The method of claim 2, wherein the method further comprises: In the S224, the signal transmission interference value of the original image The expression is: ； wherein, denotes an interference indication weight of the original image, denotes a transmit power of the interference signal, denotes a white Gaussian noise of the ground receiving end, denotes a channel gain from the airborne terminal to the ground receiving end, denotes a transmit power of the image transmission link, denotes a channel gain of the interference signal.

5. The method of claim 1, wherein, In the S24, the Gaussian kernel function weighted fusion processing on the image blocks comprises the following sub-steps: S241, collecting a gray level co-occurrence matrix of the original image and local gray level co-occurrence matrices of the image blocks; S242, determining an entropy difference of the image block according to the local gray level co-occurrence matrices and the gray level co-occurrence matrix of the original image; S243, performing clustering processing on all the entropy differences to determine a clustering cluster corresponding to the image block; S244, taking a ratio between the entropy difference of the image block and a center value of the clustering cluster to which the image block belongs as an adjustment weight of the Gaussian kernel; S245, optimizing a variance of the Gaussian kernel by using the adjustment weight of the Gaussian kernel, and completing the Gaussian kernel function weighted fusion processing on the image block by using the optimized Gaussian kernel.

6. The method of claim 5, wherein the method further comprises: In the S242, an absolute value of a difference between an entropy of each local gray level co-occurrence matrix and an entropy of the gray level co-occurrence matrix of the original image is taken as the entropy difference of the image block.

7. The method of claim 1, wherein the method further comprises: In the S3, the adaptive gamma correction is used to perform dynamic enhancement on the received latest images.

Images