Ocean front detection method based on infrared imaging of unmanned aerial vehicle and related equipment thereof

Abstract

The invention relates to an ocean front detection method based on infrared imaging of an unmanned aerial vehicle and related equipment thereof. The method comprises the steps that an initial infrared image of a target ocean area is acquired, the target ocean area comprises an ocean front area, and the infrared image is obtained through shooting based on infrared imaging equipment carried by an unmanned aerial vehicle; sequentially carrying out denoising, correction and image splicing processing on the infrared image to generate a target infrared image; feature enhancement processing is carried out on the ocean front area in the target infrared image, an enhanced infrared image is generated, and the enhanced infrared image comprises an ocean front candidate area; and detecting the ocean front in the ocean front candidate region based on a method of combining wavelet energy and an edge operator so as to determine the position and length characteristic parameters of the ocean front.

Description

Technical Field

[0001] The present application relates to the field of infrared images and ocean front technology, and in particular to an ocean front detection method based on unmanned aerial vehicle infrared imaging and related equipment. Background Art

[0002] In the field of ocean monitoring, the combination of UAV technology and remote sensing technology has developed rapidly. It has the advantages of low cost, strong flexibility and high timeliness, and provides important support for marine scientific research, climate change response and marine engineering. At the same time, infrared imaging is gradually being used in UAV ocean remote sensing due to its own advantages, and UAV infrared imagers are suitable for ocean temperature front detection.

[0003] However, infrared imagers have problems such as low contrast, unclear boundaries, and high noise. Ocean fronts have weak edges, unstable shapes, and low signal-to-noise. These factors lead to many difficulties in detecting ocean fronts in infrared images. In summary, the existing technology for detecting ocean fronts in infrared images has problems such as difficulty in detection and poor detection effect due to the limitations of infrared imagers themselves and the characteristics of ocean fronts. Summary of the invention

[0004] An ocean front detection method based on unmanned aerial vehicle infrared imaging and related equipment provided in an embodiment of the present invention at least solve the problem of difficult detection and poor detection effect in the related art in terms of ocean front detection in infrared images due to the limitations of the infrared imager itself and the characteristics of the ocean front.

[0005] According to a first aspect of an embodiment of the present invention, there is provided a method for detecting an ocean front based on infrared imaging of a drone, comprising:

[0006] Acquire an initial infrared image of a target ocean area, wherein the target ocean area includes an ocean front area, and the infrared image is obtained by taking an infrared imaging device carried by a drone;

[0007] The infrared image is subjected to denoising, correction and image stitching processing in sequence to generate a target infrared image;

[0008] Performing feature enhancement processing on the ocean front region in the target infrared image to generate an enhanced infrared image, wherein the enhanced infrared image includes the ocean front candidate region;

[0009] The ocean front in the ocean front candidate area is detected based on a method combining wavelet energy and edge operator to determine the position and length characteristic parameters of the ocean front.

[0010] According to a second aspect of an embodiment of the present invention, there is provided an ocean front detection device based on drone infrared imaging, comprising:

[0011] An acquisition module is used to acquire an initial infrared image of a target ocean area, wherein the target ocean area includes an ocean front area, and the infrared image is obtained by taking an infrared imaging device carried by a drone;

[0012] A processing module, used for performing denoising, correction and image stitching processing on the infrared image in sequence to generate a target infrared image;

[0013] A generating module, configured to perform feature enhancement processing on the ocean front region in the target infrared image to generate an enhanced infrared image, wherein the enhanced infrared image includes the ocean front candidate region;

[0014] The detection module is used to detect the ocean front in the ocean front candidate area based on a method combining wavelet energy and edge operator to determine the position and length characteristic parameters of the ocean front.

[0015] According to a third aspect of an embodiment of the present invention, there is provided an electronic device, comprising: a processor, and a memory storing a program, wherein the program comprises instructions, which, when executed by the processor, cause the processor to execute the method according to the first aspect.

[0016] According to a fourth aspect of an embodiment of the present invention, a non-transitory machine-readable medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the method according to the first aspect.

[0017] Beneficial effects of the embodiments of the present invention:

[0018] The embodiment of the present invention is based on an infrared imaging system carried by a drone, and aims at the characteristics of the weak edge, unstable morphology, small target features and the like of the ocean front, and provides an ocean front detection method based on drone infrared imaging. First, through the image preprocessing method of denoising, correction and splicing, the problems of low contrast, blurred visual effect and significant noise caused by equipment performance or imaging conditions in the infrared imaging process can be overcome. Then, the candidate area of ​​the ocean front is enhanced based on the feature enhancement method, thereby highlighting the characteristics of the ocean front and reducing the lack of significance and feature imbalance of the ocean front in a large area of ​​background. This process effectively solves the problem that the ocean front is difficult to distinguish in the infrared image, making the ocean front area more prominent in the enhanced image. Finally, the enhanced infrared image is accurately detected by combining the wavelet energy and edge operator methods. The application of wavelet energy further suppresses the background noise and reduces the interference of the background on the ocean front detection, while the edge operator accurately locates the ocean front and extracts the feature parameters through edge feature extraction. The combination of the two realizes the accurate detection and feature extraction of the ocean front area, and provides technical guarantee for the efficient acquisition of parameters such as the position and length of the ocean front. Based on the above method, it can effectively overcome the shortcomings of existing technologies in dealing with weak edges and small target features, significantly improve the accuracy and robustness of ocean front detection, and lay the foundation for the UAV infrared imaging system to realize automated and real-time monitoring of marine environmental phenomena.

[0019] The details of one or more embodiments of the invention are set forth in the following drawings and description so that other features, objects, and advantages of the invention are more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other embodiments can be obtained based on these drawings without creative work.

[0021] Figure 1 A flow chart of an ocean front detection method based on drone infrared imaging provided in an embodiment of the present invention.

[0022] Figure 2 A schematic diagram of an infrared image A provided by an embodiment of the present invention.

[0023] Figure 3 A schematic diagram of an infrared image B provided by an embodiment of the present invention.

[0024] Figure 4 A schematic diagram of an infrared image A after removing periodic stripe noise provided by an embodiment of the present invention.

[0025] Figure 5 A schematic diagram of an infrared image B after removing isolated point noise provided by an embodiment of the present invention.

[0026] Figure 6 A schematic diagram of an infrared image A after radiation direction identity correction provided by an embodiment of the present invention.

[0027] Figure 7 A schematic diagram of an infrared image B after radiation direction identity correction provided by an embodiment of the present invention.

[0028] Figure 8 A schematic diagram of an infrared image A provided in an embodiment of the present invention after removing the influence of resolution difference and wave tilt.

[0029] Fig. 9 A schematic diagram of an infrared image B provided by an embodiment of the present invention after removing the influence of resolution difference and wave tilt.

[0030] Fig.10 A schematic diagram of an infrared image A after geo-registration provided by an embodiment of the present invention.

[0031] Fig.11 A schematic diagram of an infrared image B after geo-registration provided by an embodiment of the present invention.

[0032] Fig.12 A schematic diagram of a stitched image provided by an embodiment of the present invention.

[0033] Fig.13 A schematic diagram of a spliced ​​image enhanced by bilateral filtering provided by an embodiment of the present invention.

[0034] Fig.14 A schematic diagram of a spliced ​​image enhanced by two-dimensional fast wavelet transform provided by an embodiment of the present invention.

[0035] Fig.15 A schematic diagram of a fused feature map of an ocean front candidate area provided in an embodiment of the present invention.

[0036] Fig.16 A schematic diagram of a fused feature map after Canny edge detection processing provided in an embodiment of the present invention.

[0037] Fig.17 A schematic diagram of a front line in an ocean front area provided in an embodiment of the present invention.

[0038] Fig.18 A schematic structural diagram of an ocean front detection device based on drone infrared imaging provided in an embodiment of the present invention.

[0039] Fig.19 A schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0040] Embodiments of the present embodiment will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present embodiment are shown in the accompanying drawings, it should be understood that the present embodiment can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein, which are instead provided for a more thorough and complete understanding of the present embodiment. It should be understood that the drawings and embodiments of the present embodiment are only for exemplary purposes and are not intended to limit the scope of protection of the present embodiment.

[0041] In the field of ocean monitoring, the combination of UAV technology and remote sensing technology has developed rapidly. It has the advantages of low cost, strong flexibility and high timeliness, and provides vital support for marine scientific research, climate change response, marine engineering and marine military. At the same time, infrared imaging has gradually been applied in the field of UAV marine remote sensing due to its own advantages, and UAV infrared imagers are more suitable for the detection of ocean temperature fronts. However, infrared imagers have problems such as low contrast, unclear boundaries and high noise. The ocean front has weak edges, unstable morphology and low signal-to-noise ratio. These factors make the detection of infrared image ocean fronts face many difficulties. The research and application of existing technologies in this area need to be further strengthened and deepened.

[0042] In order to solve the problems existing in the prior art, an embodiment of the present invention provides an ocean front detection method based on drone infrared imaging, so as to solve the problems in the related technology of infrared image ocean front detection, such as detection difficulties and poor detection effects caused by the limitations of the infrared imager itself and the characteristics of the ocean front.

[0043] Figure 1 The following is a flow chart of a method for detecting ocean fronts based on infrared imaging of unmanned aerial vehicles provided by an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:

[0044] Step S101, obtaining an initial infrared image of a target ocean area, wherein the target ocean area includes an ocean front area, and the infrared image is obtained by taking an infrared imaging device carried by a drone.

[0045] Step S102, performing denoising, correction and image stitching processing on the infrared image in sequence to generate a target infrared image.

[0046] Step S103, performing feature enhancement processing on the ocean front region in the target infrared image to generate an enhanced infrared image, wherein the enhanced infrared image includes the ocean front candidate region.

[0047] Step S104, detecting the ocean front in the ocean front candidate area based on a method combining wavelet energy and edge operator to determine the position and length characteristic parameters of the ocean front.

[0048] First, the acquisition of the initial infrared image is the starting step of the embodiment of the present invention, which aims to capture infrared image data of the target ocean area through the infrared imaging device carried by the drone. The target ocean area includes the ocean front area.

[0049] In this embodiment, the drone needs to be equipped with a highly sensitive infrared imaging device to support long-distance imaging and have the ability to resist environmental interference (such as strong winds and ocean water vapor). The target ocean area includes the ocean front area, and the shooting time can be selected at a time period that is conducive to capturing significant differences in the thermal gradient of the ocean front (such as early morning or sunset, etc.). Based on the above infrared imaging equipment and the target ocean area, an initial infrared image can be obtained.

[0050] Optionally, when photographing the target ocean area using an infrared imaging device carried by a drone, it is necessary to clarify the flight path and coverage of the drone, and it is also necessary to ensure that the captured images are continuous and covered without omissions.

[0051] After the initial infrared image is acquired, the infrared image can be subjected to denoising, correction and image stitching processing in sequence to generate a target infrared image.

[0052] In this embodiment, invalid information can be removed from the initial infrared image, errors caused by imaging angle, equipment characteristics, etc. can be corrected, and multiple images can be spliced ​​into a complete image.

[0053] Specifically, for periodic streak noise, specific frequency components can be detected and removed by frequency domain filtering methods (such as fast Fourier transform). For isolated point noise, median filtering or outlier detection methods based on pixel neighborhood statistics can be used. Based on the above denoising process, the distribution characteristics of the ocean front in the infrared image can be retained to avoid the loss of edge information due to excessive filtering.

[0054] Due to the change of the UAV's flight attitude and incident angle, the radiation intensity of different image areas may be different. The radiation intensity of the image can be unified through the row mean and full image mean correction methods. At the same time, the nearest neighbor interpolation and small window filtering can be used to adjust the resolution to meet the subsequent processing requirements.

[0055] In addition, multiple initial infrared images may have overlapping areas due to differences in shooting time and angle, and need to be processed by geographic coordinate alignment and pixel mean processing of overlapping areas to form a continuous target infrared image. In practical applications, window mean filtering can be used in the stitching process to smooth boundary transitions.

[0056] After generating the target infrared image, the ocean front area in the target infrared image can be feature enhanced to highlight the edge features of the ocean front and determine the ocean front candidate area in the generated enhanced infrared image to facilitate subsequent detection.

[0057] Finally, the ocean front in the candidate area is detected based on a method combining wavelet energy and edge operator to determine the position and length characteristic parameters of the ocean front.

[0058] In this embodiment, the ocean front candidate region in the enhanced infrared image is further analyzed to extract the specific position and length characteristic parameters of the ocean front.

[0059] Specifically, wavelet energy calculation is to perform multi-scale two-dimensional fast wavelet decomposition on the enhanced infrared image, calculate the wavelet energy of high-frequency components of different scales, and select the direction with the most significant wavelet energy for normalization processing. Among them, the area with high wavelet energy concentration usually corresponds to the significant edge of the ocean front area. Edge detection is based on the fusion processing of normalized wavelet energy and full-scale reconstructed enhanced data in a certain direction (horizontal direction, vertical direction or diagonal direction) of two-dimensional fast wavelet transform, and uses edge detection operator to extract the edge of the ocean front area. Furthermore, the front line can be further extracted through morphological processing (dilation, erosion, etc.), and its length and position can be calculated.

[0060] In an optional embodiment, the initial infrared image may include periodic stripe noise and isolated point noise. In this embodiment, when removing the periodic stripe noise and isolated point noise, the infrared image may be denoised based on the preset denoising processing methods corresponding to the periodic stripe noise and the isolated point noise, respectively, to generate a denoised infrared image.

[0061] Infrared imaging equipment is affected by many factors in practical applications, resulting in periodic stripe noise and isolated point noise in the collected infrared images. These noises not only interfere with the overall quality of the image, but also have an adverse effect on the detection results of ocean front features, especially in the extraction of weak feature ocean fronts. Therefore, in order to improve the detection effect, it is necessary to remove the periodic stripe noise and isolated point noise in the initial infrared image.

[0062] Specifically, infrared imaging devices are prone to periodic stripe noise in images due to the inherent characteristics of imaging sensors or data reading processes. This stripe noise is regularly distributed in a specific direction and usually covers multiple rows of pixels, causing periodic fluctuations in the grayscale value of local areas of the image, interfering with subsequent feature extraction.

[0063] To suppress this type of periodic stripe noise, the average of the three adjacent non-abnormal rows can be used to replace the value of the periodic stripe row. Specifically, the row containing stripe noise can be detected first, and the abnormal row can be replaced by calculating the average of the normal pixel rows adjacent to it, thereby effectively smoothing the periodic stripe noise while retaining the detailed information of the infrared image.

[0064] In addition, isolated point noise in infrared images is usually caused by high temperature heat sources of ships on the sea surface, weather conditions (such as local high-reflection areas caused by sunlight reflection or precipitation), equipment circuit interference, etc. These isolated points usually appear as isolated bright or dark spots in the image, which seriously affects subsequent image stitching and feature detection.

[0065] For isolated point noise, the specific location can be determined by using a method based on image gradient. Because the pixel values ​​of isolated points are significantly different from those of surrounding pixels, they will appear as abnormal gradient change points in the image gradient. After the location of the isolated point is detected, it can be filled with the mean value of nearby pixels to replace the abnormal pixel value, thereby effectively suppressing the isolated point noise and restoring the continuity of the image.

[0066] The combined application of the above two methods can significantly improve the quality of infrared images and provide high-precision basic data support for subsequent ocean front detection. Specifically, the above noise suppression method can not only effectively remove the interference of periodic streak noise and isolated point noise on infrared images, but also enhance the detail expression of the ocean front area while retaining the effective information of the image, ensuring the accuracy and robustness of subsequent detection.

[0067] In an optional embodiment, after the initial infrared image is subjected to denoising processing to obtain the denoised infrared image, the denoised infrared image may be subjected to correction processing.

[0068] In this embodiment, the mean value of each row of pixels and the mean value of all pixels in the denoised infrared image can be removed in sequence to correct the difference in radiation intensity in the denoised infrared image caused by different incident angles; and the denoised infrared image can be processed in sequence by a nearest neighbor interpolation processing method and a small window filtering processing method to unify the resolution of the denoised infrared image and generate a corrected infrared image.

[0069] Specifically, in UAV infrared imaging, due to the lens design of the imaging device and the limitations of the shooting angle, the radiation energy return values ​​of the central area and the edge area of ​​the image will be different. This difference is mainly caused by the change of the incident angle of the optical system, which destroys the consistency of the image in the radiation direction, thereby reducing the image quality. In order to solve this problem, an embodiment of the present invention provides a radiation direction identity correction method based on "row mean removal" and "full image mean removal".

[0070] In practical applications, for the denoised infrared image f(k,l), the correction process includes the following steps.

[0071] Row mean removal steps:

[0072] Calculate the mean of each row of pixels and subtract the mean of the corresponding row from each pixel value. The calculation method is shown in the following formula (1):

[0073]

[0074] Where f(k,l) is the gray value or relative brightness temperature of the pixel at the kth row and the lth column of the denoised infrared image; is the average grayscale value or relative brightness temperature value of all pixels in the kth row; f′(k,l) is the grayscale value or relative brightness temperature value of the pixel after the row mean is removed.

[0075] By subtracting the average value of the corresponding row from each pixel value, the problem of inconsistent brightness between rows caused by differences in incident angles is eliminated, making the radiation intensity distribution of each row more uniform.

[0076] Steps for removing the mean of the entire image:

[0077] For the image after row mean removal, the mean of all pixels in the image is further calculated and subtracted from each pixel value. The calculation method is shown in the following formula (2):

[0078]

[0079] Among them, f′(k,l) is the gray value of the pixel after the row mean is removed; is the average grayscale value of the pixels in the entire image; f”(k,l) is the grayscale value of the pixels after the whole image mean is removed.

[0080] The whole-image mean removal step can further balance the brightness distribution of the whole image and eliminate the global radiation deviation caused by light source non-uniformity or other system errors.

[0081] Through the processing of the above method, the inconsistency in the radiation direction is significantly improved, thereby enhancing the detectability of image features and the reliability of data.

[0082] In addition, when the UAV performs infrared imaging at different flight altitudes, the acquired infrared images usually have resolution differences due to the height difference and the resolution limitation of the imaging device. At the same time, the wave tilt on the ocean surface will introduce complex noise into the infrared image, thereby interfering with the subsequent feature detection. Therefore, the embodiment of the present invention adopts a combined method of unified resolution processing and wave tilt noise suppression to reduce the impact of the above problems on feature extraction.

[0083] In practical applications, when the resolution is uniformly processed, the nearest neighbor interpolation method can be used to uniformly adjust the resolution of the denoised infrared image. The specific calculation method is shown in the following formula (3) and formula (4):

[0084]

[0085]

[0086] Among them, srcX and srcY are the pixel coordinates in the denoised infrared image; dstX and dstY are the pixel coordinates in the corrected infrared image; srcWidth and srcHeight are the width and height of the denoised infrared image respectively; dstWidth and dstHeight are the width and height of the corrected infrared image respectively.

[0087] Through the nearest neighbor interpolation method, images of different resolutions can be quickly adjusted to a uniform scale, providing a basis for subsequent feature enhancement and detection.

[0088] Furthermore, a small window filter can be used to process the image after the resolution is unified. The small window filter effectively suppresses the high-frequency noise caused by the tilt of the waves by calculating the weighted mean of the pixel values ​​in the local window of the image, while retaining the edge and texture features in the image. After filtering, the uneven brightness and darkness problem caused by the movement of the waves in the image is significantly improved, thus laying a more stable data foundation for subsequent feature detection and analysis.

[0089] Based on the above method, the brightness consistency of the image can be improved through radiation direction correction, making the contrast of the features of the ocean front area more obvious; the noise caused by the difference in radiation direction is eliminated, and the robustness of the detection algorithm is enhanced. In addition, through unified resolution processing, the consistency of data under different imaging conditions is ensured, providing standardized input for subsequent image stitching and feature extraction; the small window filter removes the influence of wave tilt while retaining the edge features of the ocean front, enhancing the detection accuracy.

[0090] In an optional embodiment, after the denoised infrared image is corrected to obtain the corrected infrared image, the corrected infrared image can be stitched. In this embodiment, geographic alignment is achieved through bilinear interpolation, and the stitching area is processed by averaging and boundary filtering to generate a target infrared image with continuity and consistency.

[0091] The original infrared data taken by the drone only records the geographic coordinates of the image corners, image width, height and resolution. Therefore, in order to match the image data with the characteristics of the ocean phenomenon, it is necessary to calculate the geographic coordinates of any pixel in the image to complete the geo-registration.

[0092] Geo-registration is to accurately match the infrared images taken by drones with the geographic distribution of actual ocean phenomena. The original infrared image data collected by drones only contains the geographic coordinate values ​​of the four corners of the image (i.e. the latitude and longitude of the upper left corner, upper right corner, lower right corner, and lower left corner), as well as the width, height, and resolution information of the image. Since these data are not sufficient to directly match the spatial distribution characteristics of ocean phenomena, it is necessary to estimate the geographic coordinates of any point on the image through interpolation calculation.

[0093] Specifically, multiple corrected infrared images can be processed based on a bilinear interpolation method to generate first intermediate infrared images corresponding to each of the multiple corrected infrared images, wherein the multiple corrected infrared images are shot at the same time, have continuous start and end times, and have overlapping corner point geographic coordinates, and any point in the multiple first intermediate infrared images is mapped to a unified geographic coordinate system.

[0094] The key to geo-registration is to calculate the geographic coordinates of any pixel point based on the known geographic coordinates of the four corner points of the image through the bilinear interpolation method. The specific formula is as follows:

[0095] Horizontal interpolation:

[0096] Interpolate the pixel points at the upper left (x1, y1) and upper right (x2, y1) of the image and calculate the geographic coordinates in the row direction based on the following formula (5):

[0097]

[0098] Interpolate the pixel points at the lower left (x1, y2) and lower right (x2, y2) of the image and calculate the geographic coordinates in the row direction based on the following formula (6):

[0099]

[0100] Vertical interpolation:

[0101] According to the interpolation results in the row direction, the column direction is interpolated based on the following formula (7) to obtain the geographic coordinates of any point (x, y):

[0102]

[0103] The geo-referencing process converts the image coordinates of each pixel in the original data into geographic coordinates, generates a unified geo-referenced image, and provides support for subsequent stitching processing.

[0104] Since the coverage of a single drone infrared image is limited and the ocean front is usually distributed in strips, multiple images need to be stitched into a complete image to achieve comprehensive coverage of the ocean front area.

[0105] Specifically, the pixel values ​​of the overlapping areas of the plurality of first intermediate infrared images may be averaged and fused to generate a second intermediate infrared image; and then, the boundaries of the stitching areas between adjacent first intermediate infrared images may be windowed and averaged to generate a target infrared image. In this embodiment, the overlapping areas of the plurality of first intermediate infrared images are determined based on geographic coordinates.

[0106] In practical applications, we first need to select continuous images with start and end times within the same day as stitching objects based on the drone shooting time, and at the same time ensure that the geographic coordinates of the corner points of the stitched images have a certain overlapping range.

[0107] Based on the image data completed by geo-reference, the overlapping area of ​​the two images is determined by searching the geographic coordinates. The pixel values ​​in the overlapping area can be processed in the following two ways: for pixels with completely consistent geographic coordinates, the average of the corresponding pixel values ​​of the two images is directly taken as the splicing pixel value; or, for pixels with similar but not completely consistent geographic coordinates, the nearest neighbor interpolation method is used for numerical calculation.

[0108] After performing mean processing on the pixel values ​​of the overlapping area, a second intermediate infrared image is generated, and then the second intermediate infrared image can be subjected to image stitching processing.

[0109] During the stitching process, since there may be discontinuities in the radiation values ​​at the boundaries of different images, it is necessary to smooth the stitching boundaries by filtering. This embodiment may use a 3×3 window mean filtering algorithm to adjust the mutation value of the stitching area to the mean of the surrounding pixel values.

[0110] The image stitching method of the embodiment of the present invention is based on geographic registration data, uses bilinear interpolation to calculate the geographic coordinates of pixel points, determines the stitching area in combination with image time and geographic overlap information, fuses pixel values ​​through mean and interpolation algorithms, and smoothes the stitching boundaries to generate a stitching image (i.e., a target infrared image) with continuous geographic coordinate mapping, significantly reduces the boundary effect of the stitching area, and at the same time improves the continuity and recognizability of the ocean front area in the image.

[0111] In an optional embodiment, after the target infrared image is generated, feature enhancement processing may be performed on the ocean front region in the target infrared image to generate an enhanced infrared image.

[0112] First, the target infrared image can be processed by bilateral filtering to enhance the edge features of the ocean front area. Bilateral filtering includes weighted average methods based on spatial domain kernel and range kernel. The spatial domain kernel calculates the weight coefficient based on the Euclidean distance between pixels, and the range kernel calculates the weight coefficient based on the difference in pixel values.

[0113] Then, a two-dimensional fast wavelet transform is applied to the target infrared image after bilateral filtering, and the high-frequency components containing the characteristics of the ocean front are selected for enhancement. The high-frequency components include components in the horizontal, vertical or diagonal directions. The wavelet transform is based on the Haar wavelet to reconstruct the target infrared image at all scales in the horizontal, vertical or diagonal directions to obtain the enhanced ocean front signal.

[0114] In this embodiment, bilateral filtering is a nonlinear filtering method that considers both spatial domain and pixel value domain characteristics, and can effectively maintain image edge features while removing noise and achieving smoothing. Its core is to calculate weight coefficients by combining the geometric distance between pixels and the pixel value difference, perform weighted averaging on neighboring pixels, and output enhanced pixel values.

[0115] Specifically, the pixel value q(i, l) at the position (i, j) after filtering can be calculated based on the following formula (8):

[0116]

[0117] Among them, f(k,l) represents the pixel value in the neighborhood; W(i,j,k,l) ​​is the weight coefficient, which reflects the relative influence between pixels; the weight coefficient is determined by the spatial domain kernel G s and range kernel G r The product of is determined.

[0118] Spatial Domain Kernel G s The geometric distance used to characterize the pixel position is calculated based on Gaussian distribution. The calculation method is shown in the following formula (9):

[0119]

[0120] Among them, (i, j) and (k, l) are the pixel coordinates of the center point and the neighborhood point respectively, σ d Controls the decay rate of the spatial domain weights.

[0121] Range kernel G r It is used to characterize the influence of pixel value difference on weight, and is calculated based on Gaussian distribution. The calculation method is shown in the following formula (10):

[0122]

[0123] Among them, f(i,j) and f(k,l) are the pixel values ​​of the center point and the neighborhood point respectively, σ r Controls the rate at which the range weight decays.

[0124] Combined with the spatial domain kernel G s and range kernel G r , the calculation method of the weight coefficient W(i,j,k,l) ​​is shown in the following formula (11):

[0125] W(i,j,k,l)=G s (i,j,k,l)*G r (i,j,k,l) ​​(11)

[0126] The edge preservation characteristics of bilateral filtering can significantly enhance the edge features of the ocean front area and smooth the background noise. Compared with traditional filtering methods, such as Gaussian filtering, it is better at preserving the edge transitions of the ocean front area, ensuring the accuracy of subsequent ocean front feature extraction.

[0127] The ocean front region shows obvious edge or gradient jumps in the image due to the sudden changes in physical quantities such as temperature and salinity. These jump characteristics are mainly concentrated in the high-frequency components in the horizontal (H) and vertical (V) directions of the wavelet transform. By reconstructing these components at full scale, the ocean front signal can be enhanced and irrelevant information can be suppressed.

[0128] Two-dimensional fast wavelet transform can decompose an image into a low-frequency component and three high-frequency components (horizontal, vertical, and diagonal). The decomposition formula is as follows:

[0129]

[0130] in, is the low-frequency coefficient, describing the smoothing information; It is a high-frequency coefficient, describing the edge and detail information of the image; i corresponds to the horizontal (H), vertical (V), and diagonal (D) directions; is a two-dimensional scaling function; Ψ i (x,y) is a two-dimensional wavelet function.

[0131] The low frequency coefficient is calculated based on the following formula (13):

[0132]

[0133] The high frequency coefficient is calculated based on the following formula (14):

[0134]

[0135] Select the high-frequency component in the horizontal (H) or vertical (V) direction and reconstruct the image according to the full-scale expansion formula:

[0136]

[0137] In the reconstructed enhanced infrared image f′(x,y), the edge signal of the ocean front is significantly enhanced. The area where the ocean front exists in the enhanced infrared image is the candidate area of ​​the ocean front.

[0138] The multi-scale analysis capability of wavelet transform can effectively capture the edge jump characteristics of the ocean front, and enhance the image details through high-frequency component reconstruction. While suppressing background noise, the signal strength and significance of the ocean front are improved.

[0139] In an optional embodiment, when detecting the ocean front in the ocean front candidate area based on a method combining wavelet energy and edge operator to determine the position and length characteristic parameters of the ocean front, the following steps may be performed:

[0140] Firstly, based on the wavelet coefficients of the high-frequency components at each scale, the wavelet energies in the horizontal direction, the vertical direction and the diagonal direction can be calculated respectively.

[0141] In this embodiment, the wavelet coefficients corresponding to the high-frequency components of each scale are obtained as basic data for characterizing the characteristics of the local area. In practical applications, the wavelet energy in the horizontal, vertical and diagonal directions can be calculated according to the square formula of the wavelet coefficients, and the calculated energy values ​​reflect the significance of the edge or texture information of the image in each direction.

[0142] Then, the wavelet energy in the target direction is normalized to generate normalized wavelet energy, wherein the target direction is the direction in which the ocean front wavelet energy is most significant among the horizontal direction, the vertical direction and the diagonal direction.

[0143] In this embodiment, the target direction refers to the wavelet energy direction that best reflects the characteristics of the ocean front in the horizontal direction, vertical direction, and diagonal direction. The wavelet energy value in this direction is not only large in actual scenes, but also significantly conforms to the edge characteristics of the ocean front, and is clearly distinguished from interference noise or other irrelevant features in the environment.

[0144] In practical applications, when determining the target direction, it is necessary to exclude the wavelet energy of the interference data. Specifically, the wavelet energy value of the high-frequency component of the interference data (such as texture noise and complex background areas) may be large, but its distribution and change pattern are inconsistent with the characteristics of the ocean front edge, for example: the distribution is irregular and presents random fluctuations; the spatial continuity is poor and it is difficult to form a complete front line shape. Such interference directions should be excluded during the selection process to avoid being misjudged as target directions simply because of large values.

[0145] The significance of the wavelet energy of the ocean front is specifically reflected in the following aspects: it has a certain continuity in space, presenting a linear or curved structure; the energy value shows a significant peak along the front direction; the wavelet energy in the corresponding direction maintains a strong trend at multiple scales, rather than an abnormal increase at a single scale.

[0146] Combined with the above significant features, when selecting the target direction, we can compare the wavelet energy distribution and change trend in each direction, and give priority to the direction that best matches the above features.

[0147] The wavelet energy of the high-frequency component is normalized to generate normalized wavelet energy, which can be used to reduce the differences between scales and directions and ensure that features in different directions can participate in subsequent fusion calculations fairly.

[0148] Then the normalized wavelet energy can be fused with the ocean front candidate area data to obtain the fused features.

[0149] In this embodiment, the fusion feature F(x, y) is combined with the normalized wavelet energy E N The feature map generated by the pixel value f′(x,y) of the ocean front candidate region is used to integrate the significance of wavelet energy enhancement into the ocean front candidate region data, thereby strengthening the characteristics of the ocean front region.

[0150] Specifically, the fusion feature can be calculated based on the following formula (16):

[0151] F(x,y)=E N ×f′(x,y) (16)

[0152] Then, based on the fusion features, the preset edge detection operator is applied to perform edge detection in the ocean front area and the edge detection results are output.

[0153] In this embodiment, the preset edge detection operator may include a canny edge detection operator. The canny edge detection operator has high noise resistance and high detection accuracy. The edge position is located by calculating the gradient of the image pixel value change, and the false edges are removed by non-maximum suppression and double threshold processing.

[0154] Finally, the edge detection results are morphologically processed to extract the front line of the ocean front, and the position and length characteristic parameters of the ocean front are determined according to the position and length of the front line.

[0155] In this embodiment, the image processing method based on mathematical morphology generally includes corrosion, expansion, opening operation and closing operation, which is used to repair small discontinuities or noise points in the detection results and refine the shape of the front line. Among them, the position of the front line can be determined according to the longitude and latitude coordinates of the pixel points of the ocean front line, and the length of the front line can be calculated by counting the number of pixels of the front line and combining the image resolution to calculate the actual distance.

[0156] The following describes an ocean front detection method based on drone infrared imaging provided by an embodiment of the present invention in conjunction with specific embodiments.

[0157] In this embodiment, two infrared ocean front images collected by drones are used as research objects. Through preprocessing, feature enhancement, wavelet energy and edge operator combined methods, ocean front features are detected and its front position and length parameters are extracted. In this embodiment, it is assumed that the two infrared ocean front images of drones include infrared image A and infrared image B, where the schematic diagram of infrared image A is as follows: Figure 2 As shown, the schematic diagram of the infrared image B is as follows Figure 3 shown.

[0158] The following are the specific implementation steps:

[0159] Step 1: Remove periodic stripe noise

[0160] Due to the characteristics of infrared imaging equipment, there is periodic stripe noise in the image, which affects the feature extraction effect. This embodiment removes periodic stripes through the following steps:

[0161] 1) Locate the row where the periodic stripe noise is located;

[0162] 2) Use the average of the three rows of normal data before and after the stripe to replace the stripe noise row;

[0163] 3) Return the processed image, preserving the overall feature distribution of the image.

[0164] In this embodiment, the schematic diagram of the infrared image A after removing the periodic stripe noise is as follows: Figure 4 shown.

[0165] Step 2: Remove isolated point noise

[0166] Due to equipment interference and weather conditions, isolated point noise with sudden changes in grayscale values ​​appears in the image. Removal steps:

[0167] 1) Locate isolated points with significant grayscale changes by calculating the image gradient value;

[0168] 2) Replace the isolated point with the mean gray value of the 3×3 window near the isolated point.

[0169] In this embodiment, the schematic diagram of the infrared image B after removing isolated point noise is as follows: Figure 5 shown.

[0170] Step 3: Radiation direction consistency correction

[0171] Due to different incident angles during drone photography, the radiation intensity at the center and edge of the image is inconsistent. Correction method:

[0172] 1) Subtract the mean value from each row of pixels;

[0173] 2) Subtract the global mean from all pixels in the image.

[0174] In this embodiment, the schematic diagram of the infrared image A after the radiation direction identity correction is as follows: Figure 6 As shown; the schematic diagram of the infrared image B after the radiation direction identity correction is as follows Figure 7 shown.

[0175] Step 4: Remove the effects of resolution differences and wave tilt

[0176] The difference in the flying altitude of the drones leads to inconsistent image resolution, and the tilt of the waves produces high-frequency noise. This embodiment adopts:

[0177] 1) The nearest neighbor interpolation method is used to unify the resolution (adjusted to 3m);

[0178] 2) Small window filter suppresses the wave tilt noise.

[0179] In this embodiment, the schematic diagram of the infrared image A after removing the resolution difference and the influence of the wave tilt is as follows: Figure 8 As shown; the schematic diagram of infrared image B after removing the resolution difference and the influence of wave tilt is as follows Fig. 9 shown.

[0180] Step 5: Georeferencing and image stitching

[0181] There are overlapping areas in the geographic coordinates of the two drone infrared images. The following steps are used to achieve georeferencing and stitching:

[0182] 1) Based on the geographic coordinates of the four corners of the image, bilinear interpolation is used to calculate the longitude and latitude of each pixel;

[0183] 2) Take the average of the pixel values ​​in the overlapping area and perform 3×3 window mean filtering on the stitching boundary.

[0184] In this embodiment, the schematic diagram of the infrared image A after geo-registration is as follows: Fig.10 As shown; the schematic diagram of infrared image B after georeferencing is as follows Fig.11 As shown; the schematic diagram of the spliced ​​image after the infrared image A and the infrared image B are spliced ​​is as shown Fig.12 shown.

[0185] Step 6: Bilateral filtering enhancement

[0186] The bilateral filtering method that considers both the spatial domain kernel and the range kernel is adopted to preserve the edge characteristics of the ocean front and smooth the background noise:

[0187] 1) Calculate the spatial domain kernel weight based on the geometric distance between pixels;

[0188] 2) Calculate the value range kernel weight based on pixel grayscale difference;

[0189] 3) Combine the two weighted outputs to enhance the image.

[0190] In this embodiment, the schematic diagram of the stitched image after bilateral filtering enhancement is as follows: Fig.13 shown.

[0191] Step 7: Two-dimensional fast wavelet transform enhancement

[0192] Based on the Haar wavelet basis, a two-dimensional fast wavelet transform with two scales is used to further process the enhanced image:

[0193] 1) Extract high-frequency components in the horizontal (H), vertical (V) and diagonal (D) directions;

[0194] 2) Full-scale reconstruction of an image in a certain direction (the horizontal direction is taken as an example in this embodiment) to enhance the edge transition signal of the ocean front.

[0195] In this embodiment, the schematic diagram of the H direction of the spliced ​​image after being enhanced by two-dimensional fast wavelet transform is as follows: Fig.14 shown.

[0196] Step 8: Wavelet energy calculation

[0197] 1) Based on the wavelet coefficients of the high-frequency components of each scale, the wavelet energy in the horizontal, vertical and diagonal directions is calculated respectively;

[0198] 2) Normalize the wavelet energy in the direction where the ocean front wavelet energy is most significant in the horizontal, vertical and diagonal directions to generate normalized wavelet energy.

[0199] Step 9: Feature Fusion

[0200] The normalized wavelet energy is fused with the ocean front candidate area data enhanced by two-dimensional fast wavelet transform to generate a fused feature map.

[0201] In this embodiment, the H-direction fusion feature map of the ocean front candidate region is as follows: Fig.15 shown.

[0202] Step 10: Canny edge detection

[0203] The Canny edge detection operator (with the upper and lower thresholds set to 0.001 and 0.05) is applied to the fused feature map to extract the edge of the ocean front area.

[0204] In this embodiment, the schematic diagram of the fused feature map after Canny edge detection processing is as follows: Fig.16 shown.

[0205] Step 11: Morphological processing

[0206] The edge detection results are morphologically processed (dilation, erosion, and removal of small area noise) to generate a complete ocean front area and front line.

[0207] In this embodiment, the front line schematic diagram of the ocean front area is as follows: Fig.17 shown.

[0208] Step 12: Extraction of forward feature parameters

[0209] The position of the front line is determined according to the geographic coordinates of the pixel position; the length of the front line is calculated by counting the number of pixels and combining the image resolution.

[0210] The comparative experimental results show that this method can effectively suppress background noise and significantly improve the accuracy and robustness of ocean front detection by combining wavelet energy and edge operator. The extraction results of ocean front position and length parameters can be used to support climate change research, fishery resource assessment, marine ecological protection and other applications.

[0211] Based on the above-mentioned ocean front detection method based on drone infrared imaging provided by the embodiment of the present invention, the embodiment of the present invention also provides an ocean front detection device based on drone infrared imaging, such as Fig.18 As shown, the device includes an acquisition module 1801, a processing module 1802, a generation module 1803 and a detection module 1804.

[0212] The acquisition module 1801 is used to acquire an initial infrared image of a target ocean area, where the target ocean area includes an ocean front area. The infrared image is obtained by taking pictures using an infrared imaging device carried by a drone.

[0213] The processing module 1802 is used to perform denoising, correction and image stitching processing on the infrared image in sequence to generate a target infrared image.

[0214] The generating module 1803 is used to perform feature enhancement processing on the ocean front region in the target infrared image to generate an enhanced infrared image, wherein the enhanced infrared image includes the ocean front candidate region.

[0215] The detection module 1804 is used to detect the ocean front in the ocean front candidate area based on a method combining wavelet energy and edge operator to determine the position and length characteristic parameters of the ocean front.

[0216] According to an embodiment of the present invention, the initial infrared image includes periodic stripe noise and isolated point noise.

[0217] Accordingly, the processing module 1802 is specifically configured to perform denoising processing on the infrared image based on the preset denoising processing methods corresponding to the periodic stripe noise and the isolated point noise, respectively, to generate a denoised infrared image.

[0218] According to an embodiment of the present invention, the processing module 1802 is specifically used to sequentially remove the mean value of each row of pixels and the mean value of all pixels in the denoised infrared image to correct the difference in radiation intensity caused by different incident angles in the denoised infrared image; and sequentially process the denoised infrared image through a nearest neighbor interpolation processing method and a small window filtering processing method to unify the resolution of the denoised infrared image and generate a corrected infrared image.

[0219] According to an embodiment of the present invention, the processing module 1802 is specifically used to process multiple corrected infrared images based on a bilinear interpolation method to generate first intermediate infrared images corresponding to each of the multiple corrected infrared images, wherein the shooting time of the multiple corrected infrared images is the same, the start and end times are continuous, and the geographical coordinates of the corner points overlap, and any point in the multiple first intermediate infrared images is mapped to a unified geographical coordinate system; mean fusion processing is performed on the pixel values ​​of the overlapping areas of the multiple first intermediate infrared images to generate a second intermediate infrared image, and the overlapping areas of the multiple first intermediate infrared images are determined based on the geographical coordinates; and window mean filtering is performed on the boundaries of the stitching areas between adjacent first intermediate infrared images to generate a target infrared image.

[0220] According to an embodiment of the present invention, the generation module 1803 is specifically used to perform bilateral filtering processing on the target infrared image to enhance the edge features of the ocean front area. The bilateral filtering includes a weighted average method based on a spatial domain kernel and a range kernel. The spatial domain kernel calculates the weight coefficient based on the Euclidean distance between pixels, and the range kernel calculates the weight coefficient based on the pixel value difference. A two-dimensional fast wavelet transform is applied to the target infrared image after bilateral filtering, and high-frequency components containing ocean front features are selected for enhancement. The high-frequency components include components in the horizontal, vertical or diagonal directions. The wavelet transform reconstructs the target infrared image at all scales in a certain direction based on the Haar wavelet to obtain an enhanced ocean front signal.

[0221] According to an embodiment of the present invention, the detection module 1804 is specifically used to calculate the wavelet energy in the horizontal direction, vertical direction and diagonal direction based on the wavelet coefficients of the high-frequency components of each scale; normalize the wavelet energy in the target direction to generate normalized wavelet energy, the target direction is the direction in which the wavelet energy of the ocean front is most significant in the horizontal direction, vertical direction and diagonal direction; apply a preset edge detection operator based on the fusion feature to perform edge detection on the ocean front area, and output the edge detection result; perform morphological processing on the edge detection result to extract the front line of the ocean front, and determine the position and length characteristic parameters of the ocean front according to the position and length of the front line.

[0222] According to an embodiment of the present invention, the preset edge detection operator includes a canny edge detection operator.

[0223] An embodiment of the present invention further provides an electronic device, comprising: at least one processor; and a memory in communication with the at least one processor. The memory stores a computer program executable by the at least one processor, and when the computer program is executed by the at least one processor, the electronic device executes the method of the embodiment of the present invention.

[0224] An embodiment of the present invention further provides a non-transitory machine-readable medium storing a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to execute the method of the embodiment of the present invention.

[0225] An embodiment of the present invention further provides a computer program product, including a computer program, wherein the computer program, when executed by a processor of a computer, is used to enable the computer to execute the method of the embodiment of the present invention.

[0226] refer to Fig.19 , a block diagram of an electronic device that can be used as a server or client of an embodiment of the present invention will now be described, which is an example of a hardware device that can be applied to various aspects of the present invention. Electronic devices are intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0227] like Fig.19 As shown, the electronic device includes a computing unit 1901, which can store data stored in a read-only memory (ROM)

[0228] The computer program in the computer program 1902 or the computer program loaded from the storage unit 1908 into the random access memory (RAM) 1903 is used to perform various appropriate actions and processes. In the RAM 1903, various programs and data required for the operation of the electronic device can also be stored. The computing unit 1901, the ROM 1902, and the RAM 1903 are connected to each other via a bus 1904. An input / output (I / O) interface 1905 is also connected to the bus 1904.

[0229] Multiple components in the electronic device are connected to the I / O interface 1905, including: an input unit 1906, an output unit 1907, a storage unit 1908, and a communication unit 1909. The input unit 1906 can be any type of device that can input information to the electronic device, and the input unit 1906 can receive input digital or character information, and generate key signal input related to user settings and / or function control of the electronic device. The output unit 1907 can be any type of device that can present information, and can include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 1908 can include but is not limited to a disk, an optical disk. The communication unit 1909 allows the electronic device to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and can include but is not limited to a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0230] The computing unit 1901 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1901 include, but are not limited to, a CPU, a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1901 performs the various methods and processes described above. For example, in some embodiments, the method embodiments of the present invention may be implemented as a computer program, which is tangibly contained in a machine-readable medium, such as a storage unit 1908. In some embodiments, part or all of the computer program may be loaded and / or installed on an electronic device via a ROM 1902 and / or a communication unit 1909. In some embodiments, the computing unit 1901 may be configured to perform the above-described method in any other appropriate manner (e.g., by means of firmware).

[0231] The computer programs for implementing the methods of the embodiments of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer programs are executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer programs may be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0232] In the context of an embodiment of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable signal medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0233] It should be noted that the term "including" and its variations used in the embodiments of the present invention are open inclusions, that is, "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". The modifications of "one" and "multiple" mentioned in the embodiments of the present invention are illustrative rather than restrictive. Those skilled in the art should understand that unless the context clearly indicates otherwise, it should be understood as "one or more".

[0234] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of the present invention are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0235] The various steps described in the method implementation methods provided in the embodiments of the present invention may be performed in different orders and / or in parallel. In addition, the method implementation methods may include additional steps and / or omit the steps shown. The scope of protection of the present invention is not limited in this respect.

[0236] The term "embodiment" in this specification refers to specific features, structures or characteristics described in conjunction with the embodiment that can be included in at least one embodiment of the present invention. The appearance of this phrase in various places in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. The various embodiments in this specification are described in a related manner, and the same or similar parts between the various embodiments refer to each other. In particular, for the device, equipment, and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts refer to the partial description of the method embodiment.

[0237] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of patent protection. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the attached claims.

Claims

1. A method for detecting ocean fronts based on infrared imaging of unmanned aerial vehicles, characterized in that: include: Acquire an initial infrared image of a target ocean area, wherein the target ocean area includes an ocean front area, and the infrared image is obtained by taking an infrared imaging device carried by a drone; The infrared image is subjected to denoising, correction and image stitching processing in sequence to generate a target infrared image; Performing feature enhancement processing on the ocean front region in the target infrared image to generate an enhanced infrared image, wherein the enhanced infrared image includes the ocean front candidate region; The ocean front in the ocean front candidate area is detected based on a method combining wavelet energy and edge operator to determine the position and length characteristic parameters of the ocean front.

2. The method according to claim 1, characterized in that The initial infrared image includes periodic stripe noise and isolated point noise; the denoising, correction and image stitching processing are performed on the initial infrared image in sequence to generate a target infrared image, including: The infrared image is denoised based on the preset denoising methods corresponding to the periodic stripe noise and the isolated point noise, respectively, to generate a denoised infrared image.

3. The method according to claim 2, characterized in that The step of sequentially performing denoising, correction and image stitching processing on the initial infrared image to generate a target infrared image includes: Sequentially removing the mean value of each row of pixels and the mean value of all pixels in the denoised infrared image to correct the difference in radiation intensity caused by different incident angles in the denoised infrared image; and The denoised infrared image is processed in sequence by a nearest neighbor interpolation processing method and a small window filtering processing method to unify the resolution of the denoised infrared image and generate a corrected infrared image.

4. The method according to claim 3, characterized in that: The step of sequentially performing denoising, correction and image stitching processing on the initial infrared image to generate a target infrared image includes: Processing the plurality of corrected infrared images based on a bilinear interpolation method to generate first intermediate infrared images corresponding to the plurality of corrected infrared images, wherein the plurality of corrected infrared images have the same shooting time, continuous start and end times, and overlapping corner point geographic coordinates, and any point in the plurality of first intermediate infrared images is mapped to a unified geographic coordinate system; Performing mean fusion processing on pixel values ​​of overlapping areas of a plurality of the first intermediate infrared images to generate a second intermediate infrared image, wherein the overlapping areas of the plurality of the first intermediate infrared images are determined based on geographic coordinates; A window mean filtering process is performed on the boundary of the stitching area between adjacent first intermediate infrared images to generate the target infrared image.

5. The method according to claim 1, characterized in that The step of performing feature enhancement processing on the ocean front region in the target infrared image to generate an enhanced infrared image includes: Performing bilateral filtering on the target infrared image to enhance edge features of the ocean front region, the bilateral filtering includes a weighted average method based on a spatial domain kernel and a range kernel, the spatial domain kernel calculates a weight coefficient based on a Euclidean distance between pixels, and the range kernel calculates a weight coefficient based on a pixel value difference; A two-dimensional fast wavelet transform is applied to the target infrared image after bilateral filtering, and high-frequency components containing ocean front characteristics are selected for enhancement. The high-frequency components include components in the horizontal, vertical or diagonal directions. The wavelet transform reconstructs the target infrared image at full scale based on the Haar wavelet to obtain an enhanced ocean front signal.

6. The method according to claim 5, characterized in that The method based on the combination of wavelet energy and edge operator detects the ocean front in the ocean front candidate area to determine the position and length characteristic parameters of the ocean front, including: Based on the wavelet coefficients of the high-frequency components at each scale, the wavelet energies in the horizontal direction, the vertical direction and the diagonal direction are calculated respectively; Normalizing the wavelet energy in the target direction to generate normalized wavelet energy, wherein the target direction is the direction in which the ocean front wavelet energy is most significant among the horizontal direction, the vertical direction and the diagonal direction; Obtaining fusion features by fusing the normalized wavelet energy with the ocean front candidate region data; Based on the fusion features, a preset edge detection operator is applied to perform edge detection on the ocean front region, and an edge detection result is output; The edge detection result is subjected to morphological processing to extract the front line of the ocean front, and the position and length characteristic parameters of the ocean front are determined according to the position and length of the front line.

7. The method according to claim 1, characterized in that The preset edge detection operator includes a canny edge detection operator.

8. An ocean front detection device based on drone infrared imaging, characterized in that: include: An acquisition module is used to acquire an initial infrared image of a target ocean area, wherein the target ocean area includes an ocean front area, and the infrared image is obtained by taking an infrared imaging device carried by a drone; A processing module, used for performing denoising, correction and image stitching processing on the infrared image in sequence to generate a target infrared image; A generating module, configured to perform feature enhancement processing on the ocean front region in the target infrared image to generate an enhanced infrared image, wherein the enhanced infrared image includes the ocean front candidate region; The detection module is used to detect the ocean front in the ocean front candidate area based on a method combining wavelet energy and edge operator to determine the position and length characteristic parameters of the ocean front.

9. An electronic device, comprising: A processor, and a memory storing a program, wherein the program comprises instructions, which, when executed by the processor, cause the processor to perform the method according to any one of claims 1 to 7.

10. A non-transitory machine-readable medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 7.

Images