Polymorphic central point fusion cloud particle detection method

Abstract

The invention relates to a multi-form central point fused cloud particle detection method, which mainly comprises the following steps of: firstly, performing morphological processing of different levels on a cloud particle image to generate a plurality of morphological processing results; then, for each piece of morphologically processed image data, extracting different types of center points, including a geometric center point, a minimum circumscribed rectangle center point, a maximum connected region center point, a contour centroid and a minimum circumscribed circle center point; then abnormal center point filtering and fusion are carried out on center points in each maximum connected area, and a final target center point position is obtained; and projecting the central points onto the original image data to generate a predetermined number of anchor frames, establishing a deep learning detection model to predict the anchor frames, and screening the anchor frames to obtain a detection result. According to the method, the cloud particle region detection positioning precision is improved, and a reliable basis is provided for cloud microphysical parameter inversion.

Description

Technical Field

[0001] The present invention belongs to the field of cloud particle target detection, and in particular relates to rapid and efficient detection and positioning of airborne detection image data based on artificial intelligence algorithms. Background Art

[0002] The detection and location of cloud-induced precipitation particles is a key foundational issue in current meteorological research and weather modification. To deeply investigate cloud microphysical properties, such as particle size distribution, liquid water content, ice crystal concentration, particle volume, and cloud precipitation, high-resolution imaging equipment aboard aircraft is often used to provide intuitive and accurate observation and analysis of cloud particles. Airborne cloud particle imaging instruments (such as 2D-S or CPI probes) can acquire high-resolution particle images in real time, providing rich data support for the inversion of cloud microphysical parameters.

[0003] However, due to the inherent performance limitations of detection equipment and the complex physical properties of cloud particles, actual observation data often exhibits phenomena such as particle fragmentation, pixel loss, blurred edges, and interference from false targets. These problems directly reduce the accuracy of traditional particle region detection methods. Currently commonly used particle region detection and localization techniques based on single-scale or single-morphological processing methods, such as minimum bounding rectangles, template matching, or traditional morphological processing, struggle to effectively eliminate these interferences, resulting in a large number of misidentified fragmented particles or incorrect particle segmentation, which seriously reduces the reliability of cloud particle identification results. Therefore, existing methods are unable to meet the needs of cloud particle detection in complex real-world scenarios.

[0004] To improve the accuracy of cloud particle region detection and enhance the adaptability of particle localization methods for handling diverse particle morphologies and characteristics, it is necessary to comprehensively utilize morphological processing features at different scales and fuse them with particle region center points obtained using deep learning methods. This addresses the inefficiency of traditional detection methods for complex cloud particle detection. By fusing center point information from different morphological processing methods, a comprehensive center point generation scheme is formed, which can more accurately represent the true location of particles and significantly improve the accuracy of cloud particle region detection. This comprehensive multi-morphological center point fusion cloud particle detection method has important application value for further improving the accuracy of cloud particle classification and identification and enhancing the reliability of subsequent cloud microphysics research. Summary of the Invention

[0005] In view of the above-mentioned deficiencies in the prior art, the present invention proposes a multi-modal center point fusion cloud particle detection method, comprising the following steps: Step 1: Perform different levels of morphological processing on the original cloud particle image data; Step 2: Generate different types of center points in all morphologically processed cloud particle image data and fuse the center points; Step 2.1: Generate different types of center points on all erosion cloud particle image data and expansion cloud particle image data; Step 2.2: Filter out abnormal center points for the center points in each maximum connected area of ​​all erosion cloud particle image data and expansion cloud particle image data; Step 2.3: Fuse the center points retained in each maximum connected area on all erosion cloud particle image data and expansion cloud particle image data respectively; Step 3: Project the fused center points of all morphologically processed cloud particle image data onto the original cloud particle image data; Step 4: Generate an anchor frame using the center point of the original cloud particle image data as the center point of the anchor frame. Specifically: Step 4.1: Take the center point in the original cloud particle image data as the anchor box center point, and generate T anchor boxes for each anchor box center point; Step 4.2: Set the threshold , calculate the number of anchor boxes generated in each original cloud particle image data, and adjust the number of generated anchor boxes so that the number of generated anchor boxes is equal to the threshold ; Step 5: Build a deep learning model to predict the category probability, center point offset, and anchor box size offset of all generated anchor boxes; Step 6: Filter the anchor boxes according to their overlap and category likelihood to obtain the detection results.

[0006] Furthermore, the step 1: performing different levels of morphological processing on the original cloud particle image data is specifically as follows: Step 1.1: Binarize the original cloud particle image data; Step 1.2: Invert the original cloud particle image data; Step 1.3: Perform M erosion processes on the original cloud particle image data to obtain M eroded cloud particle image data; Step 1.4: Perform N expansion processes on the original cloud particle image data to obtain N expanded cloud particle image data.

[0007] Furthermore, the step 2.1: generating different types of center points on all corrosion cloud particle image data and expansion cloud particle image data, specifically: generating different types of center points on all corrosion cloud particle image data and expansion cloud particle image data, including geometric center points, minimum circumscribed rectangle center points, maximum connected area center points, contour centroids, and minimum circumscribed circle center points.

[0008] Furthermore, the step 2.2: performing abnormal center point filtering on the center points in each maximum connected area on all corrosion cloud particle image data and expansion cloud particle image data, specifically: calculating the Euclidean distance between all center points in each maximum connected area, and then calculating the average distance between each center point and other center points. Based on the average distance, the outlier method is used to screen out center points with abnormal distances from other center points, and then these center points are removed.

[0009] Furthermore, the step 2.3: fusing the center points retained in each maximum connected region on all erosion cloud particle image data and expansion cloud particle image data, specifically: first calculating the number of pixels in the maximum connected region; then, setting an adaptive threshold based on the number of pixels in the maximum connected region. , , where k is a constant, is the exponential factor, is the number of pixels in the maximum connected area, and then for all the retained center points in each maximum connected area, the Euclidean distance between them is calculated. When the distance between a pair of center points is less than the set threshold , then take the average of the coordinate values ​​of this pair of center points and merge them to get a new center point.

[0010] Furthermore, the step 4.1: taking the fusion center point in the original cloud particle image data as the anchor frame center point, generating T anchor frames for each anchor frame center point, wherein the anchor frame generated by each center point adopts T1 aspect ratios and T2 scales.

[0011] Furthermore, the step 4.2: setting the threshold , calculate the number of anchor boxes generated in each original cloud particle image data, and adjust the number of generated anchor boxes so that the number of generated anchor boxes is equal to the threshold Specifically, when the number of generated anchor boxes is less than the threshold , then by adding a virtual center point in the center of the image to generate a value equal to the threshold The number of anchor boxes; when the number of generated anchor boxes is greater than the threshold , then the most representative anchor box is selected so that the number of generated anchor boxes is equal to the threshold ; Furthermore, in step 4.2: when the number of generated anchor boxes is greater than the threshold, the most representative anchor boxes are selected so that the number of generated anchor boxes is equal to the threshold, specifically: Step 4.2.1: Group the anchor boxes of the same scale and aspect ratio into a group, calculate the intersection-and-union ratio of the anchor boxes in each group, and sort the intersection-and-union ratio of each group from large to small. Step 4.2.2: For each group of anchor boxes, select the two anchor boxes with the largest intersection-to-union ratio; in each group, randomly delete one of the two anchor boxes with the largest intersection-to-union ratio and keep the other; Step 4.2.3: By repeating step 4.2.2, gradually reduce the redundant anchor boxes until the total number is consistent with the threshold; Furthermore, the step 5: establishing a deep learning model to predict the category probability, center point offset and anchor box size offset of all generated anchor boxes, specifically: first, selecting a partially frozen ResNet-50 network as the backbone network, in which the early layers are frozen and the later layers are unfrozen for feature extraction; then using a two-layer 1x1 convolution transition layer to reduce the number of channels of the feature map from 2048 to 512, adjusting the spatial size of the feature map through an upsampling mechanism to match the number of generated dynamic anchor boxes, using a positioning layer to adjust the center point offset and size offset of each anchor box to ensure that the anchor box matches the target area more accurately, using a classification layer to classify each anchor box and predict the category probability of the anchor box, and finally the model outputs the category probability, center point offset and anchor box size offset of each anchor box; Furthermore, step 6: filtering anchor frames according to their overlap and category likelihood to obtain detection results, specifically: assigning a category confidence score to each generated anchor frame through the classification layer, sorting all anchor frames from high to low according to their category confidence scores, applying the non-maximum suppression method based on the intersection-union ratio of the anchor frames and the category confidence scores, filtering out redundant anchor frames and removing them to obtain the final detection results.

[0012] Compared with traditional cloud particle detection methods, the present invention has the following advantages, thereby solving the corresponding technical problems: 1. This paper proposes a method that combines multi-level morphological processing. This method performs multiple erosion and dilation processes on the original cloud particle image. This allows larger cloud particles in the image to be continuously highlighted and focused upon through multiple erosion processes, while also effectively focusing on smaller cloud particles through multiple dilation processes. This combined approach improves the detection adaptability of cloud particles of different sizes, resolving the issues of prior art where small targets are ignored or large targets are weakened, significantly enhancing overall target detection capabilities.

[0013] 2. The present invention proposes a method based on the fusion of different types of center points. The present invention extracts various types of center points, such as geometric center points, minimum circumscribed rectangle center points, maximum connected area center points, contour centroids, and minimum circumscribed circle center points, on images that have undergone different levels of morphological processing, thereby ensuring the full expression of the morphological characteristics of different cloud particles. By comprehensively considering various types of center points, the stability and accuracy of the target center point position are further improved.

[0014] 3. The present invention further introduces an outlier point filtering mechanism based on the distance between center points within the maximum connected area. By calculating the average distance between each center point in the area and other center points, combined with the outlier detection method, center points with abnormal positions are effectively eliminated, thereby reducing the negative impact of incorrect center points on the final detection results.

[0015] 4. This paper proposes an adaptive center point fusion method that effectively fuses center points based on the relationship between the number of pixels in each largest connected region and the distance between center points. This method finely controls the center point merging range, effectively improving the accuracy and computational efficiency of cloud particle target detection.

[0016] 5. The present invention designs an anchor frame generation and screening mechanism suitable for deep learning models. First, based on the fused center point, the number of anchor frames is adjusted to generate a predetermined number of anchor frames by combining the method of supplementing virtual center points and screening the most representative anchor frames. Then, based on the aspect ratio, scale, and overlap between anchor frames, a screening criterion is designed to prioritize the selection of anchor frames, effectively avoiding redundancy and excessive overlap of anchor frames, thereby achieving efficient standardization of the number of anchor frames and ensuring that this method can be quickly and efficiently combined with deep learning models. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 Flowchart of a multi-modal center point fusion cloud particle detection method. DETAILED DESCRIPTION

[0018] The following will clearly and completely describe the technical solution in the embodiment of the present invention in conjunction with the accompanying drawings in the embodiment of the present invention. The method includes the following steps: Step 1: Perform different levels of morphological processing on the original cloud particle image data; Step 2: Generate different types of center points in all morphologically processed cloud particle image data and fuse the center points; Step 2.1: Generate different types of center points on all erosion cloud particle image data and expansion cloud particle image data; Step 2.2: Filter out abnormal center points for the center points in each maximum connected area of ​​all erosion cloud particle image data and expansion cloud particle image data; Step 2.3: Fuse the center points retained in each maximum connected area on all erosion cloud particle image data and expansion cloud particle image data respectively; Step 3: Project the fused center points of all morphologically processed cloud particle image data onto the original cloud particle image data; Step 4: Generate an anchor frame using the center point of the original cloud particle image data as the center point of the anchor frame. Specifically: Step 4.1: Take the center point in the original cloud particle image data as the anchor box center point, and generate T anchor boxes for each anchor box center point; Step 4.2: Set the threshold , calculate the number of anchor boxes generated in each original cloud particle image data, and adjust the number of generated anchor boxes so that the number of generated anchor boxes is equal to the threshold ; Step 5: Build a deep learning model to predict the category probability, center point offset, and anchor box size offset of all generated anchor boxes; Step 6: Filter the anchor boxes according to their overlap and category likelihood to obtain the detection results.

[0019] Furthermore, the step 1: performing different levels of morphological processing on the original cloud particle image data is specifically as follows: Step 1.1: Binarize the original cloud particle image data; Step 1.2: Invert the original cloud particle image data; Step 1.3: Perform M erosion processes on the original cloud particle image data to obtain M eroded cloud particle image data; Step 1.4: Perform N expansion processes on the original cloud particle image data to obtain N expanded cloud particle image data.

[0020] Furthermore, the step 2.1: generating different types of center points on all corrosion cloud particle image data and expansion cloud particle image data, specifically: generating different types of center points on all corrosion cloud particle image data and expansion cloud particle image data, including geometric center points, minimum circumscribed rectangle center points, maximum connected area center points, contour centroids, and minimum circumscribed circle center points.

[0021] Furthermore, the step 2.2: performing abnormal center point filtering on the center points in each maximum connected area on all corrosion cloud particle image data and expansion cloud particle image data, specifically: calculating the Euclidean distance between all center points in each maximum connected area, and then calculating the average distance between each center point and other center points. Based on the average distance, the outlier method is used to screen out center points with abnormal distances from other center points, and then these center points are removed.

[0022] Furthermore, the step 2.3: fusing the center points retained in each maximum connected region on all erosion cloud particle image data and expansion cloud particle image data, specifically: first calculating the number of pixels in the maximum connected region; then, setting an adaptive threshold based on the number of pixels in the maximum connected region. , , where k is a constant, is the exponential factor, is the number of pixels in the maximum connected area, and then for all the retained center points in each maximum connected area, the Euclidean distance between them is calculated. When the distance between a pair of center points is less than the set threshold , then take the average of the coordinate values ​​of this pair of center points and merge them to get a new center point.

[0023] Furthermore, the step 4.1: taking the fusion center point in the original cloud particle image data as the anchor frame center point, generating T anchor frames for each anchor frame center point, wherein the anchor frame generated by each center point adopts T1 aspect ratios and T2 scales.

[0024] Furthermore, the step 4.2: setting the threshold , calculate the number of anchor boxes generated in each original cloud particle image data, and adjust the number of generated anchor boxes so that the number of generated anchor boxes is equal to the threshold Specifically, when the number of generated anchor boxes is less than the threshold , then by adding a virtual center point in the center of the image to generate a value equal to the threshold The number of anchor boxes; when the number of generated anchor boxes is greater than the threshold , then the most representative anchor box is selected so that the number of generated anchor boxes is equal to the threshold ; Furthermore, in step 4.2: when the number of generated anchor boxes is greater than the threshold, the most representative anchor boxes are selected so that the number of generated anchor boxes is equal to the threshold, specifically: Step 4.2.1: Group the anchor boxes of the same scale and aspect ratio into a group, calculate the intersection-and-union ratio of the anchor boxes in each group, and sort the intersection-and-union ratio of each group from large to small. Step 4.2.2: For each group of anchor boxes, select the two anchor boxes with the largest intersection-to-union ratio; in each group, randomly delete one of the two anchor boxes with the largest intersection-to-union ratio and keep the other; Step 4.2.3: By repeating step 4.2.2, gradually reduce the redundant anchor boxes until the total number is consistent with the threshold; Furthermore, the step 5: establishing a deep learning model to predict the category probability, center point offset and anchor box size offset of all generated anchor boxes, specifically: first, selecting a partially frozen ResNet-50 network as the backbone network, in which the early layers are frozen and the later layers are unfrozen for feature extraction; then using a two-layer 1x1 convolution transition layer to reduce the number of channels of the feature map from 2048 to 512, adjusting the spatial size of the feature map through an upsampling mechanism to match the number of generated dynamic anchor boxes, using a positioning layer to adjust the center point offset and size offset of each anchor box to ensure that the anchor box matches the target area more accurately, using a classification layer to classify each anchor box and predict the category probability of the anchor box, and finally the model outputs the category probability, center point offset and anchor box size offset of each anchor box; Furthermore, step 6: filtering anchor frames according to their overlap and category likelihood to obtain detection results, specifically: assigning a category confidence score to each generated anchor frame through the classification layer, sorting all anchor frames from high to low according to their category confidence scores, applying the non-maximum suppression method based on the intersection-union ratio of the anchor frames and the category confidence scores, filtering out redundant anchor frames and removing them to obtain the final detection results.

[0025] Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, other embodiments obtained by ordinary technicians in this field without making creative work are all within the scope of protection of the present invention.

Claims

1. A multi-modal center point fusion cloud particle detection method, comprising the following steps: Step 1: Perform different levels of morphological processing on the original cloud particle image data; Step 2: Generate different types of center points in all morphologically processed cloud particle image data and fuse the center points; Step 2.1: Generate different types of center points on all erosion cloud particle image data and expansion cloud particle image data; Step 2.2: Filter out abnormal center points for the center points in each maximum connected area of ​​all erosion cloud particle image data and expansion cloud particle image data; Step 2.3: Fuse the center points retained in each maximum connected area on all erosion cloud particle image data and expansion cloud particle image data respectively; Step 3: Project the fused center points of all morphologically processed cloud particle image data onto the original cloud particle image data; Step 4: Generate an anchor frame using the center point of the original cloud particle image data as the center point of the anchor frame. Specifically: Step 4.1: Take the center point in the original cloud particle image data as the anchor box center point, and generate T anchor boxes for each anchor box center point; Step 4.2: Set the threshold , calculate the number of anchor boxes generated in each original cloud particle image data, and adjust the number of generated anchor boxes so that the number of generated anchor boxes is equal to the threshold ; Step 5: Build a deep learning model to predict the category probability, center point offset, and anchor box size offset of all generated anchor boxes; Step 6: Filter the anchor boxes according to their overlap and category likelihood to obtain the detection results.

2. A multi-modal center point fusion cloud particle detection method as claimed in claim 1, characterized in that Step 1: performing different levels of morphological processing on the original cloud particle image data, specifically: Step 1.1: Binarize the original cloud particle image data; Step 1.2: Invert the original cloud particle image data; Step 1.3: Perform M erosion processes on the original cloud particle image data to obtain M eroded cloud particle image data; Step 1.4: Perform N expansion processes on the original cloud particle image data to obtain N expanded cloud particle image data.

3. A multi-modal center point fusion cloud particle detection method as claimed in claim 1, characterized in that The step 2.1: generating different types of center points on all corrosion cloud particle image data and expansion cloud particle image data, specifically: generating different types of center points on all corrosion cloud particle image data and expansion cloud particle image data, including geometric center points, minimum circumscribed rectangle center points, maximum connected area center points, contour centroids, and minimum circumscribed circle center points.

4. A multi-modal center point fusion cloud particle detection method as claimed in claim 1, characterized in that The step 2.2: performing abnormal center point filtering on the center points in each maximum connected area of ​​all corrosion cloud particle image data and expansion cloud particle image data, specifically: calculating the Euclidean distance between all center points in each maximum connected area, and then calculating the average distance between each center point and other center points. Based on the average distance, an outlier method is used to filter out center points with abnormal distances from other center points, and then these center points are removed.

5. A multi-modal center point fusion cloud particle detection method as claimed in claim 1, characterized in that Step 2.3: Fusing the center points retained in each maximum connected region on all erosion cloud particle image data and expansion cloud particle image data, specifically: first calculating the number of pixels in the maximum connected region; then, setting an adaptive threshold based on the number of pixels in the maximum connected region. , , where k is a constant, is the exponential factor, is the number of pixels in the maximum connected area, and then for all the retained center points in each maximum connected area, the Euclidean distance between them is calculated. When the distance between a pair of center points is less than the set threshold , then take the average of the coordinate values ​​of this pair of center points and merge them to get a new center point.

6. A multi-modal center point fusion cloud particle detection method as claimed in claim 1, characterized in that The step 4.1: using the fusion center point in the original cloud particle image data as the anchor frame center point, generating T anchor frames for each anchor frame center point, wherein the anchor frame generated by each center point adopts T1 aspect ratios and T2 scales.

7. A multi-modal center point fusion cloud particle detection method as claimed in claim 1, characterized in that Step 4.2: Setting the Threshold , calculate the number of anchor boxes generated in each original cloud particle image data, and adjust the number of generated anchor boxes so that the number of generated anchor boxes is equal to the threshold Specifically, when the number of generated anchor boxes is less than the threshold , then by adding a virtual center point in the center of the image to generate a value equal to the threshold The number of anchor boxes; When the number of generated anchor boxes is greater than the threshold , then the most representative anchor box is selected so that the number of generated anchor boxes is equal to the threshold .

8. A multi-modal center point fusion cloud particle detection method as claimed in claim 1, characterized in that Step 4.2: When the number of generated anchor boxes is greater than the threshold, the most representative anchor boxes are selected so that the number of generated anchor boxes is equal to the threshold. Specifically: Step 4.2.1: Group the anchor boxes of the same scale and aspect ratio into a group, calculate the intersection-and-union ratio of the anchor boxes in each group, and sort the intersection-and-union ratio of each group from large to small. Step 4.2.2: For each group of anchor boxes, select the two anchor boxes with the largest intersection-to-union ratio; in each group, randomly delete one of the two anchor boxes with the largest intersection-to-union ratio and keep the other; Step 4.2.3: By repeating step 4.2.2, gradually reduce the redundant anchor boxes until the total number is consistent with the threshold.

9. A multi-modal center point fusion cloud particle detection method as claimed in claim 1, characterized in that The step 5: establishing a deep learning model to predict the category probability, center point offset and anchor frame size offset of all generated anchor boxes, specifically: first, selecting a partially frozen ResNet-50 network as the backbone network, in which the early layers are frozen and the later layers are unfrozen for feature extraction; then using a two-layer 1x1 convolution transition layer to reduce the number of channels of the feature map from 2048 to 512, adjusting the spatial size of the feature map through an upsampling mechanism to match the number of generated dynamic anchor boxes, using a positioning layer to adjust the center point offset and size offset of each anchor box to ensure that the anchor box matches the target area more accurately, using a classification layer to classify each anchor box and predict the category probability of the anchor box, and finally the model outputs the category probability, center point offset and anchor frame size offset of each anchor box.

10. A multi-modal center point fusion cloud particle detection method as claimed in claim 1, characterized in that The step 6: filtering anchor frames according to the overlap and category possibility of the anchor frames to obtain the detection results, specifically: assigning a category confidence score to each generated anchor frame through the classification layer, sorting all anchor frames from high to low according to the category confidence scores, applying the non-maximum suppression method according to the intersection-union ratio of the anchor frames and the category confidence scores, filtering out redundant anchor frames and removing them to obtain the final detection results.

Images