Dual-channel cloud particle image recognition method and system based on transfer learning and UNET

Abstract

The present invention provides a dual-channel cloud particle image recognition method and system based on transfer learning and UNET. The method comprises obtaining a cloud particle image set; performing first feature extraction and second feature extraction on each cloud particle image in the cloud particle image set to obtain a first channel feature map and a second channel feature map corresponding to each cloud particle image; inputting the first channel feature map and the second channel feature map into a pre-trained feature adaptation model for feature space alignment to obtain an aligned first channel feature map and an aligned second channel feature map; and inputting the aligned first channel feature map and the aligned second channel feature map into a UNET network for dual-channel feature fusion and recognition to obtain a cloud particle recognition result. The present invention improves the accuracy of cloud particle recognition results.

Description

Technical Field

[0001] The present invention relates to the fields of image processing and machine learning, and in particular to a dual-channel cloud particle image recognition method and system based on transfer learning and UNET. Background Art

[0002] With the development of meteorological observation technology, cloud particle image recognition technology has been proposed. It refers to a technology that can be used to analyze cloud particle images and identify the types and characteristics of cloud particles. It has important application value in meteorological research, weather forecasting and other fields. At present, in cloud particle image recognition technology, some methods have begun to try to use deep learning models such as convolutional neural networks or basic UNET models for cloud particle image recognition, and introduce transfer learning to improve model performance. However, the existing technology still has shortcomings in the comprehensiveness of feature extraction and the effective fusion of different features. The feature extraction method is difficult to fully capture the complex characteristics of cloud particle images, and simple multi-feature splicing or fusion often cannot effectively solve the spatial distribution differences and semantic gaps between different features, resulting in the accuracy and reliability of cloud particle recognition needs to be further improved. Summary of the Invention

[0003] The present invention provides a dual-channel cloud particle image recognition method and system based on transfer learning and UNET.

[0004] In a first aspect, an embodiment of the present invention provides a dual-channel cloud particle image recognition method based on transfer learning and UNET, including: obtaining a cloud particle image set; performing image segmentation on the cloud particle image to divide it into multiple cloud particle image blocks, performing local texture feature extraction on each cloud particle image block, and generating a texture feature vector corresponding to each cloud particle image block by calculating the grayscale value change rate and directional gradient of the pixel points in each cloud particle image block; performing feature recombination on the texture feature vectors corresponding to all cloud particle image blocks, arranging multiple texture feature vectors into a two-dimensional matrix according to the spatial position relationship of the cloud particle image blocks in the original cloud particle image, and obtaining the first channel feature map; performing multi-scale region division on the cloud particle image to divide the cloud particle image into a hierarchical structure of regions of different sizes. , perform regional feature statistics on the regions in each regional hierarchy, calculate the grayscale mean, grayscale variance and spatial distribution density of the pixels in each region, and generate the regional feature vector corresponding to each region; perform feature fusion on the regional feature vectors in all regional hierarchies, and combine the regional feature vectors of regions of different scales into a feature representation of unified dimension through hierarchical feature aggregation to obtain the second channel feature map; input the first channel feature map and the second channel feature map into the pre-trained feature adaptation model for feature space alignment to obtain the aligned first channel feature map and the aligned second channel feature map; input the aligned first channel feature map and the aligned second channel feature map into the UNET network for dual-channel feature fusion and recognition to obtain the cloud particle recognition result.

[0005] In a second aspect, an embodiment of the present invention provides a computer system, comprising: a memory storing a computer program; and a processor for loading the computer program to implement the dual-channel cloud particle image recognition method based on transfer learning and UNET as described above.

[0006] The dual-channel cloud particle image recognition method based on transfer learning and UNET provided by the present invention performs a first feature extraction process and a second feature extraction process on each cloud particle image in the cloud particle image set, thereby obtaining a first channel feature map and a second channel feature map corresponding to each cloud particle image, thereby being able to obtain complementary feature information from the cloud particle image, avoiding the information loss problem that may be caused by a single feature extraction method, and providing a more comprehensive feature basis for subsequent fusion recognition. After obtaining the first channel feature map and the second channel feature map, they are input into a pre-trained feature adaptation model for feature space alignment to obtain an aligned first channel feature map and an aligned second channel feature map. This process solves the possible spatial distribution differences of the dual-channel features and ensures that the features input to the subsequent network have a consistent spatial expression scale, providing a prerequisite for effective fusion. Subsequently, the aligned dual-channel feature map is input into the UNET network for dual-channel feature fusion and recognition to obtain a cloud particle recognition result. The UNET network can make full use of the complementarity of the dual-channel features and improve the recognition ability of the cloud particle image through a targeted fusion mechanism, making the final output cloud particle recognition result more reliable. The entire process improves the accuracy of cloud particle recognition results through the organic combination of dual-channel feature extraction, feature space alignment and UNET network fusion recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 This is a flowchart of a dual-channel cloud particle image recognition method based on transfer learning and UNET provided by an embodiment of the present invention.

[0008] Figure 2 It is a schematic diagram of the composition of a computer system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0009] See also Figure 1 , Figure 1 A flowchart of a dual-channel cloud particle image recognition method based on transfer learning and UNET is provided in an embodiment of the present invention. The method can be executed by a computer system and includes the following steps:

[0010] Step S100: Acquire a cloud particle image set.

[0011] A cloud particle image collection is a collection of multiple images containing cloud particles. The cloud particle images are captured by image acquisition equipment, such as high-resolution cameras carried on meteorological satellites, meteorological aircraft, or ground-based meteorological observation stations, under various weather conditions, different geographical locations, and different times.

[0012] Step S200: performing a first feature extraction process and a second feature extraction process on each cloud particle image in the cloud particle image set, respectively, to obtain a first channel feature map and a second channel feature map corresponding to each cloud particle image.

[0013] Each cloud particle image in the cloud particle image collection is subjected to first and second feature extraction, respectively. The goal is to extract key information that reflects the cloud particle characteristics from the cloud particle image, thereby generating first and second channel feature maps. These first and second channel feature maps reflect the characteristics of the cloud particle image from different angles and dimensions, respectively. These two channel feature maps are subsequently used for cloud particle recognition, improving recognition accuracy and reliability.

[0014] As a first implementation of step S200, step S200 performs a first feature extraction process and a second feature extraction process on each cloud particle image in the cloud particle image set to obtain a first channel feature map and a second channel feature map corresponding to each cloud particle image. Specifically, the following steps S210 to S260 may be implemented:

[0015] Step S210: performing image segmentation on the cloud particle image, dividing the cloud particle image into a plurality of cloud particle image blocks.

[0016] Image segmentation of a cloud particle image is the process of dividing the entire cloud particle image into multiple smaller cloud particle image blocks. The purpose of image segmentation is to facilitate subsequent local texture feature extraction from the cloud particle image, as feature extraction on smaller image blocks can more precisely capture the local texture information of the cloud particle image. A fixed-size segmentation method can be used, for example, dividing the cloud particle image into several rectangular image blocks of equal size based on pixel size.

[0017] Step S220: extract local texture features from each cloud particle image block, and generate a texture feature vector corresponding to each cloud particle image block by calculating the grayscale value change rate and directional gradient of the pixels in each cloud particle image block.

[0018] The texture information of a cloud particle image is primarily reflected in the grayscale value changes and directional gradients of pixels. The grayscale value change rate reflects the speed of grayscale value changes between adjacent pixels, while the directional gradient reflects the direction of the grayscale value change. By calculating the grayscale value change rate and directional gradient of the pixels in each cloud particle image block, the texture features of that image block can be obtained. Specifically, for each cloud particle image block, the Sobel operator can be used to calculate the directional gradient of the pixels. The Sobel operator calculates the gradient in both the horizontal and vertical directions. Then, the grayscale value change rate is calculated based on the gradient values. The calculated grayscale value change rate and directional gradient are combined to form a vector, which is the texture feature vector corresponding to each cloud particle image block.

[0019] Step S230: Feature reorganization is performed on the texture feature vectors corresponding to all cloud particle image blocks. According to the spatial position relationship of the cloud particle image blocks in the original cloud particle image, multiple texture feature vectors are arranged into a two-dimensional matrix to obtain a first channel feature map.

[0020] The purpose of feature reorganization for the texture feature vectors corresponding to all cloud particle image blocks is to integrate the local texture features of each image block to form a first-channel feature map that reflects the texture characteristics of the entire cloud particle image. Arranging the cloud particle image blocks according to their spatial position within the original cloud particle image ensures that the texture features in the first-channel feature map correspond to the spatial structure of the original cloud particle image. Arranging multiple texture feature vectors into a two-dimensional matrix effectively reorganizes the local texture features of the cloud particle image in space.

[0021] As an implementation manner, step S230 may be specifically implemented as the following steps S231 to S239:

[0022] Step S231: obtaining the spatial coordinate information of each cloud particle image block in the original cloud particle image, where the spatial coordinate information includes the horizontal and vertical coordinate values ​​of the upper left corner pixel of the cloud particle image block in the original cloud particle image.

[0023] Spatial coordinate information clearly defines the specific location of each cloud particle image block within the original cloud particle image. The horizontal and vertical coordinates of the top-left pixel of a cloud particle image block can be determined using the rules and algorithms used for image segmentation. For example, when using fixed-size segmentation, given the starting coordinates of the cloud particle image and the size of each image block, the horizontal and vertical coordinates of the top-left pixel of each cloud particle image block can be calculated. Assuming the top-left pixel of the cloud particle image is (0, 0) and each cloud particle image block is 100×100 pixels, the top-left pixel of the first cloud particle image block (located in the top-left corner of the original cloud particle image) is (0, 0), the top-left pixel of the second cloud particle image block (located to the right of the first image block) is (100, 0), and so on.

[0024] Step S232: constructing a spatial position mapping table of the cloud particle image blocks based on the spatial coordinate information, wherein the spatial position mapping table records the association between the texture feature vector of each cloud particle image block and the corresponding spatial coordinates.

[0025] The spatial position mapping table for cloud particle image blocks, based on spatial coordinate information, is constructed to associate the texture feature vector of each cloud particle image block with its spatial position in the original cloud particle image. This spatial position mapping table can be implemented as a dictionary or hash table, where the key is the spatial coordinate of the cloud particle image block and the value is the corresponding texture feature vector. For example, for a cloud particle image block with the spatial coordinates of its top-left pixel (200, 300) and the corresponding texture feature vector [0.1, 0.2, …, 0.5], the spatial position mapping table can record this as {(200, 300): [0.1, 0.2, …, 0.5]}. By constructing this spatial position mapping table, the corresponding texture feature vector can be easily found based on the spatial coordinates during the subsequent feature vector sorting and matrix filling processes, ensuring that the order of the feature vectors is consistent with the spatial position of the cloud particle image block in the original cloud particle image.

[0026] Step S233: performing image block number statistics on the width and height directions of the original cloud particle image to determine the number of cloud particle image blocks included in the width direction and the number of cloud particle image blocks included in the height direction.

[0027] Counting the number of image blocks in the width and height directions of the original cloud particle image determines the size of the subsequent two-dimensional matrix. By counting the number of cloud particle image blocks in the width and height directions, we can determine the number of rows and columns in the two-dimensional matrix. Assuming the original cloud particle image is 1000 pixels wide and 800 pixels high, and each cloud particle image block is 100 × 100 pixels in size, then the number of cloud particle image blocks in the width direction is 1000 ÷ 100 = 10, and the number of cloud particle image blocks in the height direction is 800 ÷ 100 = 8.

[0028] Step S234: Determine the target number of rows and the target number of columns of the two-dimensional matrix according to the number of cloud particle image blocks in the width direction and the number of cloud particle image blocks in the height direction, where the target number of rows is equal to the number of cloud particle image blocks in the height direction, and the target number of columns is equal to the number of cloud particle image blocks in the width direction.

[0029] Determining the target number of rows and columns in the two-dimensional matrix based on the number of cloud particle image blocks in the width and height directions ensures that the size of the two-dimensional matrix matches the spatial layout of the cloud particle image blocks in the original cloud particle image. Because the target number of rows equals the number of cloud particle image blocks in the height direction, and the target number of columns equals the number of cloud particle image blocks in the width direction, the constructed two-dimensional matrix accurately reflects the spatial distribution of the cloud particle image blocks. For example, in the above example, if there are 10 cloud particle image blocks in the width direction and 8 in the height direction, the target number of rows in the two-dimensional matrix is ​​8, and the target number of columns is 10.

[0030] Step S235: sorting the texture feature vectors of all cloud particle image blocks in ascending order of the horizontal and vertical coordinate values ​​in the spatial position mapping table to obtain a sorted texture feature vector sequence.

[0031] Sorting the texture feature vectors of all cloud particle image blocks in ascending order of horizontal and vertical coordinate values ​​in the spatial position mapping table ensures that the order of the texture feature vectors in the two-dimensional matrix aligns with the spatial positions of the cloud particle image blocks in the original cloud particle image. A sorting algorithm, such as a quick sort algorithm, can be used to sort the keys (i.e., the spatial coordinates of the cloud particle image blocks) in the spatial position mapping table. Then, based on the sorted spatial coordinate order, the corresponding texture feature vectors are extracted to form a sorted sequence of texture feature vectors. For example, a spatial position mapping table records the spatial coordinates and corresponding texture feature vectors of multiple cloud particle image blocks. After sorting the spatial coordinates in ascending order of horizontal and vertical coordinate values, the corresponding texture feature vectors are sequentially extracted to form an ordered sequence of texture feature vectors. This sorted sequence of texture feature vectors can accurately reflect the spatial distribution of the texture features of the cloud particle image.

[0032] Step S236: Fill the texture feature vectors in the sorted texture feature vector sequence into a two-dimensional matrix frame with a target number of rows and columns in sequence, where the matrix element in the i-th row and j-th column corresponds to the texture feature vector of the i-th cloud particle image block in the height direction and the j-th cloud particle image block in the width direction in the original cloud particle image.

[0033] The purpose of sequentially filling the texture feature vectors from the sorted texture feature vector sequence into a two-dimensional matrix framework with a target number of rows and columns is to integrate the texture feature vectors of the cloud particle image blocks according to the spatial structure of the original cloud particle image to form the first channel feature map. Filling the matrix according to the rule that the matrix element in the i-th row and j-th column corresponds to the texture feature vector of the i-th cloud particle image block in the height direction and the j-th cloud particle image block in the width direction in the original cloud particle image ensures that each element in the two-dimensional matrix accurately corresponds to the texture feature of a cloud particle image block in the original cloud particle image.

[0034] Step S237: When the number of cloud particle image blocks is not equal to the product of the target number of rows and the target number of columns, the unfilled positions in the two-dimensional matrix frame are filled with zero vectors so that the filled two-dimensional matrix meets the requirements of the target number of rows and the target number of columns.

[0035] When the number of cloud particle image blocks is not equal to the product of the target number of rows and columns, it indicates that some special circumstances may have occurred during the image segmentation process, resulting in the number of cloud particle image blocks not being able to completely fill the two-dimensional matrix. In this case, it is necessary to fill the unfilled positions in the two-dimensional matrix framework with zero vectors to ensure that the filled two-dimensional matrix meets the target number of rows and columns. Zero vector filling fills the unfilled positions with vectors containing all zeros. This does not affect the texture information of the cloud particle image reflected by the filled texture feature vectors, while ensuring the integrity and standardization of the two-dimensional matrix.

[0036] Step S238: performing a feature dimension consistency check on the filled two-dimensional matrix so that the texture feature vectors of each element in the matrix have the same dimension.

[0037] The goal of checking the feature dimension consistency of the padded two-dimensional matrix is ​​to ensure that the texture feature vectors of each element in the matrix have the same dimensions. This ensures the standardization and consistency of the first channel feature map. The dimensions of the texture feature vectors may differ in different cloud particle image blocks due to differences in the image block content and feature calculation methods. This feature dimension consistency check can identify and address these dimensional inconsistencies. By traversing the two-dimensional matrix, the dimensions of the texture feature vectors of each element can be checked. If any dimensional inconsistencies are found, dimensional expansion or reduction can be used to correct them, ensuring that all texture feature vectors have the same dimensions.

[0038] Step S239: The two-dimensional matrix that passes the feature dimension consistency check is used as the first channel feature map.

[0039] The two-dimensional matrix that passed the feature dimension consistency check is used as the first channel feature map because, after a series of previous processing, the two-dimensional matrix has integrated the texture features of the cloud particle image according to the spatial structure of the original cloud particle image and ensured that the texture feature vectors of each element have the same dimension, which can accurately and comprehensively reflect the texture characteristics of the cloud particle image. This first channel feature map can provide important texture information for subsequent cloud particle recognition, improving the accuracy and reliability of cloud particle recognition.

[0040] Step S240: performing multi-scale region division on the cloud particle image, dividing the cloud particle image into a hierarchical structure of regions of different sizes.

[0041] Multi-scale region segmentation of cloud particle images aims to analyze their characteristics at different scales and levels. Regions of different sizes can reflect the characteristic information of cloud particle images at different scales. For example, larger regions can reflect the overall characteristics of the cloud particle image, while smaller regions can reflect local details. Multi-scale region segmentation of cloud particle images can be performed using a pyramid algorithm. This algorithm generates image pyramids of varying resolutions by downsampling and upsampling the image. When applying the pyramid algorithm to cloud particle images, the original cloud particle image is first used as the base layer of the pyramid. The image is then sequentially downsampled to generate image layers of varying resolutions, with each layer corresponding to a hierarchical structure of regions of varying sizes. This partitioning process divides the cloud particle image into multiple regions of varying sizes, forming a complete hierarchical structure.

[0042] Step S250: performing regional feature statistics on each region in the region hierarchy, calculating the grayscale mean, grayscale variance and spatial distribution density of the pixels in each region, and generating a regional feature vector corresponding to each region.

[0043] Regional feature statistics are performed on each region in the regional hierarchy to extract key feature information from each region. This feature information reflects the characteristic distribution of cloud particle images in different regions. The grayscale mean reflects the average grayscale value of pixels within the region, the grayscale variance reflects the dispersion of the pixel grayscale values, and the spatial distribution density reflects the distribution of pixels within the region. By calculating the grayscale mean, grayscale variance, and spatial distribution density of the pixels within each region, the regional features of that region can be obtained. Specifically, for each region, the sum of the grayscale values ​​of all pixels within the region is calculated and then divided by the number of pixels to obtain the grayscale mean. The sum of the squares of the differences between the grayscale value and the grayscale mean for each pixel is calculated and then divided by the number of pixels to obtain the grayscale variance. The number of pixels in the region is counted and divided by the region's area to obtain the spatial distribution density. The calculated grayscale mean, grayscale variance, and spatial distribution density are combined to form a vector, which is the regional feature vector corresponding to each region.

[0044] Step S260: performing feature fusion on the regional feature vectors in all regional hierarchical structures, combining the regional feature vectors of regions of different scales into a feature representation of unified dimension through hierarchical feature aggregation, and obtaining a second channel feature map.

[0045] The purpose of fusion is to integrate the regional feature information of regions at different scales to form a unified feature representation, namely the second channel feature map. Hierarchical feature aggregation is a feature fusion method that combines and aggregates regional feature vectors from regions at different scales according to certain rules. Hierarchical feature aggregation can be performed using a weighted averaging method. Regional feature vectors at different scales are assigned different weights based on their importance and contribution to cloud particle identification. These weighted averages are then calculated to produce a unified feature vector. For example, for a cloud particle image containing multiple regional hierarchies, each region in the hierarchy has a corresponding regional feature vector. These regional feature vectors are fused using a weighted averaging method to produce a unified feature vector sequence. Arranging these feature vector sequences into a matrix yields the second channel feature map. This second channel feature map comprehensively reflects the regional feature information of the cloud particle image at different scales, providing a richer and more comprehensive feature basis for subsequent cloud particle identification.

[0046] As a second implementation of step S200, step S200 can be specifically implemented as the following steps S201 to S208:

[0047] Step S201: performing edge detection on the cloud particle image to identify edge contour pixels in the cloud particle image.

[0048] Edge detection in cloud particle images aims to identify edge contour pixels within the image. These are pixels where the grayscale value of the image undergoes a sudden change, forming the edge contours of the cloud particles. Edge contour information is crucial for cloud particle identification and analysis, as the shape and boundaries of cloud particles can be determined through edge contours. The Canny edge detection algorithm can be used to detect edges in cloud particle images. The Canny edge detection algorithm first applies Gaussian smoothing to the cloud particle image to reduce noise; then calculates the image's gradient magnitude and direction; then performs non-maximum suppression to remove non-edge pixels; and finally, performs double thresholding to identify true edge pixels. The Canny edge detection algorithm accurately identifies edge contour pixels within cloud particle images, providing a foundation for subsequent edge contour feature extraction.

[0049] Step S202: constructing an edge contour map based on edge contour pixels, calculating the distance and angle between adjacent edge pixels in the edge contour map, and generating an edge contour feature vector.

[0050] Constructing an edge contour map based on edge contour pixels involves connecting the identified edge contour pixels to form a complete edge contour map. The edge contour map can intuitively demonstrate the edge shape and structure of cloud particles. Calculating the distance and angle between adjacent edge pixels in the edge contour map is done to extract edge contour feature information, which reflects the shape and boundary characteristics of cloud particles. Distance reflects the spatial distance between adjacent edge pixels, while angle reflects the directional change of the edge. By calculating the distance and angle between adjacent edge pixels, a feature vector for the edge contour is obtained. In implementation, for each edge pixel in the edge contour map, its adjacent edge pixels are found and the Euclidean distance and angle between them are calculated. The calculated distances and angles are combined to form a vector, which is the edge contour feature vector.

[0051] Step S203: performing feature expansion on the edge contour feature vector, converting the one-dimensional edge contour feature vector into a two-dimensional feature matrix by neighborhood feature interpolation, and obtaining an initial first channel feature map.

[0052] Feature expansion of the edge contour feature vector converts the one-dimensional edge contour feature vector into a two-dimensional feature matrix to better reflect the spatial distribution of cloud particle edge features. Neighborhood feature interpolation interpolates the edge contour feature vector using the feature information of neighboring pixels to obtain a two-dimensional feature matrix. Specifically, for each element in the edge contour feature vector, the eigenvalues ​​of its neighboring pixels are found. These eigenvalues ​​are then weighted averaged based on the distance and weight of the neighboring pixels to obtain the interpolated eigenvalue. These interpolated eigenvalues ​​are arranged into a two-dimensional matrix according to a specific rule to form the initial first channel feature map. For example, for an element in the edge contour feature vector, the eigenvalues ​​of its neighboring pixels are 0.1, 0.2, and 0.3, respectively. Based on the distance and weight of the neighboring pixels, the interpolated eigenvalue is calculated to be 0.2. These interpolated eigenvalues ​​are then arranged in a two-dimensional matrix to form the initial first channel feature map. This initial first channel feature map can more comprehensively and accurately reflect the spatial distribution of cloud particle edge features.

[0053] Step S204: performing feature enhancement on the initial first channel feature map, performing weighted summation of the local area for each element in the feature matrix, strengthening the spatial correlation of the edge contour features, and obtaining the first channel feature map.

[0054] Feature enhancement of the initial first-channel feature map is performed to strengthen the spatial correlation of edge contour features, enabling the first-channel feature map to more prominently reflect the edge characteristics of cloud particles. By performing a weighted summation of each element in the feature matrix, the correlation and continuity of edge contour features within the local region can be enhanced. Specifically, for each element in the feature matrix, a local region can be defined, such as a 3×3 neighborhood centered on the element. The elements within this neighborhood are then weighted summed. This weighted summation assigns different weights to the neighborhood elements based on their distance and correlation with the central element. The result of this weighted summation serves as the new value for the element, and the feature matrix is ​​updated. By performing this process on all elements in the feature matrix, an enhanced first-channel feature map is obtained. For example, for an element in the feature matrix, the values ​​of the elements in its 3×3 neighborhood are 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, and 0.9, respectively. Different weights are assigned to the neighborhood elements based on their distance and correlation with the central element, resulting in a new value of 0.5 after weighted summation. This new value is updated to the feature matrix, and after processing all elements, an enhanced first-channel feature map is obtained. This first-channel feature map can more clearly and accurately reflect the edge characteristics of cloud particles, providing more effective information for subsequent cloud particle identification.

[0055] Step S205: performing region growing on the cloud particle image, gradually merging similar pixel regions starting from the seed pixel point, and obtaining multiple cloud particle candidate regions.

[0056] Region growing for cloud particle images is an image segmentation method based on pixel similarity. Starting from a seed pixel, it gradually merges similar pixel regions, ultimately generating multiple cloud particle candidate regions. Cloud particle candidate regions are areas that may contain cloud particles. Region growing can be used to segment cloud particles from the background. The basic principle of region growing is to merge adjacent pixels into the same region based on similarities in pixel features such as grayscale value and color.

[0057] As an implementation manner, step S205 can be specifically implemented as the following steps S2051 to S2058:

[0058] Step S2051: performing grayscale threshold segmentation on the cloud particle image to determine the grayscale value range of the seed pixel point.

[0059] Grayscale threshold segmentation of cloud particle images aims to classify pixels in the image based on their grayscale values ​​and determine the grayscale value range of seed pixels. Grayscale threshold segmentation uses a grayscale threshold to classify pixels into two categories: pixels with grayscale values ​​greater than the threshold and pixels with grayscale values ​​less than the threshold. The grayscale threshold can be automatically determined using the Otsu algorithm, an adaptive threshold selection algorithm that determines the optimal grayscale threshold by maximizing the inter-class variance. When applying the Otsu algorithm to cloud particle images, the image's grayscale histogram is first calculated. The inter-class variance is then calculated based on the histogram, and the grayscale threshold that maximizes the inter-class variance is found. Based on this grayscale threshold, the pixels in the cloud particle image are classified into two categories. The grayscale value range of one category of pixels can be used as the grayscale value range of the seed pixels.

[0060] Step S2052: Selecting a plurality of initial seed pixels in the cloud particle image based on the grayscale value range.

[0061] The purpose of selecting multiple initial seed pixels within the cloud particle image based on a grayscale value range is to determine the starting points for region growing. Within a given grayscale value range, multiple pixels can be randomly selected as initial seed pixels, or they can be uniformly sampled. For example, within a range of pixels with grayscale values ​​greater than 128, multiple pixels can be uniformly selected at regular intervals as initial seed pixels. These initial seed pixels serve as the starting points for region growing, and similar pixel regions will subsequently be gradually merged around them.

[0062] Step S2053: setting a similarity judgment threshold for region growing. The similarity judgment threshold is used to control whether adjacent pixels are merged into the same region.

[0063] Setting the similarity judgment threshold for region growing is to control the region growing process and determine whether adjacent pixels are merged into the same region. The similarity judgment threshold can be set according to the characteristics of the cloud particle image and the recognition requirements. If the similarity judgment threshold is set too low, only adjacent pixels with very close grayscale values ​​to the current region pixels will be merged into the same region. The region growing process will be more stringent, and the resulting region will be smaller and more refined. If the similarity judgment threshold is set too high, more adjacent pixels will be merged into the same region. The region growing process will be more relaxed, and the resulting region will be larger and more coarse.

[0064] Step S2054: Taking each initial seed pixel point as the center, check its neighboring pixel points according to a preset search order.

[0065] The specific implementation process of region growing is to check the neighboring pixels of each initial seed pixel in a preset search order. The preset search order can be clockwise or counterclockwise, or it can be based on a specific search rule. For example, with the initial seed pixel as the center, its eight neighboring pixels are checked in a clockwise direction. For each initial seed pixel, its neighboring pixels are checked in turn to determine whether they meet the similarity judgment threshold conditions, and then decide whether to merge the neighboring pixels into the current region.

[0066] Step S2055: Calculate the average grayscale value difference between the neighborhood pixels and the current region. When the average grayscale value difference is less than the similarity judgment threshold, merge the neighborhood pixels into the current region.

[0067] The difference in average grayscale values ​​between neighboring pixels and the current region is calculated to determine whether the neighboring pixels are similar to the current region and should be merged into the current region. The average grayscale value difference reflects the degree of difference between the grayscale values ​​of the neighboring pixels and the average grayscale value of the current region. When the average grayscale value difference is less than the similarity judgment threshold, the neighboring pixel is similar to the current region and can be merged into the current region. In specific implementation, for each neighboring pixel, the absolute value of the difference between its grayscale value and the average grayscale value of the current region is calculated, and this difference is compared with the similarity judgment threshold. For example, if the average grayscale value of the current region is 130, the grayscale value of the neighboring pixel is 135, and the similarity judgment threshold is 10, the calculated average grayscale value difference is |135-130|=5, which is less than the similarity judgment threshold, and the neighboring pixel is merged into the current region.

[0068] Step S2056: Repeat the neighborhood pixel check and merging operations until no new pixels can be merged into the current area.

[0069] Repeated neighborhood pixel checking and merging operations are performed to continuously expand the current region until no new pixels can be merged into the current region. After each check of neighboring pixels and merging of pixels that meet the conditions, the average grayscale value of the current region is updated, and then new neighboring pixels are checked. This process is repeated until the boundary of the current region can no longer be expanded, that is, no new pixels can be merged into the current region. For example, for the region corresponding to an initial seed pixel, after multiple neighborhood pixel checks and merging operations, if none of the surrounding neighboring pixels meet the similarity judgment threshold, the region growth process is stopped, and the region growth process ends.

[0070] Step S2057: performing region deduplication on the growth regions corresponding to all initial seed pixels, and merging adjacent regions whose overlap ratio exceeds a preset threshold.

[0071] Deduplication is performed on the growth regions corresponding to all initial seed pixels to eliminate overlapping areas, avoiding repeated calculations and redundant information. Merging adjacent regions with an overlap ratio exceeding a preset threshold is done to combine adjacent, highly overlapping regions into a larger region, making the region division more reasonable. The overlap ratio can be calculated by calculating the ratio of the overlapping area of ​​adjacent regions to their respective areas. The preset threshold can be set based on actual conditions.

[0072] Step S2058: performing area screening on the deduplicated regions, removing regions with areas smaller than a preset area threshold, and obtaining multiple cloud particle candidate regions.

[0073] Area screening is performed on the deduplicated regions to remove areas that are too small. These areas may be noise or irrelevant, making them insignificant for cloud particle identification. The preset area threshold can be set based on the actual size of the cloud particles and the identification requirements. For example, setting the preset area threshold to 100 pixels will remove areas with an area smaller than 100 pixels. After area screening, the resulting candidate cloud particle regions are more likely to be true cloud particle regions, providing more accurate regional information for subsequent regional attribute feature extraction.

[0074] Step S206: performing regional attribute calculation on each cloud particle candidate region, obtaining the area, perimeter and shape complexity parameters of the region, and generating a regional attribute feature vector.

[0075] The purpose of calculating regional attributes for each cloud particle candidate region is to extract key attribute features of each candidate region. These features reflect the shape, size, and complexity of the cloud particles. The area of ​​the region reflects the size of the cloud particle, the perimeter reflects the length of the cloud particle boundary, and the shape complexity parameter reflects the shape complexity of the cloud particle. By calculating the area, perimeter, and shape complexity parameter of the region, a regional attribute feature vector can be obtained. In specific implementation, for each cloud particle candidate region, the area can be calculated using pixel counting, which counts the number of pixels within the region; the perimeter can be calculated using a boundary tracking algorithm, which counts the pixels along the region's boundary; and the shape complexity parameter can be calculated using methods such as the shape factor, which can be calculated from the region's area and perimeter. The calculated area, perimeter, and shape complexity parameter are combined to form a vector, which is the regional attribute feature vector.

[0076] Step S207: sorting the regional attribute feature vectors of all cloud particle candidate regions, arranging them in descending order according to the region size, and converting the sorted regional attribute feature vectors into a two-dimensional matrix to obtain an initial second channel feature map.

[0077] The purpose of sorting the regional attribute feature vectors of all cloud particle candidate regions is to arrange them by area size, thereby better reflecting the size distribution of cloud particles. After sorting them in descending order by area size, the sorted regional attribute feature vectors are converted into a two-dimensional matrix to organize them into a specific structure, forming a complete feature map. A sorting algorithm, such as the bubble sort algorithm, can be used to sort the regional attribute feature vectors in descending order by area size. The sorted regional attribute feature vectors are then arranged into a two-dimensional matrix to obtain an initial second-channel feature map. For example, given multiple cloud particle candidate regions, the regional attribute feature vectors are sorted in descending order by area size and then arranged into a two-dimensional matrix, with each row of the matrix corresponding to a regional attribute feature vector of a cloud particle candidate region. This initial second-channel feature map can intuitively demonstrate the size distribution of the regional attribute features of cloud particles.

[0078] Step S208: performing feature smoothing on the initial second channel feature map, reducing noise interference in the feature matrix by using a sliding window average filtering method, and obtaining a second channel feature map.

[0079] Feature smoothing of the initial second-channel feature map is performed to reduce noise interference in the feature matrix, making the second-channel feature map smoother and more stable. The sliding window average filter slides a window across the feature matrix, calculates the average value of the elements within the window, and replaces the value of the element at the center of the window with the average value. This effectively reduces noise and fluctuations in the feature matrix, making the feature matrix smoother. In specific implementation, for each element in the initial second-channel feature map, a sliding window (for example, a 3×3 window) is set centered on that element. The average value of all elements within the window is calculated, and this average value is used as the new value of the element. By performing this process on all elements in the feature matrix, a smoothed second-channel feature map is obtained.

[0080] Step S300: Input the first channel feature map and the second channel feature map into a pre-trained feature adaptation model for feature space alignment to obtain an aligned first channel feature map and an aligned second channel feature map.

[0081] The purpose of inputting the first and second channel feature maps into a pre-trained feature adaptation model for feature space alignment is to ensure that the first and second channel feature maps have the same scale and distribution in the feature space, facilitating subsequent effective feature fusion and recognition. Feature space alignment can eliminate feature differences between feature maps from different channels, improving feature consistency and comparability. The pre-trained feature adaptation model is pre-trained on a large amount of cloud particle image data. It can learn the mapping relationship between feature maps from different channels and map the first and second channel feature maps into the same feature space. Feature space alignment produces aligned first and second channel feature maps.

[0082] As an implementation manner, step S300 can be specifically implemented as the following steps S310 to S350:

[0083] Step S310: Input the first channel feature map into the first feature encoding subnetwork of the feature adaptation model for feature encoding to obtain a first encoding feature vector; input the second channel feature map into the second feature encoding subnetwork of the feature adaptation model for feature encoding to obtain a second encoding feature vector.

[0084] The first channel feature map is input into the first feature encoding subnetwork of the feature adaptation model for feature encoding. This is done to compress and transform the feature information of the first channel feature map, resulting in a low-dimensional feature vector, the first encoded feature vector. The first feature encoding subnetwork can adopt a convolutional neural network (CNN) architecture, consisting of multiple convolutional layers, pooling layers, and fully connected layers. The convolutional layers extract local features from the first channel feature map, the pooling layers downsample the feature map to reduce its dimensionality, and the fully connected layers convert the feature map into a one-dimensional feature vector. Through processing by the first feature encoding subnetwork, the feature information of the first channel feature map is compressed and transformed into a low-dimensional first encoded feature vector. Similarly, the second channel feature map is input into the second feature encoding subnetwork of the feature adaptation model for feature encoding, resulting in a second encoded feature vector. The structure and operating principle of the second feature encoding subnetwork are similar to those of the first feature encoding subnetwork. It compresses and transforms the feature information of the second channel feature map, resulting in a low-dimensional second encoded feature vector. These two encoded feature vectors more efficiently represent the feature information of the first and second channel feature maps.

[0085] Step S320: Input the first encoded feature vector and the second encoded feature vector into the dynamic mapping parameter generation subnetwork of the feature adaptation model to generate a feature mapping matrix; perform matrix multiplication on the first encoded feature vector and the feature mapping matrix to obtain the mapped first encoded feature vector.

[0086] The first and second encoded feature vectors are input into the dynamic mapping parameter generation subnetwork of the feature adaptation model to generate a feature mapping matrix that maps the first encoded feature vector to the same feature space as the second encoded feature vector. The dynamic mapping parameter generation subnetwork can dynamically generate the feature mapping matrix based on the feature information of the first and second encoded feature vectors. The dynamic mapping parameter generation subnetwork can adopt a multi-layer perceptron (MLP) structure consisting of multiple fully connected layers. The MLP takes the first and second encoded feature vectors as input and outputs a feature mapping matrix. Matrix multiplication is performed on the first encoded feature vector and the feature mapping matrix to obtain the mapped first encoded feature vector. This matrix multiplication maps and transforms the first encoded feature vector in the feature space, aligning it with the second encoded feature vector in the feature space. This mapping allows the first encoded feature vector to be mapped to the same feature space as the second encoded feature vector, providing a foundation for subsequent feature difference calculation and alignment.

[0087] As an implementation manner, step S320 may be specifically implemented as the following steps S321 to S326:

[0088] Step S321: performing feature concatenation on the first encoded feature vector and the second encoded feature vector to obtain a concatenated feature vector.

[0089] The purpose of concatenating the first and second encoded feature vectors is to combine the feature information of the two encoded feature vectors to form a more comprehensive feature vector, namely the concatenated feature vector. Feature concatenation sequentially connects the first and second encoded feature vectors to form a longer vector. This concatenation integrates the feature information of the first and second encoded feature vectors, providing more comprehensive feature information for the dynamic mapping parameter generation subnetwork.

[0090] Step S322: Input the concatenated feature vector into the first fully connected layer of the dynamic mapping parameter generation subnetwork for linear transformation to obtain an intermediate feature vector.

[0091] The concatenated feature vector is input into the first fully connected layer of the dynamic mapping parameter generation subnetwork for linear transformation to convert and extract features from the concatenated feature vector. The first fully connected layer is the first layer of the dynamic mapping parameter generation subnetwork and consists of multiple neurons, each connected to all elements of the concatenated feature vector. Through the linear transformation, a weighted sum operation is performed between the elements of the concatenated feature vector and the neurons in the first fully connected layer to produce an intermediate feature vector. This linear transformation transforms and extracts the feature information of the concatenated feature vector, resulting in a new feature vector, the intermediate feature vector.

[0092] Step S323: Input the intermediate feature vector into the activation function layer of the dynamic mapping parameter generation subnetwork for nonlinear transformation, input the output result of the activation function layer into the second fully connected layer of the dynamic mapping parameter generation subnetwork, and output the mapping matrix parameter vector.

[0093] The intermediate feature vector is input into the activation function layer of the dynamic mapping parameter generation subnetwork for nonlinear transformation to introduce nonlinear factors and enhance the model's expressive power. The activation function layer can use the ReLU activation function, whose formula is f(x)=max(0,x). This nonlinearly transforms the elements of the intermediate feature vector, setting negative elements to 0 and leaving positive elements unchanged. Through the ReLU activation function, the feature information of the intermediate feature vector is further transformed and extracted, enhancing the model's nonlinear expressive power. The output of the activation function layer is input into the second fully connected layer of the dynamic mapping parameter generation subnetwork. This second fully connected layer also consists of multiple neurons, each connected to the output element of the activation function layer. Through linear transformation, a weighted sum operation is performed between the output elements of the activation function layer and the neurons of the second fully connected layer, outputting the mapping matrix parameter vector. This mapping matrix parameter vector contains the parameter information required to generate the feature mapping matrix, providing the basis for subsequent feature mapping matrix generation.

[0094] Step S324: converting the mapping matrix parameter vector into a matrix of preset dimensions to obtain an initial feature mapping matrix.

[0095] Converting the mapping matrix parameter vector to a matrix of a preset dimension is to convert parameter information of the mapping matrix parameter vector into a matrix form, i.e., an initial feature mapping matrix. The number of rows and columns of the matrix of the preset dimension can be determined based on the dimensions of the first encoded feature vector and the second encoded feature vector.

[0096] As an implementation manner, step S324 can be specifically implemented as the following steps S3241 to S3245:

[0097] Step S3241: Determine the dimension of the first encoded feature vector and the dimension of the second encoded feature vector.

[0098] The dimension of the first encoded feature vector and the dimension of the second encoded feature vector are determined to clarify the target number of rows and the target number of columns of the feature mapping matrix. The dimension of the first encoded feature vector and the dimension of the second encoded feature vector can be determined by the length of the vector.

[0099] Step S3242: Based on the dimension of the first encoded feature vector and the dimension of the second encoded feature vector, determine the target number of rows and the target number of columns of the feature mapping matrix.

[0100] The target number of rows and columns of the feature mapping matrix is ​​determined based on the dimensions of the first and second encoded feature vectors to ensure that the feature mapping matrix can map the first encoded feature vector to the same feature space as the second encoded feature vector. The target number of rows of the feature mapping matrix is ​​equal to the dimension of the first encoded feature vector, and the target number of columns is equal to the dimension of the second encoded feature vector.

[0101] Step S3243: Calculate the number of elements of the mapping matrix parameter vector. When the number of elements is not equal to the product of the target number of rows and the target number of columns, perform dimension adjustment on the mapping matrix parameter vector.

[0102] The number of elements in the mapping matrix parameter vector is calculated to check whether its dimensions match the target number of rows and columns of the feature map matrix. If the number of elements is not equal to the product of the target number of rows and columns, the dimensions of the mapping matrix parameter vector need to be adjusted. Interpolation or truncation can be used to adjust the dimensions of the mapping matrix parameter vector. For example, if the target number of rows is 100 and the target number of columns is 200, then the product of the target number of rows and columns is 20,000. If the number of elements in the mapping matrix parameter vector is 15,000, which is less than 20,000, interpolation methods such as linear interpolation can be used to increase the number of elements in the mapping matrix parameter vector to 20,000. If the number of elements in the mapping matrix parameter vector is 25,000, which is greater than 20,000, truncation can be used to remove excess elements to reduce the number of elements to 20,000. This dimensional adjustment ensures that the dimensions of the mapping matrix parameter vector match the target number of rows and columns of the feature map matrix.

[0103] Step S3244: Arrange the elements of the mapping matrix parameter vector into a matrix with a target number of rows and columns in a row-first order.

[0104] The purpose of arranging the elements of the mapping matrix parameter vector in row-major order into a matrix with the target number of rows and columns is to organize the elements of the mapping matrix parameter vector into a matrix to form the initial feature map matrix. Row-major order means filling the first row of the matrix first, then the second, third, and so on until all rows are filled.

[0105] Step S3245: Transpose the arranged matrix so that the number of rows of the matrix matches the dimension of the first encoded feature vector and the number of columns of the matrix matches the dimension of the second encoded feature vector, and use the transposed matrix as the initial feature mapping matrix.

[0106] Transposing the permuted matrix adjusts the number of rows and columns to match the dimensions of the first and second coded eigenvectors. The transposition operation swaps the rows and columns of the matrix, meaning that the element in row i, column j of the original matrix becomes the element in row j, column i of the transposed matrix. This operation aligns the number of rows with the dimensions of the first coded eigenvector, and the number of columns with the dimensions of the second coded eigenvector. This transposed matrix serves as the initial eigenmapping matrix, which maps the first coded eigenvector into the same feature space as the second coded eigenvector.

[0107] Step S325: orthogonalize the initial eigenmap matrix so that the row vectors of the matrix are mutually orthogonal.

[0108] The purpose of orthogonalizing the initial feature mapping matrix is ​​to make the row vectors of the matrix orthogonal to each other, that is, the inner product of any two row vectors is 0. Orthogonalization can ensure the independence of the row vectors of the matrix, reduce the correlation between features, and improve the stability and accuracy of feature mapping. The Gram-Schmidt orthogonalization method can be used to orthogonalize the initial feature mapping matrix. The Gram-Schmidt orthogonalization method gradually orthogonalizes the row vectors of the matrix so that the row vectors of the matrix are orthogonal to each other. The orthogonalized feature mapping matrix can more effectively map the first encoded feature vector to the same feature space as the second encoded feature vector, thereby improving the quality of the feature mapping.

[0109] Step S326: performing norm normalization on the feature mapping matrix after orthogonalization processing, adjusting the value range of the matrix elements to a preset interval, and obtaining the feature mapping matrix.

[0110] The purpose of performing norm normalization on the orthogonalized feature mapping matrix is ​​to adjust the value range of the matrix elements to a preset interval, such as [0, 1] or [-1, 1]. Norm normalization makes the values ​​of the matrix elements comparable and stable, avoiding the impact of feature mapping effects caused by element values ​​being too large or too small. The L2 norm normalization method can be used to perform norm normalization on the orthogonalized feature mapping matrix. The L2 norm normalization method divides each element of the matrix by its L2 norm, which is the square root of the sum of the squares of the matrix elements. After norm normalization, a feature mapping matrix is ​​obtained, which can more stably and effectively map the first encoded feature vector to the same feature space as the second encoded feature vector.

[0111] Step S330: Calculate the feature difference value between the mapped first encoding feature vector and the second encoding feature vector.

[0112] Calculating the feature difference between the mapped first and second coded feature vectors is to measure the degree of difference between the two coded feature vectors in the feature space. The feature difference value can reflect the effectiveness of feature mapping. If the feature difference value is small, it means that the mapped first and second coded feature vectors are relatively close in the feature space, and the feature mapping effect is good. If the feature difference value is large, it means that the feature mapping effect is poor, and the network parameters of the dynamic mapping parameter generation subnetwork need to be adjusted. The feature difference between the mapped first and second coded feature vectors can be calculated using Euclidean distance. Euclidean distance is the square root of the sum of the squares of the differences between the corresponding elements of the two vectors.

[0113] Step S340: adjusting the network parameters of the dynamic mapping parameter generation sub-network based on the characteristic difference value so that the characteristic difference value satisfies a preset convergence condition.

[0114] Adjusting the network parameters of the dynamic mapping parameter generation subnetwork based on the feature difference value is intended to optimize the effect of feature mapping, so that the mapped first encoded feature vector and the second encoded feature vector are closer in the feature space, that is, the feature difference value meets the preset convergence condition. The preset convergence condition is a pre-set standard for judging the effectiveness of feature alignment. When the feature difference value is less than the preset convergence condition, it indicates that the feature mapping effect has met the requirements and network parameter adjustment stops. Optimization algorithms, such as the stochastic gradient descent algorithm, can be used to adjust the network parameters of the dynamic mapping parameter generation subnetwork. The stochastic gradient descent algorithm continuously updates the network parameters to gradually reduce the feature difference value until the preset convergence condition is met.

[0115] As an implementation manner, step S340 can be specifically implemented as the following steps S341 to S347:

[0116] Step S341: setting a preset convergence threshold of feature difference values, where the preset convergence threshold is a preset minimum difference value standard for judging the effect of feature alignment.

[0117] The purpose of setting a preset convergence threshold for feature difference values ​​is to determine the convergence standard for feature alignment, that is, the minimum difference value that the feature difference values ​​need to reach. The preset convergence threshold can be set based on actual conditions and experimental results. For example, the preset convergence threshold is set to 0.1. When the feature difference value is less than 0.1, it means that the mapped first encoded feature vector and the second encoded feature vector are very close in the feature space, and the feature alignment effect has met the requirements. The preset convergence threshold can be used as a criterion for judging whether the feature alignment has converged, providing a basis for subsequent network parameter adjustments.

[0118] Step S342: Using the feature difference value as the input of the optimization objective function, constructing an optimization objective function with the goal of reducing the feature difference value.

[0119] Taking feature differences as input to the optimization objective function and constructing an optimization objective function with the goal of reducing feature differences is to transform the feature alignment problem into an optimization problem. The optimization objective function can use a mean squared error loss function. By constructing an optimization objective function with the goal of reducing feature differences, the feature alignment problem can be transformed into a minimization optimization problem. Specifically, by adjusting the network parameters of the dynamic mapping parameter-generated subnetwork, the value of the optimization objective function is minimized, thereby reducing the feature differences.

[0120] Step S343: Calculate the gradient value of the network weight of each layer in the dynamic mapping parameter generation sub-network by the back propagation algorithm to optimize the objective function. The gradient value represents the degree of influence of the change of the network weight on the feature difference value.

[0121] The purpose of calculating the gradient of the optimization objective function with respect to the network weights of each layer in the dynamic mapping parameter generation subnetwork through the backpropagation algorithm is to determine the update direction and step size of the network weights. The backpropagation algorithm obtains the gradient value by calculating the partial derivative of the optimization objective function with respect to the network weights. The gradient value characterizes the degree to which changes in the network weights affect the feature difference value, that is, the degree to which a small change in the network weights will lead to a change in the feature difference value. For example, for the network weights of a certain layer in the dynamic mapping parameter generation subnetwork, the gradient of the optimization objective function with respect to the network weights is calculated through the backpropagation algorithm. If the gradient value is positive, it means that increasing the network weight will increase the feature difference value and the network weight needs to be reduced; if the gradient value is negative, it means that reducing the network weight will increase the feature difference value and the network weight needs to be increased.

[0122] Step S344: Based on the gradient value and the preset learning rate, the network weight of the dynamic mapping parameter generation sub-network is updated, wherein the update amount of the network weight is equal to the product of the gradient value and the learning rate.

[0123] The purpose of updating the network weights of the subnetwork generated by the dynamic mapping parameters based on the gradient value and the preset learning rate is to adjust the network parameters so that the value of the optimization objective function gradually decreases, thereby reducing the feature difference value. The preset learning rate is a hyperparameter that controls the step size of the network weight update. The amount of network weight update is equal to the product of the gradient value and the learning rate. By continuously updating the network weights, the value of the optimization objective function gradually decreases, bringing the mapped first and second encoded feature vectors closer together in the feature space.

[0124] Step S345: reapply the dynamic mapping parameter generation subnetwork after updating the network weights to the first encoded feature vector and the second encoded feature vector to generate a new feature mapping matrix.

[0125] The dynamic mapping parameter generation subnetwork, after updating the network weights, is reapplied to the first and second encoded feature vectors because the updated network weights change the structure and parameters of the dynamic mapping parameter generation subnetwork, resulting in a different feature mapping matrix. Following the previous steps for generating the feature mapping matrix, the first and second encoded feature vectors are first concatenated to obtain a concatenated feature vector. The concatenated feature vector is then input into the first fully connected layer of the dynamic mapping parameter generation subnetwork for a linear transformation to obtain an intermediate feature vector. The intermediate feature vector is then input into the activation function layer for a nonlinear transformation. The output of the activation function layer is then input into the second fully connected layer to output a mapping matrix parameter vector. The mapping matrix parameter vector is then converted to a matrix of a preset dimension and subjected to orthogonalization and norm normalization to obtain a new feature mapping matrix. This new feature mapping matrix may bring the mapped first encoded feature vector closer to the second encoded feature vector, further reducing the feature difference.

[0126] Step S346: recalculating a new feature difference value between the mapped first encoding feature vector and the second encoding feature vector based on the new feature mapping matrix.

[0127] The purpose of recalculating the new feature difference value between the mapped first encoding feature vector and the second encoding feature vector based on the new feature mapping matrix is ​​to evaluate the effect of the feature mapping after updating the network weights. The first encoding feature vector is subjected to matrix multiplication using the new feature mapping matrix to obtain a new mapped first encoding feature vector. Then, according to the previous method for calculating the feature difference value, for example, using the Euclidean distance, the new feature difference value between the newly mapped first encoding feature vector and the second encoding feature vector is calculated. The new feature difference value reflects the effect of feature alignment after updating the network weights. If the new feature difference value is smaller than the previous feature difference value, it means that updating the network weights has played a positive role in making the feature mapping more accurate.

[0128] Step S347: Compare the new feature difference value with the preset convergence threshold. When the new feature difference value is greater than the preset convergence threshold, repeat the operations of gradient value calculation, network weight update, new feature mapping matrix generation and new feature difference value calculation. When the new feature difference value is less than or equal to the preset convergence threshold, stop adjusting the network weight of the dynamic mapping parameter generation sub-network. Record the network weight of the dynamic mapping parameter generation sub-network at this time as the network weight that meets the preset convergence condition.

[0129] Comparing the size relationship between the new feature difference value and the preset convergence threshold is a key step in determining whether the feature alignment has converged. If the new feature difference value is greater than the preset convergence threshold, it means that the feature alignment has not yet met the requirements, and it is necessary to continue to adjust the network weights of the dynamic mapping parameter generation sub-network. At this time, the operations of gradient value calculation, network weight update, new feature mapping matrix generation and new feature difference value calculation are repeated, and the optimization is continuously iterated until the new feature difference value is less than or equal to the preset convergence threshold. When the new feature difference value is less than or equal to the preset convergence threshold, it means that the feature alignment has reached the preset convergence condition, and the adjustment of the network weights of the dynamic mapping parameter generation sub-network is stopped. The network weights of the dynamic mapping parameter generation sub-network at this time are recorded. These network weights can ensure that the feature mapping matrix can accurately map the first encoded feature vector to the same feature space as the second encoded feature vector, thereby realizing the alignment of the feature space.

[0130] Step S350: When the feature difference value meets the preset convergence condition, the mapped first encoded feature vector is input into the first feature decoding subnetwork of the feature adaptation model for feature decoding to obtain the aligned first channel feature map; the second encoded feature vector is input into the second feature decoding subnetwork of the feature adaptation model for feature decoding to obtain the aligned second channel feature map.

[0131] When the feature difference values ​​meet the preset convergence criteria, feature space alignment is complete. The encoded feature vectors must now be decoded and restored to feature maps. The first and second feature decoding subnetworks can employ a deconvolutional neural network (DeCNN) architecture. Deconvolutional neural networks, in contrast to convolutional neural networks, convert low-dimensional encoded feature vectors into high-dimensional feature maps. The mapped first encoded feature vector is fed into the first feature decoding subnetwork, which gradually restores the dimensionality and detail of the feature map through a series of deconvolutional and upsampling layers, resulting in an aligned first-channel feature map. Similarly, the second encoded feature vector is fed into the second feature decoding subnetwork, where it undergoes deconvolutional and upsampling layers to produce an aligned second-channel feature map. These two aligned feature maps have the same scale and distribution in the feature space, providing high-quality feature information for subsequent dual-channel feature fusion and recognition.

[0132] Step S400: Input the aligned first channel feature map and the aligned second channel feature map into the UNET network for dual-channel feature fusion and recognition to obtain cloud particle recognition results.

[0133] The final step in the cloud particle image recognition method is to feed the aligned first-channel feature map and the aligned second-channel feature map into the UNET network for dual-channel feature fusion and recognition. The UNET network is a convolutional neural network architecture with an encoder-decoder structure that effectively performs image segmentation and recognition tasks. By feeding the aligned first-channel feature map and the aligned second-channel feature map into the UNET network, the feature information of the two channels is fused, fully utilizing the complementary information of the feature maps from different channels to improve the accuracy and reliability of cloud particle recognition. After processing by the UNET network, the cloud particle recognition result is finally obtained, which can clearly identify the location and category of the cloud particles in the image.

[0134] As an implementation manner, step S400 can be specifically implemented as the following steps S410 to S470:

[0135] Step S410: Input the aligned first channel feature map to the first encoder of the UNET network for downsampling to obtain multiple first-level feature maps; input the aligned second channel feature map to the second encoder of the UNET network for downsampling to obtain multiple second-level feature maps.

[0136] The aligned first-channel feature map is fed into the first encoder of the UNET network for downsampling. This is done to gradually extract high-level features from the first-channel feature map and reduce its size. The first encoder consists of multiple convolutional and pooling layers. The convolutional layers extract local features from the feature map, while the pooling layers downsample the feature map to reduce its dimensionality. Through multiple convolutional and pooling operations, multiple first-level feature maps of different scales are generated, each with a different level of feature abstraction and size. For example, the first encoder may include four downsampling stages, with each stage halving the feature map size while increasing the level of feature abstraction. Similarly, the aligned second-channel feature map is fed into the second encoder of the UNET network for downsampling, generating multiple second-level feature maps of different scales. These first-level and second-level feature maps contain feature information from the aligned first-channel and second-channel feature maps at different levels, providing the foundation for subsequent feature concatenation and fusion.

[0137] Step S420: performing feature splicing on the plurality of first-level feature maps and the plurality of second-level feature maps according to the hierarchical correspondence to obtain a plurality of spliced-level feature maps.

[0138] The purpose of splicing multiple first-level feature maps and multiple second-level feature maps according to a hierarchical correspondence is to integrate the feature information of the two channels. Hierarchical correspondence refers to the splicing of feature maps obtained by the first encoder and the second encoder at the same downsampling stage. For example, the first-level feature map obtained by the first downsampling stage of the first encoder is spliced ​​with the second-level feature map obtained by the first downsampling stage of the second encoder to obtain the first spliced-level feature map; the first-level feature map obtained by the second downsampling stage of the first encoder is spliced ​​with the second-level feature map obtained by the second downsampling stage of the second encoder to obtain the second spliced-level feature map, and so on. Feature splicing can combine the feature information of two channels to form a more comprehensive feature map, namely the spliced-level feature map. The spliced-level feature map contains the feature information of the two channels at the same scale and can fully utilize the complementarity of feature maps from different channels.

[0139] Step S430: Input each splicing level feature map into the adaptive channel attention fusion module of the UNET network to generate a fused feature map corresponding to each level.

[0140] Each concatenated-level feature map is fed into the UNET's adaptive channel attention fusion module to further fuse the feature information of the two channels while highlighting the important feature channels. The adaptive channel attention fusion module adaptively assigns attention weights to each channel based on the feature information of the concatenated-level feature map, resulting in more efficient feature fusion.

[0141] As an implementation manner, step S430 may be specifically implemented as the following steps S431 to S436:

[0142] Step S431: Divide the splicing-level feature map into a first channel partial feature map and a second channel partial feature map, wherein the first channel partial feature map corresponds to the features obtained by encoding the aligned first channel feature map, and the second channel partial feature map corresponds to the features obtained by encoding the aligned second channel feature map.

[0143] The purpose of dividing the concatenated-level feature map into the first-channel partial feature map and the second-channel partial feature map is to process the feature information of the two channels separately. Since the concatenated-level feature map is composed of the first-level feature map and the second-level feature map, it can be divided into two parts based on the order of concatenation and the dimension of the feature map. For example, the first half of the concatenated-level feature map corresponds to the first-channel partial feature map, and the second half corresponds to the second-channel partial feature map. After this division, global average pooling operations can be performed on the first-channel partial feature map and the second-channel partial feature map to extract the global feature information of the channel.

[0144] Step S432: performing global average pooling on the partial feature map of the first channel to obtain a first channel global feature vector; performing global average pooling on the partial feature map of the second channel to obtain a second channel global feature vector.

[0145] Global average pooling is performed on the partial feature map of the first channel to extract its global feature information. Global average pooling averages all elements in each channel of the partial feature map of the first channel to obtain a scalar value. The scalar values ​​of all channels are combined to form a one-dimensional vector, the global feature vector of the first channel. Similarly, global average pooling is performed on the partial feature map of the second channel to obtain the global feature vector of the second channel. Global average pooling reduces the dimensionality of the feature map while preserving the global feature information of the channel.

[0146] Step S433: performing element-wise addition operation on the first channel global feature vector and the second channel global feature vector to obtain a fused global feature vector.

[0147] The purpose of performing element-wise addition on the first-channel global feature vector and the second-channel global feature vector is to initially fuse the global feature information of the two channels. This element-wise addition operation adds corresponding elements of the first-channel global feature vector and the second-channel global feature vector to produce a new vector, the fused global feature vector.

[0148] Step S434: Input the fused global feature vector into the fully connected layer for feature conversion to obtain a channel attention weight vector, normalize the channel attention weight vector, and obtain a normalized channel attention weight vector.

[0149] The purpose of inputting the fused global feature vector into a fully connected layer for feature transformation is to convert it into a channel attention weight vector. A fully connected layer consists of multiple neurons, each connected to all elements of the fused global feature vector. Through a linear transformation, a weighted sum operation is performed between the elements of the fused global feature vector and the neurons in the fully connected layer to obtain the channel attention weight vector. Each element of the channel attention weight vector represents the importance of the corresponding channel. To ensure that the values ​​of the channel attention weight vector elements range from [0 to 1], the channel attention weight vector is normalized.

[0150] Step S435: perform element-wise multiplication on the normalized channel attention weight vector and the splicing-level feature map to perform weighted adjustment on each channel feature in the splicing-level feature map.

[0151] The purpose of element-wise multiplication of the normalized channel attention weight vector with the concatenated-level feature map is to weight each channel feature in the concatenated-level feature map according to the importance of the channel. The element-wise multiplication operation multiplies each element of the normalized channel attention weight vector with all elements of the corresponding channel in the concatenated-level feature map to obtain a weighted concatenated-level feature map. For example, if the i-th element of the normalized channel attention weight vector is 0.8, all elements of the i-th channel in the concatenated-level feature map are multiplied by 0.8. This emphasizes the feature information of important channels and suppresses the feature information of unimportant channels.

[0152] As an implementation manner, step S435 can be specifically implemented as the following steps S4351 to S4355:

[0153] Step S4351: Determine the number of channels of the splicing-level feature map and the number of elements of the normalized channel attention weight vector.

[0154] The number of channels in the concatenated feature map and the number of elements in the normalized channel attention weight vector are determined to ensure that their dimensions are consistent, facilitating element-wise multiplication. The number of channels can be determined by checking the concatenated feature map's dimensionality. For example, if the concatenated feature map has dimensions [100, 100, 64], the number of channels is 64. Similarly, the length of the normalized channel attention weight vector is used to determine its number of elements. If the number of channels and the number of elements do not match, dimensionality adjustment is required.

[0155] Step S4352: When the number of channels is inconsistent with the number of elements, the dimension of the normalized channel attention weight vector is expanded so that the number of elements matches the number of channels.

[0156] When the number of channels is inconsistent with the number of elements, the normalized channel attention weight vector needs to be dimensionally expanded. Dimensional expansion can be performed by copying or interpolation. For example, if the number of channels is 64 and the number of elements in the normalized channel attention weight vector is 32, the elements of the normalized channel attention weight vector can be copied to bring the number of elements to 64. Through dimension expansion, the number of elements in the normalized channel attention weight vector matches the number of channels in the concatenated feature map, allowing element-wise multiplication.

[0157] Step S4353: Take each element in the normalized channel attention weight vector as the attention weight value of the corresponding channel.

[0158] Each element in the normalized channel attention weight vector is used as the attention weight value of the corresponding channel, which means that the feature information of each channel will be weighted and adjusted according to its corresponding attention weight value.

[0159] Step S4354: For each channel feature map in the concatenated level feature map, multiply each pixel value in the channel feature map by the corresponding attention weight value.

[0160] For each channel feature map in the concatenated level feature map, each pixel value in the channel feature map is multiplied by the corresponding attention weight value to achieve weighted adjustment of the channel features. By performing this multiplication operation on all pixel values ​​in each channel feature map, a weighted adjusted channel feature map is obtained.

[0161] Step S4355: Combine the weighted data of all channel feature maps into a new feature map to obtain a weighted adjusted splicing level feature map.

[0162] The purpose of combining the weighted data from all channel feature maps into a new feature map is to reassemble the weighted channel feature maps into a complete feature map. The weighted data from all channel feature maps are arranged sequentially according to the order of the channels to form a new feature map, namely the weighted concatenated-level feature map. This weighted concatenated-level feature map emphasizes the feature information of important channels and suppresses the feature information of unimportant channels, providing more targeted feature input for subsequent convolutional layer feature fusion.

[0163] Step S436: Input the weighted adjusted splicing-level feature map into the convolution layer for feature fusion to obtain a fused feature map.

[0164] The purpose of feeding the weighted, adjusted concatenated-level feature map into the convolutional layer for feature fusion is to further integrate the channel feature information and extract higher-level feature representations. The convolutional layer consists of multiple convolution kernels, each of which performs a convolution operation on the weighted, adjusted, concatenated-level feature map to extract local features. Through the convolution operation, the feature information from different channels interacts and fuses, resulting in a new feature map, the fused feature map. This fused feature map incorporates feature information from both channels and, after channel attention weighting and convolutional layer feature fusion, has higher feature representation capabilities, providing a better feature foundation for subsequent upsampling and classification.

[0165] Step S440: The deepest fusion feature map is input into the decoder of the UNET network for upsampling, and is fused with the shallow fusion feature map by jump connection to obtain a decoded feature map.

[0166] The deepest fused feature map is input into the UNET decoder for upsampling to restore the feature map size to the original image size. The decoder consists of multiple deconvolutional layers and upsampling layers. The deconvolutional layers gradually increase the feature map dimensions, while the upsampling layers increase the size of the feature maps. At each decoder stage, the upsampled feature map is skip-fused with the fused feature map of a shallower layer. Skip-fusion concatenates the upsampled and shallower fused feature maps along the channel dimension and then performs feature fusion through a convolutional layer. This combines detailed feature information from shallow layers with high-level features from deeper layers, resulting in a more accurate and detailed decoded feature map. For example, in the first decoder stage, the deepest fused feature map is upsampled and then skip-fused with the fused feature map of the penultimate layer through a skip-fusion connection. After processing through the convolutional layers, a new feature map is generated. In the second decoder stage, this new feature map is further upsampled and skip-fused with the fused feature map of the third-to-last layer through a skip-fusion connection. This process continues until a decoded feature map with the same size as the original image is obtained.

[0167] Step S450: Input the decoded feature map into the cloud particle category prediction layer of the UNET network for classification, and obtain the probability distribution of each pixel point belonging to different cloud particle categories.

[0168] The decoded feature map is input into the cloud particle category prediction layer of the UNET network for classification to determine the probability of each pixel belonging to a different cloud particle category. The cloud particle category prediction layer can adopt a fully connected layer or convolutional layer structure to convert the feature information of the decoded feature map into the probability value of each pixel belonging to a different cloud particle category. The output probability values ​​can be normalized using the Softmax function so that the sum of the probabilities of each pixel belonging to all cloud particle categories is 1. Through the processing of the cloud particle category prediction layer, the probability distribution of each pixel belonging to different cloud particle categories can be obtained, providing a basis for subsequent pixel category determination.

[0169] Step S460: Determine the cloud particle category of each pixel based on the probability distribution and generate a cloud particle category segmentation map.

[0170] Determining the cloud particle category for each pixel based on the probability distribution is done to convert the probability distribution into a specific category label. For each pixel, the cloud particle category with the highest probability value is selected as the pixel's category. For example, if the probability distribution of a pixel belonging to five different cloud particle categories is [0.1, 0.2, 0.3, 0.2, 0.2], the cloud particle category corresponding to the highest probability value of 0.3 is selected as the pixel's category. The category labels of all pixels are combined to form a two-dimensional matrix, the cloud particle category segmentation map. The cloud particle category segmentation map provides a visual representation of the distribution of cloud particles in the image. The category label of each pixel indicates the cloud particle category to which the pixel belongs.

[0171] Step S470: Post-processing the cloud particle category segmentation map to remove isolated areas with an area smaller than a preset threshold, thereby obtaining a cloud particle recognition result.

[0172] The purpose of post-processing the cloud particle category segmentation map is to further optimize the cloud particle recognition results, remove some noise and isolated areas, and make the recognition results more accurate and clear.

[0173] As an implementation manner, step S470 may be specifically implemented as the following steps S471 to S478:

[0174] Step S471: Mark the cloud particle category segmentation map, identify all interconnected pixel regions in the map through a connected region analysis algorithm, and assign a unique region identifier to each connected region.

[0175] Region labeling in the cloud particle segmentation map is performed to group and identify interconnected pixel regions. The connected region analysis algorithm can identify all interconnected pixel regions in the cloud particle segmentation map. Pixel connectivity can be defined using either four-connectedness or eight-connectedness. Four-connectedness indicates that a pixel is connected to its neighbors above, below, left, and right, while eight-connectedness indicates that a pixel is connected to its neighbors above, below, left, right, upper left, upper right, lower left, and lower right. By traversing all pixels in the cloud particle segmentation map and using either a depth-first search or a breadth-first search algorithm, interconnected pixel regions are labeled as the same region and each region is assigned a unique region identifier. For example, in a cloud particle segmentation map, if there are multiple separated cloud particle regions, the connected region analysis algorithm can label each cloud particle region with a different region identifier, facilitating subsequent area calculation and screening.

[0176] Step S472: Calculate the number of pixels contained in each connected region, and use the number of pixels as the area of ​​the connected region.

[0177] The number of pixels in each connected region is calculated to determine the area of ​​each connected region. By traversing the cloud particle category segmentation map, the number of pixels corresponding to each region identifier is counted and the number of pixels is used as the area of ​​the connected region.

[0178] Step S473: Set an area screening threshold. The area screening threshold is the minimum area standard for determining isolated areas. The area of ​​each connected area is compared with the area screening threshold. Connected areas with an area smaller than the area screening threshold are screened out and marked as isolated areas to be removed.

[0179] The area screening threshold is set to determine which connected regions are isolated regions and need to be removed. The area screening threshold can be set based on actual conditions and experimental results. For example, the area screening threshold can be set to 100 pixels. The area of ​​each connected region is compared with the area screening threshold. If the area of ​​a connected region is less than the area screening threshold, it is marked as an isolated region to be removed.

[0180] Step S474: Reset the pixel values ​​of all isolated areas to be removed in the cloud particle category segmentation map, and reset the cloud particle category of all pixels in the isolated areas to be removed to the background category.

[0181] Resetting the pixel values ​​of all isolated regions to be removed in the cloud particle segmentation map is done to remove them from the cloud particle recognition results. The cloud particle category of all pixels in the isolated region to be removed is reset to the background category, meaning that these pixels are considered not to belong to the cloud particle region. For example, in the cloud particle segmentation map, the cloud particle category of pixels in the isolated region to be removed might be 1, 2, 3, and so on. These pixels are reset to the background category 0. This pixel value reset removes isolated regions from the cloud particle recognition results, resulting in cleaner and more accurate recognition results.

[0182] Step S475: performing a morphological closing operation on the reset cloud particle category segmentation map, performing an expansion operation on the segmentation map using a structure element of a preset size and then performing an erosion operation to fill the small holes inside the cloud particle area.

[0183] Performing a morphological closing operation on the reset cloud particle segmentation map can fill small holes within the cloud particle region, making it more complete and continuous. Morphological closing operations include dilation and erosion. Dilation expands the cloud particle region outward to fill small holes, while erosion contracts the expanded region inward to restore the original shape of the cloud particle region. Pre-set structuring elements can be square, circular, or other shapes, and their size can be customized based on the characteristics of the cloud particle region and the size of the holes.

[0184] Step S476: Perform edge pixel repair on the cloud particle category segmentation map after the morphological closing operation, identify discontinuous pixel points at the edge of the cloud particle area, and correct the category of the discontinuous pixel points based on the cloud particle category of the adjacent non-edge pixels.

[0185] Edge pixel inpainting is performed on the cloud particle segmentation map after morphological closing to repair discontinuous pixels at the edge of the cloud particle region, making the cloud particle region boundary more accurate and clear. By traversing the edge pixels of the cloud particle segmentation map, discontinuous pixels—those whose cloud particle category differs from that of adjacent pixels—are identified. The category of these discontinuous pixels is then corrected based on the cloud particle category of adjacent non-edge pixels. This edge pixel inpainting improves the boundary accuracy of the cloud particle recognition results.

[0186] Step S477: Calculate the category similarity of adjacent cloud particle regions in the repaired cloud particle category segmentation map. The category similarity is calculated based on the grayscale distribution characteristics and shape characteristics of the regions.

[0187] The purpose of calculating the category similarity between adjacent cloud particle regions in the repaired cloud particle category segmentation map is to determine whether adjacent cloud particle regions belong to the same category, allowing for merging. Category similarity can be calculated based on the grayscale distribution and shape characteristics of the regions. For example, features such as the grayscale mean, grayscale variance, and shape factor of adjacent cloud particle regions can be calculated, and then category similarity can be calculated using methods such as Euclidean distance and cosine similarity. If the category similarity between adjacent cloud particle regions is high, it indicates that they likely belong to the same category and should be merged.

[0188] Step S478: When the category similarity of adjacent cloud particle areas is greater than a preset similarity threshold, the two adjacent cloud particle areas are merged into one area; the region boundaries of the merged cloud particle category segmentation map are smoothed, and the categories of the region boundary pixels are smoothly transitioned through Gaussian filtering; the cloud particle category segmentation map after the above processing is used as the cloud particle recognition result.

[0189] When the category similarity of adjacent cloud particle regions exceeds a preset similarity threshold, it indicates that the two adjacent cloud particle regions are likely to belong to the same category. Merging them into one region can improve the accuracy and rationality of cloud particle identification. The preset similarity threshold can be set based on actual conditions and experimental results; for example, a preset similarity threshold of 0.8 is set. Region boundaries are smoothed on the merged cloud particle segmentation map to make the region boundaries more natural and continuous. Gaussian filtering can be used to smooth the transition between the categories of pixels at the region boundaries. Gaussian filtering performs a weighted average of the pixel categories based on their neighborhood information, smoothing the category transition of boundary pixels. The processed cloud particle segmentation map is used as the cloud particle identification result. This cloud particle identification result undergoes processing including region labeling, area screening, morphological closing, edge pixel restoration, category similarity calculation, and region merging. It exhibits high accuracy and reliability, accurately identifying the location and category of cloud particles in the image.

[0190] It is understandable that the various algorithms involved in the above-mentioned introductions of the embodiments of the present invention, such as the Euclidean distance algorithm, the cosine distance algorithm, the pyramid algorithm, etc., can all be learned from the relevant content in the prior art. In order to save space, they will not be expanded too much in the embodiments of the present invention. In addition, when implementing the scheme of the present invention, those skilled in the art can supplement the details according to the common knowledge in this field. For example, according to the common knowledge in this field, normalization can be used to eliminate dimensional conflicts before feature fusion, interpolation can be used to eliminate dimensional differences, and thresholds can be reasonably set based on historical data, experience or business scenario requirements. The model can be trained based on a general model training method, and the number of layers in the model structure can be set based on actual needs, the activation function can be selected, etc. The present invention will no longer provide redundant introductions to the overly detailed implementation process.

[0191] See also Figure 2 , Figure 2This is a schematic diagram of the structure of a computer system provided in an embodiment of the present invention. The computer system includes at least a processor 101, a communication interface 102, and a memory 103. The processor 101, communication interface 102, and memory 103 may be connected via a bus or other means. The processor 101 (also known as the Central Processing Unit (CPU)) is the computing and control core of the computer system, capable of parsing various instructions within the computer system and processing various data within the computer system. The communication interface 102 may optionally include a standard wired interface or a wireless interface (such as Wi-Fi, a mobile communication interface, etc.), which can be used to send and receive data under the control of the processor 101. The communication interface 102 may also be used for data transmission and interaction within the computer system. The memory 103 is a storage device in the computer system for storing programs and data. It is understood that the memory 103 herein may include both the built-in memory of the computer system and, of course, the extended memory supported by the computer system. The memory 103 provides storage space, which stores the computer system's operating system, but this is not limited to this in the present invention.

[0192] In one embodiment, the processor 101 executes the dual-channel cloud particle image recognition method based on transfer learning and UNET provided in the above embodiment of the present invention by running the computer program in the memory 103.

Claims

1. A dual-channel cloud particle image recognition method based on transfer learning and UNET, characterized in that: include: Get cloud particle image collection; Performing image segmentation on the cloud particle image into a plurality of cloud particle image blocks, performing local texture feature extraction on each cloud particle image block, and generating a texture feature vector corresponding to each cloud particle image block by calculating the grayscale value change rate and directional gradient of pixels in each cloud particle image block; The texture feature vectors corresponding to all cloud particle image blocks are feature reorganized, and multiple texture feature vectors are arranged into a two-dimensional matrix according to the spatial position relationship of the cloud particle image blocks in the original cloud particle image to obtain a first channel feature map, specifically including: obtaining the spatial coordinate information of each cloud particle image block in the original cloud particle image, the spatial coordinate information including the horizontal and vertical coordinate values ​​of the upper left corner pixel of the cloud particle image block in the original cloud particle image; constructing a spatial position mapping table of the cloud particle image block based on the spatial coordinate information, the spatial position mapping table recording the association relationship between the texture feature vector of each cloud particle image block and the corresponding spatial coordinate; performing image block number statistics in the width and height directions of the original cloud particle image to determine the number of cloud particle image blocks contained in the width direction and the number of cloud particle image blocks contained in the height direction; determining the target number of rows and target number of columns of the two-dimensional matrix according to the number of cloud particle image blocks in the width direction and the number of cloud particle image blocks in the height direction, wherein the target number of rows is equal to the number of cloud particle image blocks in the height direction. The number of particle image blocks and the target number of columns are equal to the number of cloud particle image blocks in the width direction; the texture feature vectors of all cloud particle image blocks are sorted in the order of the horizontal and vertical coordinate values ​​in the spatial position mapping table from small to large to obtain a sorted texture feature vector sequence; the texture feature vectors in the sorted texture feature vector sequence are sequentially filled into a two-dimensional matrix frame of the target number of rows and target number of columns, wherein the matrix elements in the i-th row and j-th column correspond to the texture feature vectors of the i-th cloud particle image block in the height direction and the j-th cloud particle image block in the width direction in the original cloud particle image; when the number of cloud particle image blocks is not equal to the product of the target number of rows and the target number of columns, the unfilled positions in the two-dimensional matrix frame are filled with zero vectors so that the filled two-dimensional matrix meets the requirements of the target number of rows and the target number of columns; the filled two-dimensional matrix is ​​subjected to a feature dimension consistency check so that the texture feature vectors of each element in the matrix have the same dimension; the two-dimensional matrix that passes the feature dimension consistency check is used as the first channel feature map; Performing multi-scale region segmentation on the cloud particle image, dividing the cloud particle image into a regional hierarchy of different sizes, performing regional feature statistics on the regions in each regional hierarchy, calculating the grayscale mean, grayscale variance, and spatial distribution density of the pixels in each region, and generating a regional feature vector corresponding to each region; Perform feature fusion on the regional feature vectors in all regional hierarchies, combine the regional feature vectors of different scales into a feature representation of unified dimension through hierarchical feature aggregation, and obtain the second channel feature map; Inputting the first channel feature map and the second channel feature map into a pre-trained feature adaptation model for feature space alignment to obtain an aligned first channel feature map and an aligned second channel feature map; The aligned first channel feature map and the aligned second channel feature map are input into the UNET network for dual-channel feature fusion and recognition to obtain a cloud particle recognition result.

2. The method according to claim 1, wherein Processing each cloud particle image in the cloud particle image set to obtain a first channel feature map and a second channel feature map corresponding to each cloud particle image, including: Performing edge detection on the cloud particle image to identify edge contour pixels in the cloud particle image; constructing an edge contour map based on the edge contour pixel points, calculating the distance and angle between adjacent edge pixel points in the edge contour map, and generating an edge contour feature vector; Performing feature expansion on the edge contour feature vector, converting the one-dimensional edge contour feature vector into a two-dimensional feature matrix through neighborhood feature interpolation to obtain an initial first channel feature map; Performing feature enhancement on the initial first channel feature map by performing local area weighted summation on each element in the feature matrix to strengthen the spatial correlation of edge contour features, thereby obtaining the first channel feature map; Performing region growing on the cloud particle image, gradually merging similar pixel regions starting from a seed pixel point, to obtain multiple cloud particle candidate regions; Calculate the regional attributes of each cloud particle candidate region and generate a regional attribute feature vector; The regional attribute feature vectors of all cloud particle candidate regions are sorted and arranged in descending order according to the region size. The sorted regional attribute feature vectors are then converted into a two-dimensional matrix to obtain the initial second channel feature map. Feature smoothing is performed on the initial second channel feature map to obtain the second channel feature map.

3. The method according to claim 2, wherein The cloud particle image is subjected to region growing, and similar pixel regions are gradually merged starting from a seed pixel point to obtain multiple cloud particle candidate regions, including: Performing grayscale threshold segmentation on the cloud particle image to determine the grayscale value range of the seed pixel points; Selecting a plurality of initial seed pixels in the cloud particle image based on the grayscale value range; Setting a similarity judgment threshold for region growing, which is used to control whether adjacent pixels are merged into the same region; Taking each initial seed pixel as the center, check its neighboring pixels according to the preset search order; Calculate the average grayscale value difference between the neighboring pixels and the current region, and when the average grayscale value difference is less than the similarity judgment threshold, merge the neighboring pixels into the current region; Repeat the neighboring pixel check and merging operations until no new pixels can be merged into the current area; De-duplication is performed on the growth areas corresponding to all initial seed pixels, and adjacent areas whose overlap rate exceeds a preset threshold are merged; The areas after deduplication are screened by area, and areas with areas smaller than a preset area threshold are removed to obtain the multiple cloud particle candidate areas.

4. The method according to claim 1, wherein Inputting the first channel feature map and the second channel feature map into a pre-trained feature adaptation model for feature space alignment to obtain an aligned first channel feature map and an aligned second channel feature map, comprising: Inputting the first channel feature map into the first feature encoding subnetwork of the feature adaptation model for feature encoding to obtain a first encoded feature vector; inputting the second channel feature map into the second feature encoding subnetwork of the feature adaptation model for feature encoding to obtain a second encoded feature vector; Inputting the first encoded feature vector and the second encoded feature vector into a dynamic mapping parameter generation subnetwork of a feature adaptation model to generate a feature mapping matrix; performing a matrix multiplication operation on the first encoded feature vector and the feature mapping matrix to obtain a mapped first encoded feature vector; Calculating a feature difference value between the mapped first encoding feature vector and the second encoding feature vector; Adjusting the network parameters of the dynamic mapping parameter generation sub-network based on the characteristic difference value so that the characteristic difference value satisfies a preset convergence condition; When the feature difference value meets the preset convergence condition, the mapped first encoded feature vector is input into the first feature decoding subnetwork of the feature adaptation model for feature decoding to obtain the aligned first channel feature map; the second encoded feature vector is input into the second feature decoding subnetwork of the feature adaptation model for feature decoding to obtain the aligned second channel feature map.

5. The method according to claim 4, wherein The step of inputting the first encoded feature vector and the second encoded feature vector into a dynamic mapping parameter generation subnetwork of a feature adaptation model to generate a feature mapping matrix includes: Performing feature concatenation on the first encoded feature vector and the second encoded feature vector to obtain a concatenated feature vector; Inputting the concatenated feature vector into the first fully connected layer of the dynamic mapping parameter generation subnetwork for linear transformation to obtain an intermediate feature vector; Inputting the intermediate feature vector into the activation function layer of the dynamic mapping parameter generation subnetwork for nonlinear transformation, inputting the output result of the activation function layer into the second fully connected layer of the dynamic mapping parameter generation subnetwork, and outputting the mapping matrix parameter vector; Converting the mapping matrix parameter vector into a matrix of preset dimensions to obtain an initial feature mapping matrix; Orthogonalizing the initial eigenmapping matrix so that row vectors of the matrix are orthogonal to each other; The feature mapping matrix after orthogonalization is normalized, and the value range of the matrix elements is adjusted to a preset interval to obtain the feature mapping matrix.

6. The method according to claim 5, wherein The step of converting the mapping matrix parameter vector into a matrix of a preset dimension to obtain an initial feature mapping matrix includes: determining a dimension of the first encoded feature vector and a dimension of the second encoded feature vector; Determining a target number of rows and a target number of columns of a feature mapping matrix based on the dimension of the first encoded feature vector and the dimension of the second encoded feature vector; Calculating the number of elements of the mapping matrix parameter vector, and when the number of elements is not equal to the product of the target number of rows and the target number of columns, performing dimension adjustment processing on the mapping matrix parameter vector; Arrange the elements of the mapping matrix parameter vector into a matrix with the target number of rows and columns in row-major order; The arranged matrix is ​​transposed so that the number of rows of the matrix matches the dimension of the first encoded feature vector and the number of columns of the matrix matches the dimension of the second encoded feature vector, and the transposed matrix is ​​used as the initial feature mapping matrix.

7. The method according to claim 1, wherein The step of inputting the aligned first channel feature map and the aligned second channel feature map into a UNET network for dual-channel feature fusion and recognition to obtain a cloud particle recognition result includes: Inputting the aligned first channel feature map into the first encoder of the UNET network for downsampling to obtain a plurality of first-level feature maps; inputting the aligned second channel feature map into the second encoder of the UNET network for downsampling to obtain a plurality of second-level feature maps; Performing feature splicing on the plurality of first-level feature maps and the plurality of second-level feature maps according to a hierarchical correspondence to obtain a plurality of spliced-level feature maps; Input each splicing level feature map into the adaptive channel attention fusion module of the UNET network to generate the fused feature map corresponding to each level; The deepest fusion feature map is input into the decoder of the UNET network for upsampling, and is fused with the shallow fusion feature map through jump connection to obtain the decoded feature map; Input the decoded feature map into the cloud particle category prediction layer of the UNET network for classification, and obtain the probability distribution of each pixel belonging to different cloud particle categories; determining the cloud particle category of each pixel based on the probability distribution, and generating a cloud particle category segmentation map; The cloud particle category segmentation map is post-processed to remove isolated areas with an area smaller than a preset threshold, thereby obtaining the cloud particle recognition result.

8. The method according to claim 7, wherein The feature map of each splicing level is input into the adaptive channel attention fusion module of the UNET network to generate the fused feature map corresponding to each level, including: Dividing the splicing-level feature map into a first channel partial feature map and a second channel partial feature map, wherein the first channel partial feature map corresponds to features obtained by encoding the aligned first channel feature map, and the second channel partial feature map corresponds to features obtained by encoding the aligned second channel feature map; Performing global average pooling on the first channel partial feature map to obtain a first channel global feature vector; performing global average pooling on the second channel partial feature map to obtain a second channel global feature vector; Performing an element-wise addition operation on the first channel global eigenvector and the second channel global eigenvector to obtain a fused global eigenvector; Inputting the fused global feature vector into a fully connected layer for feature conversion to obtain a channel attention weight vector, and normalizing the channel attention weight vector to obtain a normalized channel attention weight vector; Performing an element-wise multiplication operation on the normalized channel attention weight vector and the splicing-level feature map to perform a weighted adjustment on each channel feature in the splicing-level feature map; The weighted adjusted splicing level feature map is input into the convolution layer for feature fusion to obtain the fused feature map.

9. A computer system, characterized in that: include: a memory storing a computer program; A processor, configured to load the computer program to implement the dual-channel cloud particle image recognition method based on transfer learning and UNET as described in any one of claims 1 to 8.

Images