Image recognition system and method for ice and snow crystal particles

Abstract

The present invention discloses an image recognition system and method for ice and snow crystal particles, the system comprising: an image acquirer, which acquires image data and preprocesses to obtain a target image; an ice and snow crystal particle feature extraction model, which receives the target image input and performs feature extraction, and is composed of an embedding unit and an adaptive large-core attention unit connected in series with four modules, wherein the embedding unit receives the target image and generates a high-dimensional embedding feature map, and the adaptive large-core transformation unit accurately captures the fine-grained spatial features of particles with the large receptive field of the large-core attention module, and introduces feature channel scale perception, so that the model focuses on key features, and then performs feature aggregation through an anti-bottleneck feedforward module; finally, the classifier receives the feature output of the recognition model, and uses the difficulty adjustment loss optimization to obtain the final recognition result. The present invention can accurately identify cloud particle images that characterize cloud microphysical processes.

Description

Technical Field

[0001] The present invention relates to the field of data recognition technology, and more particularly to an image recognition system and method for ice and snow crystal particles. Background Art

[0002] With the advancement of science and technology and the development of observation methods, human beings' understanding of ice and snow crystal particles has continued to deepen. From the initial simple observation to today's refined measurement and classification and identification, the research on ice and snow crystal particles has made significant progress. As an important component of water-forming materials in clouds, the formation, growth and final landing of ice and snow crystal particles on the ground are important research topics in the fields of meteorology and atmospheric physics. The shape, size, distribution and other characteristics of ice and snow crystal particles not only reflect the complexity of cloud microphysical processes, but also directly affect precipitation mechanisms, climate patterns and the earth's water cycle. Therefore, accurate identification of ice and snow crystal particles and cloud particle images that characterize cloud microphysical processes are of great significance for in-depth understanding of atmospheric processes, improving the accuracy of weather forecasts, and assessing the impact of climate change.

[0003] Traditional methods for identifying ice and snow crystal particles are mostly based on the identification of particle geometric characteristic parameters. The process is to first pre-process the particle image, and then manually determine the thresholds of these characteristic parameters for identification by fitting characteristic parameters such as diameter, roundness, and correlation coefficients of particle image pixels in multiple directions. Although this method has a certain rate improvement over manual screening, it needs to be adjusted based on experience for different regional environments and detection equipment, and it is difficult to cope with the complex situation of particles in a wide-area environment, including but not limited to the situation of characterizing cloud microphysical processes.

[0004] In the field of deep learning ice and snow crystal particle recognition, some advanced methods use embedded hypergraph convolution, ultra-light quantum space and other technologies to improve the model's understanding of the information of a single particle. Specifically, the hypergraph structure represents the complex relationship between pixels by constructing hyperedges, so that the model not only pays attention to the local features around each pixel, but also captures the relationship between remote vertices through hyperedge connections in the global feature space, thereby obtaining deeper feature information. The ultra-light quantum space strategy cleverly decomposes the feature map into multi-scale subspaces, implements efficient multi-frequency learning, effectively reduces the consumption of computing resources, and improves the operation efficiency of the model. These advanced technologies have been successfully integrated into deep learning models, but the following problems still exist: 1. The current methods focus on the shape recognition of single ice and snow crystal particle images, but the growth of ice crystals in natural environments often has microphysical phenomena such as particle collision and condensation. These cloud particle images that represent cloud microphysical processes often contain multiple particles. The existing models cannot accurately identify cloud particle images that represent cloud microphysical processes. 2. In nature, different types of ice and snow crystal particles and their accompanying cloud microphysical processes have very different frequencies of occurrence, which may cause the model to encounter uneven data distribution during the learning process. In addition, the complexity and recognition of visual features of different types of particles and microphysical phenomena vary, further exacerbating the difficulty of model feature learning. Summary of the invention

[0005] The present invention is proposed based on the above-mentioned needs and relies on the existing technology. The technical problem to be solved by the present invention is an image recognition system and method for ice and snow crystal particles to accurately identify cloud particle images that characterize cloud microphysical processes.

[0006] In order to solve the above problems, the present invention is implemented by adopting the following technical solutions:

[0007] Provided is an image recognition system for ice and snow crystal particles, the system comprising: an image acquirer, which acquires and preprocesses data to obtain a target image; an ice and snow crystal feature extraction recognition model, the ice and snow crystal particle feature extraction model comprising four feature extraction modules connected in sequence, the target image is input to the front end of the first feature extraction module of the recognition model, and the output of any feature extraction module will be used as the input of the next feature extraction module connected thereto; each feature extraction module comprises an embedding unit and a plurality of adaptive large-core transformation units connected thereto in sequence; the embedding unit embeds the received data and inputs the processed data into the adaptive large-core transformation unit connected thereto adjacently; the adaptive large-core transformation unit comprises an adaptive large-core attention module and an anti-bottleneck feedforward module, the adaptive large-core attention module comprises a first deep convolution block processing of the input data of the module to obtain feature data; the feature data are respectively subjected to adaptive normalization processing and deep hole convolution processing, and the two processing results are fused to obtain a fused feature; the fused The features are processed by maximum pooling and average pooling respectively, and the pooling results of the two poolings are spliced ​​to obtain aggregate features; the aggregate features are processed by convolution and activation function to obtain attention features, and the attention features are evenly divided in the channel dimension to obtain the first attention weight and the second attention weight; the fused features are split into channels to obtain the first fused features and the second fused features, and the first fused features and the second fused features are processed and spliced ​​based on the first attention weight and the second attention weight to obtain weighted feature data; the anti-bottleneck feedforward module includes processing the input data of the module through a 1×1 convolution block, a second depth convolution block, an activation layer, a discard layer and a 1×1 convolution block in sequence to obtain feedforward feature data; the output data of the adaptive large kernel transformation unit is obtained based on the weighted feature data and the feedforward feature data, which is used as the input data of the adjacent connected adaptive large kernel transformation; a classifier, the classifier is connected to the end of the ice and snow crystal particle feature extraction model, and the output data of the ice and snow crystal particle feature extraction model is input into the classifier to obtain a recognition result.

[0008] Optionally, the embedding unit in the first feature extraction module is a shallow embedding unit, and the embedding units in other feature extraction modules are fused embedding units; the shallow embedding unit includes using a first convolution block to process the target image to obtain embedded features, and performing batch normalization on the embedded features to obtain output data of the shallow embedding unit; the fused embedding unit includes performing a second convolution block and 1×1 point convolution on the output data of the previous feature extraction module to obtain a first convolution result and a second convolution result; performing batch normalization on the first convolution result to obtain a third convolution result; and inputting the third convolution result and the second convolution result into the fusion block to obtain the output data of the fused embedding unit.

[0009] Optionally, the adaptive large core transformation unit also includes a first batch normalization block, a first residual network, a second batch normalization block, a second residual network and a layer normalization block; the first batch normalization block is used to process the first data input to the adaptive large core transformation unit to obtain second data; the adaptive large core attention module processes the second data to obtain weighted feature data; the weighted feature data is added to the first data through the first residual network, and the addition result is added to the second batch normalization block; the input data of the block is processed by the second batch normalization block to obtain third data; the anti-bottleneck feedforward module processes the third data to obtain feedforward feature data; the feedforward feature data is added to the addition result through the second residual network and added to the layer normalization block to obtain the output data of the adaptive large core transformation unit.

[0010] Optionally, the ice and snow crystal particle feature extraction model includes four feature extraction modules connected in sequence, the third feature extraction module is provided with four adaptive large core transformation units connected in sequence, and the remaining feature extraction modules are provided with two adaptive large core transformation units connected in sequence.

[0011] Optionally, the classifier includes an average pooling layer and a fully connected layer, and the average pooling layer is used to perform 1×1 average pooling on the input data of the classifier in the spatial dimension; and the fully connected layer is used to map the output result of the average pooling layer to obtain the recognition result.

[0012] Optionally, the ice and snow crystal particle feature is enhanced by using the constructed difficulty balance adjustment loss function.

[0013] Take the model for optimization, and the expression of the difficulty balance adjustment loss function is: , , ,in, Indicates i The difficulty factor of samples indicates the difficulty of the model for samples i Correctly predicted as the actual category The confidence level, Represents the model for the sample i Predicted as actual class The probability of C Indicates the number of sample categories that need to be identified, Represents the model for the sample i Prediction j The probability of the class, Indicates the sample category j The frequency factor, express j The number of samples for each category, express i The number of samples for each category, Lrepresents the constructed loss function, N represents the total number of samples, The mean is 0 and the standard deviation is The normal distribution of Indicates the actual category The frequency factor of .

[0014] Optionally, the convolution kernel size of the first depth convolution block is 5×5, the convolution kernel size in the depth void convolution is 7×7, the void ratio is 3, the convolution kernel size in the aggregated feature convolution is 7×7, the convolution kernel size of the second depth convolution block is 3×3, and the activation layer applies the SiLU activation function.

[0015] Optionally, the feature data are subjected to adaptive normalization processing and deep hole convolution processing respectively, and the two processing results are fused to obtain a fused feature: , , , ,in, represents the processing result of the first depth convolution block, represents the first depthwise convolutional block of 5×5, represents the input data of the adaptive large-core attention unit, Adaptive normalization processing results, represents a learnable parameter with the same channel dimension as the feature map obtained by the first depthwise convolution, represents the 2-norm in the width and height dimensions, represents the processing result of the deep hole convolution block, represents a 7×7 deep hole convolution block, Represents fusion features.

[0016] Optionally, the convolution kernel size of the first convolution block is 7×7, the convolution kernel size of the second convolution block is 3×3, the step size is 2, and the step size is 2 in the 1×1 point convolution processing.

[0017] Provided is an image recognition method for ice and snow crystal particles, the method comprising: step one: acquiring image data and preprocessing to obtain a target image; step two: using a shallow embedding unit to embed the target image to obtain first output data; step three: inputting the first output data into a plurality of sequentially connected adaptive large kernel transformation units to obtain second output data as input data for a fusion embedding unit; capturing the particle features of the input data of the unit through the large receptive field of the adaptive large kernel transformation unit, and focusing on the features for identifying ice and snow crystal particles and characterizing cloud microphysical processes; step four: using the fusion embedding unit to perform convolution processing on the input data of the unit with convolution kernels of different sizes, and fusing the convolution results of the convolution kernels of different sizes to obtain third output data; step five: inputting the third output data into a plurality of sequentially connected adaptive large kernel transformation units to obtain fourth output data, and inputting the fourth output data into the fusion embedding unit; step six: repeating steps four to five until a preset number of repetitions is reached, and outputting fifth output data; step seven: inputting the fifth output data into a classifier to obtain a recognition result.

[0018] Compared with the prior art, the present invention provides an image recognition system and method for ice and snow crystal particles. By introducing an adaptive large kernel transformation unit with a large receptive field and adaptive regularization, the model's feature extraction capability for complex ice and snow crystal particles and microphysical phenomena is improved. The difficulty balance adjustment loss function combines the difficulty factor and the frequency factor to enhance the model's recognition effect for difficult samples and few samples, thereby achieving high-precision recognition of complex ice and snow crystal particles and cloud particle images characterizing cloud microphysical processes under natural conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of this specification. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0020] Figure 1 is an architecture diagram of an image recognition system for ice and snow crystal particles provided in this embodiment;

[0021] Figure 2 is a schematic structural diagram of a shallow embedding unit provided in this embodiment;

[0022] Figure 3 is a schematic diagram of the structure of the fusion embedding unit provided in this embodiment;

[0023] Figure 4 is a schematic diagram of the structure of the adaptive large-core attention module provided in this embodiment;

[0024] Figure 5 is a schematic diagram of the structure of the anti-bottleneck feedforward module provided in this embodiment;

[0025] Figure 6 is a schematic diagram of the structure of the adaptive large-core transformation unit provided in this embodiment;

[0026] Figure 7 It is a flow chart of an image recognition method for ice and snow crystal particles provided in this embodiment. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] To facilitate understanding of the embodiments of the present invention, further explanation will be given below with reference to specific embodiments in conjunction with the accompanying drawings. The embodiments do not constitute a limitation on the protection scope of the present invention.

[0029] Example 1

[0030] This embodiment provides an image recognition system for ice and snow crystal particles, including:

[0031] Image acquirer, acquires image data and preprocesses it to obtain the target image.

[0032] The target image may be an image of a single ice or snow crystal particle, or may be an image of cloud particles that characterizes cloud microphysical processes such as collision and adhesion of multiple ice or snow crystal particles.

[0033] Ice and snow crystal particle feature extraction model, the ice and snow crystal particle feature extraction model framework diagram is as follows Figure 1 As shown, the ice and snow crystal particle feature extraction model includes four feature extraction modules connected in sequence, the target image is input to the front end of the first feature extraction module of the recognition model, and the output of any feature extraction module will be used as the input of the next feature extraction module connected thereto; each feature extraction module includes an embedding unit and multiple adaptive large-core transformation units connected in sequence.

[0034] The embedding unit performs embedding processing on the received data, and inputs the processed data into the adaptive large-core transformation unit connected adjacent thereto.

[0035] The embedding unit in the first feature extraction module is a shallow embedding unit, which includes using the first convolution block to process the target image to obtain embedded features, and performing batch normalization processing on the embedded features to obtain output data of the shallow embedding unit.

[0036] For example, Figure 2 As shown in the figure, in the shallow embedding unit, the first convolution block with a convolution kernel size of 7×7 and a step size of 4 is used to embed the data input by the ice and snow crystal particle feature extraction model. These embeddings contain the spatial and color information of the image. Then the batch normalization layer is used to process the embedded data to ensure the stability of the embedded features.

[0037] The embedding unit in other feature extraction modules is a fusion embedding unit, which includes performing a second convolution block and a 1×1 point convolution on the output data of the previous feature extraction module to obtain a first convolution result and a second convolution result; performing batch normalization on the first convolution result to obtain a third convolution result; and inputting the third convolution result and the second convolution result into the fusion block to obtain the output data of the fusion embedding unit.

[0038] Exemplarily, the convolution kernel size of the second convolution block is 3×3, the step size is 2, and the step size is 2 in the 1×1 point convolution processing.

[0039] like Figure 3 As shown in the figure, the fusion embedding unit includes two branches, one of which is a second convolution block with a convolution kernel of 3×3, a stride of 2, and a padding of 1, and a batch normalization block to enhance and stabilize the channel features of the output features of the adaptive large kernel transformation unit; the other branch is a point convolution with a convolution kernel of 1×1 and a stride of 2 to extract more abstract feature representations from the original feature map. The design of two branches with a stride of 2 helps to downsample while extracting features, reduce the spatial dimension of features, increase the channel dimension size, map the feature dimension to a representation suitable for processing, and enhance the semantic representation. Then, the two branch features are fused through the fusion block to enhance the model's ability to capture multi-scale features in the image, which helps to improve the recognition accuracy of different forms and details of ice and snow crystal particles.

[0040] The expression of the fused embedding unit is:

[0041] in, represents the output data of the fused embedding unit, represents batch normalization, represents 3×3 convolution, z represents the input data of the fused embedding unit, Represents a 1×1 convolution.

[0042] The fusion embedding unit aggregates the important features extracted by the adaptive large kernel transformation unit and prevents information loss during the fusion process through short connections.

[0043] The adaptive large-core transformation unit includes an adaptive large-core attention module and an anti-bottleneck feedforward module, which are used to learn high-level semantic features.

[0044] The adaptive large-core attention module includes performing a first deep convolution block processing on the input data of the module to obtain feature data; performing adaptive normalization processing and deep hole convolution processing on the feature data, and fusing the two processing results to obtain fusion features; performing maximum pooling and average pooling processing on the fusion features, and splicing the pooling results of the two pooling to obtain aggregate features; performing convolution and activation function processing on the aggregate features to obtain a first attention weight and a second attention weight; performing channel splitting on the fusion features to obtain a first fusion feature and a second fusion feature, and processing and splicing the first fusion feature and the second fusion feature based on the first attention weight and the second attention weight to obtain weighted feature data.

[0045] Exemplarily, the convolution kernel size of the first depth convolution block is 5×5, the convolution kernel size in the depth hole convolution is 7×7, the hole rate is 3, and the convolution kernel size in the aggregated feature convolution is 7×7.

[0046] like Figure 4 As shown in the figure, in the adaptive large kernel attention module, a 5×5 first deep convolution block is used to preliminarily extract local details and preliminary representations from the input data of the unit to obtain feature data; in order to process these features more deeply and enhance the representation ability of the model, the feature data is divided into two parallel branches for processing: one branch uses an adaptive normalization block to adjust the scale of each channel and reduce the internal covariate offset of the feature; the other branch uses a deep hole convolution block with a convolution kernel of 7×7 and a hole rate of 3 to simulate the effect of large kernel convolution at a lower computational cost, and captures a wider and more holistic shape feature of particles or microphysical phenomena with a large receptive field; the features extracted by the two branches are added together to obtain the fused feature, which is expressed as:

[0047]

[0048]

[0049]

[0050]

[0051] in, represents the processing result of the first depth convolution block, represents the first depthwise convolutional block of 5×5, represents the input data of the adaptive large-core attention unit, Adaptive normalization processing results, represents a learnable parameter with the same channel dimension as the feature map obtained by the first depthwise convolution, represents the 2-norm in the width and height dimensions, represents the processing result of the deep hole convolution block, represents a 7×7 deep hole convolution block, Represents fusion features.

[0052] Then, the global maximum feature and global average feature of the fusion feature are obtained through the maximum pooling layer and the average pooling layer respectively, and the global average feature and the global maximum feature are spliced ​​in the channel dimension to form an aggregate feature. The aggregate feature is compressed through a 7×7 convolution, and two attention weight maps are generated through the Sigmoid activation function, namely the first attention weight and the second attention weight. The expression is:

[0053]

[0054] in, represents the first attention weight corresponding to the global maximum feature, represents the second attention weight corresponding to the global average feature, represents an evenly distributed operation in the channel dimension, Indicates the maximum pooling operation with a convolution kernel of 1×1. Indicates that the convolution kernel is an average pooling operation of 1×1. represents the connection operation in the channel dimension, represents a 7×7 large kernel convolution, Represents the sigmoid activation function.

[0055] The fused feature is divided into two parts according to half of the number of channels, namely the first fused feature and the second fused feature. The first fused feature and the second fused feature are multiplied by the corresponding attention weights respectively, and then reassembled to form the final weighted feature data:

[0056]

[0057]

[0058] in, represents the first fusion feature, represents the second fusion feature, represents weighted feature data, Represents element-wise multiplication.

[0059] The adaptive large-core attention module aims to improve the sensitivity and robustness of the model to complex feature recognition. This unit is a carefully designed network structure. After the input feature undergoes a 5×5 deep convolution, it passes through an adaptive normalization branch and uses the dimension as a learnable parameter of the number of feature channels. The 2-norm value of the feature map in the spatial domain is combined with the scale difference of the feature channel domain to learn the feature. This allows the feature map of each channel to be normalized independently, making the elements in each channel closer in value, improving the numerical stability during model training, and making the model pay more attention to the direction and relative size of the features, rather than their absolute size, so as to extract features with stable spatial state and channel importance. The deep hole convolution branch simulates the large kernel convolution in the spatial domain by combining the previous 5×5 deep convolution, which can capture a wider range of spatial context information, enhance the model's perception of global information, and strengthen the model's ability to extract spatial details. By splitting and reorganizing the fused features on the channel, extracting the attention weight of the maximum spatial feature and the attention weight of the average feature, the extraction effect of the original feature channel and space is further strengthened, so that the model can pay attention to the edge, shape, texture, and light and shadow detail features while maintaining the overall shape and semantic understanding.

[0060] The anti-bottleneck feedforward module includes processing the input data of the unit through a 1×1 convolution block, a second depth convolution block, an activation layer, a discard layer and a 1×1 convolution block in sequence to obtain feedforward feature data.

[0061] Exemplarily, the convolution kernel size of the second depth convolution block is 3×3, and the activation layer applies the SiLU activation function. Figure 5 As shown in the figure, the input data of the anti-bottleneck feedforward module is upgraded through a 1×1 convolution block, and then transformed in high-dimensional space using a 3×3 deep convolution block and an activation layer. Then, a dropout layer is used to randomly inactivate some activations to prevent overfitting, and finally, a 1×1 convolution block is used to reduce the dimension to the input feature dimension.

[0062] The anti-bottleneck feedforward module uses dimensionality increase and uses deep convolution for nonlinear transformation activation and then dimensionality reduction. Nonlinear operations are performed first in high-dimensional space to effectively reduce information loss, so that the model can capture more complex relationships between input features and improve the ability to represent data. Deep convolution can also reduce complexity and improve the reasoning speed of the model. Combined with the random inactivation of the discard layer, the adaptive adjustment enables the model to better generalize to unseen data and reduce the risk of overfitting. The expression of this process is:

[0063]

[0064] in, represents the feedforward feature data, represents a 1×1 convolution, represents a dropout layer with random dropout, represents the SiLU nonlinear activation function, represents the second depth convolutional block of 3×3, Represents the input data of the anti-bottleneck feed-forward module.

[0065] The output data of the adaptive large-core transformation unit is obtained based on the weighted feature data and the feedforward feature data, and is used as the input data of the adjacently connected adaptive large-core transformation unit.

[0066] The adaptive large kernel transformation unit also includes a first batch normalization block, a first residual network, a second batch normalization block, a second residual network and a layer normalization block.

[0067] like Figure 6 As shown, the processing process of the adaptive large core transformation unit includes:

[0068] The first batch normalization block is used to process the first data input to the adaptive large kernel transformation unit to obtain the second data; the adaptive large kernel attention module processes the second data to obtain weighted feature data; the weighted feature data is added to the first data through the first residual network to obtain a first addition result, which is expressed as:

[0069]

[0070] in, l Indicates the identification number of the adaptive large core transformation unit, Represents the adaptive large kernel transform unit l The first addition result of represents the adaptive large-core attention module, Represents the adaptive large kernel transform unit l The first data.

[0071] The first addition result is processed by the second batch normalization block to obtain third data; the anti-bottleneck feedforward module processes the third data to obtain feedforward feature data; the feedforward feature data is added to the first addition result through the second residual network to obtain a second addition result, which is expressed as:

[0072]

[0073] in, Represents the adaptive large kernel transform unit l The second addition result is represents the anti-bottleneck feed-forward module.

[0074] The second addition result is subjected to layer normalization to obtain the output data of the adaptive large kernel transformation unit, which is expressed as follows:

[0075]

[0076] in, Represents the adaptive large kernel transform unit l Output data.

[0077] In summary, the adaptive large kernel transform unit is based on the transformer structure and constitutes the core of the model. The input features are regularized through the first batch normalization block, and the large receptive field of the quasi-large kernel convolution of the adaptive large kernel attention module is used to enable the model to understand the contextual information in a wide range of images, especially the complex structures and shapes in the case of multi-particle collision and adhesion. The anti-bottleneck feedforward unit is used to perform nonlinear transformation on the feature map through three convolution blocks and SiLU nonlinear activation function, and the important features of the adaptive large kernel attention module are extended to represent, so that the model can further understand the complex patterns in the image and improve the recognition ability of different types of particles and microphysical phenomena. The batch normalized features are added to the results of the anti-bottleneck feedforward module through residual connections, and layer normalization is performed, so that the model can maintain feature stability while being easy to superimpose.

[0078] The classifier is connected to the end of the ice and snow crystal particle feature extraction model, and the output data of the ice and snow crystal particle feature extraction model is input into the classifier to obtain a recognition result.

[0079] The classifier includes an average pooling layer and a fully connected layer. The average pooling layer is used to perform 1×1 average pooling on the input data of the classifier in the spatial dimension; the fully connected layer is used to map the output result of the average pooling layer to obtain the recognition result.

[0080] Preferably, the ice and snow crystal particle feature extraction model includes four feature extraction modules connected in sequence, the third feature extraction module is provided with four adaptive large core transformation units connected in sequence, and the remaining feature extraction modules are provided with two adaptive large core transformation units connected in sequence.

[0081] In order to further enhance the recognition effect of particles with different difficulties and sample data and cloud particle images representing cloud microphysical processes, a difficulty balance adjustment loss function is proposed to perform the ice and snow crystal particle feature

[0082] Extract model learning optimization, the difficulty balance adjustment loss function is:

[0083]

[0084]

[0085]

[0086] in, Indicates i The difficulty factor of samples indicates the difficulty of the model for samples i Correctly predicted as the actual category The confidence level, Represents the model for the sample i Predicted as actual class The probability of C Indicates the number of sample categories that need to be identified, Represents the model for the sample i Prediction j The probability of the class, Indicates the sample category j The frequency factor, express j The number of samples for each category, express i The number of samples for each category, L represents the constructed loss function, N represents the total number of samples, The mean is 0 and the standard deviation is The normal distribution of Indicates the actual category The frequency factor of .

[0087] Difficulty Balance Adjustment Loss Function The difficulty balance adjustment loss function adopts a two-pronged strategy to comprehensively optimize the model. Specifically, the model is optimized from two aspects: the number of samples and the difficulty of recognition through the frequency factor and the difficulty factor. The frequency factor adjusts the weights of different categories, giving greater importance to those categories with fewer samples, so that the model can learn the uncommon characteristics of ice and snow crystal particles more deeply, which helps to alleviate the problem of few samples of some categories of ice and snow crystal particles in natural conditions. The difficulty factor dynamically calculates the weight according to the difference between the predicted value of the correct category of each sample and the average of the predicted values ​​of all samples. It will give greater weight to those samples that are difficult to identify, prompting the model to focus more on these samples during the training process, which will help to further improve the recognition effect of complex types of particles and cloud particle images that characterize cloud microphysical processes.

[0088] The target image is input into the best model trained by using the difficulty balance adjustment loss function, and the final recognition result is obtained by the classifier. In this embodiment, the model parameters with the best recognition effect are obtained after 200 rounds of training.

[0089] Compared with the prior art, this embodiment provides an image recognition system for ice and snow crystal particles. It introduces an adaptive large kernel transformation unit with a large receptive field and adaptive regularization to improve the model's feature extraction capability for complex ice and snow crystal particles and cloud particle images representing cloud microphysical processes. The difficulty balance adjustment loss function combines the difficulty factor and the frequency factor to enhance the model's recognition effect for difficult samples and few samples, thereby achieving high-precision recognition of complex ice and snow crystal particles and cloud particle images representing cloud microphysical processes under natural conditions.

[0090] Example 2

[0091] This embodiment provides an image recognition method for ice and snow crystal particles, which implements an image recognition system for ice and snow crystal particles described in Example 1. The process is as follows: Figure 7 As shown, including:

[0092] S1 acquires image data and preprocesses it to obtain a target image.

[0093] S2 uses a shallow embedding unit to embed the target image to obtain first output data.

[0094] S3 inputs the first output data into a plurality of adaptive large kernel transformation units connected in sequence to obtain second output data as input data of a fusion embedding unit; captures the particle features of the input data of the unit through the large receptive field of the adaptive large kernel transformation unit, and focuses on identifying ice and snow crystal particles and characterizing cloud microphysical processes.

[0095] S4 uses the fused embedding unit to perform convolution processing on the input data of the unit with convolution kernels of different sizes, and fuses the convolution results of the convolution kernels of different sizes to obtain the third output data.

[0096] S5 inputs the third output data into a plurality of sequentially connected adaptive large-core transformation units to obtain fourth output data, and inputs the fourth output data into a fusion embedding unit.

[0097] S6 repeats S4 to S5 until a preset number of repetitions is reached, and outputs the fifth output data.

[0098] S7 inputs the fifth output data into a classifier to obtain a recognition result.

[0099] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An image recognition system for ice and snow crystal particles, characterized in that: include: An image acquirer acquires image data and preprocesses it to obtain a target image; An ice and snow crystal particle feature extraction model, wherein the ice and snow crystal particle feature extraction model comprises four feature extraction modules connected in sequence, wherein the target image is input to the front end of the first feature extraction module of the ice and snow crystal particle feature extraction model, and the output of any feature extraction module will be used as the input of the next feature extraction module connected thereto; Each feature extraction module includes an embedding unit and a plurality of adaptive large-core transformation units connected in sequence; The embedding unit generates embedded feature data from the received data, and inputs the processed data into the adaptive large-core transformation unit connected thereto; The adaptive large core transformation unit includes an adaptive large core attention module and an anti-bottleneck feedforward module, wherein the adaptive large core attention module includes performing a first deep convolution block processing on the input data of the module to obtain feature data; The feature data are respectively subjected to adaptive normalization processing and deep hole convolution processing, and the two processing results are fused to obtain fused features; the fused features are respectively subjected to maximum pooling and average pooling processing, and the pooling results of the two pooling processes are spliced ​​to obtain aggregated features; Performing convolution and activation function processing on the aggregated features to obtain an attention feature, and averaging the attention feature in the channel dimension to obtain a first attention weight and a second attention weight; Performing channel splitting on the fused feature to obtain a first fused feature and a second fused feature, and processing and splicing the first fused feature and the second fused feature based on the first attention weight and the second attention weight to obtain weighted feature data; The anti-bottleneck feedforward module includes processing the input data of the module through a 1×1 convolution block, a second depth convolution block, an activation layer, a discard layer and a 1×1 convolution block in sequence to obtain feedforward feature data; Based on the weighted feature data and the feedforward feature data, output data of the adaptive large kernel transformation unit is obtained as input data of the adjacently connected adaptive large kernel transformation unit; A classifier, wherein the classifier is connected to the end of the ice and snow crystal particle feature extraction model, and the output data of the ice and snow crystal particle feature extraction model is input into the classifier to obtain a recognition result; The embedding unit in the first feature extraction module is a shallow embedding unit, and the embedding units in other feature extraction modules are fused embedding units; The shallow embedding unit includes processing the target image using a first convolution block to obtain an embedding feature, and performing batch normalization processing on the embedding feature to obtain output data of the shallow embedding unit; The fusion embedding unit includes performing a second convolution block and a 1×1 point convolution process on the output data of the previous feature extraction module respectively to obtain a first convolution result and a second convolution result; performing batch normalization process on the first convolution result to obtain a third convolution result; and inputting the third convolution result and the second convolution result into the fusion block to obtain the output data of the fusion embedding unit; The adaptive large kernel transform unit further includes a first batch normalization block, a first residual network, a second batch normalization block, a second residual network and a layer normalization block; Processing the first data input to the adaptive large kernel transformation unit using the first batch normalization block to obtain second data; The adaptive large core attention module processes the second data to obtain weighted feature data; adding the weighted feature data to the first data through the first residual network, and adding the addition result to the second batch normalization block; Processing the input data of the block using a second batch normalization block to obtain third data; The anti-bottleneck feedforward module processes the third data to obtain feedforward feature data; The feedforward feature data is added to the addition result through the second residual network and added to the layer normalization block to obtain the output data of the adaptive large kernel transformation unit.

2. The image recognition system for ice and snow crystal particles according to claim 1, characterized in that: The ice and snow crystal particle feature extraction model includes four feature extraction modules connected in sequence, the third feature extraction module is provided with four adaptive large core transformation units connected in sequence, and the remaining feature extraction modules are provided with two adaptive large core transformation units connected in sequence.

3. The image recognition system for ice and snow crystal particles according to claim 1, characterized in that: The classifier includes an average pooling layer and a fully connected layer. The average pooling layer is used to perform 1×1 average pooling on the input data of the classifier in the spatial dimension; and the fully connected layer is used to map the output result of the average pooling layer to obtain a recognition result.

4. The image recognition system for ice and snow crystal particles according to claim 1, characterized in that: The constructed difficulty balance adjustment loss function is used to optimize the ice and snow crystal particle feature extraction model. The expression of the difficulty balance adjustment loss function is: in, Indicates i The difficulty factor of samples indicates the difficulty of the model for samples i Correctly predicted as the actual category The confidence level, Represents the model for the sample i Predicted as actual class The probability of C Indicates the number of sample categories that need to be identified, Represents the model for the sample i Prediction j The probability of the class, Indicates the sample category j The frequency factor, express j The number of samples for each category, express i The number of samples for each category, L represents the constructed loss function, N represents the total number of samples, The mean is 0 and the standard deviation is The normal distribution of Indicates the actual category The frequency factor of .

5. The image recognition system for ice and snow crystal particles according to claim 1, characterized in that: The convolution kernel size of the first depth convolution block is 5×5, the convolution kernel size in the depth void convolution is 7×7, the void ratio is 3, the convolution kernel size in the aggregated feature convolution is 7×7, the convolution kernel size of the second depth convolution block is 3×3, and the activation layer applies the SiLU activation function.

6. The image recognition system for ice and snow crystal particles according to claim 5, characterized in that: The feature data are subjected to adaptive normalization processing and deep hole convolution processing respectively, and the two processing results are fused to obtain fusion features: in, represents the processing result of the first depth convolution block, represents the first depthwise convolutional block of 5×5, represents the input data of the adaptive large-core attention unit, Adaptive normalization processing results, represents a learnable parameter with the same channel dimension as the feature map obtained by the first depthwise convolution, represents the 2-norm in the width and height dimensions, represents the processing result of the deep hole convolution block, represents a 7×7 deep hole convolution block, Represents fusion features.

7. The image recognition system for ice and snow crystal particles according to claim 1, characterized in that: The convolution kernel size of the first convolution block is 7×7, the convolution kernel size of the second convolution block is 3×3, the step size is 2, and the step size is 2 in the 1×1 point convolution processing.

8. A method for image recognition of ice and snow crystal particles, realizing an image recognition system for ice and snow crystal particles according to any one of claims 1 to 7, characterized in that: include: Step 1: Acquire image data and preprocess to obtain the target image; Step 2: embedding the target image using a shallow embedding unit to obtain first output data; Step 3: Input the first output data into a plurality of sequentially connected adaptive large-core transformation units to obtain second output data as input data of a fusion embedding unit; capture the particle features of the input data of the unit through the large receptive field of the adaptive large-core transformation unit, and focus on the features of identifying ice and snow crystal particles and their microphysical phenomena; Step 4: Use the fused embedding unit to perform convolution processing on the input data of the unit with convolution kernels of different sizes, and fuse the convolution results of the convolution kernels of different sizes to obtain the third output data; Step 5: inputting the third output data into a plurality of sequentially connected adaptive large-core transformation units to obtain fourth output data, and inputting the fourth output data into a fusion embedding unit; Step 6: Repeat step 4 to step 5 until the preset number of repetitions is reached, and output the fifth output data; Step 7: Input the fifth output data into a classifier to obtain a recognition result.

Images