A polarized image recognition and pixel-level classification method

Abstract

The present application belongs to the technical field of polarization imaging, image classification, pixel-level classification and deep learning, and particularly relates to a polarization image recognition and pixel-level classification method. The method can distinguish polarization images from non-polarization images, and further recognize five types of polarization images, including I polarization images, Q polarization images, U polarization images, DOP polarization images and AOP polarization images. First, since the characteristics of Q polarization images and U polarization images are similar, the present application first classifies the two types of polarization images into one type of QU polarization image, and uses a polarization image five-classification ResNet50 residual network model to perform five-classification training of I, QU, DOP, AOP and non-polarization images. Second, the QU polarization image is subjected to image sharpening processing, which enhances the difference in polarization characteristics of the two types of images, and then a polarization image two-classification ResNet50 residual network model is used to perform two-classification training of Q polarization images and U polarization images, thereby improving the classification accuracy.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of polarization imaging, image classification, pixel-level classification and deep learning, and particularly relates to a polarization image recognition and pixel-level classification method. BACKGROUND

[0002] Polarization is one of the basic characteristics of light, and polarization imaging technology is an observation means based on polarization characteristic information. When natural light is irradiated onto the surface of different substances, different states of polarized light are generated by reflection of the objects, and the differences in the polarization characteristics of the reflected light reflect the physical information such as the surface material, roughness and shape of the object. By using this polarization characteristic, it is possible to distinguish substances that are difficult to distinguish with the naked eye. Therefore, by using polarization characteristic information, it is possible to better extract target objects from a complex background and improve the accuracy of target recognition and extraction. Polarization imaging technology has a relatively wide application in the fields of target detection, material identification and medical imaging.

[0003] A polarization imaging acquisition system will acquire multiple intensity images at different polarization angles, generally four images at 0°, 45°, 90° and 135°. By using the Stokes formula for calculation, three polarization images with polarization information, I, Q and U, can be obtained, where I represents the light intensity, Q represents the intensity difference between the horizontal and vertical polarization directions, and U represents the intensity difference between the two diagonal polarization directions. Further calculation can obtain the degree of polarization (DOP) and the angle of polarization (AOP) images. The above five polarization images are widely used in the fields of polarization feature extraction, polarization image fusion and polarization image target detection. However, the current polarization image application is relatively limited. The polarization image is different from the ordinary gray-scale image, and the polarization characteristics of the Q and U polarization images are similar. Certain polarization knowledge is required to distinguish the five types of polarization images, namely, I polarization image, Q polarization image, U polarization image, DOP polarization image and AOP polarization image, and non-polarization image, so as to interpret the polarization information in the image, which greatly limits the application range.

[0004] Image classification is a basic task in the field of computer vision at present, and is also the basis for deep visual tasks such as target detection, image segmentation and target tracking. Different features reflected in image information are used as the basis to realize the differentiation of different types of images. In recent years, deep learning convolutional neural networks have been widely applied in the field of image classification. By learning a large number of training samples and iteratively optimizing the weight parameters of the network model, a network model with specific classification function can be obtained. This model can effectively extract image features and use them as the basis for image classification.

[0005] Pixel-level classification is to identify and classify the object corresponding to a pixel point and its adjacent pixel points according to the feature information of the pixel point, and then to classify each point of the whole image, and finally to obtain the classification result of each pixel. Pixel-level classification is mainly applied to target detection, information extraction and other fields. Researchers can identify and extract target objects from collected images, and it is widely used in industrial manufacturing, military detection, intelligent sensing and other fields. With the improvement of identification accuracy requirements and the increase of background complexity in application scenarios, the images captured by ordinary cameras cannot well support target extraction and identification in some cases, and more feature information needs to be obtained to improve the accuracy of target identification and extraction in complex natural environment background. SUMMARY

[0006] The technical problem of the present application is to overcome the shortcomings of the prior art and provide a polarization image recognition and pixel-level classification method. First, the method can identify and distinguish five types of polarization images, I, Q, U, DOP and AOP, and non-polarization images. Then, the pixel-level classification of polarization images is realized by using polarization features, and better classification accuracy can be obtained.

[0007] The technical solution of the present application is:

[0008] A polarization image recognition and pixel-level classification method comprises the following steps:

[0009] S1: input the polarization images and non-polarization images of unknown categories of the target scene into the polarization image six-classification network model for six-classification, to obtain six-classification images of the target scene, and the six-classification images are respectively I polarization images, Q polarization images, U polarization images, DOP polarization images, AOP polarization images and non-polarization images;

[0010] S2: stack the I polarization images, Q polarization images, U polarization images, DOP polarization images and AOP polarization images in the six-classification images obtained in step S1 together to form five-channel polarization images of the target scene;

[0011] S3: input the five-channel polarization images of the target scene stacked in step S2 into a pixel-level classification network model to judge the category of each point of the five-channel polarization images of the target scene, and complete the pixel-level classification of the five-channel polarization images of the target scene.

[0012] In step S1, the specific steps of the six-classification method are:

[0013] S101: establish a data set for polarization image classification task by using a sufficient number of I polarization images, Q polarization images, U polarization images, DOP polarization images, AOP polarization images and non-polarization images with known categories, wherein the calculation formula of I polarization images, Q polarization images and U polarization images is:

[0014]

[0015] where I0, I 45 , I 90 and I 135 are images of different polarization angles taken by a polarization camera;

[0016] Calculate the degree of polarization DOP image and the polarization angle AOP image:

[0017]

[0018]

[0019] S102: Train the polarization image five-classification ResNet50 residual network model, classify the Q polarization image and the U polarization image obtained in step S101 as a QU polarization image, establish an I polarization image, a QU polarization image, a DOP polarization image, an AOP polarization image and a non-polarization image five-classification task data set, and the true value label set of the five-classification task data set is N1 is the number of images in the five-classification task data set, and the five-classification task data set is input into the polarization image five-classification ResNet50 residual network model to perform an I, QU, DOP, AOP and non-polarization image five-classification task, and the classification prediction value set of the five-classification task data set is obtained The prediction value of each image is a vector with a length of M1, M1 is the number of classes of the five-classification task, and each element is the probability of the image being predicted as each class;

[0020] S103: According to the true value set Y 1 and the prediction value set , construct a cross-entropy loss function

[0021] where N1 is the number of images in the five-classification task data set, c is the image class, M1 is the number of classes in the five-classification task data set, y ic is a symbol function 0 or 1, which takes 1 if the true class of sample i is equal to c, and 0 otherwise, p ic is the predicted probability that sample i belongs to class c, and the true value set Y 1 and the prediction value set whether the loss function value between the true value set Y and the prediction value set satisfies the preset requirement, if yes, the current five-classification network model is the final polarized image five-classification ResNet50 residual network model, the training of the I, QU, DOP, AOP and non-polarized image five-classification ResNet50 residual network model is completed, and step S105 is entered, if not, step S104 is entered;

[0022] S104: updating the parameter weight in the polarized image five-classification ResNet50 residual network model according to the set rule, re-executing steps S102-S103, updating the prediction value set of the five-classification task data set and recalculating the loss function value between the true value set Y and the prediction value set 1 of the five-classification task data set until the preset requirement is met;

[0023] S105: performing edge sharpening processing on the QU polarized image;

[0024] S106: training the polarized image two-classification ResNet50 residual network model, the QU polarized image after the edge sharpening in step S105 is used as the data set of the Q and U polarized image two-classification task, and the true value label set of the two-classification task data set is N2 is the number of images in the two-classification task data set, the two-classification task data set is input into the polarized image two-classification ResNet50 residual network model to perform the Q and U image two-classification task, and the prediction value set of the two-classification task data set is obtained the prediction value of each image is a vector with a length of M2, M2 is the number of two-classification task categories, and each element is the probability that the image is predicted as each category;

[0025] S107: constructing a cross-entropy loss function according to the Q and U polarized image true value set Y 2 and the prediction value set

[0026]

[0027] wherein N2 is the number of images in the two-classification task data set, c is the image category, M2 is the number of categories in the two-classification task data set, y ic is a symbol function 0 or 1, which is 1 if the real category of sample i is equal to c, and 0 otherwise, p ic is the prediction probability that the observation sample i belongs to the category c, the true value set Y 2 and the prediction value set ​​whether the loss function value between the true value set Y and the prediction value set Y satisfies the preset requirement, if yes, the current binary classification network model is the final Q and U polarization image binary classification ResNet50 residual network model, the training of the Q and U polarization image binary classification ResNet50 residual network model is completed, and step S109 is entered, if not, step S108 is entered;

[0028] S108: updating the parameter weight in the polarization image binary classification ResNet50 residual network model according to the set rule, re-executing steps S106-S107, updating the prediction value set Y of the binary classification task data set and re-calculating the loss function value between the true value set Y and the prediction value set Y until the preset requirement is satisfied. 2 and the prediction value set Y

[0029] S109: inputting the polarization image and the non-polarization image of the unknown category of the target scene into the trained I, QU, DOP, AOP and non-polarization image five-classification ResNet50 residual network model first, performing five-classification prediction, performing image edge sharpening processing on the polarization image with the prediction result of QU, then inputting the polarization image into the trained Q and U polarization image binary classification ResNet50 residual network model, performing Q and U image classification prediction, and finally obtaining the category of all images of the target scene, thereby realizing the identification of the I polarization image, the Q polarization image, the U polarization image, the DOP polarization image, the AOP polarization image and the non-polarization image of the target scene.

[0030] In step S102, the ResNet50 residual classification network model comprises 49 convolution layers and 1 fully connected layer, first performs convolution operation on the input image, then passes through 4 residual blocks, and finally performs fully connected operation to output the classification result.

[0031] The training of the ResNet50 residual network model is iterated for a total of 260 times, the learning rate of the first 200 iterations is 0.0001, the learning rate of the last 60 iterations is 0.00001, and the Batch Size is 8.

[0032] In step S102, the Q polarization image and the U polarization image are first combined into a QU polarization image for five-classification training, then the QU polarization image is subjected to edge sharpening processing to enhance the polarization characteristic difference between the Q polarization image and the U polarization image, and then the Q polarization image and the U polarization image are classified.

[0033] In S105, the method for performing edge sharpening processing on the QU polarization image is to use the Sobel operator edge sharpening algorithm to enhance the polarization feature difference between the Q polarization image and the U polarization image.

[0034] ​The step S106, the ResNet50 residual classification network model network contains 49 convolution layers and 1 full connection layer, first carries out convolution operation to input image, then passes through 4 residual blocks, finally carries out full connection operation and outputs classification result, the ResNet50 residual network model is trained and iterates 260 times totally, in order to obtain better training effect, the learning rate of the first 200 iterations is 0.0001, the learning rate of the last 60 iterations is 0.00001, and Batch Size is 8.

[0035] The step S3, the pixel-level classification method of five-channel polarization image is specifically:

[0036] S301: prepare a plurality of known scene polarization images, stack the I, Q, U, DOP and AOP five polarization images of the same scene together to form a five-channel polarization image pixel-level classification task data set, manually label all objects in the polarization image data set according to the polarization image, and form the true value label of the pixel-level classification task data set. The size of the pixel-level classification task data set true value label is consistent with the spatial size of the five-channel polarization image, and different intensity values represent different objects.

[0037] S302: training five-channel polarization image pixel-level classification ResNet18 residual network model, inputting the five-channel polarization image pixel-level classification task data set stacked in step S301 into the pixel-level classification ResNet18 residual network model for training, inputting the five-channel image of 15x15 pixel points centered on the target pixel point into the pixel-level classification ResNet18 residual network model each time, outputting the label prediction value of the target pixel point, and finally obtaining the pixel-level classification task data set prediction value set consistent with the spatial size of the five-channel polarization image.

[0038] S303: according to the true value label set and the prediction value set of the pixel-level classification data set, the loss function is constructed, whether the loss function value between the true value set and the prediction value set of the pixel-level classification task data set meets the preset requirement is judged, if it meets, the current pixel-level classification network model is the final pixel-level classification ResNet18 residual network model, the training of the five-channel polarization image pixel-level classification ResNet18 residual network model is completed, and step S305 is entered, if it does not meet, step S304 is entered.

[0039] S304: updating the parameter weight in the pixel-level classification ResNet18 residual network model according to the set rule, re-executing steps S302-S303, updating the prediction value set of the pixel-level classification task data set, and re-calculating the loss function value between the true value set and the prediction value set of the pixel-level classification task data set until the preset requirement is met.

[0040] S305: The target scene five-channel polarization image is first subjected to image segmentation preprocessing, the image segmentation preprocessing divides one image into several regions according to the polarization characteristics of the five channels, the image of each channel in the five-channel polarization image is respectively subjected to summation and averaging operation according to the segmentation region, then the obtained average value is filled back into the corresponding segmentation region to cover the original data, and the target scene five-channel polarization image after segmentation preprocessing is constructed;

[0041] S306: The target scene five-channel polarization image after segmentation preprocessing is input into the trained five-channel polarization image pixel-level classification ResNet18 residual network model, the point-by-point input into the network wastes operation resources, therefore, only 15*15 pixel points of the five-channel image centered on the center pixel point in each segmentation region are input, the predicted value result of the center pixel point is output, the predicted value result is used as the label of the entire segmentation region, the pixel points with a distance less than 7 to the boundary in each segmentation region are separately input into the network for calculation, and finally the pixel-level classification result of the entire target scene five-channel polarization image is obtained.

[0042] In the step S302, the ResNet18 residual network is composed of multiple residual units, includes 17 convolutional layers and 1 fully connected layer, and the finally output result is the target category information corresponding to each pixel point of the entire image.

[0043] In the step S305, the image segmentation preprocessing divides the image into several mutually disjoint regions according to the intensity value and spatial texture feature, and the original polarization image is preliminarily segmented by using an image segmentation algorithm.

[0044] Beneficial effects

[0045] 1. The application provides a polarization image classification and recognition method based on a ResNet50 residual network deep learning model, which can distinguish polarization images from non-polarization images, and further recognize five types of polarization images, namely I polarization images, Q polarization images, U polarization images, DOP polarization images and AOP polarization images with different polarization characteristics. First, since the Q polarization image and the U polarization image have similar characteristics, the application classifies the two types of polarization images into one type of QU polarization image, and uses a polarization image five-classification ResNet50 residual network model to perform I, QU, DOP, AOP and non-polarization image five-classification training. Secondly, the QU polarization image is subjected to image sharpening processing, which enhances the difference in polarization characteristics between the two types of images, and then a polarization image two-classification ResNet50 residual network model is used to perform Q polarization image and U polarization image two-classification training, thereby improving the classification accuracy.

[0046] 2. The application provides a polarization image pixel-level classification method based on a five-channel polarization image pixel-level classification ResNet18 residual network model, which utilizes a deep learning network to learn the polarization information of known scene target objects, relies on different polarization characteristic information of different objects, combines an image segmentation algorithm, identifies the object categories of each pixel point of the target scene polarization image, and realizes pixel-level classification of the polarization image.

[0047] 3. The application provides a polarization image recognition and pixel-level classification method, which utilizes a deep learning network to identify polarization images and non-polarization images, and further distinguishes five types of polarization images, namely I, Q, U, DOP and AOP, which have different polarization characteristics. Considering that the polarization characteristics of Q polarization images and U polarization images are similar, the Q and U images are first combined into one type, and then five classification of I, QU, DOP, AOP and non-polarization images is performed, and then the edge sharpening method is used to enhance the polarization characteristic difference of the Q and U polarization images, and then two classification of QU polarization images is performed, which can effectively improve the classification accuracy of the polarization image. Then, the recognized polarization images are stacked together, and the next step of polarization image pixel-level classification is performed, the polarization information of known scene target objects is learned by using a deep learning network, the object categories of each pixel point of the target scene polarization image are identified by relying on different polarization characteristic information of different objects, and the pixel-level classification of the polarization image is realized.

[0048] 4. The application aims to identify polarization images and non-polarization images by a deep learning method, and further distinguish polarization images with different polarization characteristics, so that researchers can better use and identify polarization images, and then perform the next step of pixel-level classification by using the recognized polarization images, and identify the categories of objects corresponding to each pixel point in the image according to the polarization characteristics. BRIEF DESCRIPTION OF DRAWINGS

[0049] Figure 1 A polarization image recognition and pixel-level classification method flowchart is provided for the embodiments of the application.

[0050] Figure 2 A ResNet50 residual network model structure diagram is provided.

[0051] Figure 3 A comparison diagram of Q and U polarization images with similar polarization characteristics under the same scene is provided.

[0052] Figure 4 An image segmentation preprocessing result diagram is provided.

[0053] Figure 5 A residual unit structure model diagram in the ResNet residual network is provided. DETAILED DESCRIPTION

[0054] The application will be described in detail below with reference to the drawings and examples.

[0055] A polarization image recognition and pixel-level classification method, specifically comprising the following steps:

[0056] S1: inputting the polarization image and non-polarization image of an unknown class of a target scene into a polarization image six-classification network model, and the polarization image six-classification network model performing six classification, i.e., identifying and distinguishing five types of polarization images of the target scene, including I polarization image, Q polarization image, U polarization image, DOP polarization image and AOP polarization image, and non-polarization image according to polarization characteristics;

[0057] S2: stacking the five types of polarization images of the target scene, including I, Q, U, DOP and AOP, distinguished in step S1, together to form a five-channel polarization image of the target scene, and taking the five-channel polarization image of the target scene as input data for pixel-level classification;

[0058] S3: inputting the five-channel polarization image of the target scene stacked in step S2 into a pixel-level classification network model, and the pixel-level classification network model judging the class of each point of the five-channel polarization image to complete pixel-level classification of the five-channel polarization image of the target scene, and completing polarization image recognition and pixel-level classification;

[0059] In step S1, the polarization image six-classification network model considers that the polarization characteristics of Q polarization image and U polarization image are similar, first merges the Q polarization image and the U polarization image into one type to obtain a QU polarization image, then performs five classification of I, QU, DOP, AOP and non-polarization image, and then enhances the polarization characteristic difference of the Q and U two types of polarization images through an edge sharpening processing method to perform two classification of the QU polarization image;

[0060] The specific steps of the polarization image six-classification method are as follows:

[0061] S101: preparing a sufficient number of I, Q, U, DOP, AOP and non-polarization images of known classes to establish a data set for a polarization image classification task, wherein the calculation method of the I, Q and U three types of polarization images is as follows:

[0062]

[0063] wherein I0, I 45 , I 90 and I 135 are four images of different polarization angles shot by a polarization camera, and the I, Q and U three images are calculated according to the above formula, and then a polarization degree (DOP) image and a polarization angle (AOP) image are calculated:

[0064]

[0065]

[0066] S102: Train the polarized image five-classification ResNet50 residual network model, classify the polarized images with similar Q and U features obtained in step S101 into a class of QU polarized images, establish an I, QU, DOP, AOP and non-polarized image five-classification task data set, and the true value label set of the five-classification task data set is N1 is the number of images in the five-classification task data set, the five-classification task data set is input into the polarized image five-classification ResNet50 residual network model to perform an I, QU, DOP, AOP and non-polarized image five-classification task, and the classification prediction value set of the five-classification task data set is obtained The prediction value of each image is a vector with a length of M1, M1 = 5, which is the number of classes of the five-classification task, and each element is the probability of the image being predicted as each class;

[0067] S103: According to the true value set Y 1 and the prediction value set , a cross entropy (Cross Entropy) loss function is constructed

[0068]

[0069] Wherein, N1 is the number of images in the five-classification task data set, c is the image class, M1 is the number of classes of the five-classification task data set, M1 = 5, y ic is a symbol function (0 or 1), which takes 1 if the true class of sample i is equal to c, and 0 otherwise, p ic is the predicted probability that sample i belongs to class c, and whether the loss function value between the true value set Y 1 and the prediction value set meets the preset requirement is judged, if it meets, the current five-classification network model is the final polarized image five-classification ResNet50 residual network model, the training of the I, QU, DOP, AOP and non-polarized image five-classification ResNet50 residual network model is completed, and step S105 is entered, if it does not meet, step S104 is entered;

[0070] S104: Update the parameter weights in the polarized image five-classification ResNet50 residual network model according to the set rule, re-execute steps S102-S103, update the prediction value set of the five-classification task data set and re-calculate the true value set Y 1 and the prediction value set the loss function value between Y and P until a preset requirement is met;

[0071] S105: performing edge sharpening processing on the QU polarization image, considering that the polarization characteristics of the Q and U polarization images are similar, an edge sharpening operation is adopted to enhance the polarization characteristic difference between the Q and U polarization images, and the edge sharpening processing method is various, and a Sobel operator edge sharpening algorithm is adopted to enhance the polarization characteristic difference between the Q and U polarization images;

[0072] S106: training the polarization image binary classification ResNet50 residual network model, the QU polarization image after the edge sharpening processing in step S105 is used as a data set of a Q and U polarization image binary classification task, and a true value label set of the binary classification task data set is N2 is the number of images in the binary classification task data set, the binary classification task data set is input into the polarization image binary classification ResNet50 residual network model to perform a Q and U image binary classification task, and a predicted value set of the binary classification task data set is obtained the predicted value of each image is a vector with a length of M2, M2=2, and is the number of binary classification task categories, and each element is the probability that the image is predicted to be each category;

[0073] S107: constructing a cross-entropy loss function according to the Q and U polarization image true value set Y 2 and the predicted value set P

[0074]

[0075] wherein N2 is the number of images in the binary classification task data set, c is the image category, M2 is the number of categories in the binary classification task data set, M2=2, y ic is a symbol function (0 or 1), and takes 1 if the real category of the sample i is equal to c, and takes 0 otherwise, p ic is the predicted probability that the observation sample i belongs to the category c, and whether the loss function value between the true value set Y 2 and the predicted value set P meets a preset requirement is judged, if the preset requirement is met, the current binary classification network model is the final Q and U polarization image binary classification ResNet50 residual network model, the training of the Q and U polarization image binary classification ResNet50 residual network model is completed, and step S109 is entered, if the preset requirement is not met, step S108 is entered;

[0076] ​S108: update the parameter weights in the polarization image binary classification ResNet50 residual network model according to the set rules, re-execute steps S106-S107, update the predicted value set of the binary classification task data set and re-calculate the loss function value between the true value set Y 2 and the predicted value set until the preset requirements are met.

[0077] S109: the polarization image and non-polarization image of unknown class of target scene are first input into the trained I, QU, DOP, AOP and non-polarization image five-classification ResNet50 residual network model, five-classification prediction is performed, the polarization image with a prediction result of QU is subjected to image edge sharpening processing, and then input into the trained Q and U polarization image binary classification ResNet50 residual network model, Q and U image classification prediction is performed, and finally the class of all images of the target scene is obtained, realizing the identification of the I polarization image, Q polarization image, U polarization image, DOP polarization image, AOP polarization image and non-polarization image of the target scene.

[0078] In steps S102 and S106, the ResNet50 residual classification network model is as shown in Figure 2 The network contains 49 convolutional layers and 1 fully connected layer, first performs convolution operation on the input image, then passes through 4 residual blocks (ResidualBlock), each residual block contains multiple residual units, and finally performs full connection operation to output the classification result. Compared with other networks, the ResNet50 residual network has more layers, can better obtain the feature information of the image and perform classification according to the feature information, and does not have problems such as local optimal solution and gradient disappearance, and has better performance on many image classification data sets. The ResNet50 residual network model is trained for a total of 260 times. In order to obtain better training effect, the learning rate of the first 200 iterations is 0.0001, the learning rate of the last 60 iterations is 0.00001, and the Batch Size is 8.

[0079] Further, in step S102, the polarization features of the Q and U two types of polarization images are similar. As can be seen from the formula in step S101, the Q polarization image is the difference between the intensity of the 0° polarization image and the 90° polarization image, and the U polarization image is the difference between the intensity of the 45° polarization image and the 135° polarization image. When the polarization information in the shooting scene is not obvious, the polarization features in the Q polarization image and the U polarization image are similar, as shown in Figure 3As shown, it is difficult to distinguish between the two, so first combine the Q polarization image and the U polarization image into a QU polarization image for five-class training, then perform edge sharpening on the QU polarization image to enhance the difference in polarization characteristics, and then classify the Q polarization image and the U polarization image.

[0080] Further, the pixel-level classification method of the five-channel polarization image in step S3 is specifically:

[0081] S301: Prepare polarization images of a plurality of known scenes, stack the I, Q, U, DOP, and AOP five polarization images of the same scene together to form a five-channel polarization image pixel-level classification task data set, manually label all objects in the polarization image data set according to the polarization image, and form the true value label of the pixel-level classification task data set. The size of the pixel-level classification task data set true value label is consistent with the spatial size of the five-channel polarization image, and different intensity values represent different objects.

[0082] S302: Train the five-channel polarization image pixel-level classification ResNet18 residual network model, input the five-channel polarization image pixel-level classification task data set stacked in step S301 into the pixel-level classification ResNet18 residual network model for training, consider the information contained by each pixel point and its surrounding pixel points, input the five-channel image of 15x15 pixel points centered on the target pixel point into the pixel-level classification ResNet18 residual network model each time, output the label prediction value of the target pixel point, and finally obtain a pixel-level classification task data set prediction value set consistent with the spatial size of the five-channel polarization image.

[0083] S303: Construct a loss function according to the true value label set and the prediction value set of the pixel-level classification data set, determine whether the loss function value between the true value set and the prediction value set of the pixel-level classification task data set meets the preset requirement, if yes, the current pixel-level classification network model is the final pixel-level classification ResNet18 residual network model, the training of the five-channel polarization image pixel-level classification ResNet18 residual network model is completed, and step S305 is entered, if not, step S304 is entered.

[0084] S304: Update the parameter weights in the pixel-level classification ResNet18 residual network model according to the set rule, re-execute steps S302-S303, update the prediction value set of the pixel-level classification task data set, and re-calculate the loss function value between the true value set and the prediction value set of the pixel-level classification task data set until the preset requirement is met.

[0085] S305: First perform image segmentation preprocessing on the target scene five-channel polarization image, which divides an image into several regions according to the polarization characteristics of the five channels, such asFigure 4 The image of each channel in the five-channel polarization image is respectively summed and averaged according to the segmentation area, and then the obtained average value is filled back into the corresponding segmentation area to cover the original data, thereby forming a five-channel polarization image of the target scene after segmentation preprocessing.

[0086] S306: inputting the five-channel polarization image of the target scene after segmentation preprocessing into the trained ResNet18 residual network model for pixel-level classification, considering that the values of each segmentation area of the image after segmentation preprocessing are the same, and that point-by-point input into the network wastes computational resources, therefore, only the five-channel image of 15*15 pixel points centered on the center pixel point in each segmentation area is input into the network, and the predicted value result of the center pixel point is output as the label of the entire segmentation area. It is also considered that the image segmentation preprocessing is not accurate in edge extraction of some objects, therefore, the pixel points in each segmentation area with a distance less than 7 to the boundary are separately input into the network for calculation to obtain more accurate edge information. Finally, the pixel-level classification result of the entire image of the five-channel polarization image of the target scene is obtained.

[0087] Further, the ResNet18 residual network in step S302 is composed of multiple residual units, including 17 convolutional layers and 1 fully connected layer. Compared with the ResNet50 residual network with a large number of layers, the ResNet18 residual network has fewer layers and is suitable for pixel-level classification networks with less input data. It should be noted that the output of the ResNet18 polarization image pixel-level classification network is different from that of the ResNet50 polarization image six-classification network. The output of the polarization image six-classification network is the class information of each image, while the output of the polarization image pixel-level classification network is the target class information of each pixel point in the entire image.

[0088] Further, the image segmentation preprocessing in step S305 is to divide the image into several mutually disjoint regions according to the intensity value, spatial texture and other features. The images in the same region have similar features, while the images in different regions have obvious feature differences. The original polarization image can be preliminarily segmented by using the image segmentation algorithm. The segmentation result is mainly based on edge information and spatial feature differences between different objects, and the image segmentation preprocessing makes up for the problem of insufficient learning of spatial information by the pixel-level classification network.

[0089] Embodiment

[0090] Please refer to Figure 1 , Figure 1 A polarization image recognition and pixel-level classification method is provided in the embodiment of the application. The technical scheme of the method is as follows:

[0091] S1: input the polarization image and non-polarization image of unknown categories of the target scene into the polarization image six-classification network model, and the polarization image six-classification network model performs six classification, that is, identifies and distinguishes five categories of I polarization image, Q polarization image, U polarization image, DOP polarization image and AOP polarization image of the target scene and non-polarization image according to the polarization characteristics;

[0092] S2: stack the five categories of I, Q, U, DOP and AOP polarization images of the target scene distinguished in step S1 together to form a five-channel polarization image of the target scene, and the five-channel polarization image of the target scene is used as input data for pixel-level classification;

[0093] S3: input the five-channel polarization image of the target scene stacked in step S2 into a pixel-level classification network model, the pixel-level classification network model judges the category of each point of the five-channel polarization image, completes the pixel-level classification of the five-channel polarization image of the target scene, and completes the polarization image recognition and pixel-level classification;

[0094] In step S1, the polarization image six-classification network model considers that the polarization characteristics of Q polarization image and U polarization image are similar, first combines the Q polarization image and the U polarization image into one category to obtain a QU polarization image, then performs five classification of I, QU, DOP, AOP and non-polarization image, and then enhances the polarization characteristic difference of the Q and U two categories of polarization images through an edge sharpening processing method to perform two classification of the QU polarization image;

[0095] The specific steps of the polarization image six-classification method are as follows:

[0096] S101: prepare a sufficient number of I, Q, U, DOP, AOP and non-polarization images of known categories to establish a data set for a polarization image classification task, wherein the calculation method of the I, Q and U three categories of polarization images is as follows:

[0097]

[0098] wherein I0, I 45 , I 90 and I 135 are four images of different polarization angles taken by a polarization camera. The I, Q and U three images are calculated according to the above formula, and then a polarization degree (DOP) image and a polarization angle (AOP) image are calculated:

[0099]

[0100]

[0101] S102: Train the polarized image five-classification ResNet50 residual network model, classify the polarized images with similar Q and U features calculated in step S101 into a class of QU polarized images, establish an I, QU, DOP, AOP, and non-polarized image five-classification task dataset, and the true value label set of the five-classification task dataset is N1 is the number of images in the five-classification task dataset, and the five-classification task dataset is input into the polarized image five-classification ResNet50 residual network model to perform an I, QU, DOP, AOP, and non-polarized image five-classification task, and the classification prediction value set of the five-classification task dataset is obtained The prediction value of each image is a vector with a length of M1, M1=5, which is the number of classes of the five-classification task, and each element is the probability of the image being predicted as each class;

[0102] S103: According to the true value set Y 1 and the prediction value set , a cross entropy (Cross Entropy) loss function is constructed

[0103]

[0104] , wherein N1 is the number of images in the five-classification task dataset, c is the image class, M1 is the number of classes of the five-classification task dataset, M1=5, y ic is a symbol function (0 or 1), which takes 1 if the true class of sample i is equal to c, and 0 otherwise, p ic is the predicted probability that sample i belongs to class c, and whether the loss function value between the true value set Y 1 and the prediction value set meets the preset requirement is judged, if it meets the requirement, the current five-classification network model is the final polarized image five-classification ResNet50 residual network model, the training of the I, QU, DOP, AOP, and non-polarized image five-classification ResNet50 residual network model is completed, and step S105 is entered, if it does not meet the requirement, step S104 is entered;

[0105] S104: Update the parameter weights in the polarized image five-classification ResNet50 residual network model according to the set rule, re-execute steps S102-S103, update the prediction value set of the five-classification task dataset , and re-calculate the loss function value between the true value set Y 1 and the prediction value set until the preset requirement is met;

[0106] S105: Edge sharpening processing is performed on the QU polarization image. Considering that the polarization characteristics of the Q and U polarization images are similar, edge sharpening operation is used to enhance the polarization characteristic difference between the Q and U polarization images. The edge sharpening processing method is various, and the Sobel operator edge sharpening algorithm is used to enhance the polarization characteristic difference between the Q and U polarization images;

[0107] S106: Training of the polarization image binary classification ResNet50 residual network model. The edge sharpened QU polarization image in step S105 is used as the data set of the Q and U polarization image binary classification task, and the true value label set of the binary classification task data set is N2 is the number of images in the binary classification task data set. The binary classification task data set is input into the polarization image binary classification ResNet50 residual network model to perform the Q and U image binary classification task, and the predicted value set of the binary classification task data set is obtained The predicted value of each image is a vector with a length of M2, M2=2, which is the number of binary classification task categories, and each element is the probability of the image being predicted as each category;

[0108] S107: According to the Q and U polarization image true value set Y 2 and the predicted value set , the cross-entropy loss function is constructed

[0109]

[0110] , wherein N2 is the number of images in the binary classification task data set, c is the image category, M2 is the number of categories in the binary classification task data set, M2=2, y ic is a symbol function (0 or 1), which takes 1 if the true category of sample i is equal to c, and 0 otherwise, p ic is the predicted probability that sample i belongs to category c, and the loss function value between the true value set Y 2 and the predicted value set is determined whether it meets the preset requirement, if it meets, the current binary classification network model is the final Q and U polarization image binary classification ResNet50 residual network model, the training of the Q and U polarization image binary classification ResNet50 residual network model is completed, and step S109 is entered, if it does not meet, step S108 is entered;

[0111] S108: According to the set rule, the parameter weight in the polarization image binary classification ResNet50 residual network model is updated, steps S106-S107 are re-executed, the predicted value set of the binary classification task data set is updated , and the true value set Y 2 and the predicted value set of the binary classification task data set are recalculated loss function value between the target scene unknown class and the other classes until a preset requirement is met;

[0112] S109: The polarization image and the non-polarization image of the target scene unknown class are first input into the trained I, QU, DOP, AOP and non-polarization image five-classification ResNet50 residual network model for five-classification prediction. The polarization image with a prediction result of QU is subjected to image edge sharpening processing, and then input into the trained Q and U polarization image two-classification ResNet50 residual network model for classification prediction of Q and U images. Finally, the class of all images of the target scene is obtained, and the identification of the five types of polarization images, i.e., the I polarization image, the Q polarization image, the U polarization image, the DOP polarization image and the AOP polarization image, and the non-polarization image of the target scene is realized.

[0113] In steps S102 and S106, the ResNet50 residual classification network model is as shown in Figure 2 The network contains 49 convolutional layers and 1 fully connected layer. First, the input image is subjected to convolutional operation, then passes through 4 residual blocks (ResidualBlock), each residual block contains multiple residual units, and finally performs full connection operation to output the classification result. Compared with other networks, the ResNet50 residual network has a relatively deep layer, can better obtain the feature information of the image and classify according to the feature information, and does not have problems such as local optimal solution and gradient disappearance. It has relatively outstanding performance on many image classification data sets. The ResNet50 residual network model is trained for a total of 260 iterations. In order to obtain better training effect, the learning rate of the first 200 iterations is 0.0001, the learning rate of the last 60 iterations is 0.00001, and the Batch Size is 8.

[0114] Further, in step S102, the polarization features of the Q and U two types of polarization images are similar. As can be seen from the formula in step S101, the Q polarization image is the difference between the intensity of the 0° polarization image and the intensity of the 90° polarization image, and the U polarization image is the difference between the intensity of the 45° polarization image and the intensity of the 135° polarization image. When the polarization information in the shooting scene is not obvious, the polarization features in the Q polarization image and the U polarization image are similar. As shown in Figure 3 It is difficult to distinguish between the two, so the Q polarization image and the U polarization image are first combined into one type of QU polarization image for five-classification training. Then, the QU polarization image is subjected to edge sharpening processing to enhance the difference in polarization characteristics between the two, and then the Q polarization image and the U polarization image are classified.

[0115] Further, the edge sharpening processing method used in step S105 is various, and the application adopts a Sobel operator edge sharpening algorithm to enhance the polarization feature difference of the Q and U images. In order to verify the effectiveness of edge sharpening in improving the classification accuracy of Q and U images, the application makes a contrast experiment: using a residual network to train the QU data set with Sobel sharpening processing and the QU data set without sharpening processing respectively, ensuring that other parameters are the same, and comparing the classification accuracies of the two. The experimental results are as follows: the Q and U image classification accuracy of the test set is 76.92% when trained on the Sobel sharpening processed data set; and the Q and U image classification accuracy of the test set is only 38.46% when trained on the data set without sharpening processing. It can be seen that sharpening processing can increase the feature difference of Q and U polarization images, and effectively improve the classification accuracy of Q and U polarization images.

[0116] Further, the pixel-level classification method of the five-channel polarization image in step S3 is specifically:

[0117] S301: Prepare polarization images of a plurality of known scenes, stack the I, Q, U, DOP and AOP five polarization images of the same scene together to form a five-channel polarization image pixel-level classification task data set, manually label all objects in the polarization image data set according to the polarization image, and form a true value label of the pixel-level classification task data set. The size of the true value label of the pixel-level classification task data set is consistent with the spatial size of the five-channel polarization image, and different intensity values represent different objects.

[0118] S302: Train the five-channel polarization image pixel-level classification ResNet18 residual network model, input the five-channel polarization image pixel-level classification task data set stacked in step S301 into the pixel-level classification ResNet18 residual network model for training, consider the information contained by each pixel point and its surrounding pixel points, input the five-channel image of 15x15 pixel points centered on the target pixel point into the pixel-level classification ResNet18 residual network model each time, output the label prediction value of the target pixel point, and finally obtain a pixel-level classification task data set prediction value set consistent with the spatial size of the five-channel polarization image;

[0119] S303: Construct a loss function according to the true value label set and the prediction value set of the pixel-level classification data set, determine whether the loss function value between the true value set and the prediction value set of the pixel-level classification task data set meets the preset requirement, if yes, the current pixel-level classification network model is the final pixel-level classification ResNet18 residual network model, the training of the five-channel polarization image pixel-level classification ResNet18 residual network model is completed, and step S305 is entered, if not, step S304 is entered;

[0120] S304: update the parameter weights in the pixel-level classification ResNet18 residual network model according to the set rules, re-execute steps S302-S303, update the predicted value set of the pixel-level classification task data set, and re-calculate the loss function value between the true value set and the predicted value set of the pixel-level classification task data set until the preset requirement is met;

[0121] S305: the target scene five-channel polarization image is first subjected to image segmentation preprocessing. The image segmentation preprocessing divides an image into several regions according to the polarization characteristics of the five channels, as shown in FIG. 5. The image of each channel in the five-channel polarization image is respectively subjected to summation and averaging operation according to the segmentation region, and then the obtained average value is filled back into the corresponding segmentation region to cover the original data, thereby forming the target scene five-channel polarization image after segmentation preprocessing; Figure 4

[0122] S306: the five-channel polarization image after segmentation preprocessing is input into the trained five-channel polarization image pixel-level classification ResNet18 residual network model. Considering that the values of each segmentation region after segmentation preprocessing are the same, point-by-point input into the network wastes computational resources, so only the five-channel image of 15x15 pixel points centered on the center pixel point in each segmentation region is input, and the predicted value result of the center pixel point is output as the label of the entire segmentation region. It is also considered that the image segmentation preprocessing is not accurate in extracting the edges of some objects, so the pixel points in each segmentation region with a distance less than 7 to the boundary are separately input into the network for calculation to obtain more accurate edge information. Finally, the pixel-level classification result of the entire image of the target scene five-channel polarization image is obtained.

[0123] Further, the image segmentation preprocessing in step S305 divides the image into several mutually disjoint regions according to the intensity value, spatial texture and other characteristics, the images in the same region have similar characteristics, and the images in different regions have obvious characteristic differences. The original polarization image can be preliminarily segmented by using an image segmentation algorithm. The segmentation result mainly depends on the edge information and the spatial feature difference between different objects, and the image segmentation preprocessing compensates for the insufficient learning of spatial information by the pixel-level classification network.

[0124] ​Further, the ResNet18 residual network in step S302 is composed of a plurality of residual units, including 17 convolutional layers and 1 fully connected layer. Compared with the ResNet50 residual network with a deeper layer, the ResNet18 residual network has fewer layers and is suitable for a pixel-level classification network with less input data. It should be noted that the output of the ResNet18 polarized image pixel-level classification network is different from that of the ResNet50 polarized image six-classification network. The output of the polarized image six-classification network is the class information of each image, while the output of the polarized image pixel-level classification network is the target class information of each pixel in the whole image.

[0125] Further, the network models commonly used for image classification at present include LeNet, AlexNet, VggNet, GoogLeNet and ResNet, all of which have good classification results on many public data sets. However, as the requirement for classification accuracy is improved, researchers pay more attention to the extraction of some detailed features in images. In order to obtain better classification results, the training weight parameters are constantly increased, and the neural network model is developing in the direction of being deeper and deeper. However, as the number of network layers increases, the objective function is more likely to fall into a local optimal solution, and at the same time, as the number of layers increases, the gradient vanishing problem is more serious, so that the network parameters cannot be effectively learned. The ResNet residual network mentioned above can well solve these problems. The residual network is a very effective network for alleviating the gradient vanishing problem, which greatly improves the depth of the network that can be effectively trained.

[0126] Further, the ResNet residual network used in the present application is composed of a plurality of residual units. The residual unit can be realized in the form of skip-layer connection, as shown in Figure 5 , that is, the input of the unit is directly added to the output of the unit, and then activated.

[0127] The residual network directly jumps over a plurality of layers to introduce the data output of a certain layer in the front several layers into the input part of the data layer in the rear, which means that the content of the feature layer in the rear will have a part of linear contribution from the front certain layer. It can be expressed by formula as:

[0128] H(x)=F(x)+x

[0129] Wherein, x is the input of the residual unit, F(x) is the residual, H(x) is the sum of the two, and then the activation function is:

[0130] x1=f(H(x))

[0131] where f() is the activation function ReLU, x1 is the output of the residual unit and also the input of the next residual unit. Residual networks can be easily implemented with mainstream automatic differentiation deep learning frameworks, and directly use BP algorithm to update the gradient of the loss with respect to some low-level output.

[0132] Of course, the present application can have other various embodiments, without departing from the spirit and essence of the present application, and those skilled in the art can certainly make various corresponding changes and modifications according to the present application, but these corresponding changes and modifications should all belong to the protection scope of the claims attached to the present application.

Claims

1. A polarization image recognition and pixel-level classification method, characterized in that... Includes the following steps: S1: Input the polarization and non-polarization images of the target scene of unknown category into the polarization image six-classification network model for six-classification to obtain the six-classification images of the target scene. The six-classification images are I polarization image, Q polarization image, U polarization image, DOP polarization image, AOP polarization image and non-polarization image. S2: Stack the five polarization images (I polarization image, Q polarization image, U polarization image, DOP polarization image and AOP polarization image) from the six-class image obtained in step S1 together to form a five-channel polarization image of the target scene. S3: Input the five-channel polarization image of the target scene obtained in step S2 into the pixel-level classification network model to determine the category of each point of the five-channel polarization image of the target scene, and complete the pixel-level classification of the five-channel polarization image of the target scene. In step S1, the specific steps of the six-classification method are as follows: S101: Construct a dataset for the polarization image classification task using a sufficient number of I-polarization images, Q-polarization images, U-polarization images, DOP-polarization images, AOP-polarization images, and unpolarized images with known categories. The calculation formulas for I-polarization images, Q-polarization images, and U-polarization images are as follows: in, , , and Images taken by a polarization camera at different polarization angles; Calculate the degree of polarization (DOP) image and the angle of polarization (AOP) image: S102: Train a ResNet50 residual network model for five-class classification of polarization images. Classify the Q-polarization and U-polarization images obtained in step S101 into a single class, QU-polarization images. Establish a five-class classification task dataset comprising I-polarization images, QU-polarization images, DOP-polarization images, AOP-polarization images, and non-polarization images. The ground truth label set for the five-class classification task dataset is as follows: , Given the number of images in the five-class classification task dataset, the dataset is input into a ResNet50 residual network model for five-class classification of polarized images to perform I, QU, DOP, AOP, and non-polarized image five-class classification tasks, resulting in a set of classification prediction values ​​for the five-class classification task dataset. The predicted value for each image They are all of length The vector, The number of categories for a five-class classification task, where each element represents the probability that the image is predicted to be in each category; S103: Based on the truth set and predicted value set Constructing the cross-entropy loss function : in, The number of images in the five-class classification task dataset. For image categories, The number of classes in the five-class classification task dataset. The sign function is 0 or 1, if the sample The true category equals Then take 1, otherwise take 0. For observation samples Category The predicted probabilities are used to determine the true value set of the five-class classification task dataset. With the set of predicted values If the loss function values ​​between the two classes meet the preset requirements, then the current five-class network model is the final polarized image five-class ResNet50 residual network model. The training of the I, QU, DOP, AOP and non-polarized image five-class ResNet50 residual network models is completed, and the process proceeds to step S105. If the requirements are not met, the process proceeds to step S104. S104: Update the parameter weights in the five-class ResNet50 residual network model for polarization image classification according to the set rules, and repeat steps S102-S103 to update the predicted value set of the five-class classification task dataset. And recalculate the true value set of the five-class classification task dataset. and predicted value set The loss function value between the values ​​is calculated until the preset requirements are met; S105: Sharpen the edges of the QU polarization image; S106: Train the ResNet50 residual network model for binary classification of polarization images. The Q and U polarization images after edge sharpening in step S105 are used as the dataset for the Q and U polarization image binary classification task. The set of ground truth labels for the binary classification task dataset is as follows: , Given the number of images in the binary classification task dataset, the dataset is input into a ResNet50 residual network model for polarization image binary classification to perform Q and U image binary classification tasks, resulting in a set of predicted values ​​for the binary classification task dataset. The predicted value for each image They are all of length The vector, Let be the number of categories for the binary classification task, where each element represents the probability that the image is predicted to be in each category; S107: Based on the truth set of Q and U polarization images and predicted value set Constructing the cross-entropy loss function : in, The number of images in the binary classification task dataset. For image categories, The number of classes in the binary classification task dataset. The sign function is 0 or 1, if the sample The true category equals Then take 1, otherwise take 0. For observation samples Category Predicted probabilities, determining the true set With the set of predicted values If the loss function values ​​between the two meet the preset requirements, then the current binary classification network model is the final Q and U polarization image binary classification ResNet50 residual network model. The training of the Q and U polarization image binary classification ResNet50 residual network model is completed, and the process proceeds to step S109. If the requirements are not met, the process proceeds to step S108. S108: Update the parameter weights in the ResNet50 residual network model for polarization image binary classification according to the set rules, and repeat steps S106-S107 to update the predicted value set of the binary classification task dataset. And recalculate the set of truth values ​​for the binary classification task dataset. and predicted value set The loss function value between the values ​​is calculated until the preset requirements are met; S109: First, input the unknown polarized and unpolarized images of the target scene into the trained ResNet50 five-class residual network model for I, QU, DOP, AOP and unpolarized images for five-class classification prediction. Then, sharpen the edges of the polarized images predicted as QU. After that, input them into the trained ResNet50 two-class residual network model for Q and U polarized images for classification prediction of Q and U images. Finally, obtain the category of all images in the target scene, realize the recognition of five types of polarized and unpolarized images in the target scene: I polarized image, Q polarized image, U polarized image, DOP polarized image, AOP polarized image and unpolarized image.

2. The polarization image recognition and pixel-level classification method according to claim 1, characterized in that: In step S102, the ResNet50 residual classification network model contains 49 convolutional layers and 1 fully connected layer. First, the input image is convolved, then passed through 4 residual blocks, and finally a fully connected operation is performed to output the classification result.

3. The polarization image recognition and pixel-level classification method according to claim 2, characterized in that: The ResNet50 residual network model was trained in 260 iterations. The learning rate was 0.0001 for the first 200 iterations and 0.00001 for the last 60 iterations. The batch size was 8.

4. The polarization image recognition and pixel-level classification method according to claim 1, characterized in that: In step S102, the Q-polarized image and the U-polarized image are first merged into a single class, QU-polarized image, for five-class classification training. Then, the edges of the QU-polarized image are sharpened to enhance the difference in polarization characteristics between the two images. Finally, the Q-polarized image and the U-polarized image are classified.

5. The polarization image recognition and pixel-level classification method according to claim 1, characterized in that: In step S105, the method for edge sharpening the QU polarization image is as follows: the Sobel operator edge sharpening algorithm is used to enhance the difference in polarization features between the Q polarization image and the U polarization image.

6. The polarization image recognition and pixel-level classification method according to claim 1, characterized in that: In step S106, the ResNet50 residual classification network model contains 49 convolutional layers and 1 fully connected layer. First, the input image is convolved, then passed through 4 residual blocks, and finally a fully connected operation is performed to output the classification result. The ResNet50 residual network model is trained for a total of 260 iterations. To obtain better training results, the learning rate is 0.0001 for the first 200 iterations and 0.00001 for the last 60 iterations. The batch size is 8.

7. The polarization image recognition and pixel-level classification method according to claim 1, characterized in that: In step S3, the pixel-level classification method for the five-channel polarization image is specifically as follows: S301: Prepare polarization images of multiple known scenes. Overlay five polarization images (I, Q, U, DOP, AOP) of the same scene to form a five-channel polarization image pixel-level classification task dataset. Manually label all objects in the polarization image dataset according to the polarization images to form the ground truth labels of the pixel-level classification task dataset. The size of the ground truth labels of the pixel-level classification task dataset is consistent with the spatial size of the five-channel polarization images. Different intensity values ​​represent different objects. S302: Train the five-channel polarization image pixel-level classification ResNet18 residual network model. Input the five-channel polarization image pixel-level classification task dataset obtained in step S301 into the pixel-level classification ResNet18 residual network model for training. Each time, the five-channel image of 15×15 pixels centered on the target pixel is input into the pixel-level classification ResNet18 residual network model, and the label prediction value of the target pixel is output. Finally, the pixel-level classification task dataset prediction value set with the same spatial size as the five-channel polarization image is obtained. S303: Construct a loss function based on the set of ground truth labels and the set of predicted values ​​of the pixel-level classification dataset. Determine whether the loss function value between the set of ground truth labels and the set of predicted values ​​of the pixel-level classification task dataset meets the preset requirements. If it does, the current pixel-level classification network model is the final pixel-level classification ResNet18 residual network model. Complete the training of the five-channel polarization image pixel-level classification ResNet18 residual network model and proceed to step S305. If it does not meet the requirements, proceed to step S304. S304: Update the parameter weights in the pixel-level classification ResNet18 residual network model according to the set rules, re-execute steps S302-S303, update the predicted value set of the pixel-level classification task dataset, and recalculate the loss function value between the true value set and the predicted value set of the pixel-level classification task dataset until the preset requirements are met. S305: First, perform image segmentation preprocessing on the five-channel polarization image of the target scene. The image segmentation preprocessing divides an image into several regions according to the polarization characteristics of the five channels. Then, the image of each channel in the five-channel polarization image is summed and averaged according to the segmented region. After that, the average value is filled back into the corresponding segmented region to cover the original data and form the five-channel polarization image of the target scene after segmentation preprocessing. S306: The pre-processed five-channel polarization image of the target scene is input into the trained five-channel polarization image pixel-level classification ResNet18 residual network model. Inputting the network point by point wastes computational resources, so only the five-channel image of 15×15 pixels centered on the center pixel is input into each segmentation region. The predicted value of the center pixel is output and used as the label of the entire segmentation region. Pixels in each segmentation region whose distance to the boundary is less than 7 are fed into the network for calculation separately. Finally, the pixel-level classification result of the entire five-channel polarization image of the target scene is obtained.

8. The polarization image recognition and pixel-level classification method according to claim 7, characterized in that: In step S302, the ResNet18 residual network consists of multiple residual units, including 17 convolutional layers and 1 fully connected layer. The final output is the target category information corresponding to each pixel in the entire image.

9. A polarization image recognition and pixel-level classification method according to claim 7 or 8, characterized in that: In step S305, the image segmentation preprocessing divides the image into several non-overlapping regions based on intensity values ​​and spatial texture features, and uses an image segmentation algorithm to perform preliminary segmentation on the original polarization image.

Images