A method and device for discriminating fish-eye image occlusion categories

Abstract

The present application discloses a method and device for discriminating the occlusion categories of fisheye images. The method includes: inputting a fisheye image into a fisheye image occlusion category discrimination model to output a target fisheye image. The establishment process of the fisheye image occlusion category discrimination model is as follows: obtaining a number of fisheye captured images, performing label processing and pixelization processing on them to obtain a number of fisheye label images, constructing a BN network layer and introducing it into the original U_Net network structure to obtain a batch-normalized U_Net network structure. Using the fisheye captured images as training samples and the fisheye label images as sample labels, training through the batch-normalized U_Net network structure to obtain the fisheye image occlusion category discrimination model. It can be seen that the fisheye image occlusion category discrimination model can assist in identifying the direct satellite signals corresponding to the sky area, the reflected satellite signals corresponding to the building area, and the diffracted satellite signals corresponding to the tree area, providing more conditions for precise satellite positioning.

Description

Technical Field

[0001] This application relates to the field of obstacle recognition, and more specifically, to a method and device for discriminating fish-eye image occlusion categories. Background Art

[0002] At present, the global satellite navigation system can provide users equipped with satellite communication equipment with real-time and detailed positioning information. The positioning effect is better in areas with good satellite signal reception, such as highways, expressways, and the wild suburbs, but the positioning performance in urban areas is quite different. Due to the dense urban environment, such as multipath environments like urban canyons and urban forests, the satellite signal is blocked by urban buildings and trees, resulting in serious deviation of satellite positioning.

[0003] Nowadays, support vector machines or other semantic segmentation algorithms can be used to perform binary classification segmentation (sky area and non-sky area) on sky fish-eye images containing building occlusion and tree occlusion, so as to eliminate non-direct satellite signals corresponding to the non-sky area as occlusion signals. However, non-direct signals are divided into reflected signals and diffracted signals. If the diffracted signals are weighted down and pseudo-range modification operations are performed, they can still become useful satellite signals.

[0004] Therefore, in densely populated urban areas, classifying the occlusion categories of sky fish-eye images, mining the diffracted signals with reference value in satellite signals, and providing more conditions for satellite precise positioning are issues that need attention. Summary of the Invention

[0005] In view of the above problems, this application is proposed to provide a method and device for discriminating fish-eye image occlusion categories, mining the diffracted signals with reference value in satellite signals, and providing more conditions for satellite precise positioning.

[0006] To achieve the above objective, the following specific solutions are proposed:

[0007] A method for discriminating fish-eye image occlusion categories, comprising:

[0008] Inputting a fish-eye image into a pre-established fish-eye image occlusion category discrimination model, and outputting a target fish-eye image with occlusion classification information;

[0009] The establishment process of the fish-eye image occlusion category discrimination model includes:

[0010] Obtaining a plurality of fish-eye captured images, and performing label processing and pixelization processing on the plurality of fish-eye captured images to obtain a plurality of fish-eye label images, each fish-eye label image containing at least one label among a sky label, a building label, and a tree label;

[0011] Construct a batch normalization (BN) network layer for insertion into the original U-Net network structure;

[0012] Insert the BN network layer into the original U-Net network structure to obtain a batch normalization U-Net network structure;

[0013] Use the several fisheye-captured images as training samples, and use the corresponding fisheye label images of each fisheye-captured image as the sample labels of the fisheye-captured image, and train a fisheye image occlusion category discrimination model through the batch normalization U-Net network structure.

[0014] Optionally, the construction of the batch normalization (BN) network for insertion into the original U-Net network structure includes:

[0015] Construct a batch normalization (BN) network for insertion into the original U-Net network structure using the following formula:

[0016] ;

[0017] Where, is the regression value of the k-th dimensional input feature received by the batch normalization (BN) network, is the average value of the k-th dimensional input feature, is the mean square error of the k-th dimensional input feature, is the output feature y of the k-th dimensional input feature.

[0018] Optionally, the insertion of the BN network layer into the original U-Net network structure to obtain a batch normalization U-Net network structure includes:

[0019] Double the number of feature channels for downsampling in the original U-Net network structure to obtain a first U-Net network structure;

[0020] Halve the number of feature channels for upsampling in the first U-Net network structure to obtain a second U-Net network structure;

[0021] Insert the BN network layer after each convolutional layer in the second U-Net network structure to obtain a batch normalization U-Net network structure.

[0022] Optionally, after performing label processing and pixelization processing on several of the fisheye-captured images to obtain several fisheye label images, it further includes:

[0023] Perform data augmentation on several of the fisheye-captured images to obtain several data-augmented fisheye sample images;

[0024] Using the images captured by the several fish eyes as training samples, and using the fish eye label image corresponding to each fish eye captured image as the sample label of this fish eye captured image, training the batch normalization U_Net network structure to obtain a fish eye image occlusion category discrimination model, including:

[0025] Using the several fish eye sample images as training samples, and using the fish eye label image corresponding to each fish eye captured image as the sample label of this fish eye captured image, training the batch normalization U_Net network structure to obtain a fish eye image occlusion category discrimination model.

[0026] Optionally, using the several fish eye sample images as training samples, and using the fish eye label image corresponding to each fish eye captured image as the sample label of this fish eye captured image, training through the batch normalization U_Net network structure to obtain a fish eye image occlusion category discrimination model, including:

[0027] Using the several fish eye sample images as target training samples, and using the fish eye label image corresponding to each fish eye captured image as the sample label of this fish eye captured image, training through the batch normalization U_Net network structure under the constraint of a loss function, where the loss function is:

[0028] ;

[0029] where, when the category determined by the batch normalization U_Net network structure for each fish eye label image is the same as the category to which each fish eye sample image belongs, M is 0, otherwise M is 1, is the probability that the category to which the prediction result of each fish eye label image belongs is the same as the category c to which this fish eye label image belongs, is the proportion of fish eye sample images with the occlusion category of category c in the target training samples, is the first proportionality coefficient, and its calculation formula is: ;

[0030] where, is the number of pixels with the occlusion category of category c in each fish eye sample image, is the number of pixels in this fish eye sample image.

[0031] Optionally, using the several fish eye sample images as target training samples, and using the fish eye label image corresponding to each fish eye captured image as the sample label of this fish eye captured image, training through the batch normalization U_Net network structure under the constraint of a loss function, including:

[0032] Using the several fisheye sample images as target training samples, and taking the fisheye label image corresponding to each fisheye captured image as the sample label of this fisheye captured image, under the constraint of the loss function, a fisheye image occlusion category discrimination model is trained through an iterative learning rate optimization algorithm and the batch normalization U_Net network structure. The iterative learning rate optimization algorithm is as follows:

[0033] ;

[0034] where t is the iteration round, is the parameter to be iterated for identifying the occlusion category, is the gradient obtained by taking the derivative of the objective function of the parameter to be iterated with respect to the parameter to be iterated, is the first moment of, is the second moment of, is the bias correction of, is the bias correction of, is the first preset iteration optimization coefficient, is the second preset iteration optimization coefficient, is the first-order bias correction factor, is the second-order bias correction factor.

[0035] Optionally, using the several fisheye sample images as target training samples, and taking the fisheye label image corresponding to each fisheye captured image as the sample label of this fisheye captured image, under the constraint of the loss function, a fisheye image occlusion category discrimination model is trained through an iterative learning rate optimization algorithm and the batch normalization U_Net network structure, including:

[0036] Using the several fisheye sample images as target training samples, and taking the fisheye label image corresponding to each fisheye captured image as the sample label of this fisheye captured image, under the constraint of the loss function and preset training parameters, a fisheye image occlusion category discrimination model is trained through an iterative learning rate optimization algorithm and the batch normalization U_Net network structure. The preset training parameters include the number of training rounds and the learning speed of the iterative learning rate optimization algorithm.

[0037] Optionally, after inputting the fisheye image into the pre-established fisheye image occlusion category discrimination model and outputting the target fisheye image with occlusion classification information, it further includes:

[0038] Obtaining the elevation angle and azimuth angle of each satellite being photographed, as well as the platform heading angle of each satellite;

[0039] Determine the projected satellite coordinates of each satellite on the target fisheye image according to the elevation angle, azimuth angle and platform heading angle of each satellite;

[0040] Project each projected satellite coordinate onto the target fisheye image to classify the satellite signals of each satellite.

[0041] Optionally, determining the projected satellite coordinates of each satellite on the target fisheye image according to the elevation angle, azimuth angle and platform heading angle of each satellite includes:

[0042] Use the following formula set to calculate the projected satellite coordinates of each satellite on the target fisheye image:

[0043] ;

[0044] where, is the projected abscissa of satellite Sat on the target fisheye image, is the projected ordinate of satellite Sat on the target fisheye image, is the preset image reference coordinate, is the platform heading angle of satellite Sat, is the azimuth angle of satellite Sat, is the second proportionality coefficient, and its calculation formula is:

[0045] ;

[0046] where, is the focal length of the calibration camera for photographing satellite Sat, is the elevation angle of satellite Sat, and x is the distance in the horizontal direction of the elevation angle of satellite Sat at the focal point of the calibration camera when the calibration camera is aligned with the sky.

[0047] A discriminant device for the occlusion category of a fisheye image, comprising:

[0048] A discriminant model application unit, configured to input a fisheye image into a pre-established fisheye image occlusion category discriminant model, and output a target fisheye image with occlusion classification information;

[0049] A fisheye image preprocessing unit, configured to obtain a plurality of fisheye captured images, and perform label processing and pixelization processing on the plurality of fisheye captured images to obtain a plurality of fisheye label images, and each fisheye label image includes at least one of a sky label, a building label and a tree label;

[0050] A BN network layer construction unit, configured to construct a batch normalization BN network layer for inserting into the original U_Net network structure;

[0051] A network structure update unit, configured to insert the BN network layer into the original U_Net network structure to obtain a batch normalization U_Net network structure;

[0052] A discriminant model training unit, configured to use the several fisheye captured images as training samples, use the fisheye label image corresponding to each fisheye captured image as the sample label of the fisheye captured image, and train through the batch normalization U_Net network structure to obtain a fisheye image occlusion class discriminant model.

[0053] Optionally, the BN network layer construction unit includes:

[0054] A BN network layer construction subunit, configured to construct a batch normalization BN network for inserting into the original U_Net network structure by using the following formula:

[0055] ;

[0056] Wherein, is the regression value of the k-th dimensional input feature received by the batch normalization BN network, is the average value of the k-th dimensional input feature, is the mean square error of the k-th dimensional input feature, is the output feature y of the k-th dimensional input feature.

[0057] Optionally, the network structure update unit includes:

[0058] A downsampling channel increasing unit, configured to double the number of feature channels for downsampling in the original U_Net network structure to obtain a first U_Net network structure;

[0059] An upsampling channel reducing unit, configured to halve the number of feature channels for upsampling in the first U_Net network structure to obtain a second U_Net network structure;

[0060] A BN network layer insertion unit, configured to insert the BN network layer after each convolutional layer in the second U_Net network structure to obtain a batch normalization U_Net network structure.

[0061] Optionally, the device further includes:

[0062] A data augmentation unit, configured to perform label processing and pixelization processing on several of the fisheye captured images by the fisheye image preprocessing unit to obtain several fisheye label images, and perform data augmentation on several of the fisheye captured images to obtain several data-augmented fisheye sample images;

[0063] The discriminant model training unit includes:

[0064] An enhanced data training unit, which uses the several fisheye sample images as training samples, and uses the fisheye label image corresponding to each fisheye captured image as the sample label of this fisheye captured image, to train the batch normalization U_Net network structure, and obtain a fisheye image occlusion category discrimination model.

[0065] Optionally, the enhanced data training unit includes:

[0066] A loss constraint training unit, which uses the several fisheye sample images as target training samples, and uses the fisheye label image corresponding to each fisheye captured image as the sample label of this fisheye captured image, and under the constraint of a loss function, trains through the batch normalization U_Net network structure to obtain a fisheye image occlusion category discrimination model. The loss function is:

[0067] ;

[0068] wherein, when the category determined by the batch normalization U_Net network structure for each fisheye label image is the same as the category of each fisheye sample image, M is 0, otherwise M is 1, is the probability that the category of the prediction result of each fisheye label image is the same as the category c of this fisheye label image, is the proportion of fisheye sample images with the category c in the target training samples, is: ;

[0069] wherein, is the number of pixels with the occlusion category being category c in each fisheye sample image, is the number of pixels of this fisheye sample image.

[0070] Optionally, the loss constraint training unit includes:

[0071] An optimization learning training unit, which uses the several fisheye sample images as target training samples, and uses the fisheye label image corresponding to each fisheye captured image as the sample label of this fisheye captured image, and under the constraint of a loss function, trains through an iterative learning rate optimization algorithm and the batch normalization U_Net network structure to obtain a fisheye image occlusion category discrimination model. The iterative learning rate optimization algorithm is:

[0072] ;

[0073] wherein, t is the number of iterative rounds, is the parameter to be iterated for identifying the occlusion category, is the gradient obtained by taking the derivative of the objective function of the parameter to be iterated with respect to the parameter to be iterated, is the first moment of is the second moment of is the bias correction of is the bias correction of is the first preset iterative optimization coefficient, is the second preset iterative optimization coefficient, is the first-order bias correction factor, is the second-order bias correction factor.

[0074] Optionally, the optimization learning and training unit includes:

[0075] [[ID=2⑨]]A training parameter constraint training unit, which uses the several fisheye sample images as target training samples, takes the fisheye label image corresponding to each fisheye captured image as the sample label of the fisheye captured image, and under the constraints of the loss function and preset training parameters, trains a fisheye image occlusion category discrimination model through an iterative learning rate optimization algorithm and the batch normalization U_Net network structure. The preset training parameters include the number of training rounds and the learning speed of the iterative learning rate optimization algorithm.

[0076] Optionally, the device further includes:

[0077] A satellite azimuth determination unit, which is configured to obtain the elevation angle and azimuth angle of each satellite, and the platform heading angle of each satellite after the discrimination model training unit inputs the fisheye image into a pre-established fisheye image occlusion category discrimination model and outputs a target fisheye image with occlusion classification information;

[0078] A satellite projection coordinate determination unit, which is configured to determine the projected satellite coordinates of each satellite on the target fisheye image according to the elevation angle, azimuth angle and platform heading angle of each satellite;

[0079] A satellite projection unit, which is configured to project each projected satellite coordinate onto the target fisheye image to classify the satellite signals of each satellite.

[0080] With the above technical solution, in this application, the fisheye image is input into a pre-established fisheye image occlusion category discrimination model, and a target fisheye image with occlusion classification information is output. Among them, the establishment process of the fisheye image occlusion category discrimination model includes: obtaining a number of fisheye captured images, performing label processing and pixelization processing on the number of fisheye captured images to obtain a number of fisheye label images, and each fisheye label image contains at least one of the sky label, building label, and tree label. Further, a batch normalization (BN) network layer for inserting into the original U_Net network structure is constructed, and the BN network layer is inserted into the original U_Net network structure to obtain a batch normalization U_Net network structure. Using the number of fisheye captured images as training samples, and using the fisheye label image corresponding to each fisheye captured image as the sample label of the fisheye captured image, the fisheye image occlusion category discrimination model is trained through the batch normalization U_Net network structure. Thus, it can be seen that using the fisheye captured images containing sky regions, building regions, and tree regions as materials for training the model, the fisheye image occlusion category discrimination model is trained, enabling the fisheye image occlusion category discrimination model to effectively distinguish the sky regions, building regions, and tree regions in the fisheye captured images, thereby assisting in identifying the direct satellite signals corresponding to the sky regions, the reflected satellite signals corresponding to the building regions, and the diffracted satellite signals corresponding to the tree regions, and further mining the valuable diffracted signals in the satellite signals to provide more conditions for precise satellite positioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] By reading the detailed description of the preferred embodiments below, various other advantages and benefits will become clear to those of ordinary skill in the art. The drawings are only for the purpose of showing the preferred embodiments and are not considered to be a limitation of this application. Moreover, throughout the drawings, the same reference numerals are used to represent the same components. In the drawings:

[0082] Figure 1 It is a schematic flowchart of a method for establishing a fisheye image occlusion category discrimination model provided by an embodiment of this application;

[0083] Figure 2 It is an effect diagram of a fisheye captured image provided by an embodiment of this application;

[0084] Figure 3 It is an effect diagram of a fisheye captured image after label processing provided by an embodiment of this application;

[0085] Figure 4 It is a schematic diagram of the original U_Net network structure provided by an embodiment of this application;

[0086] Figure 5Schematic diagram of a U_Net network structure incorporating a BN network layer provided by an embodiment of the present application;

[0087] Figure 6 Schematic diagram of the structure of a device for discriminating the occlusion categories of fish-eye images provided by an embodiment of the present application;

[0088] Figure 7 Schematic diagram of the structure of a device for discriminating the occlusion categories of fish-eye images provided by an embodiment of the present application. Detailed implementation manners

[0089] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0090] The solution of the present application can be implemented based on a terminal with data processing capabilities, and the terminal can be a computer, a server, the cloud, etc.

[0091] Next, the method for discriminating the occlusion categories of fish-eye images in the present application may include the following steps:

[0092] Input the fish-eye image into a pre-established discrimination model for the occlusion categories of fish-eye images, and output a target fish-eye image with occlusion classification information.

[0093] Specifically, in combination with Figure 1 as described above, the establishment process of the discrimination model for the occlusion categories of fish-eye images may include:

[0094] Step S110: Obtain a number of fish-eye captured images, and perform label processing and pixelization processing on the number of fish-eye captured images to obtain a number of fish-eye label images.

[0095] Specifically, fish-eye images of the sky can be captured using a camera in a complex multipath environment in the city. The camera for capturing the fish-eye images of the sky can be vertically upward, and then a number of fish-eye captured images captured from the camera can be obtained.

[0096] Furthermore, the image format captured by the camera can be in jpg format, and the resolution can be 6048×4024.

[0097] It is understandable that in order to avoid the impact of using lossy jpg format images on the final result of model training, before inputting several fisheye captured images for training, the jpg format images need to be converted into png format images. At the same time, considering the limited gpu video memory, the image resolution can be reduced from 6048×4024 to 2048×1024.

[0098] Furthermore, each fisheye captured image can be labeled through the labelme software.

[0099] Among them, each fisheye labeled image contains at least one label among the sky label, building label, and tree label.

[0100] For example Figure 2 , Figure 2 shows a schematic diagram of a fisheye captured image, Figure 3 is Figure 2 the fisheye labeled image after label processing and pixelization processing for the fisheye captured image shown. Among them, the pixel values of the sky area, the building area, and the tree area can be different. For example, the pixel value of the sky area is 0, the pixel value of the building area is 1, and the pixel value of the tree area is 2.

[0101] Step S120: Construct a batch normalization BN network layer for insertion into the original U_Net network structure.

[0102] Step S130: Insert the BN network layer into the original U_Net network structure to obtain a batch normalization U_Net network structure.

[0103] Specifically, the original U_Net network structure is as Figure 4 shown. It can be composed of a contracting path and an expanding path. The contracting path follows the typical architecture of a convolutional network. Each step in it can include two repeated 3*3 convolutions. After each convolution, there is a rectified linear unit ReLU and a 2*2 max pooling operation with a stride of 2 for downsampling. Each step in the expanding path can include upsampling of the feature map, followed by a 2*2 convolution, and two 3*3 convolutions. After each convolution, there is a rectified linear unit ReLU. The last layer uses a 1*1 convolution to map each component feature vector to the required number of classes.

[0104] The batch normalization U_Net network structure after inserting the BN network layer is as Figure 5As shown, the original U_Net network structure is combined with the BN layer. The input image first passes through a combination of 2 convolutional layers + BN + activation layer to generate a feature map with 64 channels. The 64-channel feature map passes through 4 max-pooling layers + 2×(convolutional layer + BN + activation layer) to generate a 1024-dimensional feature map with a resolution 1 / 16 of the input image. Then, through 4 transposed convolutional layers + 2×(convolutional layer + BN + activation layer), the 1024-channel feature map is upsampled to a 64-channel feature map of the original image size, and the intermediate feature maps of the same dimension generated in the contraction path are spliced into the intermediate feature maps of the same dimension in the expansion path using Skip Connection. Here, Skip Connection is connected in the form of concatenation, and the shallower feature layer and the deeper feature layer are concatenated and input into the next layer of the network. This operation can effectively reduce the problem of gradient disappearance. Finally, through 1 convolution, the concatenated 64-dimensional feature map is used to generate the final segmentation image.

[0105] It can be understood that after inserting the BN network layer into the original U_Net network structure, the convergence speed of the obtained batch-normalized U_Net network structure is faster. A larger initial learning rate can be selected to improve the learning speed, or the dependence of the network on parameter initialization can be weakened.

[0106] Step S140: Using the images taken by the several fisheye cameras as training samples, and using the fisheye label image corresponding to each fisheye image as the sample label of the fisheye image, a fisheye image occlusion category discrimination model is trained through the batch-normalized U_Net network structure.

[0107] The method for discriminating the occlusion category of fisheye images provided in this embodiment outputs a target fisheye image with occlusion classification information by inputting a fisheye image into a pre-established fisheye image occlusion category discrimination model. The process of establishing the fisheye image occlusion category discrimination model includes: obtaining a number of fisheye captured images, performing label processing and pixelization processing on the number of fisheye captured images to obtain a number of fisheye label images, and each fisheye label image contains at least one label among a sky label, a building label, and a tree label. Further, a batch normalization BN network layer for inserting into the original U_Net network structure is constructed, and the BN network layer is inserted into the original U_Net network structure to obtain a batch normalization U_Net network structure. Using the number of fisheye captured images as training samples and using the fisheye label image corresponding to each fisheye captured image as the sample label of the fisheye captured image, a fisheye image occlusion category discrimination model is trained through the batch normalization U_Net network structure. Thus, by using fisheye captured images containing sky regions, building regions, and tree regions as materials for training the model, a fisheye image occlusion category discrimination model is trained, enabling the fisheye image occlusion category discrimination model to effectively distinguish the sky region, building region, and tree region in the fisheye captured image, thereby assisting in identifying the direct satellite signal corresponding to the sky region, the reflected satellite signal corresponding to the building region, and the diffracted satellite signal corresponding to the tree region, and further excavating the valuable diffracted signals in the satellite signal to provide more conditions for precise satellite positioning.

[0108] In some embodiments of the present application, the process of step S120, constructing a batch normalization BN network layer for inserting into the original U_Net network structure, is introduced. This process may include:

[0109] Use the following formula to construct a batch normalization BN network for inserting into the original U_Net network structure:

[0110] ;

[0111] Where, is the regression value of the k-th dimensional input feature received by the batch normalization BN network, is the average value of the k-th dimensional input feature, is the mean square error of the k-th dimensional input feature, is the output feature y of the k-th dimensional input feature.

[0112] It is understandable that, as a form of regularization, the BN layer can solve the problem of selecting the regularization term parameters in overfitting and improve the generalization ability of the network. Among them, due to the large number of parameters in the deep convolutional network and the large computational amount, if the training samples are insufficient, it is very easy to cause overfitting. To prevent overfitting, a dropout layer can also be introduced into the network. During the training process, a part of the neurons are randomly selected according to a certain probability, and only the calculated weights are retained without being updated, and they will be updated during the next iteration. Randomly setting some nodes not to work can avoid the situation where some features only take effect under fixed combinations, break the correlation of the fixed combinations, and consciously let the network learn some general commonalities, thereby improving the generalization ability of the model and preventing the network from overfitting.

[0113] During training, a dropout probability is usually set, that is, each output node is set to 0 (making it not work) with this probability. The expression of the neuron weight under the dropout probability can be:

[0114] ;

[0115] Among them, is the set dropout probability, is the Bernoulli random number, and z is the basic weight score of the neuron.

[0116] The method for discriminating the category of fish-eye image occlusion provided in this embodiment constructs a batch normalization BN network layer for inserting into the original U_Net network structure, which can reduce the network's dependence on initialization and improve the training speed.

[0117] In some embodiments of the present application, the process of step S130, inserting the BN network layer into the original U_Net network structure to obtain the batch normalization U_Net network structure, is introduced. This process may include:

[0118] S1. Double the number of feature channels of the downsampling in the original U_Net network structure to obtain the first U_Net network structure.

[0119] S2. Reduce the number of feature channels of the upsampling in the first U_Net network structure by half to obtain the second U_Net network structure.

[0120] It is understandable that the downsampling convolutional layer can be cascaded with the corresponding cropped feature map in the contraction path.

[0121] S3. Insert the BN network layer after each convolutional layer in the second U_Net network structure to obtain the batch normalization U_Net network structure.

[0122] For example Figure 5 , the arrow below each '+' sign represents each convolutional layer, and the '+' sign represents the BN network layer.

[0123] The performance index evaluation of the batch normalization U_Net network structure for the image segmentation algorithm applied to fisheye images can be calculated by analyzing the relationship between the true value and the predicted value, and calculating the precision, recall, and accuracy of the image segmentation algorithm. Assuming three-class semantic segmentation is implemented, the confusion matrix can be listed as:

[0124]

[0125] Taking architecture as an example, its precision, recall, and accuracy can be calculated by the following formulas:

[0126] ;

[0127] ;

[0128] ;

[0129] The method for discriminating the occlusion category of fisheye images provided in this embodiment increases the feature channels in the downsampling and reduces the feature channels in the upsampling in the original U_Net network structure, and introduces the BN network layer into the original U_Net network structure, which is beneficial to reducing the network's dependence on initialization, improving the training speed, and thus beneficial to improving the accuracy of the model in segmenting occlusions.

[0130] Considering avoiding overfitting when training the fisheye image occlusion category discrimination model, improving the generalization ability of the model, and avoiding sample imbalance, the training samples for training the fisheye image occlusion category discrimination model can be data-augmented. Based on this, in some embodiments of the present application, after performing label processing and pixelization processing on a number of the fisheye captured images to obtain a number of fisheye label images, a number of the fisheye captured images are data-augmented to obtain a number of data-augmented fisheye sample images.

[0131] Specifically, the dataset can be pixel-by-pixel transformed through the fast training data augmentation library albumentations, such as blurring, downsampling, Gaussian dotting, Gaussian blurring, motion blurring, RGB conversion, random fogging, etc., or spatial transformation (which will also transform the target) can be performed, such as cropping, flipping, random cropping, etc.

[0132] Based on this, the process of using the images captured by the several fish eyes as training samples, taking the fish eye label image corresponding to each fish eye captured image as the sample label of this fish eye captured image, and training the batch normalization U_Net network structure to obtain a fish eye image occlusion category discrimination model mentioned in the above embodiments may include:

[0133] Use the several fish eye sample images as training samples, take the fish eye label image corresponding to each fish eye captured image as the sample label of this fish eye captured image, and train the batch normalization U_Net network structure to obtain a fish eye image occlusion category discrimination model.

[0134] In some embodiments of the present application, the process of using the several fish eye sample images as training samples, taking the fish eye label image corresponding to each fish eye captured image as the sample label of this fish eye captured image, and training to obtain a fish eye image occlusion category discrimination model through the batch normalization U_Net network structure mentioned in the above embodiments is introduced. This process may include:

[0135] Use the several fish eye sample images as target training samples, take the fish eye label image corresponding to each fish eye captured image as the sample label of this fish eye captured image, and under the constraint of the loss function, train to obtain a fish eye image occlusion category discrimination model through the batch normalization U_Net network structure.

[0136] Specifically, the loss function is:

[0137] ;

[0138] wherein, when the category determined by the batch normalization U_Net network structure for each fish eye label image is the same as the category to which each fish eye sample image belongs, M is 0, otherwise M is 1, is the probability that the category to which the prediction result of each fish eye label image belongs is the same as the category c to which this fish eye label image belongs, is the proportion of fish eye sample images with the occlusion category of category c in the target training samples, is the first proportionality coefficient, and its calculation formula is: ;

[0139] wherein, is the number of pixels with the occlusion category of category c in each fish eye sample image, is the number of pixels in this fish eye sample image.

[0140] It can be understood that the loss function uses weighted cross entropy. Compared with general cross entropy, weighted cross entropy can obtain better results in the case of unbalanced sample quantities.

[0141] In some embodiments of the present application, a process of using the above-mentioned several fisheye sample images as target training samples, using the fisheye label image corresponding to each fisheye captured image as the sample label of the fisheye captured image, and training a fisheye image occlusion category discrimination model through the batch normalization U_Net network structure under the constraint of a loss function is introduced. This process may include:

[0142] Using the above-mentioned several fisheye sample images as target training samples, using the fisheye label image corresponding to each fisheye captured image as the sample label of the fisheye captured image, and training a fisheye image occlusion category discrimination model through an iterative learning rate optimization algorithm and the batch normalization U_Net network structure under the constraint of a loss function.

[0143] Specifically, the iterative learning rate optimization algorithm is as follows:

[0144] ;

[0145] where t is the iteration round, is the parameter to be iterated for identifying the occlusion category, is the gradient obtained by taking the derivative of the objective function of the parameter to be iterated with respect to the parameter to be iterated, is the first moment of is the second moment of is the bias correction of is the bias correction of is the first preset iteration optimization coefficient, is the second preset iteration optimization coefficient, is the first-order bias correction factor, is the second-order bias correction factor.

[0146] It can be understood that the iterative learning rate optimization algorithm can be implemented through an Adam optimizer. Adam is a first-order optimization algorithm that can replace the traditional stochastic gradient descent process. It can dynamically adjust the learning rate of each parameter to be iterated using the first-moment estimate and second-moment estimate of the gradient, so it can stably adjust each parameter to be iterated.

[0147] In some embodiments of the present application, the process of using the above-mentioned several fisheye sample images as target training samples, using the fisheye label image corresponding to each fisheye captured image as the sample label of the fisheye captured image, and training a fisheye image occlusion category discrimination model through an iterative learning rate optimization algorithm and the batch normalization U_Net network structure under the constraint of a loss function will be introduced. This process may include:

[0148] Using the above-mentioned several fisheye sample images as target training samples, using the fisheye label image corresponding to each fisheye captured image as the sample label of the fisheye captured image, and training a fisheye image occlusion category discrimination model through an iterative learning rate optimization algorithm and the batch normalization U_Net network structure under the constraint of a loss function and preset training parameters. The preset training parameters include the number of training epochs and the learning rate of the iterative learning rate optimization algorithm.

[0149] Specifically, the value of the number of training epochs can be preset to 100, the number of data samples captured each time for training can be preset to 2, the initial value of the learning rate of the Adam optimizer can be set to 10^-4, and after 25 rounds of training, the learning rate of the Adam optimizer can be adjusted to 10^-5.

[0150] Considering that the fisheye image occlusion category discrimination model receives a fisheye image as input and outputs a target fisheye image, and the output target fisheye image carries occlusion classification information, which can assist in distinguishing satellite signals among the three categories of direct satellite signals, reflected satellite signals, and diffracted satellite signals. In some embodiments of the present application, after the above-mentioned process of inputting a fisheye image into a pre-established fisheye image occlusion category discrimination model and outputting a target fisheye image with occlusion classification information, the following process may be included:

[0151] S1. Obtain the elevation angle and azimuth angle of each satellite being photographed, as well as the platform heading angle of each satellite.

[0152] Specifically, the elevation angle and azimuth angle of each satellite, as well as the platform heading angle of each satellite, can be calculated according to the ephemeris.

[0153] S2. Determine the projected satellite coordinates of the satellite on the target fisheye image according to the elevation angle, azimuth angle, and platform heading angle of each satellite.

[0154] Specifically, the process of determining the projected satellite coordinates of the satellite on the target fisheye image according to the elevation angle, azimuth angle, and platform heading angle of each satellite may include:

[0155] Use the following set of equations to calculate the projected satellite coordinates of each satellite on the target fisheye image:

[0156] ;

[0157] Wherein, is the projected abscissa of the satellite Sat on the target fisheye image, is the projected ordinate of the satellite Sat on the target fisheye image, is the preset image reference coordinate, is the platform heading angle of the satellite Sat, is the azimuth angle of the satellite Sat, is the second proportionality coefficient, and its calculation formula is:

[0158] ;

[0159] Wherein, is the focal length of the calibration camera for photographing the satellite Sat, and this focal length can be in pixels, is the elevation angle of the satellite Sat, and x is the horizontal distance of the elevation angle of the satellite Sat at the focus of the calibration camera when the calibration camera is aligned with the sky.

[0160] S3. Project each projected satellite coordinate onto the target fisheye image to classify the satellite signals of each satellite.

[0161] Next, the apparatus for discriminating the occlusion category of the fisheye image provided in the embodiment of the present application will be described. The apparatus for discriminating the occlusion category of the fisheye image described below can be correspondingly referred to the method for discriminating the occlusion category of the fisheye image described above.

[0162] Refer to Figure 6 , Figure 6 which is a schematic structural diagram of an apparatus for discriminating the occlusion category of the fisheye image disclosed in the embodiment of the present application.

[0163] As Figure 6 shown, the apparatus may include:

[0164] A discrimination model application unit 11, configured to input a fisheye image into a pre-established fisheye image occlusion category discrimination model, and output a target fisheye image with occlusion classification information;

[0165] A fisheye image preprocessing unit 12, configured to obtain a plurality of fisheye captured images, and perform label processing and pixelization processing on the plurality of fisheye captured images to obtain a plurality of fisheye label images, and each fisheye label image includes at least one of a sky label, a building label, and a tree label;

[0166] The BN network layer construction unit 13 is used to construct a batch normalization (BN) network layer for insertion into the original U-Net network structure;

[0167] The network structure update unit 14 is used to insert the BN network layer into the original U-Net network structure to obtain a batch normalization U-Net network structure;

[0168] The discriminant model training unit 15 is used to use the plurality of fisheye captured images as training samples, use the fisheye label image corresponding to each fisheye captured image as the sample label of the fisheye captured image, and train through the batch normalization U-Net network structure to obtain a fisheye image occlusion category discriminant model.

[0169] Optionally, the BN network layer construction unit includes:

[0170] The BN network layer construction subunit is used to construct a batch normalization BN network for insertion into the original U-Net network structure by using the following formula:

[0171] ;

[0172] Wherein, is the regression value of the k-th dimensional input feature received by the batch normalization BN network, is the average value of the k-th dimensional input feature, is the mean square error of the k-th dimensional input feature, is the output feature y of the k-th dimensional input feature.

[0173] Optionally, the network structure update unit includes:

[0174] The downsampling channel increasing unit is used to double the number of feature channels for downsampling in the original U-Net network structure to obtain a first U-Net network structure;

[0175] The upsampling channel reducing unit is used to reduce the number of feature channels for upsampling in the first U-Net network structure by half to obtain a second U-Net network structure;

[0176] The BN network layer insertion unit is used to insert the BN network layer after each convolutional layer in the second U-Net network structure to obtain a batch normalization U-Net network structure.

[0177] Optionally, the device further includes:

[0178] A data augmentation unit, configured to perform label processing and pixelization processing on a plurality of the fisheye captured images by the fisheye image preprocessing unit to obtain a plurality of fisheye label images, and perform data augmentation on the plurality of the fisheye captured images to obtain a plurality of data-augmented fisheye sample images;

[0179] The discriminant model training unit includes:

[0180] An enhanced data training unit, configured to use the plurality of fisheye sample images as training samples, use the fisheye label image corresponding to each fisheye captured image as the sample label of the fisheye captured image, and train the batch normalization U_Net network structure to obtain a fisheye image occlusion category discriminant model.

[0181] Optionally, the enhanced data training unit includes:

[0182] A loss constraint training unit, configured to use the plurality of fisheye sample images as target training samples, use the fisheye label image corresponding to each fisheye captured image as the sample label of the fisheye captured image, and train the batch normalization U_Net network structure under the constraint of a loss function to obtain a fisheye image occlusion category discriminant model. The loss function is:

[0183] ;

[0184] wherein, when the category determined by the batch normalization U_Net network structure for each fisheye label image is the same as the category of each fisheye sample image, M is 0; otherwise, M is 1. is the probability that the category of the prediction result of each fisheye label image is the same as the category c of the fisheye label image. is the proportion of the fisheye sample images with the category c in the target training samples. is: ;

[0185] wherein, is the number of pixels with the occlusion category c in each fisheye sample image. is the number of pixels in the fisheye sample image.

[0186] Optionally, the loss constraint training unit includes:

[0187] An optimization learning training unit, configured to use the plurality of fisheye sample images as target training samples, use the fisheye label image corresponding to each fisheye captured image as the sample label of the fisheye captured image, and train the batch normalization U_Net network structure under the constraint of a loss function through an iterative learning rate optimization algorithm. The iterative learning rate optimization algorithm is:

[0188] ;

[0189] where t is the iteration round, is the parameter to be iterated for identifying occlusion categories, is the gradient obtained by differentiating the objective function of the parameter to be iterated with respect to the parameter to be iterated, is the first moment of, is the second moment of, is the bias correction of, is the bias correction of, is the first preset iterative optimization coefficient, is the second preset iterative optimization coefficient, is the first-order bias correction factor, is the second-order bias correction factor.

[0190] Optionally, the optimization learning and training unit includes:

[0191] A training parameter constraint training unit, which uses the several fisheye sample images as target training samples, takes the fisheye label image corresponding to each fisheye captured image as the sample label of the fisheye captured image, and trains a fisheye image occlusion category discrimination model through an iterative learning rate optimization algorithm and the batch normalization U_Net network structure under the constraints of a loss function and preset training parameters. The preset training parameters include the number of training rounds and the learning speed of the iterative learning rate optimization algorithm.

[0192] Optionally, the device further includes:

[0193] A satellite azimuth determination unit, which is configured to obtain the elevation angle and azimuth angle of each satellite, and the platform heading angle of each satellite, after the discrimination model training unit inputs a fisheye image into a pre-established fisheye image occlusion category discrimination model and outputs a target fisheye image with occlusion classification information;

[0194] A satellite projection coordinate determination unit, which is configured to determine the projected satellite coordinates of each satellite on the target fisheye image according to the elevation angle, azimuth angle and platform heading angle of each satellite;

[0195] A satellite projection unit, which is configured to project each projected satellite coordinate onto the target fisheye image to classify the satellite signals of each satellite.

[0196] The device for discriminating the occlusion category of a fisheye image provided by an embodiment of the present application can be applied to a device for discriminating the occlusion category of a fisheye image, such as a terminal: a mobile phone, a computer, etc. Optionally, Figure 7 shows a hardware structure block diagram of a device for discriminating the occlusion category of a fisheye image. Referring to Figure 7 , the hardware structure of a device for discriminating the occlusion category of a fisheye image may include: at least one processor 1, at least one communication interface 2, at least one memory 3, and at least one communication bus 4;

[0197] In an embodiment of the present application, the number of the processor 1, the communication interface 2, the memory 3, and the communication bus 4 is at least one, and the processor 1, the communication interface 2, and the memory 3 complete mutual communication through the communication bus 4;

[0198] The processor 1 may be a central processing unit CPU, or a specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present invention, etc.;

[0199] The memory 3 may include a high-speed RAM memory, and may also include a non-volatile memory, etc., such as at least one disk memory;

[0200] Wherein, the memory stores a program, and the processor can call the program stored in the memory, and the program is used for:

[0201] Input a fisheye image into a pre-established fisheye image occlusion category discrimination model, and output a target fisheye image with occlusion classification information;

[0202] The establishment process of the fisheye image occlusion category discrimination model includes:

[0203] Obtain a number of fisheye captured images, and perform label processing and pixelization processing on the number of fisheye captured images to obtain a number of fisheye label images, and each fisheye label image includes at least one label among a sky label, a building label, and a tree label;

[0204] Construct a batch normalization BN network layer for inserting into the original U_Net network structure;

[0205] Insert the BN network layer into the original U_Net network structure to obtain a batch normalization U_Net network structure;

[0206] Using the images captured by the several fish eyes as training samples, taking the fish eye label image corresponding to each fish eye captured image as the sample label of the fish eye captured image, and training through the batch normalization U_Net network structure to obtain a fish eye image occlusion category discrimination model.

[0207] Optionally, the refinement function and expansion function of the program can refer to the above description.

[0208] An embodiment of the present application also provides a storage medium, which can store a program suitable for a processor to execute. The program is used for:

[0209] Inputting a fish eye image into a pre-established fish eye image occlusion category discrimination model, and outputting a target fish eye image with occlusion classification information;

[0210] The establishment process of the fish eye image occlusion category discrimination model includes:

[0211] Obtaining several fish eye captured images, and performing label processing and pixelization processing on the several fish eye captured images to obtain several fish eye label images. Each fish eye label image contains at least one label among a sky label, a building label, and a tree label;

[0212] Constructing a batch normalization BN network layer for inserting into the original U_Net network structure;

[0213] Inserting the BN network layer into the original U_Net network structure to obtain a batch normalization U_Net network structure;

[0214] Using the several fish eye captured images as training samples, taking the fish eye label image corresponding to each fish eye captured image as the sample label of the fish eye captured image, and training through the batch normalization U_Net network structure to obtain a fish eye image occlusion category discrimination model.

[0215] Optionally, the refinement function and expansion function of the program can refer to the above description.

[0216] Finally, it should also be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprising", "including" or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0217] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The embodiments can be combined as needed, and the same or similar parts can be referred to each other.

[0218] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but rather to the broadest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for discriminating the category of fish-eye image occlusion, characterized in that, Including: Input the fisheye image into a pre-established fisheye image occlusion category discrimination model, and output a target fisheye image with occlusion classification information; The establishment process of the fisheye image occlusion category discrimination model includes: Obtain a number of fisheye captured images, and perform label processing and pixelization processing on the number of fisheye captured images to obtain a number of fisheye label images, and each fisheye label image contains at least one label among the sky label, building label, and tree label; Construct a batch normalization (BN) network layer for insertion into the original U_Net network structure; Insert the BN network layer into the original U_Net network structure to obtain a batch normalization U_Net network structure; Use the number of fisheye captured images as training samples, and use the fisheye label image corresponding to each fisheye captured image as the sample label of the fisheye captured image, and train through the batch normalization U_Net network structure to obtain a fisheye image occlusion category discrimination model; The construction of the batch normalization (BN) network for insertion into the original U_Net network structure includes: Use the following formula to construct a batch normalization (BN) network for insertion into the original U_Net network structure: Among them, x (k) is the regression value of the k-th dimensional input feature received by the batch normalization BN network, β (k) is the average value of the k-th dimensional input feature, γ (k) is the mean square error of the k-th dimensional input feature, y (k) is the output feature y of the k-th dimensional input feature; The insertion of the BN network layer into the original U_Net network structure to obtain a batch normalization U_Net network structure includes: Double the number of feature channels for downsampling in the original U_Net network structure to obtain a first U_Net network structure; Reduce the number of feature channels for upsampling in the first U_Net network structure by half to obtain a second U_Net network structure; Insert the BN network layer after each convolutional layer in the second U_Net network structure to obtain a batch normalization U_Net network structure; After performing label processing and pixelization processing on the number of fisheye captured images to obtain a number of fisheye label images, it further includes: Perform data augmentation on the number of fisheye captured images to obtain a number of data-augmented fisheye sample images; Use the number of fisheye captured images as training samples, and use the fisheye label image corresponding to each fisheye captured image as the sample label of the fisheye captured image, and train the batch normalization U_Net network structure to obtain a fisheye image occlusion category discrimination model, including: Use the number of fisheye sample images as training samples, and use the fisheye label image corresponding to each fisheye captured image as the sample label of the fisheye captured image, and train the batch normalization U_Net network structure to obtain a fisheye image occlusion category discrimination model.

2. The method according to claim 1, characterized in that, Use the number of fisheye sample images as training samples, and use the fisheye label image corresponding to each fisheye captured image as the sample label of the fisheye captured image, and train through the batch normalization U_Net network structure to obtain a fisheye image occlusion category discrimination model, including: Using the several fish-eye sample images as target training samples, taking the fish-eye label image corresponding to each fish-eye captured image as the sample label of this fish-eye captured image, and under the constraint of the loss function, training a fish-eye image occlusion category discrimination model through the batch normalization U-Net network structure, where the loss function is: Among them, when the category determined by the batch normalization U_Net network structure for each fish-eye label image is the same as the category to which each fish-eye sample image belongs, M is 0; otherwise, M is 1, and p c is the probability that the category to which the prediction result of each fish-eye label image belongs is the same as the category c to which the fish-eye label image belongs, and y c is the proportion of fish-eye sample images with the category c in the target training samples, and w c is the first proportionality coefficient, and its calculation formula is: w c = (N - N c ) / N where N c is the number of pixels with occlusion category c in each fisheye sample image, and N is the number of pixels in the fisheye sample image.

3. The method according to claim 2, wherein Using the several fish-eye sample images as target training samples, taking the fish-eye label image corresponding to each fish-eye captured image as the sample label of this fish-eye captured image, and under the constraint of the loss function, training a fish-eye image occlusion category discrimination model through the batch normalization U-Net network structure, including: Using the several fish-eye sample images as target training samples, taking the fish-eye label image corresponding to each fish-eye captured image as the sample label of this fish-eye captured image, and under the constraint of the loss function, training a fish-eye image occlusion category discrimination model through the iterative learning rate optimization algorithm and the batch normalization U-Net network structure, where the iterative learning rate optimization algorithm is: where t is the iteration round, and Δθ t is the parameter to be iterated for identifying the occlusion category, g t is the gradient obtained by taking the derivative of the objective function of the parameter to be iterated with respect to the parameter to be iterated, m t is the first moment of g t and n t is the second moment of g t . is the bias correction of m t , is the bias correction of n t . η is the first preset iteration optimization coefficient, ε is the second preset iteration optimization coefficient, μ is the first-order bias correction factor, and ν is the second-order bias correction factor.

4. The method according to claim 3, characterized in that, Using the several fish-eye sample images as target training samples, taking the fish-eye label image corresponding to each fish-eye captured image as the sample label of this fish-eye captured image, and under the constraint of the loss function, training a fish-eye image occlusion category discrimination model through the iterative learning rate optimization algorithm and the batch normalization U-Net network structure, including: Using the several fish-eye sample images as target training samples, taking the fish-eye label image corresponding to each fish-eye captured image as the sample label of this fish-eye captured image, and under the constraints of the loss function and preset training parameters, training a fish-eye image occlusion category discrimination model through the iterative learning rate optimization algorithm and the batch normalization U-Net network structure, where the preset training parameters include the number of training epochs and the learning rate of the iterative learning rate optimization algorithm.

5. The method according to any one of claims 1-4, characterized in that, After inputting the fish-eye image into the pre-established fish-eye image occlusion category discrimination model and outputting the target fish-eye image with occlusion classification information, it further includes: Obtaining the elevation angle and azimuth angle of each satellite, as well as the platform heading angle of each satellite; Determining the projected satellite coordinates of each satellite on the target fish-eye image according to the elevation angle, azimuth angle and platform heading angle of each satellite; Projecting each projected satellite coordinate onto the target fish-eye image to classify the satellite signals of each satellite.

6. The method according to claim 5, characterized in that, Determining the projected satellite coordinates of each satellite on the target fish-eye image according to the elevation angle, azimuth angle and platform heading angle of each satellite, including: Using the following set of equations to calculate the projected satellite coordinates of each satellite on the target fish-eye image: Among them, is the projected abscissa of the satellite Sat on the target fisheye image, is the projected ordinate of the satellite Sat on the target fisheye image, (x c , y c ) is the preset image reference coordinate, α platform is the platform heading angle of the satellite Sat, α Sat is the azimuth angle of the satellite Sat, is the second proportionality coefficient, and its calculation formula is: where f pix is the focal length of the calibration camera of the imaging satellite Sat, ε sat is the elevation angle of the satellite Sat, and x is the distance in the horizontal direction from the elevation angle of the satellite Sat at the focal point of the calibration camera when the calibration camera is aimed at the sky.

7. A discriminant device for the category of fish-eye image occlusion, characterized in that, Applied to the method for discriminating the fish-eye image occlusion category as described in claim 1, the device includes: A discrimination model application unit, configured to input a fish-eye image into the pre-established fish-eye image occlusion category discrimination model and output a target fish-eye image with occlusion classification information; The fish-eye image preprocessing unit is used to obtain a plurality of fish-eye captured images, perform label processing and pixelization processing on the plurality of fish-eye captured images, and obtain a plurality of fish-eye label images. Each fish-eye label image contains at least one label among a sky label, a building label, and a tree label; The BN network layer construction unit is used to construct a batch normalization BN network layer for insertion into the original U_Net network structure; The network structure update unit is used to insert the BN network layer into the original U_Net network structure to obtain a batch normalization U_Net network structure; The discriminant model training unit is used to use the plurality of fish-eye captured images as training samples, use the fish-eye label image corresponding to each fish-eye captured image as the sample label of the fish-eye captured image, and train a fish-eye image occlusion category discriminant model through the batch normalization U_Net network structure.

Images