Polarization navigation course angle estimation method based on neural network

Abstract

The invention relates to a polarization navigation course angle estimation method based on a neural network, an end-to-end classification neural network is constructed, the input of the classification neural network is an urban sky scene polarization image, and the output of the classification neural network is a sun meridian angle. According to the network, a plurality of steps of polarization mode segmentation, restoration and the like are omitted, the sun meridian angle is directly estimated from an urban polarization image with a sheltered area, and the course angle estimation process is greatly simplified. Besides, polarization modes of different scenes, different shielding degrees, different weather and illumination conditions and different shielding types are adopted as a data set to train the network, the generalization ability of the neural network can also be improved, the effect of the neural network in the scenes of different shielding degrees is ensured, and the robustness of the neural network is improved. And particularly, the course angle estimation precision in an urban environment with dense shielding is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of bionic polarization navigation and positioning, and in particular to a polarization navigation heading angle estimation method based on a neural network. Background Art

[0002] Bionic polarization navigation is a novel auxiliary navigation method that mimics the natural behavior of organisms in extracting directional information from the polarization patterns of skylight to estimate the heading angle of a moving vehicle. Due to its advantages of zero cumulative error, immunity to electromagnetic interference, and high concealment, bionic polarization navigation has broad application prospects for both ground-based and low-altitude unmanned moving vehicles.

[0003] With the steady advancement of my country's new urbanization construction, the modern urban environment with its ever-expanding area, increasingly dense population and increasingly complex road environment has become an important stage for bionic polarization navigation technology to play a role. Figure 1 As shown in the figure, achieving urban bionic polarization navigation requires using a polarization camera to capture polarized images of the city sky. From these images, the polarization angle distribution of the urban skylight is calculated and the solar meridian angle is extracted from the skylight polarization angle distribution. Given the known time and the longitude and latitude of the surface observation point, the solar meridian angle is fixed. The difference between the estimated solar meridian angle from the captured polarized image of the city sky and the true solar meridian angle is the vehicle's heading angle.

[0004] However, the polarization pattern of skylight in urban areas differs from that in open areas. Urban areas often contain densely packed high-rise buildings, which block portions of the sky, creating a unique urban polarization pattern. In this urban polarization pattern, the correct skylight polarization pattern is significantly compressed, and the blocked area introduces a significant amount of interference noise, significantly reducing the accuracy of conventional heading angle estimation methods.

[0005] To overcome the damage caused by occluded areas to the original skylight polarization pattern and achieve accurate estimation of the vehicle heading angle in urban environments, several polarization pattern restoration methods have been proposed, such as the Chinese invention patents CN117928565A and CN117809016A. However, these methods rely on segmenting the occluded area, then restoring the correct skylight polarization pattern within the occluded area, and then estimating the heading angle from the restored skylight polarization pattern. These methods have two drawbacks. First, they require segmenting and repairing the polarization angle distribution, and then estimating the solar meridian angle from the restored polarization distribution, making the heading angle estimation algorithm lengthy and consuming a large amount of time and computing resources. Second, these methods are more effective for urban polarization patterns with light occlusion, but less effective for urban polarization patterns with heavy occlusion. This is because the heading angle extraction accuracy of these methods depends largely on the effectiveness of the polarization pattern restoration. The restoration algorithm has difficulty recovering the original skylight polarization pattern in severely occluded scenes, which greatly reduces the effectiveness of heading angle extraction. Summary of the Invention

[0006] In order to solve the technical problems existing in the above-mentioned prior art, the purpose of the present invention is to provide a polarization navigation heading angle estimation method based on a neural network, which can estimate the heading angle from the polarization pattern of a city with dense occlusion, reduce the lengthy steps such as the segmentation and repair of the polarization pattern, and improve the adaptability of the heading angle estimation method to the polarization pattern of a heavily occluded city.

[0007] To achieve the above-mentioned object of the invention, the present invention provides a method for estimating a heading angle of polarization navigation based on a neural network, comprising the following steps:

[0008] Obtain polarized images of the city sky taken by a polarization camera;

[0009] Inputting the polarization image into an end-to-end neural network model to directly output an estimated value of the solar meridian angle;

[0010] The end-to-end neural network model is trained using a polarization image dataset containing occluded scenes, and its network structure includes:

[0011] Dilated convolution layer: Separates sub-images with different polarization directions from the original polarization image;

[0012] Fixed calculation layer: The polarization angle distribution is calculated using a physical formula, which is expressed as:

[0013]

[0014] Where I(A), I(B), I(C), and I(D) represent four 1224 × 1024 pixel images composed of pixels with four different polarization directions of angles A, B, B, and D, respectively, and A, B, C, and D are the directions of four mutually orthogonal angles;

[0015] Feature extraction layer: deep feature learning of polarization angle distribution;

[0016] Classification output layer: outputs the discretized solar meridian angle classification results.

[0017] According to a technical solution of the present invention, the dilated convolution layer uses four 2×2 convolution kernels with a step size of 2 to extract sub-images of polarization directions at four angles of A, B, B, and D respectively.

[0018] According to a technical solution of the present invention, the parameters of the fixed calculation layer remain unchanged during the training process, and the output polarization angle distribution is reduced to 224×224 pixels through center clipping and mean filtering.

[0019] According to a technical solution of the present invention, the training process of the end-to-end neural network model includes:

[0020] Building the end-to-end neural network model for urban polarization navigation heading angle estimation;

[0021] Construct a simulated data set;

[0022] The end-to-end neural network model is trained using a simulated dataset.

[0023] According to a technical solution of the present invention, the simulation data set is constructed by a virtual-real fusion method, including:

[0024] Generate ideal polarization angle distribution based on Rayleigh scattering theory;

[0025] Calibrate the internal parameters of the polarization camera and establish the imaging model of the polarization camera;

[0026] A small number of city sky images were collected on-site using a polarization camera, and an image processing algorithm was used to segment the valid sky area and invalid occluded area in the image to generate a city occlusion mask.

[0027] According to the imaging model, the ideal model of atmospheric polarization pattern is projected onto a two-dimensional plane to generate an ideal skylight polarization angle distribution map;

[0028] Apply the occlusion mask to the ideal skylight polarization angle distribution map and inject noise into the occluded area;

[0029] By changing the position of the sun, a large number of urban skylight polarization angle distributions are obtained. The urban polarization angle distribution and the solar meridian angle value exist in pairs, forming an urban atmospheric polarization pattern dataset.

[0030] According to a technical solution of the present invention, the noise includes at least one of Gaussian noise and salt and pepper noise, and the occlusion mask covers at least one occlusion type of buildings, clouds, and leaves.

[0031] According to a technical solution of the present invention, the end-to-end neural network model is trained using a simulated data set, the simulated data set of the urban polarization navigation heading angle estimation neural network is input into the neural network, the loss function between the estimated value and the true value of the solar meridian angle is calculated, and the loss function is continuously optimized using an iterative optimization method to minimize the loss;

[0032] The loss function between the estimated solar meridian angle and the true value is expressed as:

[0033]

[0034] Among them, y i is the one-hot encoding of the true label, so that the j-th probability y j is 1, and the rest are 0. Represents the probability value of the i-th dimension in the predicted probability distribution, x i Represents the data in the i-th node of the linear output layer, x j Represents the data in the jth node of the linear output layer.

[0035] According to a technical solution of the present invention, the end-to-end neural network model adopts Adam optimizer, and the initial learning rate is set to 10 -3 , and decayed according to the cosine annealing strategy, the decay period was set to 50, and the minimum value of the learning rate was set to 10 -6 .

[0036] According to a technical solution of the present invention, the feature extraction layer is a residual network, comprising a convolutional layer, a maximum pooling layer, a residual block and a mean pooling layer;

[0037] The depth of the residual network is 18 layers (ResNet18), and the output layer is connected to a global average pooling layer.

[0038] According to one aspect of the present invention, an electronic device is proposed, comprising: one or more processors, one or more memories, and one or more computer programs; wherein the processor is connected to the memory, and the one or more computer programs are stored in the memory. When the electronic device is running, the processor executes the one or more computer programs stored in the memory, so that the electronic device performs the neural network-based polarization navigation heading angle estimation method as described in any one of the above technical solutions.

[0039] According to one aspect of the present invention, a computer-readable storage medium is proposed for storing computer instructions. When the computer instructions are executed by a processor, a neural network-based polarization navigation heading angle estimation method as described in any one of the above technical solutions is implemented.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] The present invention proposes a neural network-based method for estimating heading angles for polarization navigation. This method transforms heading angle estimation from a traditional feature extraction problem into a neural network multi-classification problem. This method constructs an end-to-end classification neural network whose input is a polarized image of a city sky scene and whose output is the solar meridian angle. This method, when applied to urban polarization navigation, eliminates the need for segmentation and repair of obscured areas. Instead, it estimates the solar meridian angle directly from the polarized image of the city with obscured areas, significantly simplifying the heading angle estimation process and saving computational time.

[0042] The present invention no longer relies on the effects of segmentation and repair algorithms, but instead utilizes the excellent prediction characteristics of neural networks. The accuracy of heading angle extraction in severely occluded scenes will be greatly improved.

[0043] The urban polarization navigation heading angle estimation method of the present invention is a data-driven method. It only needs to use polarization patterns of different scenes, different occlusion levels, different weather and lighting conditions, and different occlusion types as data sets during the training process to improve the generalization ability of the neural network and ensure the effect of the neural network in scenes with different occlusion levels, making it easy to migrate and apply the method in other scenes.

[0044] The present invention uses an urban atmospheric polarization pattern simulation generation method to construct a simulation data set. The urban atmospheric polarization pattern simulation generation method is a data generation method that integrates virtual and real elements. Compared with existing theoretical modeling methods, this method can obtain diverse urban polarization pattern data. Compared with existing field measurement methods, this method can easily and quickly obtain a large amount of urban polarization pattern data with true values. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be derived from these drawings without inventive effort.

[0046] Figure 1 This is a schematic diagram of the principle of urban bionic polarization navigation;

[0047] Figure 2 This is a schematic diagram of the neural network structure for urban polarization navigation heading angle estimation according to the present invention;

[0048] Figure 3 This is a flow chart of the end-to-end urban polarization navigation heading angle estimation neural network simulation data set construction method of the present invention;

[0049] Figure 4 This is a schematic diagram of the neural network structure for urban polarization navigation heading angle estimation according to an embodiment of the present invention;

[0050] Figure 5 An ideal model of atmospheric polarization mode established based on Rayleigh scattering theory according to an embodiment of the present invention;

[0051] Figure 6 The polarized image of the city sky and the city occlusion mask captured by the polarization camera according to an embodiment of the present invention;

[0052] Figure 7 Schematic diagram of the softmax function principle of an embodiment of the present invention;

[0053] Figure 8 This is a schematic diagram of the training results of the neural network for end-to-end urban polarization navigation heading angle estimation according to an embodiment of the present invention. DETAILED DESCRIPTION

[0054] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0055] like Figures 1 to 8 As shown, a method for estimating a heading angle of polarization navigation based on a neural network of the present invention comprises the following steps:

[0056] Obtain polarized images of the city sky taken by a polarization camera;

[0057] Inputting the polarization image into an end-to-end neural network model to directly output an estimated value of the solar meridian angle;

[0058] The end-to-end neural network model is trained using a polarization image dataset containing occluded scenes, and its network structure includes:

[0059] Dilated convolution layer: Separates sub-images with different polarization directions from the original polarization image;

[0060] Fixed calculation layer: The polarization angle distribution is calculated using a physical formula, which is expressed as:

[0061]

[0062] Where I(A), I(B), I(C), and I(D) represent four 1224 × 1024 pixel images composed of pixels with four different polarization directions of angles A, B, B, and D, respectively, and A, B, C, and D are the directions of four mutually orthogonal angles;

[0063] In this formula, A, B, C, and D are the directions of the micro-polarizers used in polarization cameras, usually 0°, 45°, 90°, and 135°. Therefore, the physical formula can be changed to:

[0064]

[0065] Mutually orthogonal angles can be understood as adjacent angles being 45°.

[0066] Feature extraction layer: deep feature learning of polarization angle distribution;

[0067] Classification output layer: outputs the discretized solar meridian angle classification results.

[0068] The estimation of heading angle is transformed from a traditional feature extraction problem into a neural network multi-classification problem. An end-to-end classification neural network is constructed, whose input is a polarized image of an urban sky scene and whose output is the solar meridian angle. This network eliminates multiple steps such as polarization pattern segmentation and repair, and directly estimates the solar meridian angle from polarized images of cities with occluded areas, greatly simplifying the heading angle estimation process. In addition, using polarization patterns from different scenes, different occlusion levels, different weather and lighting conditions, and different occlusion types as a dataset for training the network can also improve the generalization ability of the neural network, ensuring its effectiveness in scenes with different occlusion levels, especially improving the accuracy of heading angle estimation in urban environments with dense occlusion.

[0069] The present invention constructs an end-to-end neural network model, which includes a void convolution layer, a fixed calculation layer, a feature extraction layer, and a classification output layer in sequence. It can directly output a heading angle estimation value from the city sky polarization image without relying on polarization mode segmentation, repair, or external input in traditional methods. It can simplify the heading angle estimation process, greatly improve computing efficiency, effectively enhance the estimation accuracy in heavy occlusion scenarios, strengthen the model generalization ability, reduce the system implementation cost, solve the problem of polarization mode destruction caused by occlusion of dense urban buildings, and provide a robust and efficient autonomous navigation solution for carriers such as drones and ground robots.

[0070] In some embodiments of the present invention, the training process of the end-to-end neural network model includes:

[0071] Step S1: building the end-to-end neural network model for urban polarization navigation heading angle estimation;

[0072] The end-to-end urban polarization navigation heading angle estimation neural network input is the urban sky polarization image, and the output is the solar meridian angle. The network structure diagram is as follows Figure 2 As shown in the figure, it mainly includes input layer, hole convolution layer, calculation layer, intermediate layer and output layer.

[0073] The input layer is the first layer of the neural network and receives a polarized image of the city sky. This image is captured vertically upward by a polarization camera mounted on a mobile vehicle. The polarization camera uses an internal polarization imaging sensor to generate images. The resulting polarization image consists of pixels with four different polarization directions: 0°, 45°, 90°, and 135°.

[0074] The polarization camera consists of two parts: the Lucid TTRI050S-QC camera and the FUJINFILM FE185C057HA-1 fisheye lens. The polarization camera relies on the internal polarization imaging sensor to capture a 2448-pixel × 2048-pixel polarization image, which is composed of pixels with four different polarization directions of 0°, 45°, 90°, and 135° arranged in a cross pattern.

[0075] After the city sky polarization image is input, it first enters the dilated convolution layer. The dilated convolution kernel size is 2 pixels × 2 pixels, and the stride is 2 pixels. It is defined as follows:

[0076]

[0077] K 90 , K 45 , K 135 The subscripts of K0 and K1 represent the polarization direction extracted by the convolution kernel. 90As an example, in order to extract the light intensity information in the 90° polarization direction, it is necessary to compare the original light intensity image with the K 90 Perform convolution with a step size of 2. The calculation process is shown in the formula:

[0078] I out (90°) i,j =IMG*K 90 =1×img 2i-1,2j-1 +0×img 2i-1,2j +0×img 2i,2j-1 +0×img 2i,2j ,i=1,2,3,…,1024; j=1,2,3,…,1224.

[0079] Among them, I out (90°) i,j I(0°), I(45°), I(90°), and I(135°) are four 1224×1024 pixel images, denoted by I(0°), I(45°), I(90°), and I(135°), respectively.

[0080] After the output from the dilated convolutional layer, it enters the computation layer. The computation layer is a matrix operation layer. The parameters involved in the operation are fixed and do not change with the neural network training. The matrix operation performed by the computation layer can be expressed as:

[0081]

[0082] AoP is the calculated skylight polarization angle distribution. The calculated AoP matrix size is 1224 pixels × 1024 pixels. To improve training and computation speed, the AoP matrix is ​​preprocessed by center-cropping and then performing a mean filter. This reduces random noise interference and computational complexity, resulting in a final size of 224 × 224 pixels.

[0083] After output from the computational layer, the data enters the intermediate layer. The intermediate layer is the ResNet18 residual network, which is the core of the urban polarization navigation heading angle estimation neural network. The ResNet18 residual network consists of 18 layers, including convolutional layers, maximum pooling layers, residual blocks, and mean pooling layers.

[0084] Preferably, the structure and specific parameters of each component of the neural network are shown in Figure 4 .

[0085] After being output from the middle layer, it enters the linear output layer. The output layer is the last layer of the neural network.

[0086] Preferably, the resolution of the solar meridian angle estimation is designed to be 1°, the number of input nodes of the linear layer is 512, and the number of output nodes is 360. The labels of these nodes are discrete 0°, 1°, 2°...359°, respectively indicating that the solar meridian angle in the input image is equal to 0° to 359°.

[0087] Step S2: constructing a simulation data set;

[0088] Since it is difficult to quickly and easily obtain a large amount of true and diverse urban polarization pattern data using either theoretical modeling or field measurements, this method uses the urban atmospheric polarization pattern simulation generation method to construct a simulated dataset.

[0089] like Figure 3 As shown in Figure 2, constructing a dataset using the urban atmospheric polarization model simulation generation method includes the following steps:

[0090] (1) Based on the atmospheric scattering theory, an ideal model of atmospheric polarization pattern is established.

[0091] Preferably, if Figure 5 As shown in the figure, the Earth's atmosphere is modeled into a three-dimensional hemispherical model based on the Rayleigh scattering theory. The direction angle and altitude angle of the sun's position are input to simulate and generate the ideal skylight polarization angle distribution.

[0092] (2) Calibrate the internal parameters of the polarization camera and establish the imaging model of the polarization camera.

[0093] Preferably, the polarization camera consists of a Lucid TTRI050S-QC camera and a FUJINFILMFE185C057HA-1 fisheye lens. Since the lens used in the imaging measurement system is a fisheye lens, a 12×9 checkerboard calibration plate with a side length of 30 mm is used to calibrate the imaging system using the Kannala fisheye imaging model. The intrinsic parameter matrix of the imaging system is obtained as follows:

[0094]

[0095] Among them, f x and f y Respectively represent the imaging focal length of the imaging system in the horizontal and vertical directions, c x and c y Represents the coordinates of the imaging center point. The four distortion coefficients of the imaging system are:

[0096] [k1,k2,k3,k4]=[0.0202837, -0.007755907, -0.000595663, 0.00234646].

[0097] At this point, the internal parameters of the imaging system have been clarified and the imaging model of the polarization camera has been established.

[0098] (3) Use polarization cameras to collect a small amount of city sky images, and use image processing algorithms to segment the effective sky area and invalid occluded area in the image to generate a city occlusion mask. The effect is as follows: Figure 6 shown.

[0099] (4) According to the imaging model, the ideal atmospheric polarization model is projected onto a two-dimensional plane to generate an ideal skylight polarization angle distribution map. By simply changing the sun position in the ideal atmospheric polarization model, a large number of ideal skylight polarization angle distributions and the corresponding solar meridian angle values ​​can be easily and quickly obtained.

[0100] (5) The urban occlusion mask is used to divide the sky area and the occlusion area in the skylight polarization angle distribution, and noise is introduced into the occlusion area to generate an urban skylight polarization angle distribution that includes both valid sky areas and invalid occlusion areas.

[0101] In some embodiments of the present invention, the noise includes at least one of Gaussian noise and salt and pepper noise, and the occlusion mask covers at least one occlusion type of buildings, clouds, and leaves.

[0102] (6) A large number of urban skylight polarization angle distributions are obtained by changing the sun's position. The urban polarization angle distribution and the solar meridian angle value exist in pairs, forming an urban atmospheric polarization pattern dataset.

[0103] Step S3: Using a simulated data set to train the end-to-end neural network model.

[0104] Since the parameters of the dilated convolutional layer and computational layer of the urban polarization navigation heading angle estimation neural network are fixed and do not change with network training, only the remaining intermediate layers and output layers need to be trained.

[0105] The simulated dataset of the urban polarization navigation heading angle estimation neural network is input into the neural network, and the loss function between the estimated value and the true value of the solar meridian angle is calculated. The iterative optimization method is used to continuously optimize the loss function to minimize the loss.

[0106] To calculate the loss function between the estimated solar meridian angle of the neural network output layer and the true value, it is necessary to quantify the difference between the output layer output and the true value. The following steps are included:

[0107] (1) Connect a softmax activation function after the output layer to The output of each node is converted into a dimensional probability distribution, and the sum of the probabilities is 1.

[0108] (2) Use one-hot encoding to construct the true value into a dimensional probability distribution.

[0109] (3) Calculate the maximum likelihood function of two probability distributions to measure the similarity between the two probability distributions.

[0110] Preferably, if Figure 7 As shown in Figure 1, the softmax activation function converts the output results of the 360 ​​output nodes of the output layer into a 360-dimensional probability distribution, and the sum of the probabilities is 1. Specifically for the urban bionic polarization navigation heading angle estimation neural network proposed in this paper, the calculation formula of the softmax function is as follows:

[0111]

[0112] in,

[0113]

[0114] n represents the node label, x n Represents the data in the nth node of the linear output layer, Represents the probability value of the nth dimension in the predicted probability distribution, and

[0115] Preferably, for an input data, assuming its true label is j, use one-hot encoding to construct a true probability distribution y0, y1...y 359 , so that the j-th probability y j is 1, and the rest are 0.

[0116] Preferably, in order to measure the similarity between two probability distributions, the maximum likelihood loss function is used, which is calculated as follows:

[0117]

[0118] Among them, y i is the one-hot encoding of the true label, so that the j-th probability y j is 1, and the rest are 0. Represents the probability value of the i-th dimension in the predicted probability distribution.

[0119] For a batch of data, assuming that the amount of data in the batch is N, the mean of all loss function results in the batch is used as the loss function value of the batch training, which is calculated as follows:

[0120]

[0121] Preferably, the Adam optimization method is used to optimize the neural network weights. The initial learning rate of the Adam optimization method is set to 10 -3 The learning rate is decayed using cosine annealing, the decay period is set to 50, and the minimum value of the learning rate is set to 10 -6 .

[0122] Thus, an end-to-end urban polarization navigation heading angle estimation neural network is obtained. The input is the polarization image of the urban sky scene, and the output is the solar meridian angle. Figure 8 This visualizes the training results. train_accuracy represents the accuracy of the entire training set after loading and traversing all the training data in each training round. This is the ratio of the number of correctly predicted images in the training set to the total number of images in the training set. test_accuracy represents the accuracy of the heading angle estimation for all data in the validation set. This is the ratio of the number of correctly predicted images in the validation set to the total number of images in the validation set. Based on the performance of these parameters, the network is considered to have converged well between 23k and 30k training rounds, with test_accuracy for the corresponding training batches reaching a maximum value of over 91%.

[0123] According to one aspect of the present invention, an electronic device is proposed, comprising: one or more processors, one or more memories, and one or more computer programs; wherein the processor is connected to the memory, and the one or more computer programs are stored in the memory. When the electronic device is running, the processor executes the one or more computer programs stored in the memory, so that the electronic device performs the neural network-based polarization navigation heading angle estimation method as described in any one of the above technical solutions.

[0124] According to one aspect of the present invention, a computer-readable storage medium is proposed for storing computer instructions. When the computer instructions are executed by a processor, a neural network-based polarization navigation heading angle estimation method as described in any one of the above technical solutions is implemented.

[0125] Computer-readable storage media may include any medium capable of storing or transmitting information. Examples of computer-readable storage media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, and the like. The code segments may be downloaded via a computer network such as the Internet, an intranet, and the like.

[0126] The present invention proposes a neural network-based method for estimating heading angles for polarization navigation. This method constructs an end-to-end neural network, transforming the heading angle estimation problem into a neural network multi-classification problem. The input is a polarized image of a city sky scene, and the output is the solar meridian angle. By constructing a dataset and training the neural network, the resulting end-to-end neural network can directly estimate heading angles from polarized images of city sky scenes with occlusions, avoiding complex steps such as polarization pattern segmentation and restoration.

[0127] Furthermore, by adjusting the data set used for training, the neural network has good generalization capabilities for different scenes, different occlusion levels, different weather and lighting conditions, and different occlusion types, and can adapt to diverse application scenarios.

[0128] Furthermore, it should be noted that the present invention may be provided as a method, apparatus, or computer program product. Thus, embodiments of the present invention may take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention may take the form of a computer program product embodied on one or more computer-usable storage media containing computer-usable program code.

[0129] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the process in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0130] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1a process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, so that a series of operation steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for implementing the process in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0131] It should also be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or terminal device comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or terminal device comprising the element.

[0132] Finally, it should be noted that the above is a preferred embodiment of the present invention. It should be noted that although the preferred embodiment of the present invention has been described, it is clear that those skilled in the art, once they understand the basic inventive concept of the present invention, can make various improvements and modifications without departing from the principles of the present invention. Such improvements and modifications should also be considered as within the scope of protection of the present invention. Therefore, the appended claims are intended to be interpreted as including the preferred embodiment and all changes and modifications that fall within the scope of the embodiments of the present invention.

Claims

1. A method for estimating heading angle of polarization navigation based on neural network, characterized in that: The following steps are involved: Obtain polarized images of the city sky taken by a polarization camera; Inputting the polarization image into an end-to-end neural network model to directly output an estimated value of the solar meridian angle; The end-to-end neural network model is trained using a polarization image dataset containing occluded scenes, and its network structure includes: Dilated convolution layer: Separates sub-images with different polarization directions from the original polarization image; Fixed calculation layer: The polarization angle distribution is calculated using a physical formula, which is expressed as: Where I(A), I(B), I(C), and I(D) represent four 1224 × 1024 pixel images composed of pixels with four different polarization directions of angles A, B, B, and D, respectively, and A, B, C, and D are the directions of four mutually orthogonal angles; Feature extraction layer: deep feature learning of polarization angle distribution; Classification output layer: outputs the discretized solar meridian angle classification results.

2. The method according to claim 1, characterized in that The dilated convolution layer uses four 2×2 convolution kernels with a step size of 2 to extract sub-images of the polarization directions of four angles A, B, B, and D respectively.

3. The method according to claim 1, characterized in that The parameters of the fixed calculation layer remain unchanged during the training process, and the output polarization angle distribution is reduced to 224×224 pixels through center cropping and mean filtering.

4. The method according to claim 1, wherein The training process of the end-to-end neural network model includes: Building the end-to-end neural network model for urban polarization navigation heading angle estimation; Construct a simulated data set; The end-to-end neural network model is trained using a simulated dataset.

5. The method according to claim 4, characterized in that The simulation data set is constructed by a virtual-real fusion method, including: Generate ideal polarization angle distribution based on Rayleigh scattering theory; Calibrate the internal parameters of the polarization camera and establish the imaging model of the polarization camera; A small number of city sky images were collected on-site using a polarization camera, and an image processing algorithm was used to segment the valid sky area and invalid occluded area in the image to generate a city occlusion mask. According to the imaging model, the ideal model of atmospheric polarization pattern is projected onto a two-dimensional plane to generate an ideal skylight polarization angle distribution map; Apply the occlusion mask to the ideal skylight polarization angle distribution map and inject noise into the occluded area; By changing the position of the sun, a large number of urban skylight polarization angle distributions are obtained. The urban polarization angle distribution and the solar meridian angle value exist in pairs, forming an urban atmospheric polarization pattern dataset.

6. The method according to claim 5, characterized in that The noise includes at least one of Gaussian noise and salt and pepper noise, and the occlusion mask covers at least one occlusion type of buildings, clouds, and leaves.

7. The method according to claim 4, characterized in that The end-to-end neural network model is trained using a simulated data set. The simulated data set of the urban polarization navigation heading angle estimation neural network is input into the neural network. The loss function between the estimated solar meridian angle and the true value is calculated. The loss function is continuously optimized using an iterative optimization method to minimize the loss. The loss function between the estimated solar meridian angle and the true value is expressed as: Among them, y i is the one-hot encoding of the true label, so that the j-th probability y j is 1, and the rest are 0. Represents the probability value of the i-th dimension in the predicted probability distribution, x i Represents the data in the i-th node of the linear output layer, x j Represents the data in the jth node of the linear output layer.

8. The method according to claim 1, characterized in that The end-to-end neural network model uses the Adam optimizer, and the initial learning rate is set to 10 -3 , and decayed according to the cosine annealing strategy, the decay period was set to 50, and the minimum value of the learning rate was set to 10 -6 .

9. An electronic device, characterized in that: include: One or more processors, one or more memories, and one or more computer programs; wherein the processor is connected to the memory, and the one or more computer programs are stored in the memory. When the electronic device is running, the processor executes the one or more computer programs stored in the memory to enable the electronic device to perform the neural network-based polarization navigation heading angle estimation method as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that Used to store computer instructions, which, when executed by a processor, implement the polarization navigation heading angle estimation method based on a neural network as described in any one of claims 1 to 8.

Images