NLOS satellite detection method, device and system, and storage medium

Abstract

The invention discloses an NLOS satellite detection method, device and system, and a storage medium, and the method comprises the steps: remarkably improving the segmentation precision of a fisheye image boundary region through introducing sky / leaf / building three-classification semantic priori, and combining a master-slave segmentation collaborative learning mechanism, dual-branch pseudo-label consistency constraint and explicit boundary modeling, and obtaining a fisheye image boundary region. NLOS false detection and missing detection in a leaf shielding scene are reduced, so that GNSS positioning precision and robustness in an urban canyon environment are improved.

Description

Technical Field

[0001] This invention belongs to the field of satellite navigation technology, specifically relating to an NLOS satellite detection method, device, system, and storage medium. Background Technology

[0002] In urban canyons, tree-lined roads, and densely populated areas with high-rise buildings, GNSS signals are susceptible to obstruction and multipath effects, with pseudorange deviations reaching tens of meters or even higher. This leads to unstable positioning solutions, error heavy-tailing, and reduced availability. To improve positioning performance in urban environments, recent years have seen the development of schemes that utilize fisheye or panoramic cameras to acquire upward-looking "sky images" to assist in LOS / NLOS satellite detection and multi-sensor fusion positioning.

[0003] Existing methods have generally developed along two main lines: one is geometric visibility analysis and shadow matching based on 3D city models, using the geometric relationship between building models and satellites to eliminate NLOS observations; the other is sky segmentation and skyline matching based primarily on fisheye images, assessing satellite visibility through binary sky masks and ephemeris projection overlap. These methods have improved urban positioning accuracy to some extent, but with the increasing complexity of urban scene structures, seasonal canopy changes, and temporary occlusions, existing methods have revealed the following limitations: 1. Insufficient boundary segmentation accuracy: Satellite projection is often located at the boundary between the sky and buildings or trees. If there is a pixel-level deviation in the sky-ground boundary, the original LOS will be misjudged as NLOS, or the real NLOS will be misjudged as LOS. The error is concentrated in the boundary zone.

[0004] 2. Insufficient category granularity: Most methods use a binary classification of "sky / non-sky", classifying leaves, branches, etc., into "non-sky" or "buildings", ignoring the characteristics of leaves such as semi-transparency and fine structure of branches and leaves, making it difficult to accurately depict the true distribution of LOS / NLOS in complex occlusion scenes.

[0005] 3. Lack of high-quality three-class classification data and unified protocol: Most existing fisheye segmentation datasets are binary classification, lacking high-quality three-class classification labeled data and unified evaluation metrics that systematically cover "sky / leaves / buildings". The learning-based methods are limited in their ability to model complex boundaries and thin structures.

[0006] 4. Incomplete learning-based methodologies: In engineering practice, end-to-end machine learning / computer vision solutions for fisheye NLOS detection are still scarce, lacking standard data protocols, unified evaluation systems, and reproducible master-slave segmentation collaborative learning frameworks and public implementations. Summary of the Invention

[0007] To address the problems existing in the prior art, this invention provides an NLOS satellite detection method, device, system, and storage medium. By introducing a three-class semantic prior of "sky / leaves / buildings" and combining a master-slave segmentation collaborative learning mechanism, a two-branch pseudo-label consistency constraint, and explicit boundary modeling, the segmentation accuracy of fisheye image boundary regions is significantly improved, and the false detection and false detection of NLOS in foliage occlusion scenarios are reduced, thereby improving the GNSS positioning accuracy and robustness in urban canyon environments.

[0008] To achieve the above objectives, the present invention provides the following solution: An NLOS satellite detection method, comprising: Step 1: Obtain urban scene images captured by the fisheye camera installed on the upper side, divide the images into labeled training sets and unlabeled / weakly labeled training sets containing three semantic categories: "sky / leaves / buildings", and perform size normalization and geometric and lighting enhancement on the images. Step 2: Based on the labeled training set, construct the upstream guided segmentation main network and output pixel-level three-class semantic probability maps and corresponding edge response maps; Step 3: Construct a collaborative segmentation subnetwork containing two lightweight branches. Based on the soft probability output, multi-layer feature field, and contour response of the upstream guiding segmentation main network, obtain the trained three-class fisheye segmentation model. Step 4: Input the fisheye image to be tested into the three-class fisheye segmentation model to obtain the optimized three-class mask; Step 5: Obtain the azimuth and elevation angles of each satellite. Based on the angle-radius mapping in the fisheye imaging model and the preset radial distortion parameters, transform the satellite orientation into image plane pixel coordinates. Based on the category label of the pixel in the optimized three-class mask, determine the satellites located in the "sky" or "leaf" area as LOS, and determine the satellites located in the "building" area or that are occluded as NLOS, thus completing the NLOS satellite detection based on fisheye images.

[0009] Preferably, in step 2, an upstream guided segmentation main network is constructed, and a hierarchical self-attention feature extraction and stepwise recovery decoding structure is used to train the labeled training set for three-class semantic segmentation, outputting pixel-level three-class semantic probability maps and corresponding edge response maps.

[0010] Preferably, in step 3, a collaborative segmentation sub-network with two lightweight branches is constructed. Each branch adopts a compact encoder-decoder structure and an explicit contour auxiliary head is set on the output side. On the labeled training set, the three-class classification output is constrained by a combined loss based on pixel class log-likelihood and region overlap, and the contour auxiliary head is constrained by the contour ground truth derived from geometric labels. On the unlabeled / weakly labeled training set, a bidirectional pseudo-label consistency constraint is introduced between the two branches. Combined with the soft probability output, multi-layer feature field and contour response from the upstream guiding main network, output alignment, feature alignment and contour alignment constraints are applied to the collaborative segmentation sub-network. The network parameters are updated under pixel-level confidence mask filtering to obtain the trained three-class fisheye segmentation model.

[0011] Preferably, in step 4, the fisheye image to be tested is input into the three-class fisheye segmentation model to obtain a pixel-level semantic mask and contour response map of "sky / leaves / buildings". The semantic mask is then subjected to post-processing of suppressing isolated bright areas inside buildings and refining leaf areas based on color cues to obtain an optimized three-class mask.

[0012] The present invention also provides an NLOS satellite detection device, comprising: The first processing module is used to acquire urban scene images captured by the fisheye camera installed on the upper side, divide the images into labeled training sets and unlabeled / weakly labeled training sets containing three semantic categories: "sky / leaves / buildings", and perform size normalization and geometric and lighting enhancement on the images. The second processing module is used to construct the upstream guided segmentation main network based on the labeled training set, and output pixel-level three-class semantic probability maps and corresponding edge response maps; The third processing module is used to construct a collaborative segmentation sub-network containing two lightweight branches. Based on the soft probability output, multi-layer feature field, and contour response of the upstream guiding segmentation main network, a trained three-class fisheye segmentation model is obtained. The fourth processing module is used to input the fisheye image to be tested into the three-class fisheye segmentation model to obtain an optimized three-class mask; The fifth processing module is used to acquire the azimuth and elevation angles of each satellite. Based on the angle-radius mapping in the fisheye imaging model and the preset radial distortion parameters, the satellite orientation is transformed into image plane pixel coordinates. According to the category label of the pixel in the optimized three-class mask, satellites located in the "sky" or "leaf" area are identified as LOS, and satellites located in the "building" area or that are obscured are identified as NLOS, thus completing the NLOS satellite detection based on fisheye images.

[0013] Preferably, the second processing module constructs an upstream guided segmentation main network, and uses a hierarchical self-attention feature extraction and stepwise recovery decoding structure to perform three-class semantic segmentation training on the labeled training set, outputting pixel-level three-class semantic probability maps and corresponding edge response maps.

[0014] Preferably, the third processing module constructs a collaborative segmentation sub-network containing two lightweight branches. Each branch adopts a compact encoder-decoder structure and sets an explicit contour auxiliary head on the output side. On the labeled training set, the three-class classification output is constrained by a combined loss based on pixel class log-likelihood and region overlap, and the contour auxiliary head is constrained by the contour ground truth derived from geometric labels. On the unlabeled / weakly labeled training set, a bidirectional pseudo-label consistency constraint is introduced between the two branches, and the soft probability output, multi-layer feature field, and contour response from the upstream guiding main network are combined to apply output alignment, feature alignment, and contour alignment constraints to the collaborative segmentation sub-network. The network parameters are updated under pixel-level confidence mask filtering to obtain the trained three-class fisheye segmentation model.

[0015] Preferably, the fourth processing module inputs the fisheye image to be tested into the three-class fisheye segmentation model to obtain a pixel-level semantic mask and contour response map of "sky / leaves / buildings". Then, it performs post-processing on the semantic mask to suppress isolated bright areas inside buildings and refine leaf areas based on color cues, thereby obtaining an optimized three-class mask.

[0016] The present invention also provides an NLOS satellite detection system, comprising: a memory and a processor, wherein the memory stores a computer program executed by the processor, and the computer program executes an NLOS satellite detection method when run by the processor.

[0017] The present invention also provides a storage medium storing a computer program that executes the NLOS satellite detection method when running.

[0018] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention explicitly separates "leaves" from the traditional "non-sky" category and establishes a three-class semantic prior of "sky / leaves / buildings". It can distinguish complex situations such as open field of view, foliage occlusion and building occlusion in the image domain, providing a more granular and interpretable visual prior for star-by-star LOS / NLOS discrimination and significantly improving the accuracy of visibility discrimination.

[0019] 2. This invention adopts a multi-scale master-slave segmentation collaborative learning framework. A high-capacity master guiding network provides semantic and boundary priors, and then the representation is compressed into a lightweight slave network through category response transfer, multi-scale feature transfer and boundary response transfer. This enables the slave model to obtain global and boundary indicators that are close to or even better than the master model under the conditions of limited parameter scale and inference latency, thus balancing accuracy and deployment efficiency.

[0020] 3. This invention uses a dual-branch pseudo-label consistency constraint and pixel-level confidence mask to stably mine pseudo-label information in the presence of a large amount of unlabeled / weakly labeled data, suppress the negative impact of low-confidence noise labels on training, and effectively improve the model's generalization ability under cross-scene, complex lighting and seasonal tree canopy change conditions.

[0021] 4. This invention strengthens boundary modeling by using independent edge discrimination branches, boundary response migration, and consistency constraints applied only within the boundary zone. Combined with denoising of isolated islands inside buildings and color-driven leaf region refinement post-processing, it effectively eliminates false detection of bright small blocks inside buildings and bidirectional misclassification at the tree-sky boundary, making LOS / NLOS discrimination more robust in high-risk boundary areas.

[0022] This invention directly couples a three-class semantic mask with satellite orientation using fisheye projection geometry, making it usable as a front-end visibility filtering module in existing GNSS positioning systems. It significantly reduces CEP and 3D positioning errors without relying on detailed 3D city models, demonstrating good engineering feasibility and widespread application value. Attached Figure Description

[0023] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart of the NLOS satellite detection method according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the main segmentation network in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of the dual-branch student segmentation network and the boundary auxiliary head in an embodiment of the present invention; Figure 4 This is a flowchart of fisheye image reasoning and post-processing in an embodiment of the present invention; Figure 5 This is a schematic diagram of the training process of "main-auxiliary segmentation network + dual-branch pseudo-label consistency constraint + multi-scale representation transfer" in an embodiment of the present invention; Figure 6The image shown is a mask of the sky / leaves / buildings obtained by model inference in an embodiment of the present invention. Figure 7 This is a comparison chart of the model evaluation metrics in this embodiment of the invention with common baseline models; Figure 8 This is a schematic diagram of satellite LOS / NLOS discrimination assisted by three-class semantic mask in an embodiment of the present invention. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0026] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0027] Example 1 This invention provides an NLOS satellite detection method, comprising: Step 1: Obtain urban scene images captured by the fisheye camera installed on the upper side, divide the images into labeled training sets and unlabeled / weakly labeled training sets containing three semantic categories: "sky / leaves / buildings", and perform size normalization and geometric and lighting enhancement on the images. Step 2: Based on the labeled training set, construct the upstream guided segmentation main network and output pixel-level three-class semantic probability maps and corresponding edge response maps; Step 3: Construct a collaborative segmentation subnetwork containing two lightweight branches. Based on the soft probability output, multi-layer feature field, and contour response of the upstream guiding segmentation main network, obtain the trained three-class fisheye segmentation model. Step 4: Input the fisheye image to be tested into the three-class fisheye segmentation model to obtain the optimized three-class mask; Step 5: Obtain the azimuth and elevation angles of each satellite. Based on the angle-radius mapping in the fisheye imaging model and the preset radial distortion parameters, transform the satellite orientation into image plane pixel coordinates. Based on the category label of the pixel in the optimized three-class mask, determine the satellites located in the "sky" or "leaf" area as LOS, and determine the satellites located in the "building" area or that are obscured as NLOS, thus completing the NLOS satellite detection based on fisheye images.

[0028] As one embodiment of the present invention, in step 2, an upstream guided segmentation main network is constructed, and a hierarchical self-attention feature extraction and stepwise recovery decoding structure is used to perform three-class semantic segmentation training on the labeled training set, outputting pixel-level three-class semantic probability maps and corresponding edge response maps.

[0029] In one embodiment of the present invention, step 3 involves constructing a collaborative segmentation sub-network comprising two lightweight branches. Each branch employs a compact encoder-decoder structure and an explicit contour auxiliary head is set on the output side. On the labeled training set, the three-class classification output is constrained by a combined loss based on pixel category log-likelihood and region overlap, and the contour auxiliary head is constrained by the contour ground truth derived from geometric annotations. On the unlabeled / weakly labeled training set, a bidirectional pseudo-label consistency constraint is introduced between the two branches. Combined with the soft probability output, multi-layer feature field, and contour response from the upstream guiding main network, output alignment, feature alignment, and contour alignment constraints are applied to the collaborative segmentation sub-network. The network parameters are updated under pixel-level confidence mask filtering to obtain the trained three-class fisheye segmentation model.

[0030] As one embodiment of the present invention, in step 4, the fisheye image to be tested is input into the three-class fisheye segmentation model to obtain the pixel-level semantic mask and contour response map of "sky / leaves / buildings". Then, the semantic mask is sequentially processed by suppressing isolated bright areas inside buildings and refining leaf areas based on color cues to obtain an optimized three-class mask.

[0031] Example 2 like Figure 1 , 5 As shown, the present invention provides an NLOS satellite detection method, comprising: Step S1: Data Acquisition and Preprocessing A top-view fisheye camera was fixedly mounted on the roof of a vehicle or near a GNSS antenna to collect urban scene images at different times, in different weather conditions, and in different neighborhoods (including high-rise building areas, tree-lined roads, etc.), resulting in a raw fisheye image set: ; For each input image Perform size normalization and standardization processing, such as uniformly scaling to... It performs data augmentation operations including random horizontal flipping, brightness / contrast perturbation, Gaussian noise, and slight blurring. The preprocessing operator is denoted as... The enhanced image is obtained: ; For labeled images, three semantic categories, "sky / leaves / buildings," are manually and meticulously labeled to generate grayscale masks. Different categories correspond to different grayscale values, for example: ; And through a preset mapping table Convert grayscale values ​​to category numbers: ; This results in a labeled set: ; And sets of unlabeled / weakly labeled items: ; Step S2: Training the main segmentation network like Figure 2 As shown, this example constructs a large-capacity master segmentation network to learn high-quality semantics and boundary priors on fully labeled data.

[0032] The main segmentation network can be formally represented as: ; in To split the main model, Its parameters, For pixel x, it belongs to category The predicted probability, The probability of boundary pixels.

[0033] The network structure includes: Encoder: Employs a hierarchical self-attention structure, downsampling the input image step by step to... Extracting multi-scale features ; Decoder: Employs a top-down, stepwise upsampling structure and fuses shallow details with deep semantics through skip connections, outputting a three-class logits feature map. ; Boundary Auxiliary Branch: Boundary prediction branches are introduced at several decoding scales, and the boundary probability map is obtained through 1×1 convolution and upsampling. ; in, This is the Sigmoid function.

[0034] The final pixel probability is calculated using softmax: ; For labeled samples The pixel-level semantic cross-entropy loss is defined as: ; in Let be the set of image pixels. To improve inter-class balance and region overlap quality, Dice loss is introduced: ; Semantic labels are converted into binary boundary ground truths using edge operators (such as Sobel or morphological gradient). Binary cross-entropy loss is applied to the boundary branches: ; The training objective of the main segmentation network is obtained by combining the results: ; in These are the weighting coefficients.

[0035] Through minimize on The parameters of the trained main network are obtained by using an iterative optimization algorithm (such as AdamW). .

[0036] Step S3: Co-training and Representation Transfer of Downstream Segmentation Networks like Figure 3 As shown, this embodiment constructs two downstream segmentation branches with identical structures but independent parameters, and combines them with the main network output to achieve multi-loss collaborative training of a lightweight network.

[0037] Each branch can be represented as: ; in To adapt branches for downstream applications, For three-class classification probabilities, This represents the boundary probability.

[0038] The downstream segmentation network encoder employs a multi-scale feature extraction structure (e.g., a hierarchical Transformer encoder), while the decoder uses a lightweight full MLP or a small convolutional upsampling structure. At the decoder end, the following settings are configured: 1. Three-class semantic header: outputs logits With softmax probability: ; 2. Boundary Auxiliary Head: Outputs boundary probabilities: ; for For each subordinate branch, calculate the semantic cross-entropy and Dice loss separately: ; ; Boundary truth As mentioned earlier, the boundary branches of subordinate splitting branches are constrained by binary cross-entropy. ; The supervised loss on the labeled path is: ; for Construct two different data augmented views from the same original image: ; The inputs are fed into the two splitting branches respectively to obtain the predicted probabilities. Define a pixel-level confidence mask based on the main network prediction: ; in, This is the confidence threshold. Only when... Unsupervised constraints are applied within the high confidence region.

[0039] Use the high-confidence predicted class of one branch as the pseudo-label of the other branch, for example: ; Define the bi-branch pseudo-label consistency loss as: ; like Figure 5 As shown, in order to reuse the high-quality semantics and boundary priors learned by the main network in a lightweight downstream adaptation network, this embodiment introduces multi-path representation transfer constraints.

[0040] (1) Class Probability Soft Target Transfer make and Temperature is introduced into the logits of the main network and the downstream subordinate branches, respectively. Define soft probability: ; ; Constructing the soft target migration loss: ; (2) Multi-scale feature representation transfer Extracting multi-scale feature maps from the decoding stages of the main network and subordinate networks

[0041] and Through 1×1 mapping After aligning the channels, construct a multi-scale feature transfer loss: ; (3) Boundary probability transition With the main network boundary probability For soft targets, guide the boundary prediction of the two-way splitting branches: ; The representation transfer loss between the master and slave networks is obtained by combining the results: ; The overall training objective of the subordinate network is: ; in This is for loss weights. A phased warming and threshold annealing strategy can be used during training: 1. In the early training phase, with a smaller and lower confidence threshold Start up to avoid gradient oscillations caused by unstable pseudo-labels; 2. Gradually increase with each training round. And improve This leads to pseudo-labels and transfer constraints gradually dominating the learning of high-confidence regions.

[0042] Through minimize on This yields subordinate partitioning model parameters with a moderate parameter size suitable for deployment. In actual deployment, the branch with better performance can be selected as the final three-class classification segmentation model.

[0043] Step S4: Reasoning and Post-processing like Figure 4 As shown, in the inference stage, this embodiment uses a trained downstream adaptation segmentation model to segment and post-process fisheye images of arbitrary resolution.

[0044] For input fisheye image Perform deterministic scaling and normalization: ; in For fixed interpolation and normalization operators (no random enhancement required).

[0045] Using the subordinate partitioning model Calculate the three-class classification probability and the boundary probability: ; And generate the initial semantic label mask by taking the category with the highest probability: ; For regions predicted as "sky", all white connected components are extracted using 8-neighbor connectivity analysis. For each connected component, its reliability is determined based on an area threshold, whether it reaches the image boundary, and the statistical analysis of surrounding pixel categories. For internal island regions determined to be unreliable, the pixel labels within them are rewritten from "sky" to "buildings," and small-scale dilation-fill operations can be used to smooth the boundaries.

[0046] For regions predicted as "leaves" and their unilateral expansion bands along the sky direction, HSV or other color space features of the original color image are extracted. A brightness-saturation joint threshold criterion is designed to rewrite pixels satisfying the "approximate sky" color mode from "leaves" to "sky". Subsequently, morphological opening and closing operations and small connected component filtering are performed on the boundary region between leaves and sky to remove isolated noise pixels. The above processing yields a final three-class mask that is topologically connected, has smooth boundaries, and better fits the real scene. The final output is as follows Figure 6 The classification prediction results are shown.

[0047] like Figure 7 As shown, the model in this example significantly outperforms existing methods in key category recognition. For the "sky" and "leaves" categories, which directly determine satellite LOS / NLOS classification, the IoU-Sky and IoU-Tree metrics of this method reach 0.9898 and 0.9023 respectively, both significantly higher than those of U-Net, DeepLab-Pro, PSONet+OCR, FPN feature pyramid network, and SegFormer. Simultaneously, the F1-Sky metric reaches 0.8000, demonstrating greater stability in terms of sky region integrity and boundary coherence. In terms of overall performance, this method achieves an mIoU of 0.9300 and a pixel accuracy of 0.9508 with only about 3.72M trainable parameters. Compared to DeepLab-Pro+ and SegFormer, which have 29.81M and 13.23M parameters respectively, the model size is reduced by approximately 3–8 times, while achieving higher or comparable accuracy in key categories, balancing segmentation quality and model complexity. Combined with subsequent boundary refinement and geometric projection, the model proposed in this application has outstanding comprehensive advantages in extracting priors of "sky / leaves / buildings" in fisheye images and supporting satellite LOS / NLOS discrimination, making it suitable for deployment in resource-constrained vehicle terminals or small GNSS receivers.

[0048] Step S5: Satellite projection and LOS / NLOS discrimination like Figure 8 As shown, this embodiment combines the three-class mask with the satellite orientation geometry for satellite-by-satellite LOS / NLOS discrimination.

[0049] Let the azimuth and elevation angles of the i-th satellite in the receiver coordinate system at a certain epoch be respectively... Its unit direction vector can be written as: ; superscript This indicates the coordinate system of the "receiver".

[0050] Given the external parametric matrix of the camera-antenna rigid body Under the premise that the general space vector in receiver coordinates Vector in camera coordinate system satisfy: ; Therefore, the first The unit direction vector of the satellite in the camera coordinate system is: ; superscript This represents the "camera" coordinate system. Let the camera's optical axis be the coordinate system. The axis, the unit vector is The angle between the satellite's direction and the optical axis is: ; Its polar angle on the imaging plane is: ; in They are respectively exist Xianghe The component of the direction.

[0051] Under the equidistant fisheye model, the radius of the satellite on the ideal image plane is: ; in This is the equivalent focal length. Combined with the principal point position... Polar angle determined by azimuth angle The pixel coordinates of the satellite in the image can be obtained. Approximately: ; If necessary, it can be within the ideal radius Based on this, a radial distortion correction term is superimposed, and the radius is corrected by a higher-order polynomial to improve the projection accuracy.

[0052] In the final three-class mask Above, with The pixel category at that location is used as the occlusion criterion for this satellite: ; The rules stipulate that when c is in the category of "sky" or "leaves", the satellite is considered to be in the open field of view or visible through gaps in the tree canopy, and is marked as LOS; when c is "building" or is located outside the effective field of view of the image, the satellite is considered to be hard-obstructed by buildings or other structures, and is marked as NLOS.

[0053] Therefore, the discriminant function is defined as follows: ; It can be further used for the selection, weighting and quality control of GNSS observations, such as shielding NLOS pseudoranges and reducing the weight of low-confidence satellites, thereby improving CEP and 3D positioning errors in urban canyon environments.

[0054] Through the above steps, this embodiment realizes a complete link from fisheye image input, three-class semantic segmentation, boundary refinement, to satellite geometric projection and LOS / NLOS discrimination.

[0055] Example 3 The present invention also provides an NLOS satellite detection device, comprising: The first processing module is used to acquire urban scene images captured by the fisheye camera installed on the upper side, divide the images into labeled training sets and unlabeled / weakly labeled training sets containing three semantic categories: "sky / leaves / buildings", and perform size normalization and geometric and lighting enhancement on the images. The second processing module is used to construct the upstream guided segmentation main network based on the labeled training set, and output pixel-level three-class semantic probability maps and corresponding edge response maps; The third processing module is used to construct a collaborative segmentation sub-network containing two lightweight branches. Based on the soft probability output, multi-layer feature field, and contour response of the upstream guiding segmentation main network, a trained three-class fisheye segmentation model is obtained. The fourth processing module is used to input the fisheye image to be tested into the three-class fisheye segmentation model to obtain an optimized three-class mask; The fifth processing module is used to acquire the azimuth and elevation angles of each satellite. Based on the angle-radius mapping in the fisheye imaging model and the preset radial distortion parameters, the satellite orientation is transformed into image plane pixel coordinates. According to the category label of the pixel in the optimized three-class mask, satellites located in the "sky" or "leaf" area are identified as LOS, and satellites located in the "building" area or that are obscured are identified as NLOS, thus completing the NLOS satellite detection based on fisheye images.

[0056] As one embodiment of the present invention, the second processing module constructs an upstream guided segmentation main network, and uses a hierarchical self-attention feature extraction and stepwise recovery decoding structure to perform three-class semantic segmentation training on the labeled training set, and outputs pixel-level three-class semantic probability maps and corresponding edge response maps.

[0057] In one embodiment of the present invention, the third processing module constructs a collaborative segmentation sub-network comprising two lightweight branches. Each branch adopts a compact encoder-decoder structure and sets an explicit contour auxiliary head on the output side. On the labeled training set, the three-class classification output is constrained by a combined loss based on pixel category log-likelihood and region overlap, and the contour auxiliary head is constrained by the contour ground truth derived from geometric labels. On the unlabeled / weakly labeled training set, a bidirectional pseudo-label consistency constraint is introduced between the two branches, and the output alignment, feature alignment, and contour alignment constraints are applied to the collaborative segmentation sub-network by combining the soft probability output, multi-layer feature field, and contour response from the upstream guiding main network. The network parameters are updated under pixel-level confidence mask filtering to obtain the trained three-class fisheye segmentation model.

[0058] As one embodiment of the present invention, the fourth processing module inputs the fisheye image to be tested into the three-class fisheye segmentation model to obtain a pixel-level semantic mask and contour response map of "sky / leaves / buildings". Then, the semantic mask is sequentially processed by suppressing isolated bright areas inside buildings and refining leaf areas based on color cues to obtain an optimized three-class mask.

[0059] Example 4 The present invention also provides an NLOS satellite detection system, comprising: a memory and a processor, wherein the memory stores a computer program executed by the processor, and the computer program executes an NLOS satellite detection method when run by the processor.

[0060] Example 5 The present invention also provides a storage medium storing a computer program that executes the NLOS satellite detection method when running.

[0061] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. An NLOS satellite detection method, characterized in that, include: Step 1: Obtain urban scene images captured by the fisheye camera installed on the upper side, divide the images into labeled training sets and unlabeled / weakly labeled training sets containing three semantic categories: "sky / leaves / buildings", and perform size normalization and geometric and lighting enhancement on the images. Step 2: Based on the labeled training set, construct the upstream guided segmentation main network and output pixel-level three-class semantic probability maps and corresponding edge response maps; Step 3: Construct a collaborative segmentation subnetwork containing two lightweight branches. Based on the soft probability output, multi-layer feature field, and contour response of the upstream guiding segmentation main network, obtain the trained three-class fisheye segmentation model. Step 4: Input the fisheye image to be tested into the three-class fisheye segmentation model to obtain the optimized three-class mask; Step 5: Obtain the azimuth and elevation angles of each satellite. Based on the angle-radius mapping in the fisheye imaging model and the preset radial distortion parameters, transform the satellite orientation into image plane pixel coordinates. Based on the category label of the pixel in the optimized three-class mask, satellites located in the "sky" or "leaf" area are identified as LOS, and satellites located in the "building" area or that are occluded are identified as NLOS, thus completing the NLOS satellite detection based on fisheye images.

2. The NLOS satellite detection method as described in claim 1, characterized in that, In step 2, an upstream guided segmentation main network is constructed. Using a hierarchical self-attention feature extraction and stepwise recovery decoding structure, the labeled training set is trained for three-class semantic segmentation, and pixel-level three-class semantic probability maps and corresponding edge response maps are output.

3. The NLOS satellite detection method as described in claim 2, characterized in that, In step 3, a collaborative segmentation subnetwork with two lightweight branches is constructed. Each branch adopts a compact encoder-decoder structure and an explicit contour auxiliary head is set on the output side. On the labeled training set, the three-class classification output is constrained by a combined loss based on pixel class log likelihood and region overlap, and the contour auxiliary head is constrained by the contour ground truth derived from geometric labels. On the unlabeled / weakly labeled training set, a bidirectional pseudo-label consistency constraint is introduced between the two branches. Combined with the soft probability output, multi-layer feature field and contour response from the upstream guiding main network, output alignment, feature alignment and contour alignment constraints are applied to the collaborative segmentation sub-network. The network parameters are updated under pixel-level confidence mask filtering to obtain the trained three-class fisheye segmentation model.

4. The NLOS satellite detection method as described in claim 1, characterized in that, In step 4, the fisheye image to be tested is input into the three-class fisheye segmentation model to obtain the pixel-level semantic mask and contour response map of "sky / leaves / building". The semantic mask is then subjected to post-processing of suppressing isolated bright areas inside buildings and refining leaf areas based on color cues to obtain the optimized three-class mask.

5. An NLOS satellite detection device, characterized in that, include: The first processing module is used to acquire urban scene images captured by the fisheye camera installed on the upper side, divide the images into labeled training sets and unlabeled / weakly labeled training sets containing three semantic categories: "sky / leaves / buildings", and perform size normalization and geometric and lighting enhancement on the images. The second processing module is used to construct the upstream guided segmentation main network based on the labeled training set, and output pixel-level three-class semantic probability maps and corresponding edge response maps; The third processing module is used to construct a collaborative segmentation sub-network containing two lightweight branches. Based on the soft probability output, multi-layer feature field, and contour response of the upstream guiding segmentation main network, a trained three-class fisheye segmentation model is obtained. The fourth processing module is used to input the fisheye image to be tested into the three-class fisheye segmentation model to obtain an optimized three-class mask; The fifth processing module is used to acquire the azimuth and elevation angles of each satellite. Based on the angle-radius mapping in the fisheye imaging model and the preset radial distortion parameters, the satellite orientation is transformed into image plane pixel coordinates. According to the category label of the pixel in the optimized three-class mask, satellites located in the "sky" or "leaf" area are identified as LOS, and satellites located in the "building" area or that are occluded are identified as NLOS, thus completing the NLOS satellite detection based on fisheye images.

6. The NLOS satellite detection device as described in claim 5, characterized in that, The second processing module constructs an upstream guided segmentation main network, and uses a hierarchical self-attention feature extraction and stepwise recovery decoding structure to perform three-class semantic segmentation training on the labeled training set, outputting pixel-level three-class semantic probability maps and corresponding edge response maps.

7. The NLOS satellite detection device method as described in claim 6, characterized in that, The third processing module constructs a collaborative segmentation subnetwork with two lightweight branches. Each branch adopts a compact encoder-decoder structure and sets an explicit contour auxiliary head on the output side. On the labeled training set, the three-class classification output is constrained by a combined loss based on pixel category log likelihood and region overlap, and the contour auxiliary head is constrained by the contour ground truth derived from geometric labels. On the unlabeled / weakly labeled training set, a bidirectional pseudo-label consistency constraint is introduced between the two branches. Combined with the soft probability output, multi-layer feature field and contour response from the upstream guiding main network, output alignment, feature alignment and contour alignment constraints are applied to the collaborative segmentation sub-network. The network parameters are updated under pixel-level confidence mask filtering to obtain the trained three-class fisheye segmentation model.

8. The NLOS satellite detection device as described in claim 7, characterized in that, The fourth processing module inputs the fisheye image to be tested into the three-class fisheye segmentation model to obtain the pixel-level semantic mask and contour response map of "sky / leaves / building". The semantic mask is then subjected to post-processing of suppressing isolated bright areas inside buildings and refining leaf areas based on color cues to obtain the optimized three-class mask.

9. An NLOS satellite detection system, characterized in that, include: A memory and a processor, wherein the memory stores a computer program executed by the processor, the computer program performing the NLOS satellite detection method as described in any one of claims 1-4 when executed by the processor.

10. A storage medium, characterized in that, The storage medium stores a computer program that, when executed, performs the NLOS satellite detection method as described in any one of claims 1-4.

Images