An image processing method based on efficient and fused panoramic glass precision network
By using a high-efficiency and fused panoramic glass segmentation network, the problems of accuracy and hardware cost in glass segmentation in panoramic images are solved, achieving efficient and accurate glass segmentation, which is suitable for autonomous driving and depth perception systems.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- WUYI UNIV
- Filing Date
- 2026-01-29
- Publication Date
- 2026-06-02
AI Technical Summary
Existing deep learning methods suffer from reduced accuracy and high hardware costs when processing glass segmentation in panoramic images, making it difficult to achieve efficient and accurate glass segmentation.
A high-efficiency parallel-convolutional fusion panoramic glass segmentation network is adopted. Through a lightweight network architecture, a feature extraction network, a parallel convolutional fusion depth-separable module, a convolutional dual-branch fusion tuning module, and a parallel-convolutional cumulative fusion fine tuning module are used to process the transparency, reflectivity, and geometric distortion of the glass, thereby generating a high-precision glass segmentation mask.
It significantly improves the glass segmentation accuracy in complex scenes, reduces the number of parameters and computational overhead, lowers hardware costs, and achieves accurate segmentation of single panoramic RGB images.
Smart Images
Figure CN122135025A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of computer vision and deep learning technology, and in particular to a semantic segmentation method for glass objects in panoramic images, specifically an image processing method based on an efficient and fused panoramic glass segmentation network. Background Technology
[0002] Accurate segmentation of glass objects in the environment is a key step in improving the performance of vision systems such as autonomous driving and depth perception. However, most current mainstream deep learning segmentation methods are based on traditional perspective images, which have a limited field of view and are inadequate for handling glass targets of varying scales and distances in open scenes.
[0003] While panoramic imaging can provide a comprehensive global environmental perception without blind spots, glass areas in panoramic images face significant challenges: First, glass has excellent light transmission capabilities, and its appearance is highly unstable, fluctuating with changes in the observation angle and environment; second, reflections on the glass surface can distort or even replicate background information, leading to visual feature confusion; finally, the drastic geometric deformations produced by panoramic imaging (such as rectangular projection distortion) interact with the physical properties of the glass itself, constituting an extremely complex visual analysis problem.
[0004] Existing glass segmentation methods (such as TransLab and Trans2Seg) largely rely on reflection and boundary cues, resulting in a significant drop in accuracy when dealing with severe distortions in panoramic images. Furthermore, while some multimodal methods (such as RGB-T and RGB-P) incorporate thermal infrared or polarization information, they increase hardware costs and computational burden, making large-scale deployment in practical applications difficult. Therefore, achieving efficient and accurate glass segmentation using only a single panoramic RGB image is a pressing technical challenge. Summary of the Invention
[0005] To address the shortcomings of existing technologies, this invention proposes an image processing method based on a high-efficiency and fused panoramic glass precision network, aiming to solve the problems of transparency, reflectivity, and geometric distortion of glass in panoramic images through a lightweight network architecture.
[0006] The technical solution of this invention is: an image processing method based on a high-efficiency parallel-fusion panoramic glass precision network, characterized in that the method includes the following steps: S1) Obtain the panoramic RGB image to be processed, and use a feature extraction network to extract multi-level feature maps of the panoramic RGB image; S2) The multi-level feature map is enhanced with transparency features using a high-efficiency parallel convolution depth separable module. Edge cues of the transparent medium are extracted through multi-scale parallel perception and depth separable convolution, and the first enhanced feature map is output. S3) The first enhanced feature map is subjected to reflection interference suppression using a high-efficiency roll-to-roll dual-branch fusion module, and the dynamic enhancement of features is achieved through a dual attention guidance mechanism, and the second enhanced feature map is output. S4) Use the efficient parallel-conversion fusion fine-tuning module to perform geometric distortion correction on the second enhanced feature map, and use the learnable resampling characteristics of transposed convolution to establish a mapping from the distorted feature space to the corrected feature space, and output the third enhanced feature map. S5) Perform multi-scale fusion on the third enhanced feature map to generate the glass segmentation mask corresponding to the panoramic RGB image.
[0007] In the above method, the feature extraction network is a lightweight backbone network (PoolFormer) based on pooling operations, which is used to extract multi-level semantic features while reducing the number of parameters.
[0008] In the above method, the processing of the efficient parallel convolution depth-separable module in step S2) includes: The input feature map is fed into four parallel 3×3 convolutional branches for multi-scale feature sampling; The outputs of the four branches are concatenated along the channel dimension and batch normalized to reduce feature variance. We use 3×3 depthwise separable convolutions to refine the depth dimension information of the concatenated features; Channel integration and dimensionality reduction are performed using a 1×1 convolutional layer to output the first enhanced feature map.
[0009] In the above method, the processing procedure of the high-efficiency roll-to-roll dual-branch fusion modulation module in step S3) includes: A 1×1 transposed convolution is used to adjust the spatial dimension of the input features to provide spatial context information; The adjusted features are divided into upper branches and lower branches; Among them, the upper branch generates channel attention weights through global average pooling and continuous 1×1 convolutions to suppress reflection noise; The lower branch generates spatially guided weights through 1×1 convolutions to enhance the features of the real glass region; The upper branch output and the lower branch output are multiplied element-wise and then smoothed through a 3×3 convolutional layer to output the second enhanced feature map.
[0010] In the above method, the processing steps of the efficient parallel transfer cumulative touchdown fine-tuning module in step S4) include: The input feature map is fed into four parallel branches, where the first branch is the identity mapping branch, and the second, third and fourth branches are transposed convolution branches with different sampling perspectives. The outputs of the four branches are accumulated step by step through cascaded addition nodes to integrate geometric correction information from different transformation perspectives; The accumulated features are then processed by channel splicing and non-linear activation. The final feature mapping is performed through a 1×1 convolutional layer, outputting the third enhanced feature map.
[0011] In the above method, in step S5), the fusion process adopts a cascaded fusion strategy and combines empty convolution operation to expand the receptive field, thereby integrating the global contextual information of the panoramic image.
[0012] The above method further includes: using the cross-entropy loss function to train the efficient and fused panoramic glass precision network end-to-end.
[0013] The beneficial effects of this invention are as follows: 1. Highly targeted: By using three specially designed modules to address the transparency, reflectivity, and panoramic distortion of glass respectively, the segmentation accuracy in complex scenes is significantly improved. 2. High efficiency and lightweight: It adopts depthwise separable convolution and lightweight backbone network, which greatly reduces the number of parameters and computational overhead while maintaining high accuracy, making it easy to deploy on embedded devices. 3. No auxiliary modality required: Accurate segmentation can be achieved with only a single panoramic RGB image, reducing hardware costs and data acquisition difficulties. Attached Figure Description
[0014] Figure 1 This is a schematic diagram of the overall architecture of an image processing method based on a high-efficiency and fused panoramic glass precision network provided by the present invention; Figure 2 This is a schematic diagram of the structure of the efficient parallel convolution depth separable module in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of the high-efficiency roll-to-roll dual-branch fusion modulation module in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of the high-efficiency parallel-to-cumulative fine-tuning module in an embodiment of the present invention. Detailed Implementation
[0015] The specific embodiments of the present invention will now be further described in conjunction with the technical solution:
[0016] The present invention proposes an image processing method based on a high-efficiency fused panoramic glass precision network, the core of which lies in reprocessing the features extracted by the backbone network through three cascaded feature enhancement modules.
[0017] Example 1: Feature Extraction and Transparency Enhancement (S1-S2) First, acquire a panoramic RGB image (e.g., 2048×512 resolution). Extract multi-level features using the PoolFormer backbone network. In stage S2, to address the issues of large feature variance and weak edges caused by the transparency of glass, the feature map is fed into a highly efficient parallel convolutional depthwise separable module. This module captures edge cues at different scales through four parallel 3×3 convolutional branches, and then refines the features with extremely low computational cost using depthwise separable convolutions, outputting the first enhanced feature map. This step effectively distinguishes transparent glass from a completely empty background.
[0018] Example 2: Suppression of reflection interference (S3) The first enhanced feature map is input into an efficient transconvolutional dual-branch fusion module. Since glass surfaces often have complex reflective textures, this module utilizes 1×1 transposed convolution to expand the spatial context and employs a dual attention mechanism for adjustment. The channel attention weights generated by the upper branch identify and suppress reflective noise features, while the spatial guidance weights generated by the lower branch focus on the actual glass region. After fusion of the two branches, a second enhanced feature map is output, significantly improving the model's robustness in highly reflective environments.
[0019] Example 3: Geometric Distortion Correction and Fusion (S4-S5) The second enhanced feature map is input into an efficient and cumulative fusion fine-tuning module. Addressing the severe distortion near the poles in panoramic images, this module utilizes three parallel transposed convolutional branches to learn non-uniform spatial redistribution operations, simulating the inverse transformation of panoramic projection. By progressively accumulating correction features from different viewpoints, geometric fine-tuning of the distorted region is achieved. Finally, the corrected feature maps are integrated at multiple scales using a cascaded fusion strategy, and the receptive field is expanded using empty convolution, ultimately outputting a high-precision glass segmentation mask.
[0020] Experimental data show that the proposed method achieves an IoU of 91.37% and a MAE as low as 0.0060 on the PanoGlass V2 dataset, significantly outperforming existing semantic segmentation and glass segmentation models, thus verifying the effectiveness of the invention.
[0021] The embodiments of the present invention disclosed above are merely illustrative of the invention. These embodiments do not exhaustively describe all details, nor do they limit the invention to any specific implementation. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the scope of the present invention should be included within the protection scope of the present invention.
Claims
1. An image processing method based on a high-efficiency, fused panoramic glass precision network, characterized in that, The method includes the following steps: S1) Obtain the panoramic RGB image to be processed, and use a feature extraction network to extract multi-level feature maps of the panoramic RGB image; S2) The multi-level feature map is enhanced with transparency features using a high-efficiency parallel convolution depth separable module. Edge cues of the transparent medium are extracted through multi-scale parallel perception and depth separable convolution, and the first enhanced feature map is output. S3) The multi-level feature map is subjected to reflection interference suppression using a high-efficiency roll-to-roll dual-branch fusion module, and the feature is dynamically enhanced through a dual attention guidance mechanism, and a second enhanced feature map is output. S4) The multi-level feature map is geometrically distorted by using the efficient parallel-conversion fusion fine-tuning module, and a mapping from the distorted feature space to the corrected feature space is established by using the learnable resampling characteristics of transposed convolution, and the third enhanced feature map is output. S5) Perform multi-scale fusion on the first enhanced feature map, the second enhanced feature map and the third enhanced feature map to generate the glass segmentation mask corresponding to the panoramic RGB image.
2. The image processing method based on a high-efficiency parallel-fusion panoramic glass precision network according to claim 1, characterized in that, The processing steps of the efficient parallel convolution depth-separable modules in step S2) include: The input feature map is fed into four parallel 3×3 convolutional branches for multi-scale feature sampling; The outputs of the four branches are concatenated along the channel dimension and batch normalized to reduce feature variance. We use 3×3 depthwise separable convolutions to refine the depth dimension information of the concatenated features; Channel integration and dimensionality reduction are performed using a 1×1 convolutional layer to output the first enhanced feature map.
3. The image processing method based on a high-efficiency parallel-fusion panoramic glass precision network according to claim 1, characterized in that, The processing steps of the high-efficiency roll-to-roll dual-branch fusion module in step S3) include: A 1×1 transposed convolution is used to adjust the spatial dimension of the input features to provide spatial context information; The adjusted features are divided into upper branches and lower branches; Among them, the upper branch generates channel attention weights through global average pooling and continuous 1×1 convolutions to suppress reflection noise; The lower branch generates spatially guided weights through 1×1 convolutions to enhance the features of the real glass region; The upper branch output and the lower branch output are multiplied element-wise and then smoothed through a 3×3 convolutional layer to output the second enhanced feature map.
4. The image processing method based on a high-efficiency parallel-fusion panoramic glass precision network according to claim 1, characterized in that, The processing steps of the high-efficiency parallel transfer cumulative fusion fine-tuning module in step S4) include: The input feature map is fed into four parallel branches, where the first branch is the identity mapping branch, and the second, third and fourth branches are transposed convolution branches with different sampling perspectives. The outputs of the four branches are accumulated step by step through cascaded addition nodes to integrate geometric correction information from different transformation perspectives; The accumulated features are then processed by channel splicing and non-linear activation. The final feature mapping is performed through a 1×1 convolutional layer, outputting the third enhanced feature map.
5. The image processing method based on a high-efficiency parallel-fusion panoramic glass precision network according to claim 1, characterized in that, The feature extraction network is a lightweight backbone network based on pooling operations, used to extract multi-level semantic features while reducing the number of parameters.
6. The image processing method based on a high-efficiency parallel-fusion panoramic glass precision network according to claim 1, characterized in that, In step S5), the fusion process adopts a cascaded fusion strategy and combines empty convolution operation to expand the receptive field, thereby integrating the global contextual information of the panoramic image.
7. The image processing method based on a high-efficiency parallel-fusion panoramic glass precision network according to claim 1, characterized in that, The method further includes: using the cross-entropy loss function to train the efficient and fused panoramic glass precision network end-to-end.