Rail image illumination normalization preprocessing method and system
By embedding a differentiable physical renderer into a deep neural network, the problem of unstable rail imaging under complex outdoor lighting conditions is decoupled from the essential features, thus improving the robustness of the machine vision inspection system and the downstream inspection performance.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- KUNMING UNIVERSITY
- Filing Date
- 2026-03-20
- Publication Date
- 2026-06-05
AI Technical Summary
Existing technologies suffer from drastic changes in the appearance of rail images under complex and dynamic outdoor lighting conditions, leading to unstable performance and poor reliability of machine vision inspection systems. Traditional methods cannot effectively distinguish between changes in lighting and material, while deep learning methods lack physical constraints, resulting in insufficient generalization ability.
A differentiable physical renderer is constructed and embedded in a deep neural network. By decoupling lighting from essential features through a physical reflection model, a normalized image under standard lighting is generated. Weakly supervised training is performed using lighting invariance consistency loss to reduce data dependence.
This achieves precise separation between illumination and essential features, improves the robustness and reliability of the detection system, reduces data acquisition costs, and enhances the performance of downstream detection algorithms.
Smart Images
Figure CN122156033A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the interdisciplinary field of computer vision, image processing, and intelligent operation and maintenance of rail transit. Specifically, it relates to a method and system for preprocessing rail surface images with illumination normalization, integrating a physical reflection model and a deep learning framework. This aims to address the problem of drastic changes in the appearance of rail images under complex and dynamic outdoor sunlight conditions. By physically modeling the reflection characteristics of the rail surface and embedding a differentiable physical rendering process as a strong constraint into a neural network, this invention can decouple illumination from essential features in a single image, generating a standardized image with constant illumination conditions and faithful defect information. This provides a stable and robust input for downstream defect detection algorithms, significantly improving the reliability and accuracy of the system in real outdoor scenarios. Background Technology
[0002] Automatic detection technology for rail surface defects based on machine vision is a key means to realize intelligent railway operation and maintenance and ensure transportation safety. However, in actual outdoor deployment, this technology is severely affected by the dynamic changes in natural lighting. Changes in the sun's position, intensity, weather, and time cause irregular changes in the brightness, contrast, color, and texture of the rail surface image. These appearance fluctuations caused by lighting are closely coupled with changes in the physical state of the rail itself (such as defects and oxidation) in the image, seriously interfering with the detection model that relies on feature consistency, resulting in unstable performance and poor generalization ability.
[0003] Currently, there are two main types of methods for processing image illumination changes, but both have significant shortcomings in the specific scenario of rail inspection: The first category consists of traditional image enhancement methods, such as histogram equalization, homomorphic filtering, and the Retinex series of algorithms. These methods improve visual effects by manipulating global or local statistical properties of the image, but lack modeling of the imaging physics (especially the reflection mechanism of light interacting with the surface). When processing rail images, they cannot distinguish whether pixel variations originate from illumination or material / defects. Therefore, while suppressing uneven illumination, they easily damage the real defect edges and textures, introducing noise or artifacts, resulting in limited and uncontrollable effects.
[0004] The second category consists of deep learning-based image transfer methods, such as generative adversarial networks and their variants. These methods attempt to achieve lighting style transfer through data-driven approaches, but still face three major challenges: First, supervised learning requires a large number of strictly paired cross-lighting images of the same scene, which is almost impossible to obtain in outdoor steel rail scenes; second, the models are mostly black boxes, failing to incorporate prior knowledge of the physical reflection of steel rails as metal surfaces (such as specular highlights and Fresnel effects), leading to physically unreasonable outputs under extreme conditions such as strong light and shadow; third, without strong physical and content constraints, they are prone to distorting image content to adapt to style, resulting in distortion of defective features.
[0005] Existing technologies either compromise essential features due to a lack of physical guidance or suffer from insufficient generalization ability due to the absence of data and physical constraints. Therefore, there is an urgent need for a rail image preprocessing method that can integrate physical priors, reduce data dependence, and achieve precise separation of illumination and essential features, thereby fundamentally improving the robustness of visual inspection systems in complex lighting environments. Summary of the Invention
[0006] The purpose of this invention is to address the aforementioned problems by providing a method and system for normalizing the illumination of rail images. This solves the fundamental problem of unstable performance and poor reliability in machine vision-based rail defect detection systems operating under complex outdoor sunlight conditions due to drastic changes in the illumination conditions of the input image. This application constructs a differentiable forward physically-based rendering process and embeds it as a physical bottleneck layer into a deep neural network. This guides the network to decouple the illumination changes from the essential properties of the rail surface in a single image and reconstruct a normalized image under a standard illumination reference domain.
[0007] The technical solution of the present invention is as follows: A method for preprocessing rail images with illumination normalization includes the following steps: Acquire images of the rail surface to be processed, collected under complex outdoor sunlight conditions; The rail surface image is input into a pre-trained feature extraction network to extract multi-scale depth features, and the physical material parameter vector of the rail surface and the spatial variation of the illumination distribution map are estimated in parallel based on the depth features. The physical material parameter vector and the illumination distribution map are input into a differentiable physical renderer built based on a micro-surface reflection model for forward rendering to generate a physically rendered image. Calculate the pixel-level residual between the rail surface image and the physically rendered image; By fusing multi-scale depth features with pixel-level residuals, an image reconstruction network generates an illumination-normalized image under a preset standard illumination reference domain.
[0008] Furthermore, the differentiable physical renderer is implemented based on a microsurface bidirectional reflection distribution function model, which is relevant to pixel positions. Rendering results The calculation is as follows: , , in, The diffuse reflectance coefficient is... These are the normal distribution function, Fresnel term, and geometric occlusion function, respectively. These are the surface normal, the viewing direction, and the lighting direction, respectively. For spatially varying light distribution, Ambient lighting; surface roughness and half-length vector Used for calculation , This is a preset small constant used to avoid division by zero errors in numerical calculations and to ensure numerical stability during the rendering process.
[0009] Furthermore, the normal distribution function Using the GGX model: , in, For surface roughness parameters, It is a half-range vector.
[0010] Furthermore, the feature extraction network, differentiable physical renderer, and image reconstruction network are jointly trained through the following steps: Construct a dataset containing unpaired images of the same rail scene acquired under different natural lighting conditions; Define the overall loss function:
[0011] in, For the loss of uniformity of illumination invariance, For physical prior loss, For reconstruction losses; , and These are the weighting coefficients; A phased training strategy is adopted. First, the feature extraction network and the reconstruction network are fixed, and the physical material parameters, illumination distribution estimation and differentiable physical renderer related parameters are trained. Then, all network parameters are defixed, and the overall loss function is used for end-to-end joint fine-tuning.
[0012] Furthermore, the loss of uniformity in illumination invariance. The calculation method is as follows: Select two images of the same scene with different lighting from the dataset. and The normalized result is obtained after network processing. and ,but: , in, It is a pre-trained, fixed-parameter deep convolutional neural network used to extract high-level semantic features of images.
[0013] Furthermore, the physical material parameters include at least the diffuse reflectance coefficient. Specular reflectance coefficient Surface roughness Furthermore, its value is constrained by physical prior loss and is within a reasonable range based on the actual material properties of the rail.
[0014] Furthermore, the illumination distribution map is an RGB three-channel two-dimensional image with the same resolution as the input image; the feature extraction network includes a shared encoder backbone and two parallel branches: a fully connected layer branch for regressing the physical material parameter vector, and an upsampled convolutional decoder branch for estimating the two-dimensional illumination map.
[0015] Furthermore, the image reconstruction network adopts a U-Net structure with skip connections, and its encoder part shares weights with the encoder of the feature extraction network; multi-scale depth features and size-adjusted pixel-level residuals are channel-stitched and fused at the corresponding level of the decoder; based on the fused features, the image reconstruction network outputs an illumination-normalized image with the same resolution as the input.
[0016] This application also includes a rail image illumination normalization preprocessing system, which implements a rail image illumination normalization preprocessing method, including: The image acquisition module is used to acquire images of the rail surface to be processed. The feature extraction and estimation module includes a pre-trained feature extraction network, which is used to extract the depth features of the rail surface image and estimate the physical material parameters and the illumination distribution information of the current scene based on the depth features. The physically rendered module includes a differentiable physically rendered renderer, which generates physically rendered images based on physical material parameters and lighting distribution information; The residual calculation module is used to calculate the residual between the rail surface image and the physically rendered image; The image reconstruction module includes a pre-trained image reconstruction network used to fuse depth features and residuals to generate an illumination-normalized image under a preset standard illumination reference domain.
[0017] This application also includes a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a rail image illumination normalization preprocessing method.
[0018] Compared with existing technologies, the advantages of this invention are: 1. Achieved precise information decoupling based on physical guidance. By embedding a differentiable physical renderer as a physical prior constraint into the network, this invention establishes a clear decomposition paradigm of "appearance = material × lighting". Guided by physical laws, the network learns and can more accurately decompose the input image into "the appearance changes determined by dynamic lighting" and "the essential invariant parts determined by the inherent properties of the rail itself". This effectively eliminates lighting interference such as highlights and shadows while preserving and even highlighting subtle geometric and textural features related to defects to the greatest extent possible. 2. Significantly reduces the stringent requirements for training data, improving practicality. This invention creatively utilizes the prior knowledge that "the content of the same physical scene should be consistent under different lighting conditions," and achieves effective training under conditions where there is absolutely no pixel-level paired data by designing an illumination invariance consistency loss. This weakly supervised / self-supervised training paradigm allows the model to be trained or fine-tuned using easily obtainable unpaired image sequences collected by inspection vehicles passing through the same road segment at different times, greatly reducing data collection costs and annotation difficulties, and improving the engineering practicality and deployability of the method; 3. It endows the model with physical interpretability and stronger generalization ability. The physical material parameters (such as roughness) and lighting maps estimated internally by the network have clear physical meaning, making the entire processing transparent and interpretable. The introduction of physical priors not only ensures the physical rationality of the output results (such as correctly handling metallic highlights), but more importantly, when the model faces extreme or special lighting conditions not present in the training data, it can perform reasonable extrapolation and interpolation based on the embedded physical model, demonstrating better generalization ability and scene robustness compared to purely data-driven models. 4. Significantly and directly improves the performance of downstream defect detection tasks. The rail images output after preprocessing by this invention have their illumination conditions normalized to a stable and consistent reference domain, fundamentally solving the performance fluctuation problem of detection algorithms caused by inconsistent input data distribution (domain shift). This provides a significantly improved input for any subsequent vision-based defect detection, classification, or segmentation algorithms. Experiments show that using images preprocessed by this invention significantly improves key indicators such as the mean accuracy (mAP) of downstream detection models, effectively reducing the false negative and false positive rates under complex lighting conditions, and improving the overall system reliability. Attached Figure Description
[0019] Figure 1 This is a schematic diagram of the overall process of the physical-guided rail image illumination normalization preprocessing method of this application.
[0020] Figure 2 This diagram illustrates the core architecture and data flow of the feature extraction network, the differentiable physical renderer, and the image reconstruction network.
[0021] Figure 3 This is a schematic diagram of the unpaired training dataset consisting of different sunlight images of the same rail section used for model training in this application, and the principle of its illumination invariance consistency loss.
[0022] Figure 4 This is a qualitative comparison diagram of the processing effects of this application and existing preprocessing methods on simulated and real complex sunlight-irradiated rail images.
[0023] Figure 5 This is a schematic diagram showing a quantitative comparison of the processing effects of this application and existing preprocessing methods on simulated and real complex sunlight-irradiated rail images. Detailed Implementation
[0024] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0025] The features and performance of the present invention will be further described in detail below with reference to embodiments.
[0026] Please see Figure 1-5 A rail image illumination normalization preprocessing method includes an offline training phase and an online inference phase: Offline training phase. This phase trains a holistic model (PhysNet) that includes a feature extraction network, a differentiable physical renderer, and an image reconstruction network. The specific steps include: Construct a training set. Collect multiple images of the same section of rail under different natural lighting conditions (such as different times of day, seasons, and weather), forming an unpaired image sequence. These images share the same scene content and essential properties of the rail, but have different lighting conditions.
[0027] Model architecture construction and initialization: Building the overall model (PhysNet). Its core sub-modules include: Feature extraction and estimation network: An encoder backbone network is used to extract multi-scale deep features. Based on these features, a physical material parameter vector (such as diffuse reflectance coefficient) is regressed in parallel through a fully connected layer. Mirror coefficient Roughness (etc.), and estimates the spatially varied two-dimensional illumination map through an upsampling convolution branch. .
[0028] Differentiable Physical Renderer: Based on a micro-surface bidirectional reflection distribution function model, it receives the aforementioned physical material parameters and lightmap as input, combines surface geometry information (such as normals), performs forward rendering calculations, and outputs a physically rendered image that should theoretically appear under the current lighting conditions. All operations in this renderer are differentiable.
[0029] Image Reconstruction Network: An encoder-decoder structure with skip connections for fusing multi-scale feature maps provided by the feature extraction network with the computed residual image. Finally, a normalized illumination image was reconstructed. .
[0030] Design a multi-task joint loss function. The loss function is crucial for guiding model learning and should include at least: Reconstruction losses ( Constraint Physical Rendering Image Compared with the original input image Similarity at the pixel level or perceptual level ensures that the physical model can effectively fit the current actual lighting effect.
[0031] Physical prior loss ( To prevent the network from converging to physically meaningless solutions, boundary constraints are imposed on the physical material parameters estimated by the network (e.g., limiting the diffuse reflection coefficient to a certain range).
[0032] Loss of uniformity in illumination invariance ( For two randomly selected images of the same rail scene with different lighting from the training dataset... and After overall network processing, the normalized output is obtained. and The loss is as close as possible to the high-level semantic feature space (e.g., features extracted via a pre-trained VGG network). This loss is central to enabling training on unpaired data, forcing the network to learn to strip away the changing lighting and extract and retain the essential content of the rails that is invariant to the lighting.
[0033] Joint model optimization: Using the training dataset and multi-task loss function, the parameters of the overall model are iteratively optimized end-to-end through the backpropagation algorithm until the model converges.
[0034] Online inference phase: The trained model is used to perform real-time normalization processing on the new rail images, specifically including the following steps: Image input. Obtain an image of the surface of an outdoor steel rail to be processed. .
[0035] Feature extraction and physical parameter estimation. Input the trained feature extraction network and simultaneously obtain its multi-scale deep features. Physical material parameter vector and spatial variation lighting diagram .
[0036] Physically-oriented forward rendering. and Input a trained differentiable physical renderer to generate the corresponding physically rendered image. .
[0037] Residual calculation. Calculate the pixel-level residual between the input image and the rendered image. The residual includes parts that the physical model cannot fully explain, typically including high-frequency texture details, sensor noise, and, most importantly, real-world defect information.
[0038] Image reconstruction and normalization output. Multi-scale features provided by the feature extraction network are then processed. With residual The images are fused together, input into the trained image reconstruction network, and finally generated a normalized image under a preset standard illumination reference domain (such as uniform diffuse light at noon on a cloudy day). This image can be directly and reliably input into subsequent defect detection algorithms.
[0039] This application also includes a rail image illumination normalization preprocessing system, comprising: Image acquisition module, used to capture or receive images of the rail surface; The data processing module integrates or runs an overall model trained in the offline training phase, which is used to execute the steps in the online inference phase and output a lighting-normalized image. The results output module is used to transmit the illumination-normalized image to a storage device or a subsequent defect detection and analysis module.
[0040] In another embodiment, this application discloses a complete training and inference scheme based on the fusion of surface physical models and deep networks, the core architecture of which is as follows: Figure 2 As shown, the overall process is as follows: Figure 1 As shown, Figure 1 A in the diagram represents the offline training phase (i.e., model learning). Figure 1 Figure B is a flowchart illustrating the online inference phase (i.e., model application). It includes: Training data collection. Along a fixed railway line, under the premise of ensuring that the camera pose (mounted on the inspection vehicle and precisely positioned) is fixed, multiple image acquisitions are conducted in different seasons, different weather conditions (sunny, cloudy, partly cloudy), and multiple typical time periods within a day (early morning, noon, evening). This results in multiple unpaired image sequences of the same physical section under different natural lighting conditions. This embodiment constructs a dataset including over 12,000 high-resolution rail surface images, covering a variety of complex and realistic lighting conditions such as low-angle strong light in the morning, overhead light and highlights at noon, backlight in the evening, uniform diffused light on cloudy days, and interspersed light spots under tree shadows.
[0041] Data preprocessing. To improve training efficiency and model stability, all images were uniformly scaled to a resolution of 512×512 pixels. Before inputting the images into the network, they were normalized based on the mean (0.485, 0.456, 0.406) and standard deviation (0.229, 0.224, 0.225) of the ImageNet dataset. To enhance the model's generalization ability, data augmentation was applied online during the training phase, including random color jitter (fine-tuning of brightness, contrast, and saturation), Gaussian noise addition, and small-amplitude random horizontal flipping.
[0042] Model Architecture Details: The overall model (PhysNet) consists of three core sub-modules: Feature extraction network (Encoder & Estimation Heads): Encoder backbone: A ResNet-34 pre-trained on ImageNet is used as the backbone network. The last global average pooling layer and fully connected layer are removed, and the first four stages of convolutional layers are retained as feature extractors, outputting multi-scale feature maps. .
[0043] Physical parameter estimation head: After the final feature map (16×16×512) output by the encoder, a global average pooling layer is applied to flatten it into a 512-dimensional vector, followed by two fully connected layers (256 and 8 neurons respectively). The final output is an 8-dimensional physical material parameter vector. . Specifically, this includes: three-dimensional diffuse albedo. (3D, corresponding to RGB channels), one-dimensional specular reflection coefficient (1D) Surface roughness (1D) and 2D anisotropic parameters and a two-dimensional residual vector (Used to capture unmodeled material properties).
[0044] Illumination estimation head: Employs a lightweight decoder architecture. The feature maps from the second stage (stage 2, 128×128×128) and fourth stage (stage 4, 16×16×512) of the encoder are upsampled and concatenated channel by channel. After passing through several convolutional layers and bilinear upsampling layers, the final output is an RGB three-channel spatial variation illumination map with the same resolution (512×512) as the input image. .
[0045] Differentiable physics renderer. To achieve physics guidance and ensure end-to-end trainability, this embodiment implements a differentiable renderer based on a microfacet reflection model (Micorofacet BRDF). Its core mathematical formula is as described above, using the GGX normal distribution function (GGX). Smith joint geometric occlusion function ( ) and Schlick approximate Fresnel term ( Surface normal information This is crucial for rendering. In this embodiment, a combination of data-driven and simplified prior methods is used: for the straight rail waist region, a fixed vertical upward normal is preset (e.g., Simultaneously, it allows the network to learn from features through a small, learnable fully connected layer. A globally small normal perturbation vector (3D) is predicted to simulate the subtle changes in the macroscopic curvature of the rail surface and the shooting angle. This approach avoids complex 3D modeling while maintaining physical plausibility. (Light direction) Then from the estimated illumination diagram In this process, a dominant direction is obtained by simply estimating the illumination intensity gradient of the image region, or simplified to a learnable global vector. The rendering process can be represented by the following mathematical formula: For each pixel position p in the image: Get input parameters: Physical material parameters ,in: Diffuse reflectance coefficient (RGB three channels); Specular reflection coefficient; Surface roughness parameter; Light distribution RGB light intensity varying in space; Surface normal : Normal direction of rail surface (unit vector); Observation direction : The direction from a surface point to the camera (unit vector); Light direction : Direction of the main light source (unit vector).
[0046] Calculate the intermediate vector: Half-length vectors , Calculate each component of the BRDF: Normal distribution function Using the GGX distribution: , Geometric occlusion function Using the Smith joint shadow occlusion function: ,in This is the Smith highly correlated occlusion function.
[0047] Fresnel term The Schlick approximation is used: .
[0048] Combination BRDF: .
[0049] Calculation and rendering results: .
[0050] The surface normal information can be obtained through one of the following methods: Pre-calculation method: Obtain a 3D model of the rail section through laser scanning or motion recovery structure technology, and render the normal map.
[0051] Learning method: The normal map is used as a learnable parameter and optimized together with the network during training.
[0052] Simplified approach: For straight rail sections, a fixed vertical upward normal is used as an approximation.
[0053] All computational operations in this renderer are designed to be differentiable, ensuring that gradients can be backpropagated through the rendering process to the physical material parameters and lighting distribution estimation network, achieving end-to-end joint optimization.
[0054] Image reconstruction network (Decoder). It employs a U-Net-like symmetric encoder-decoder structure. The encoder part shares ResNet-34 weights with the feature extraction network. The decoder part consists of multiple upsampling modules, each including a bilinear upsampling layer, a convolutional layer, a batch normalization layer, and a ReLU activation function. The key feature fusion mechanism is as follows: the multi-scale feature maps output from different stages of the feature extraction network (e.g., stage 2, stage 3, stage 4) are compared with the residual maps downsampled to the corresponding sizes. The data is stitched together along the channel dimension and then input into the corresponding level of the decoder. This design ensures that high-frequency residual details (including defect information) are effectively incorporated into the reconstruction process. The decoder ultimately outputs an RGB image with the same resolution as the input. .
[0055] Model training strategies: such as Figure 1 and Figure 3 As shown, the network is jointly trained using the following strategy: Training data organization: Images of the same road segment are collected and organized into sequences according to road segment. In each training iteration, a sequence is randomly selected, and two images with different lighting conditions are randomly drawn from that sequence as a training sample pair. ).
[0056] Optimization objective and loss function: Overall loss function It is a weighted sum of the three core losses:
[0057] In this embodiment, the following settings are provided: , , .
[0058] Reconstruction losses The image is rendered using a smooth L1 loss constraint. With input image Proximity: .
[0059] Physical prior loss We employ Huber loss with boundaries to constrain physical parameters within a reasonable range. For example: , , , .
[0060] Loss of uniformity due to illumination invariance For input pairs Normalized outputs are obtained through the network respectively. and We extracted features from its ReLU33 layers using a VGG-16 network pre-trained on ImageNet with its parameters frozen. Calculate the distance in the feature space: .
[0061] Training process details: Optimizer: Using the AdamW optimizer, the initial learning rate is... The weight decays to .
[0062] Training Strategy: A two-stage training method is employed. In the first stage (the first 50 rounds), the weights of the feature extraction encoder and image reconstruction decoder are fixed, and only the physical parameter estimation head, lighting estimation head, and renderer-related parameters are trained, enabling the physical model to quickly fit the basic lighting effects. In the second stage (the remaining 150 rounds), all parameters are unfrozen, and the complete physical model is used. End-to-end joint fine-tuning is performed to enable the network to learn to optimize feature decomposition and reconstruction under physical constraints.
[0063] Hardware and time consumption: The total training time was approximately 48 hours on a workstation equipped with two NVIDIA RTX 3090 GPUs.
[0064] Online Inference and Results: After the model training is complete, its online inference process is as follows: Figure 1 As shown in B.
[0065] The average processing time for a 512x512 image is approximately 65 milliseconds (RTX 3090 GPU). To verify the performance, comparative experiments were conducted on a separate test set (results are illustrated in the figure below). Figure 4 ): Qualitative results: such as Figure 4 As shown, the method of this invention can effectively eliminate heavy shadows when processing images with extreme backlighting, making the concealed transverse cracks clearly visible without introducing artifacts. While the CLAHE method improves overall contrast, noise in shadow areas is amplified, and defect edges are distorted. The CycleGAN method changes the image style, but the defect texture undergoes unrealistic changes.
[0066] Quantitative results: such as Figure 5 As shown in the table below, the key data is as follows.
[0067] Table 1 Quantitative Key Data Table
[0068] The complete method of this invention achieves the best results in both color fidelity (PSNR) and downstream defect detection performance (mAP). This demonstrates that the introduction of the physical model not only helps the network better understand the relationship between illumination and imaging (resulting in higher PSNR), but more importantly, guides the network to retain essential features crucial for defect detection, thereby improving the performance of the final application. While CycleGAN and its ablation version (Ours w / o Physics) can also reduce brightness differences, they lack physical constraints and therefore do not retain features as well as the complete method.
[0069] In another specific embodiment, this application discloses a lightweight version suitable for embedded devices, which makes lightweight improvements to the previous embodiment considering resource-constrained scenarios deployed in mobile inspection devices or vehicle-mounted edge computing units (such as Jetson AGX Xavier).
[0070] Model simplification: The backbone network is replaced with MobileNetV3-Small.
[0071] physical parameter vector The dimensions were compressed from 8 to 5. (3D) , ), removing minor parameters such as anisotropy.
[0072] The spatially varying lighting map is simplified into a 9-dimensional global lighting vector. It includes a 3D main light source direction, a 3D main light source intensity (RGB), and a 3D ambient light intensity. The lighting estimation head is correspondingly changed to a small fully connected layer.
[0073] Differentiable renderers are simplified to using this global illumination vector for rendering, which greatly reduces the amount of computation.
[0074] Knowledge distillation: The fully trained model from the previous embodiment is used as the "Teacher" network, and this lightweight model is used as the "Student" network. When training the Student network, in addition to the original... , , In addition to the losses, add a "characteristic distillation loss": , in, and These are feature maps of the intermediate layers of the decoder for both the student and teacher networks. This loss allows the student network to mimic the internal representation of the teacher network.
[0075] Results: The lightweight model has only 12% of the parameters of the full model (approximately 2.1M). On Jetson AGX Xavier, after acceleration with TensorRT, the inference speed reaches 28 FPS, meeting real-time requirements. On the same test set, the downstream detection model driven by it has an mAP of 0.786, only about 3.2% lower than the full model.
[0076] In another specific implementation, a complete workflow for integration into a rail intelligent inspection system is disclosed. This embodiment describes how to integrate the method of the present invention (preferably a lightweight version) into a practical "vehicle-ground-cloud" collaborative rail intelligent inspection system.
[0077] System Architecture: Front-end acquisition unit: Installed at the bottom of the integrated inspection train, it includes a high-speed linear / area array camera, a supplementary laser, and a high-precision GNSS / IMU combined positioning module.
[0078] Edge processing unit: a high-performance industrial control computer in a vehicle or an embedded AI computing box (such as Jetson AGX Xavier), which has the lightweight preprocessing model of this invention built in, and is responsible for real-time image illumination normalization.
[0079] Defect Analysis Unit: Can be located on an onboard unit or a ground server, running high-performance defect detection and classification models (such as YOLOv5 or Faster R-CNN variants), and receiving normalized images for analysis.
[0080] Data Management and Decision Center: A cloud server platform that aggregates all inspection data to perform defect review, trend analysis, health status assessment, and maintenance decision support.
[0081] End-to-end workflow: The inspection vehicle operates at a speed of 80-100 km / h, and the camera continuously acquires images of the rail surface in line scan or frame trigger mode, while the GNSS / IMU synchronously records the precise position and attitude.
[0082] The acquired raw images and their location timestamps are sent to the edge processing unit in real time. The preprocessing model within the unit completes the illumination normalization processing of a single image within 200 milliseconds.
[0083] The normalized image, along with its location information, is fed into the defect analysis unit. The detection model identifies the defect type (such as cracks or spalling), location (image coordinates), and size.
[0084] The defect report (including category, confidence level, and image coordinates) is bound to GNSS coordinates and transmitted back to the data management and decision-making center in real time via 5G or a dedicated wireless network, automatically generating alarms and updating the "health record" of the rails in that section.
[0085] System Value: By integrating the preprocessing module of this invention, the system achieves a leap from "raw perception" to "reliable cognition." Field tests show that during periods of strong sunlight at midday in summer and low illumination at dusk in winter, the overall defect detection rate (Recall) of the system is improved compared to the system without this preprocessing module, the false alarm rate is reduced, and the automation level and reliability of inspection operations are enhanced.
[0086] The embodiments described above merely illustrate specific implementation methods of this application, and while the descriptions are detailed and specific, they should not be construed as limiting the scope of protection of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the technical solution of this application, and these modifications and improvements all fall within the scope of protection of this application.
Claims
1. A method for preprocessing rail images with illumination normalization, characterized in that, Includes the following steps: Acquire images of the rail surface to be processed, collected under complex outdoor sunlight conditions; The rail surface image is input into a pre-trained feature extraction network to extract multi-scale depth features, and the physical material parameter vector of the rail surface and the spatial variation of the illumination distribution map are estimated in parallel based on the depth features. The physical material parameter vector and the illumination distribution map are input into a differentiable physical renderer built based on a micro-surface reflection model for forward rendering to generate a physically rendered image. Calculate the pixel-level residual between the rail surface image and the physically rendered image; By fusing multi-scale image features and pixel-level residuals, an illumination-normalized image under a preset standard illumination reference domain is generated through an image reconstruction network.
2. The rail image illumination normalization preprocessing method according to claim 1, characterized in that, The differentiable physical renderer is implemented based on a microsurface bidirectional reflection distribution function model, which is relevant to pixel positions. Rendering results The calculation is as follows: ， ， in, The diffuse reflectance coefficient is... These are the normal distribution function, Fresnel term, and geometric occlusion function, respectively. These are the surface normal, the viewing direction, and the lighting direction, respectively. For spatially varying light distribution, Ambient lighting; surface roughness and half-length vector Used for calculation , This is a preset small constant used to avoid division by zero errors in numerical calculations and to ensure numerical stability during the rendering process.
3. The rail image illumination normalization preprocessing method according to claim 2, characterized in that, Normal distribution function Using the GGX model: ， in, For surface roughness parameters, It is a half-range vector.
4. The rail image illumination normalization preprocessing method according to claim 1, characterized in that, The feature extraction network, differentiable physical renderer, and image reconstruction network are jointly trained through the following steps: Construct a dataset containing unpaired images of the same rail scene acquired under different natural lighting conditions; Define the overall loss function: in, For the loss of uniformity of illumination invariance, For physical prior loss, For reconstruction losses; , and These are the weighting coefficients; A phased training strategy is adopted. First, the feature extraction network and the image reconstruction network are fixed, and the relevant parameters of the physical material parameter estimation, illumination distribution estimation and differentiable physical rendering module are trained. Then, the fixed parameters of all networks are removed, and the overall loss function is used for end-to-end joint fine-tuning.
5. The rail image illumination normalization preprocessing method according to claim 4, characterized in that, Loss of uniformity due to illumination invariance The calculation method is as follows: Select two images of the same scene with different lighting from the dataset. and The normalized result is obtained after network processing. and ,but: ， in, It is a pre-trained, fixed-parameter deep convolutional neural network used to extract high-level semantic features of images.
6. The method for preprocessing rail image illumination normalization according to claim 1, characterized in that, The physical material parameters include at least the diffuse reflectance coefficient. Specular reflectance coefficient Surface roughness Furthermore, its value is constrained by physical prior loss and is within a reasonable range based on the actual material properties of the rail.
7. The rail image illumination normalization preprocessing method according to claim 1, characterized in that, The illumination distribution map is an RGB three-channel two-dimensional image with the same resolution as the input image; the feature extraction network contains a shared encoder backbone and two parallel branches: a fully connected layer branch for regressing the physical material parameter vector, and an upsampled convolutional decoder branch for estimating the two-dimensional illumination map.
8. The rail image illumination normalization preprocessing method according to claim 1, characterized in that, The image reconstruction network adopts a U-Net structure with skip connections, and its encoder part shares weights with the encoder of the feature extraction network; multi-scale image features and size-adjusted pixel-level residuals are channel-stitched and fused at the corresponding level of the decoder; based on the fused features, the image reconstruction network outputs an illumination-normalized image with the same resolution as the input.
9. A rail image illumination normalization preprocessing system, characterized in that, The method for preprocessing rail image illumination normalization as described in any one of claims 1-8 includes: The image acquisition module is used to acquire images of the rail surface to be processed. The feature extraction and estimation module includes a pre-trained feature extraction network, which is used to extract the depth features of the rail surface image and estimate the physical material parameters and the illumination distribution information of the current scene based on the depth features. The physically rendered module includes a differentiable physically rendered renderer, which generates physically rendered images based on physical material parameters and lighting distribution information; The residual calculation module is used to calculate the residual between the rail surface image and the physically rendered image; The image reconstruction module includes a pre-trained image reconstruction network used to fuse image features and residuals to generate an illumination-normalized image under a preset standard illumination reference domain.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements a rail image illumination normalization preprocessing method as described in any one of claims 1-8.