Carriage multi-region passenger flow detection method based on fisheye distortion adaptive correction

Abstract

The invention provides a compartment multi-region passenger flow detection method based on fisheye distortion adaptive correction, and solves the problem of feature scale inconsistency of different regions of a fisheye image through three core steps of construction of a continuous radial mapping function, distortion mode perception and feature reconstruction and feature separation enhancement based on density adaptation. According to the method, a continuous radial mapping function is adopted to replace traditional discrete region division, and self-adaptive receptive field design is achieved; a distortion mode learning and feature transformation parameter matrix is introduced to accurately adapt to actual lens distortion; through a density adaptive feature separation operator and a sparse calculation optimization strategy, the problem of feature confusion in a crowded environment is effectively solved, and the passenger flow detection precision of a compartment edge area is remarkably improved.

Description

A method for multi-zone passenger flow detection in train carriages based on adaptive fisheye distortion correction Technical Field

[0001] This invention relates to passenger flow detection, and more particularly to a multi-area passenger flow detection method for train carriages based on adaptive correction of fisheye distortion. Background Technology

[0002] With the rapid development of urban rail transit systems and the continuous increase in passenger flow, accurate passenger flow detection technology is of great significance for improving operational efficiency, optimizing passenger experience, and ensuring safe operation. Passenger flow detection not only provides data support for train scheduling and capacity allocation but also helps management departments monitor carriage congestion in real time and respond promptly to emergencies. Especially during peak hours, accurate passenger density analysis can effectively alleviate congestion, optimize platform management strategies, and improve overall service quality. Fisheye camera-based passenger flow detection technology, due to its wide-angle field of view and ease of installation, has become an ideal solution for passenger flow monitoring within carriages.

[0003] Currently, passenger flow detection technologies in train carriages mainly fall into two categories: traditional computer vision methods and deep learning methods. Traditional methods often employ techniques such as background subtraction, optical flow analysis, and feature point tracking to count passenger flow by detecting moving targets. For example, Bertozzi et al. proposed a pedestrian and vehicle detection and tracking algorithm based on fisheye images, which uses image segmentation and contour extraction to identify pedestrians. Deep learning methods mainly use object detection networks such as YOLO and Faster R-CNN for passenger identification. Zhang et al. proposed applying Tiny YOLO to detect passengers inside buses, while Li et al. explored using fisheye cameras for top-view passenger counting. For the fisheye distortion problem, a common solution is to perform distortion correction before detection, such as using partition fitting methods or multinomial models to preprocess the image. Some studies have also attempted to perform detection directly on the original distorted image, such as the AST (Ring Swing Transformer) method, which adapts to the radial distortion characteristics of fisheye images through a special window design.

[0004] However, existing technologies still face significant challenges in passenger flow detection using fisheye cameras in train carriages. The most prominent problem is the imbalance in detection accuracy caused by the inconsistency in feature scale across different regions of the fisheye image. Due to the radial distortion characteristics of fisheye lenses, passengers at the same distance exhibit drastically different scales and deformation characteristics in the central and peripheral regions of the image—passengers in the central region maintain a relatively normal proportion, while those in the peripheral regions show severe "stretching" and "compression" deformation. This prevents existing detection networks based on uniform feature extraction and scale processing from effectively handling passenger features in different regions simultaneously. Passenger flow detection accuracy in peripheral regions is significantly lower than in the central region, with a decrease of over 30% in crowded environments. Furthermore, most existing methods employ discrete region segmentation strategies for feature processing, which can easily lead to discontinuities at region boundaries, affecting the detection accuracy in transitional regions. Especially in crowded train carriage environments, the problems of feature confusion and occlusion among passengers are further exacerbated. Existing technologies lack adaptive feature enhancement mechanisms for different passenger flow densities, making it difficult to maintain stable detection performance in highly dynamic train carriage environments. Summary of the Invention

[0005] The purpose of this invention is to provide a multi-area passenger flow detection method for train carriages based on adaptive correction of fisheye distortion, in order to solve at least one technical problem existing in the prior art.

[0006] According to one aspect of this application, a method for multi-area passenger flow detection in a train carriage based on fisheye distortion adaptive correction includes: acquiring original images from a fisheye camera inside the carriage, preprocessing them, and generating a preprocessed image; extracting features from the preprocessed image using a pre-constructed radial mapping function to form a preliminary feature map; transforming the preliminary feature map using a pre-constructed feature transformation parameter matrix to form a distortion-adaptive feature representation; extracting the passenger flow density distribution in each area of ​​the carriage from the preprocessed image, and optimizing the distortion-adaptive feature representation based on it to obtain an enhanced feature representation; and performing passenger flow target detection and precise positioning accordingly to generate a passenger flow detection result containing passenger location information.

[0007] Beneficial effects: effectively solves the problem of feature confusion in crowded environments and significantly improves the accuracy of passenger flow detection in the edge areas of carriages. Attached Figure Description

[0008] Figure 1 is a flowchart of the present invention.

[0009] Figure 2 is a flowchart of the process of generating a preliminary feature map according to the present invention.

[0010] Figure 3 is a flowchart of the construction of the continuous radial mapping function in this invention.

[0011] Figure 4 is a flowchart of the process for forming the enhanced flow feature map according to the present invention. Detailed Implementation

[0012] According to another aspect of this application, a method for multi-area passenger flow detection in a train carriage based on fisheye distortion adaptive correction includes: S1. Acquiring an original image I_orig from a fisheye camera inside the carriage, and preprocessing the original image I_orig to obtain a preprocessed image I_pre; S2. Constructing a radial mapping function based on the optical characteristics of the fisheye camera, and using the radial mapping function to achieve continuous feature scale adaptive extraction on the preprocessed image I_pre to obtain a preliminary feature map F_init; S3. Based on the preprocessed image I_pre and the preliminary feature map F_init, performing distortion pattern perception and feature reconstruction to obtain a distortion adaptation feature map F_adapt; S4. Based on the distortion adaptation feature map F_adapt and the preprocessed image I_pre, performing density-adaptive feature separation enhancement to obtain an optimized feature map F_sparse; S5. Based on the optimized feature map F_sparse, performing multi-area passenger flow detection and localization to obtain a passenger flow detection result D_result; S6. Based on the passenger flow detection result D_result, performing passenger flow statistics and density analysis to obtain a passenger flow statistics report R_report.

[0013] As shown in Figures 1 to 4, according to another aspect of this application, a method for multi-area passenger flow detection in a train carriage based on adaptive fisheye distortion correction is also provided, comprising: S1. Acquiring images from a fisheye camera inside the carriage, and obtaining preprocessed image data through image enhancement and noise reduction processing; S11. Acquiring a real-time image sequence from the fisheye camera installed inside the carriage, and recording each frame as the original image I_orig, with an image resolution of W×H pixels; S12. Analyzing the noise characteristics of the original image I_orig, calculating the image noise level σ_noise, selecting the optimal noise reduction parameters based on σ_noise, and performing adaptive noise reduction processing using bilateral filtering to obtain a denoised image I_denoise; S13. Calculating the illumination distribution map L_map inside the carriage based on the denoised image I_denoise, and performing illumination compensation on the denoised image I_denoise using the contrast-limited adaptive histogram equalization (CLAHE) method to obtain an illumination-equalized image I_equal; S14. The illumination-equalized image I_equal is converted to a different color space while retaining both RGB and the converted HSV information. These two images are then combined to generate a multi-channel preprocessed image I_pre, which serves as the input for subsequent processing.

[0014] S2. Radial Mapping Function Construction and Adaptive Extraction of Continuous Feature Scale: A radial mapping function is constructed based on the optical characteristics of a fisheye camera to achieve adaptive extraction of continuous feature scale, resulting in a preliminary feature map; S21. Radial Mapping Function Parameter Initialization: Based on the intrinsic parameter matrix K_camera and distortion coefficients D_coeff of the fisheye camera, the image center point C(cx, cy) is determined, and a radial distance calculation model is established; S22. Continuous Radial Mapping Function Construction: For the specific environment of the carriage, a continuous radial mapping function R(d) is constructed, where d is the distance from the pixel point to the image center C, and R(d) satisfies: R(d) = α·d^γ + β·d + δ; where α, β, γ, and δ are parameters to be optimized. The optimal parameter values ​​are determined by analyzing the characteristics of the fisheye image of the carriage, generating a radial mapping parameter set R_params; S23. Adaptive receptive field design: Based on the radial mapping function R(d) and the radial mapping parameter set R_params, an adaptive receptive field deformable convolutional layer is designed. For a pixel at position (x,y), its receptive field size F(x,y) is: F(x,y) = F_base·(1 + λ·R(sqrt((x-cx)^2 + (y-cy)^2))); where F_base is the base receptive field size and λ is the receptive field adjustment coefficient, resulting in the receptive field distribution map F_map; S24. Continuous feature scale extraction: The deformable convolution guided by the receptive field distribution map F_map is used to extract features from the preprocessed image I_pre, generating a preliminary feature map F_init. This feature map maintains consistent spatial dimensions but can adapt to changes in feature scale in different regions.

[0015] S3. Distortion Pattern Perception and Feature Reconstruction: Learn the distortion pattern of an actual fisheye lens, generate a feature transformation parameter matrix, and perform spatial adaptive transformation on the preliminary feature map to obtain a distortion-adapted feature representation; S31. Distortion Pattern Estimation: Based on the preprocessed image I_pre and the radial mapping parameter set R_params, construct a distortion pattern estimation network, learn the actual distortion characteristics of the carriage fisheye camera, and obtain the distortion characteristic tensor D_tensor; S32. Generation of Feature Transformation Parameter Matrix: Based on the distortion characteristic tensor D_tensor, calculate the transformation parameter matrix M(i,j) for each position (i,j) in the feature space, where the formula for calculating the feature transformation parameter matrix M is: M(i,j) = f_transform(D_tensor, i, j, F_init); where f_transform is the feature transformation function, comprehensively considering distortion characteristics and local feature distribution, to generate the feature transformation parameter matrix set M_params; S33. Adaptive Feature Reconstruction: Apply the feature transformation parameter matrix set M_params to perform spatial adaptive transformation on the initial feature map F_init to achieve distortion correction in the feature space, resulting in the distortion-adapted feature map F_adapt; S34. Implementation of Bidirectional Feature Flow Mechanism: Establish a feature flow channel to allow feature information in the distortion-adapted feature map F_adapt to flow freely between different scales and regions, enhancing information exchange between regions. Through convolutional layers and feature convergence operations, feature sharing between different regions is achieved, resulting in the enhanced flow feature map F_flow.

[0016] S4. Density-Adaptive Feature Separation Enhancement: Evaluate passenger flow density in each area, dynamically adjust feature separation and enhancement strategies based on density distribution, achieve feature optimization under congested conditions, and obtain enhanced feature representations; S41. Fast Local Density Evaluation: Use a lightweight density evaluation network to process the preprocessed image I_pre to generate passenger flow density maps D_crowd for each area of ​​the carriage, where the density value ρ(i,j) at each location (i,j) represents the passenger flow density of that area; S42. Construction of Density-Adaptive Feature Separation Operator: Based on the passenger flow density map D_crowd, construct the density-adaptive feature separation operator D_op(ρ) = ω_1·ρ^2 + ω_2·ρ + ω_3; where ω_1, ω_2, and ω_3 are parameters, dynamically adjusted by analyzing the degree of feature confusion under different densities, generating a feature separation parameter set D_params; S43. High-density region feature enhancement: Apply the feature separation parameter set D_params to the enhanced flow feature map F_flow to enhance feature contrast and suppress background interference in high-density regions, resulting in a density-enhanced feature map F_density; S44. Feature channel attention mechanism implementation: Based on the passenger flow density map D_crowd, calculate the density-related channel attention weights A_channel, apply them to the density-enhanced feature map F_density, increase the weights of density-related channels, and obtain an attention-enhanced feature map F_attention; S45. Sparse computation optimization: Based on the passenger flow density map D_crowd, determine the high-density key regions R_key that require complete feature processing and the low-density regions R_simple that can be simplified, apply feature processing strategies of different complexities to different regions, and integrate the processing results to obtain an optimized feature map F_sparse.

[0017] S5. Multi-region passenger flow detection and localization: Passenger flow target detection and precise localization are performed based on enhanced feature representation, generating passenger flow detection results; S51. Multi-scale target candidate box generation: Based on the optimized feature map F_sparse, an improved region proposal network is used to generate a multi-scale target candidate box set B_proposals, each candidate box containing location, size, and confidence information; S52. Distortion-aware target representation optimization: Based on the candidate box position and distortion characteristic tensor D_tensor, distortion-aware shape adjustment is applied to each candidate box to generate optimized candidate boxes B_refined that better fit the actual distortion, including deformed rectangles, ellipses, or polygon representations; S53. Feature integration and target classification: Features corresponding to the optimized candidate box B_refined region in the optimized feature map F_sparse are extracted and integrated, and a classifier is used to determine whether each candidate box contains passengers, obtaining the target score map S_obj; S54. Non-maximum suppression and precise target localization: Apply the non-maximum suppression algorithm to the target score map S_obj and the optimized candidate box B_refined to eliminate duplicate detections and obtain the final passenger flow detection result D_result, which includes the precise location, pose and confidence of each detected passenger.

[0018] S6. Passenger Flow Statistics and Density Analysis: Statistically count the number of passengers in each area, analyze the passenger flow distribution density in the carriages, and generate a passenger flow statistics report.

[0019] S61. Multi-area passenger flow statistics: Based on the passenger flow detection results D_result, the carriage space is divided into multiple functional areas, such as entrance / exit areas, seating areas, and standing areas. The number of passengers detected in each area is counted to obtain the regional passenger flow statistics table T_count. S62. Temporal smoothing: A temporal smoothing algorithm is applied to the regional passenger flow statistics table T_count across multiple consecutive frames to reduce detection fluctuations and improve statistical stability, resulting in a smoothed passenger flow statistics table T_smooth. S63. Passenger flow density distribution analysis: Based on the smoothed passenger flow statistics table T_smooth and the physical dimensions of the carriage space, the passenger flow density distribution of each area is calculated, generating a passenger flow density distribution map D_distribution. S64. Passenger flow statistics report generation: The smoothed passenger flow statistics table T_smooth and the passenger flow density distribution map D_distribution are integrated to generate a passenger flow statistics report R_report containing numerical statistics and visualization charts, providing a basis for decision-making in carriage passenger flow management.

[0020] According to one aspect of this application, step S22 specifically comprises: S221. Fisheye distortion model analysis: Read K_camera and D_coeff from the radial mapping parameter initialization step, calculate the theoretical distortion model of the standard fisheye lens, and obtain the theoretical distortion curve C_theory; S222. Multi-sample distortion characteristic extraction: Select multiple representative samples from the pre-stored car fisheye sample library I_samples, extract the curvature change law of the straight structure (such as seat edge, window frame) in each sample, and obtain the actual distortion sampling point set P_distort; S223. Distortion difference analysis and compensation: Compare the theoretical distortion curve C_theory with the actual distortion sampling point set P_distort, calculate the difference vector Δd_vector between the two at different radial distances, and use it for subsequent compensation; S224. Parameter optimization initialization: Set the initial values ​​of the parameters of the radial mapping function R(d) = α·d^γ + β·d + δ, where α_init = 0.05, β_init = 1.0, γ_init = 2.0, δ_init = 0, forming the initial parameter vector θ_init = [α_init, β_init, γ_init, δ_init]; S225. Iterative optimization of mapping function parameters: Using gradient descent, based on the actual distortion sampling point set P_distort and the difference vector Δd_vector, the initial parameter vector θ_init is iteratively optimized. The optimization objective is to minimize the mapping error, resulting in the optimized parameter vector θ_opt = [α_opt, β_opt, γ_opt, δ_opt]; S226. Construction and verification of continuous mapping function: Using the optimized parameter vector θ_opt, the final continuous radial mapping function R(d) = α_opt·d^γ_opt + β_opt·d + δ_opt is constructed. Its effectiveness is verified through the correction test of the carriage structure lines, generating a complete radial mapping parameter set R_params = {α_opt, β_opt, γ_opt, δ_opt,C}, where C is the coordinate of the image center point.

[0021] According to one aspect of this application, step S23 specifically comprises: S231. Radial distance field calculation: Read the coordinates C(cx, cy) of the image center point in the radial mapping parameter set R_params, calculate the distance from each pixel (x, y) of the image to the center, and generate a radial distance field D_field; S232. Determination of basic receptive field size: Based on the average size of passengers in the carriage and the detection requirements, set the basic receptive field size F_base = 3×3 in the central region as a reference value for receptive field adjustment; S233. Calculation of receptive field adjustment coefficient: Analyze the variation law of passenger size in different areas of the carriage, and combine the variation curve of the radial mapping function R(d) to establish a calculation method for the receptive field adjustment coefficient λ: λ = λ_max · (1 - exp(-k· (d_norm - d_thresh))); where d_norm is the normalized radial distance, d_thresh is the threshold distance, k is the attenuation coefficient, and λ_max is the maximum adjustment coefficient, and generate an adjustment coefficient mapping table λ_map; S234. Adaptive Receptive Field Size Mapping: Read the radial distance field D_field and the adjustment coefficient mapping table λ_map. For each pixel (x,y), calculate its receptive field size F(x,y) = F_base·(1 + λ(x,y)·R(D_field(x,y))), generating the receptive field size mapping F_map for the entire image; S235. Deformed Convolution Kernel Design: Design deformed convolution kernels based on the receptive field size mapping F_map. For each pixel (x,y), the sampling position of the convolution kernel is dynamically adjusted according to F(x,y), generating the convolution kernel sampling offset O_offset; S236. Position Adaptive Convolution Implementation: Construct a position adaptive convolution layer, read the convolution kernel sampling offset O_offset, and implement adaptive convolution operations at each position to ensure that the receptive field size mapping F_map can directly guide the feature extraction process.

[0022] According to one aspect of this application, S32, the generation of the feature transformation parameter matrix specifically includes: S321. Distortion characteristic analysis: reading the distortion characteristic tensor D_tensor and extracting three components: radial distortion, tangential distortion, and irregular distortion, denoted as D_radial, D_tangential, and D_irregular, respectively; S322. Standard registration point generation: uniformly distributing a set of registration points on a distortion-free ideal plane to form a standard registration point set P_standard, used to track the effect of feature transformation; S323. Mapping of registration points under distortion influence: forward mapping the standard registration point set P_standard through the distortion characteristic tensor D_tensor to obtain the distortion-affected registration point set P_distorted; S324. Feature space grid partitioning: dividing the feature map space into M×N grid cells, each cell corresponding to a region on the feature map, forming a feature grid G_feature; S325. Local Affine Transformation Estimation: For each grid cell (i,j) in the feature grid G_feature, the local affine transformation matrix A(i,j) is estimated based on the standard registration point set P_standard and the corresponding distorted registration point set P_distorted within its region. This matrix includes rotation, scaling, and shearing components. S326. Deformation Field Interpolation Calculation: Based on the local affine transformation matrix A(i,j) of the grid cell, the precise transformation parameters are calculated for each position (x,y) on the feature map using bilinear interpolation, forming a continuous deformation field V_field. S327. Feature Transformation Parameter Matrix Integration: The feature distribution information of the deformation field V_field and the preliminary feature map F_init is integrated. The final feature transformation parameter matrix set M_params is generated through the transformation function f_transform. This set contains parameters for translation, rotation, scaling, and deformation components, which are used for subsequent feature reconstruction.

[0023] According to one aspect of this application, step S33 specifically comprises: S331. Feature reconstruction region partitioning: Based on the transformation intensity in the feature transformation parameter matrix set M_params, the feature space is divided into a high-transformation region R_high, a medium-transformation region R_mid, and a low-transformation region R_low, forming a region partitioning map R_map; S332. Multi-scale feature pyramid construction: A three-layer feature pyramid is constructed on the initial feature map F_init to generate a multi-scale feature set F_pyramid = {F_s1, F_s2, F_s3}, which is used to adapt to feature reconstruction with different transformation intensities; S333. Region adaptive feature sampling: Based on the region partitioning map R_map and the feature transformation parameter matrix set M_params, adaptive sampling is performed on the features in the multi-scale feature set F_pyramid. A fine sampling strategy is used for high-transformation regions, and a coarse sampling strategy is used for low-transformation regions, resulting in adaptively sampled features F_sample; S334. Transformation Matrix Application and Feature Reconstruction: The feature transformation parameter matrix set M_params is applied to the adaptive sampling feature F_sample to perform spatial adaptive transformation, thereby correcting distortion in the feature space and obtaining the reconstructed feature map F_recon; S335. Feature Consistency Optimization: The inter-region feature continuity index C_index of the reconstructed feature map F_recon is calculated, and feature smoothing transition processing is applied at the region boundaries to ensure the spatial continuity of features, resulting in the optimized reconstructed feature map F_recon_opt; S336. Multi-level Feature Fusion: The optimized reconstructed feature map F_recon_opt and the initial feature map F_init are combined through a weighted fusion mechanism to retain the effective information in the original features while correcting the distortion effect, generating the final distortion-adapted feature map F_adapt.

[0024] According to one aspect of this application, S42, the construction of the density adaptive feature separation operator specifically includes: S421. Passenger flow density gradient analysis: Read the passenger flow density map D_crowd, calculate the density gradient ∇D_crowd, identify areas with drastic density changes, and form a density gradient map G_density; S422. Feature confusion assessment: Based on the passenger flow density map D_crowd and the density gradient map G_density, calculate the feature confusion level at different density levels and establish a confusion-density mapping relationship C_map; S423. Construction of a dynamic parameter adjustment model: Based on the confusion-density mapping relationship C_map, establish a dynamic adjustment model for the parameters of the feature separation operator: ω_1 = a1 · exp(b1 · C_avg) + c1. ω_2 = a2 · C_max + b2 · C_avg + c2. ω_3 = a3 · (1 - C_max) + c3.

[0025] Where C_avg is the average confusion level, C_max is the maximum confusion level, and a1, b1, c1, etc. are preset constants, generating the parameter adjustment model M_adjust; S424. Local density adaptive parameter calculation: Based on the parameter adjustment model M_adjust and the local passenger flow density map D_crowd, calculate the feature separation operator parameters ω_1(i,j), ω_2(i,j), and ω_3(i,j) at each location (i,j), forming the parameter space distribution map P_dist; S425. Separation operator kernel function design: Based on the parameter space distribution map P_dist, design the feature separation operator kernel function D_op(i,j,ρ) = ω_1(i,j)·ρ^2 + ω_2(i,j)·ρ + ω_3(i,j) for each location (i,j), generating the separation operator kernel function set K_func; S426. Feature separation parameter set generation: Integrate the kernel function set K_func of the separation operator and the parameter space distribution map P_dist to construct a complete feature separation parameter set D_params, which contains the separation intensity, direction and smoothing factor at each position, for subsequent feature separation enhancement.

[0026] According to one aspect of this application, S44, the feature channel attention mechanism is implemented as follows: S441. Channel-density correlation analysis: Read the density-enhanced feature map F_density and the passenger flow density map D_crowd, analyze the correlation between each feature channel and the passenger flow density, and generate a channel correlation vector R_corr; S442. Channel importance score: Based on the channel correlation vector R_corr, calculate the importance score of each feature channel for passenger flow detection, and generate a channel importance vector I_chan; S443. Density-aware channel weight generation: Combine the passenger flow density map D_crowd and the channel importance vector I_chan, construct a density-aware channel weight calculation function: w_j = σ(I_chan[j] · (1 + α · D_norm)); where w_j is the weight of the j-th channel, I_chan[j] is the importance score of the channel, D_norm is the normalized passenger flow density, α is the adjustment coefficient, and σ is the sigmoid activation function, to obtain the channel weight vector W_chan; S444. S445. Spatial Adaptive Attention Map Construction: Based on the spatial distribution of the passenger flow density map D_crowd, attention weights for spatial locations are generated to form a spatial attention map A_spatial; S446. Channel Attention Feature Enhancement: The channel weight vector W_chan is applied to each channel of the density enhancement feature map F_density to perform channel-level attention enhancement, resulting in a channel enhancement feature map F_chan_att; S447. Spatial-Channel Attention Fusion: The spatial attention map A_spatial and the channel enhancement feature map F_chan_att are integrated, and the final attention enhancement feature map F_attention is generated through a joint attention mechanism of spatial location and channel dimension.

[0027] According to one aspect of this application, S45. Sparse computation optimization, specifically: S451. Passenger flow density threshold analysis: Read the passenger flow density map D_crowd, and determine the boundary thresholds T_high and T_low for high density, medium density, and low density through an adaptive threshold algorithm, generating a density threshold set T_thresh; S452. Key region identification: Based on the density threshold set T_thresh and the passenger flow density map D_crowd, divide the image region into a high-density key region R_key that requires complete feature processing, a medium-density region R_mid that requires medium processing, and a low-density region R_simple that can be simplified, forming a processing region map R_proc; S453. Computational complexity allocation: Based on the processing region map R_proc, allocate computational resources and processing complexity to each region, apply complete feature separation and enhancement processing to the high-density key region, and apply simplification processing to the low-density region, generating a computational complexity map C_comp; S454. Sparse kernel function design: Design corresponding feature processing kernel functions for regions with different processing complexities. Use the full D_op operator for high-complexity regions and the simplified D_op_simple operator for low-complexity regions, forming a multi-complexity kernel function set K_sparse; S455. Region adaptive feature processing: Read the attention-enhanced feature map F_attention, and apply feature processing of corresponding complexity to different regions according to the processing region map R_proc and the multi-complexity kernel function set K_sparse, generating a region processing feature map F_region; S456. Boundary consistency preservation: Apply smooth transition processing at the boundaries of regions with different processing complexities to ensure the spatial continuity of features and prevent unnatural feature gaps between regions, obtaining the final optimized feature map F_sparse.

[0028] According to one aspect of this application, S52. Optimization of the distortion-aware target representation, specifically: S521. Candidate box distortion evaluation: Read the multi-scale target candidate box set B_proposals and the distortion feature tensor D_tensor, calculate the degree of distortion affecting each candidate box, and generate a distortion evaluation index E_dist; S522. Representation selection strategy: Based on the value of the distortion evaluation index E_dist, select the most suitable representation for each candidate box: use standard rectangles for regions with small distortions, rotated rectangles for regions with medium distortions, and ellipses or polygons for regions with high distortions, obtaining a representation mapping table F_repr; S523. Rectangular representation optimization: For candidate boxes selected from the representation mapping table F_repr using standard rectangular representations, adjust their position and size parameters according to the distortion feature tensor D_tensor to obtain an optimized set of rectangular boxes B_rect; S524. S525. Rotation Rectangle Representation Optimization: For candidate boxes selected from the representation mapping table F_repr, the principal direction deflection angle θ caused by distortion is calculated, generating an optimized rotation box set B_rot with position, size, and rotation angle parameters; S526. Ellipse Representation Optimization: For candidate boxes selected from the representation mapping table F_repr, the major axis, minor axis length, and rotation angle are calculated based on the distortion characteristic tensor D_tensor, generating an optimized ellipse set B_ellip; S527. Polygon Representation Generation: For candidate boxes in severely distorted regions, multiple control points are generated on the box boundary through adaptive sampling to form a polygon representation that fits the distorted contour, generating an optimized polygon set B_poly; S528. Multi-Representation Integration: The optimized rectangle box set B_rect, optimized rotation box set B_rot, optimized ellipse set B_ellip, and optimized polygon set B_poly are integrated to form a unified optimized candidate box B_refined, with each candidate box using a representation most suitable for its distortion level. Example

[0029] S1. Fisheye Image Acquisition and Preprocessing S11. Fisheye Image Acquisition: This embodiment uses a fisheye camera installed at the center of the subway car's ceiling to acquire images inside the car. The camera model is CX-F185, with a resolution of 1920×1080 pixels and a field of view of 185°, capable of covering almost the entire car space. A single frame of the original image I_orig is shown in Figure 1 (figure omitted).

[0030] S12. Image Noise Assessment and Adaptive Denoising: First, noise assessment is performed on the original image I_orig. The noise level σ_noise is calculated as follows: σ_noise = √(1 / (W×H)∑∑(I_orig(x,y) - I_blur(x,y))²); where: I_orig(x,y) is the original image pixel value at position (x,y); I_blur(x,y) is the image pixel value after blurring using a 5×5 Gaussian kernel (σ=1.5); W=1920 is the image width; H=1080 is the image height; σ_noise is the estimated noise standard deviation.

[0031] In this case, the calculated noise level σ_noise = 8.7. Based on the noise level, the bilateral filter parameters are selected as follows: when σ_noise < 5, the spatial parameter σ_d = 3 and the range parameter σ_r = 0.1 are set; when 5 ≤ σ_noise < 10, σ_d = 5 and σ_r = 0.15 are set; when σ_noise ≥ 10, σ_d = 7 and σ_r = 0.2 are set. In this example, σ_noise = 8.7, therefore σ_d = 5 and σ_r = 0.15 are chosen, with a window size of 11×11, resulting in the denoised image I_denoise.

[0032] S13. Compensation for Uneven Illumination: Illumination inside subway cars is often uneven and needs to be compensated. First, calculate the illumination distribution map L_map inside the car: L_map(x,y) = (0.2989×R(x,y) + 0.5870×G(x,y) + 0.1140×B(x,y)); where: R(x,y), G(x,y), and B(x,y) are the RGB three-channel values ​​at position (x,y) respectively; L_map(x,y) is the illumination value at position (x,y).

[0033] Next, the contrast-limited adaptive histogram equalization (CLAHE) method is applied to divide the image into 8×8 grids, the contrast limit parameter clip_limit is set to 3.0, and illumination compensation is performed to obtain the illumination-equalized image I_equal.

[0034] S14. Image Preprocessing Improvement: The illumination equalization image I_equal is converted from the RGB color space to the HSV color space, resulting in three channels: H (hue), S (saturation), and V (luminance). Then, the RGB and HSV information are combined to generate a 6-channel preprocessed image I_pre = [R, G, B, H, S, V], which serves as the input for subsequent processing.

[0035] S2. Radial Mapping Function Construction and Adaptive Extraction of Continuous Feature Scale S21. Radial Mapping Function Parameter Initialization: First, the intrinsic parameter matrix K_camera and distortion coefficients D_coeff of the fisheye camera are obtained through camera calibration: K_camera = [800.5, 0, 959.7; 0, 801.2, 539.5; 0, 0,1]. D_coeff = [-0.3651, 0.1185, -0.00015, 0.00021, -0.0321]. The first row and third column element 959.7 and the second row and third column element 539.5 of K_camera represent the image center point coordinates cx and cy, respectively.

[0036] S22. Construction of Continuous Radial Mapping Function: S221. Fisheye Distortion Model Analysis: Based on the intrinsic parameter matrix K_camera and the distortion coefficients D_coeff, the theoretical distortion curve C_theory of the standard fisheye lens is calculated. For pixel coordinates (x,y), the normalized radial distance r to the center point is calculated as follows: r = √((x-cx)²+(y-cy)²) / max(W,H); where: x and y are pixel coordinates; cx=959.7 and cy=539.5 are the coordinates of the image center point; W=1920 is the image width; H=1080 is the image height; r is the normalized radial distance, with a value range of [0,1].

[0037] Based on the distortion coefficient D_coeff, the theoretical distortion curve C_theory is expressed as:

[0038] S222. Multi-sample distortion feature extraction: Select 5 representative samples from the pre-stored car body fisheye sample library I_samples, and extract the curvature variation pattern of linear structures (such as seat edges and window frames) in each sample. In each sample image, select 10 structural lines that should theoretically be straight (such as seat edges), sample 20 points on each line, record the coordinates of these points, and form the actual distortion sampling point set P_distort.

[0039] S223. Distortion Difference Analysis and Compensation: Compare the theoretical distortion curve C_theory with the actual distortion sampling point set P_distort, and calculate the difference vector Δd_vector between the two at different radial distances: The difference vector calculation in step S223 is: Δd_vector(r_i) = P_distort(r_i) - C_theory(r_i); where: r_i is the i-th radial distance sampling point; P_distort(r_i) is the distortion value of the actual distortion sampling point set at r_i; C_theory(r_i) is the distortion value of the theoretical distortion curve at r_i; Δd_vector(r_i) is the difference vector at r_i.

[0040] S224. Parameter Optimization Initialization: Set the initial values ​​of the parameters of the radial mapping function R(d), where d is the distance from the pixel to the center point of the image: R(d) = α·d^γ + β·d + δ; where: d is the distance from the pixel to the center point of the image; α is the coefficient of the nonlinear term, with an initial value of α_init=0.05; β is the coefficient of the linear term, with an initial value of β_init=1.0; γ is the exponent of the power term, with an initial value of γ_init=2.0; δ is the constant term, with an initial value of δ_init=0; R(d) is the value of the radial mapping function.

[0041] S225. Iterative Optimization of Mapping Function Parameters: Using gradient descent, based on the actual distortion sampling point set P_distort and the difference vector Δd_vector, the initial parameter vector θ_init = [0.05, 1.0, 2.0, 0] is iteratively optimized. The optimization objective is to minimize the mapping error: E(θ) = (1 / N)·∑(R(d_i,θ) - (C_theory(r_i) +Δd_vector(r_i)))²; where: θ=[α,β,γ,δ] is the parameter vector; d_i is the distance from the i-th sampling point to the image center; R(d_i,θ) is the radial mapping function value calculated using parameter θ; N is the number of sampling points; E(θ) is the optimization objective function value, representing the mapping error. The learning rate η=0.01 is set, the maximum number of iterations is 200, and the iteration is terminated early when the error change is less than 1e-5 for 5 consecutive iterations. After 138 iterations, the optimized parameter vector θ_opt = [0.078, 0.925, 1.85, 0.012] was obtained.

[0042] S226. Construction and Verification of Continuous Mapping Function: The final continuous radial mapping function is constructed using the optimized parameter vector θ_opt: R(d) = 0.078·d^1.85 + 0.925·d + 0.012; where: d is the distance from the pixel to the center of the image; R(d) is the value of the radial mapping function.

[0043] The effectiveness of the correction test was verified by the straight lines of the carriage structure. The straight lines of 10 carriages were selected and the R(d) function was applied for correction. The average straightness error after correction was calculated to be 2.1 pixels, which is 69.1% lower than the 6.8 pixel error using the standard distortion model.

[0044] S23. Adaptive Receptive Field Design: S231. Radial Distance Field Calculation: Based on the coordinates C (cx=959.7, cy=539.5) of the image center point, calculate the distance from each pixel (x,y) to the center: D_field(x,y) = √((x-cx)²+(y-cy)²); where: x and y are pixel coordinates; cx=959.7 and cy=539.5 are the coordinates of the image center point; D_field(x,y) is the distance value from the position (x,y) to the image center.

[0045] S232. Determination of basic sensing field size: Based on the average size of passengers in the carriage (approximately 160cm high and 50cm wide) and detection requirements, the basic sensing field size of the central area is set as F_base = 3×3, which serves as a reference value for sensing field adjustment.

[0046] S233. Calculation of Receptive Field Adjustment Coefficient: Analyze the passenger scale variation patterns in different areas of the carriage and establish a calculation method for the receptive field adjustment coefficient λ: λ(d_norm) = λ_max·(1-exp(-k·(d_norm-d_thresh))); where: d_norm=D_field(x,y) / max(W,H) is the normalized radial distance with a value range of [0,1]; d_thresh=0.2 is the threshold distance, representing the central region range; k=5.0 is the attenuation coefficient, controlling the adjustment rate; λ_max=2.0 is the maximum adjustment coefficient; λ(d_norm) is the receptive field adjustment coefficient.

[0047] The calculated adjustment coefficient mapping table λ_map has the following characteristics: when d_norm≤0.2, λ is close to 0, and the receptive field maintains the basic size; when d_norm increases, λ gradually increases, and the maximum value is close to λ_max=2.0.

[0048] S234. Adaptive Receptive Field Size Mapping: Based on the receptive field adjustment coefficient λ_map and the radial mapping function R(d), calculate the receptive field size of each pixel (x,y): F(x,y) = F_base·(1+λ(d_norm)·R(D_field(x,y))); where: F_base=3 is the base receptive field size; λ(d_norm) is the receptive field adjustment coefficient at position (x,y); R(D_field(x,y)) is the radial mapping function value at position (x,y); and F(x,y) is the receptive field size at position (x,y).

[0049] In the central region of the image (d_norm≤0.2), F(x,y)≈3×3; in the edge region (d_norm≈0.8), F(x,y) can reach about 9×9, which allows feature extraction in the edge region to capture sufficient contextual information.

[0050] S235. Deformed Convolution Kernel Design: Design deformed convolution kernels based on the receptive field size mapping F_map. For each pixel (x,y), design the offset O_offset(x,y,i,j) = [(F(x,y) / F_base)-1]·[i-1,j-1] for its convolution kernel sampling position; where: F(x,y) is the receptive field size at position (x,y); F_base=3 is the base receptive field size; i,j∈{0,1,2} are the coordinate indices in the 3×3 convolution kernel; O_offset(x,y,i,j) is the sampling offset of the convolution kernel at position (x,y) at position (i,j).

[0051] For example, for a pixel with a receptive field size F(x,y)=6, when calculating the convolution, the sampling offset at position (0,0) is (-(6 / 3-1)·1, -(6 / 3-1)·1)=(-1,-1), that is, the actual sampling position is offset from the original position by (-1,-1) pixels.

[0052] S236. Position Adaptive Convolution Implementation: Construct a position adaptive convolution layer to implement adaptive convolution operation at each position. For the input feature map X and the convolution kernel weight W, the output feature map Y is calculated as follows: Y(x,y,c_out) =∑∑∑W(i,j,c_in,c_out)·X(x+i+O_offset(x,y,i,j)[0], y+j+O_offset(x,y,i,j)[1], c_in); where: X is the input feature map; W is the convolution kernel weight; c_in is the input channel index; c_out is the output channel index; O_offset is the sampling offset; Y is the output feature map.

[0053] In this way, the receptive field size mapping F_map directly guides the feature extraction process, enabling edge regions to adaptively use a larger receptive field, thereby capturing more contextual information and compensating for feature distortion caused by aberrations.

[0054] S24. Continuous Feature Scale Extraction: Feature extraction is performed on the preprocessed image I_pre using deformable convolutions guided by the receptive field distribution map F_map. The specific steps are as follows: A 4-layer feature extraction network is constructed, each layer containing a 3×3 position-adaptive convolution with a stride of 1 and padding of 1, and output channels of 64, 128, 128, and 128 respectively. ReLU activation and batch normalization are applied after each convolution. A 2×2 max pooling operation with a stride of 2 is used between the 2nd and 3rd layers.

[0055] The feature map of the last layer is upsampled to the same resolution as the input to obtain the initial feature map F_init, with dimensions [128, H, W].

[0056] Through the above process, F_init can adapt to changes in feature scale across different regions while maintaining spatial dimensionality consistency. Compared to traditional fixed receptive field methods, the feature quality of edge regions is significantly improved, providing a solid foundation for subsequent distortion-adaptive feature reconstruction.

[0057] S3. Distortion Pattern Perception and Feature Reconstruction S31. Distortion Pattern Estimation: Based on the preprocessed image I_pre and the radial mapping parameter set R_params, a distortion pattern estimation network is constructed to learn the actual distortion characteristics of the car's fisheye camera.

[0058] Design an encoder-decoder network structure. The encoder contains four downsampled convolutional blocks, each containing two 3×3 convolutional layers and one 2×2 max pooling layer. The number of output channels are 64, 128, 256, and 512, respectively.

[0059] The decoder contains four upsampled convolutional blocks, each containing a 2×2 transposed convolutional layer and two 3×3 convolutional layers, with output channels of 256, 128, 64, and 32, respectively. Skip connections are used to connect the features of the encoder to the features of the corresponding layers in the decoder. The last layer uses a 1×1 convolution with 9 output channels to generate a distortion feature tensor D_tensor with dimensions [9, H, W].

[0060] The nine channels of the D_tensor represent: Channels 0-2: Radial distortion component D_radial, indicating the degree of distortion along the radial direction, with dimensions [3, H, W]. Channels 3-5: Tangential distortion component D_tangential, indicating the degree of distortion along the tangential direction, with dimensions [3, H, W]. Channels 6-8: Irregular distortion component D_irregular, indicating aberrant distortions that do not follow the conventional distortion model, with dimensions [3, H, W].

[0061] S32. Generation of Feature Transformation Parameter Matrix: S321. Distortion Characteristic Analysis: Extract three distortion components from the distortion characteristic tensor D_tensor: D_radial = D_tensor[0:3, :, :]. D_tangential = D_tensor[3:6, :, :]. D_irregular = D_tensor[6:9, :, :].

[0062] S322. Standard Registration Point Generation: 16×16 registration points are uniformly distributed on a distortion-free ideal plane to form a standard registration point set P_standard. The coordinates of each registration point are calculated as follows: P_standard(m,n) = [m·(W-1) / 15, n·(H-1) / 15]; where: m,n∈{0,1,...,15} are the row and column indices; W=1920 is the image width; H=1080 is the image height; P_standard(m,n) is the coordinate of the (m,n)th standard registration point.

[0063] S323. Mapping of registration points under distortion: The standard registration point set P_standard is forward mapped through the distortion characteristic tensor D_tensor to obtain the distorted registration point set P_distorted(m,n) = P_standard(m,n) + D_radial(P_standard(m,n))·v_r + D_tangential(P_standard(m,n))·v_t + D_irregular(P_standard(m,n)); where: P_standard(m,n) are the coordinates of the standard registration points; D_radial, D_tangential, and D_irregular are the radial distortion, tangential distortion, and irregular distortion components, respectively; v_r is the radial unit vector; v_t is the tangential unit vector; and P_distorted(m,n) are the coordinates of the distorted registration points.

[0064] S324. Feature Space Grid Partitioning: The feature map space is divided into 8×8 grid cells, each cell corresponding to a region on the feature map, forming a feature grid G_feature. The size of each grid cell is W / 8×H / 8 pixels.

[0065] S325. Local Affine Transformation Estimation: For each grid cell (i,j) in the feature grid G_feature, based on the standard registration point set P_standard and the corresponding distorted registration point set P_distorted within its region, estimate the local affine transformation matrix A(i,j) = argmin_A ∑∑||A·P_standard(m,n) - P_distorted(m,n)||²; where: (m,n) are the indices of all registration points within the grid cell (i,j); A is a 2×3 affine transformation matrix containing rotation, scaling, and shearing components; A(i,j) is the optimal affine transformation matrix for the grid cell (i,j).

[0066] Solving the above optimization problem using the least squares method yields the local affine transformation matrix A(i,j) for each mesh element. For example, for mesh element (4,4), the calculated affine transformation matrix is: A(4,4) = [1.02, 0.01, -0.5; 0.02, 0.98, 0.8].

[0067] S326. Deformation Field Interpolation Calculation: Based on the local affine transformation matrix A(i,j) of the mesh element, calculate the precise transformation parameters for each position (x,y) on the feature map using bilinear interpolation to form a continuous deformation field V_field(x,y) = ∑∑w(i,j,x,y)·A(i,j); where: w(i,j,x,y) is the bilinear interpolation weight of position (x,y) relative to mesh element (i,j); A(i,j) is the affine transformation matrix of mesh element (i,j); and V_field(x,y) is the deformation field value at position (x,y).

[0068] The bilinear interpolation weight w(i,j,x,y) is calculated as follows: w(i,j,x,y) = max(0, 1-|x / g_w-i|)·max(0, 1-|y / g_h-j|); where: g_w=W / 8 is the grid cell width; g_h=H / 8 is the grid cell height; i,j are the grid cell indices; x,y are the pixel coordinates; w(i,j,x,y) is the interpolation weight.

[0069] S327. Feature Transformation Parameter Matrix Integration: Integrate the feature distribution information of the deformation field V_field and the preliminary feature map F_init to generate the final feature transformation parameter matrix set M_params(x,y) = [tx, ty, θ, sx, sy, shx,shy]; where: tx, ty are translation parameters, extracted from the third column of V_field(x,y); θ is the rotation angle, extracted from the rotation component of V_field(x,y); sx, sy are scaling parameters, extracted from the scaling component of V_field(x,y); shx, shy are shearing parameters, extracted from the shearing component of V_field(x,y); M_params(x,y) is the feature transformation parameter matrix at position (x,y).

[0070] For example, for position (960, 540), its feature transformation parameter matrix is: M_params(960,540) = [0.2, 0.3, 0.01, 1.01, 0.99, 0.01, 0.02].

[0071] This indicates that at this position, it needs to be translated horizontally by 0.2 pixels, vertically by 0.3 pixels, rotated by 0.01 radians, scaled horizontally by 1.01 times, scaled vertically by 0.99 times, clipped horizontally by 0.01 times, and clipped vertically by 0.02 times.

[0072] S33. Adaptive Feature Reconstruction: S331. Feature Reconstruction Region Partitioning: Based on the transformation intensity in the feature transformation parameter matrix set M_params, the feature space is divided into a high-transformation region R_high, a medium-transformation region R_mid, and a low-transformation region R_low, forming a region partitioning map R_map. Transformation intensity S_transform(x,y) = |θ(x,y)| + |sx(x,y)-1| + |sy(x,y)-1| + |shx(x,y)| + |shy(x,y)|; where θ, sx, sy, shx, and shy are parameters in M_params; S_transform(x,y) is the transformation intensity at position (x,y). Based on the transformation intensity S_transform, the following rule is used to partition the region: when S_transform(x,y) < 0.05, it is classified as a low-transformation region R_low. When 0.05 ≤ S_transform(x,y) < 0.15, it is classified as the mid-transform region R_mid. When S_transform(x,y) ≥ 0.15, it is classified as the high-transform region R_high.

[0073] S332. Multi-scale Feature Pyramid Construction: Construct a three-layer feature pyramid from the initial feature map F_init: F_s1 = F_init, original resolution feature, size [128, H, W]. F_s2 = MaxPool(F_s1, kernel_size=2, stride=2), 1 / 2 resolution feature, size [128, H / 2, W / 2]. F_s3 = MaxPool(F_s2, kernel_size=2, stride=2), 1 / 4 resolution feature, size [128, H / 4, W / 4].

[0074] S333. Region Adaptive Feature Sampling: Based on the region partitioning map R_map and the feature transformation parameter matrix set M_params, adaptive sampling is performed on the features in the multi-scale feature set F_pyramid: High-transformation regions R_high undergo fine sampling using F_s1, with a sampling grid size of 1×1 pixels. Medium-transformation regions R_mid undergo medium sampling using F_s2, with a sampling grid size of 2×2 pixels. Low-transformation regions R_low undergo coarse sampling using F_s3, with a sampling grid size of 4×4 pixels. The resulting adaptively sampled feature F_sample has the same size as F_init.

[0075] S334. Application of Transformation Matrix and Feature Reconstruction: The feature transformation parameter matrix set M_params is applied to the adaptive sampling feature F_sample to perform a spatial adaptive transformation: F_recon(x,y,c) = ∑∑w(u,v,x,y,M_params(x,y))·F_sample(u,v,c); w(u,v,x,y,M_params(x,y)) is the sampling weight calculated based on the feature transformation parameter matrix; u,v are the sampling position coordinates; c is the feature channel index; F_recon is the reconstructed feature map. After the transformation, the radial distortion of feature points in the edge region is effectively corrected, and the degree of feature deformation is reduced by an average of 65.3%, providing a more accurate feature representation for subsequent object detection.

[0076] S335. Feature Consistency Optimization: Calculate the inter-region feature continuity index C_index(x,y) = (1 / N)·∑∑||F_recon(x,y,c) - F_recon(x+dx,y+dy,c)|| in the reconstructed feature map F_recon; where: dx,dy∈{-1,0,1} is the neighborhood offset; N is the number of neighboring pixels; c is the feature channel index; C_index(x,y) is the feature continuity index at position (x,y), and the smaller the value, the better the continuity.

[0077] Apply feature smoothing transition processing at the region boundary: F_recon_opt(x,y,c) = (1-α(x,y))·F_recon(x,y,c) + α(x,y)·(1 / N)·∑∑F_recon(x+dx,y+dy,c); where: α(x,y) is the smoothing coefficient, which is proportional to C_index(x,y); dx,dy∈{-1,0,1} is the neighborhood offset; F_recon_opt is the optimized reconstructed feature map.

[0078] For example, for a boundary point (1200, 600), assuming its continuity index C_index(1200,600)=0.35, the smoothing coefficient α(1200,600)=min(0.5, 0.35·2)=0.5, that is, the features of this point will be 50% from the original features and 50% from the average of the neighborhood features.

[0079] S336. Multi-level feature fusion: The optimized reconstructed feature map F_recon_opt and the preliminary feature map F_init are combined through a weighted fusion mechanism: F_adapt(x,y,c) = β·F_recon_opt(x,y,c) + (1-β)·F_init(x,y,c); where: β is the fusion weight, which is dynamically adjusted for different regions; F_recon_opt is the optimized reconstructed feature map; F_init is the preliminary feature map; and F_adapt is the distortion adaptation feature map.

[0080] The fusion weight β is dynamically adjusted according to the transformation intensity of position (x,y): when the transformation intensity S_transform(x,y) is high, β is close to 1, and the reconstructed features are used preferentially; when the transformation intensity is low, β is close to 0.5, and the two features are used in a balanced manner.

[0081] S34. Implementation of Bidirectional Feature Flow Mechanism: A feature flow channel is established on the distortion-adapted feature map F_adapt, allowing feature information to flow freely between different scales and regions. A 4×4 region partition is constructed, with each region containing a local feature aggregation node. A bidirectional feature flow network is designed, including both "bottom-up" and "top-down" flows. Bottom-up flow: Features are transferred from fine-grained to coarse-grained through 3×3 convolution + 2×2 max pooling. Top-down flow: Features are transferred from coarse-grained to fine-grained through 2×2 transposed convolution. Lateral connection: Feature exchange between regions of the same level is established through 1×1 convolution. Through the feature flow mechanism, feature information between different regions is effectively shared, enhancing the feature consistency between edge regions and the central region, resulting in an enhanced flow feature map F_flow, with the same dimension as F_adapt.

[0082] S4. Enhanced Feature Separation Based on Density Adaptation S41. Fast Local Density Evaluation: A lightweight density evaluation network is used to process the preprocessed image I_pre to generate passenger flow density maps D_crowd for each area of ​​the carriage. A lightweight fully convolutional network is constructed, containing 5 convolutional layers with output channels of 32, 64, 32, 16, and 1 respectively. A 3×3 convolutional kernel is used with a stride of 1 and padding of 1. 2×2 max pooling is used after the 2nd and 4th layers. The last layer uses a 1×1 convolution to output a single-channel density map, which is normalized to the [0,1] interval by a Sigmoid activation function. The network output dimension is [1, H, W], representing the passenger flow density value ρ(i,j) at each location. In this embodiment, the density value of high-density areas is close to 1, and the density value of low-density areas is close to 0.

[0083] S42. Construction of Density Adaptive Feature Separation Operator: S421. Passenger Flow Density Gradient Analysis: Calculate the density gradient of the passenger flow density map D_crowd. Where: D_crowd is the passenger flow density map; This is a density gradient map, containing gradient components in both horizontal and vertical directions.

[0084] Based on gradient magnitude Identify regions of drastic density changes and generate density gradient maps. ;in: , These represent the horizontal and vertical components of the density gradient, respectively; G_density is the density gradient magnitude map.

[0085] S422. Feature Confusion Assessment: Based on the passenger flow density map D_crowd and the density gradient map G_density, assess the feature confusion level at different density levels: C(ρ) = k1·ρ² + k2·ρ·G_avg(ρ) + k3·G_max(ρ); where: ρ is the density value; G_avg(ρ) is the average gradient magnitude of the region with density value ρ; G_max(ρ) is the maximum gradient magnitude of the region with density value ρ; k1=0.6, k2=0.3, k3=0.1 are weighting coefficients; C(ρ) is the feature confusion level corresponding to the density value ρ.

[0086] The confusion level C(ρ) corresponding to different density values ​​ρ was statistically analyzed, and a confusion level-density mapping relationship C_map was established. Experiments show that when ρ=0.2, C(0.2)=0.15; when ρ=0.5, C(0.5)=0.38; and when ρ=0.8, C(0.8)=0.72. This indicates that the feature confusion level increases non-linearly with increasing density.

[0087] S423. Construction of Dynamic Parameter Adjustment Model: Based on the confusion-density mapping relationship C_map, a dynamic adjustment model for the feature separation operator parameters is established: ω_1 = 0.8·exp(2.5·C_avg) + 0.1; ω_2 = 0.6·C_max +0.3·C_avg + 0.1; ω_3 = 0.4·(1-C_max) + 0.1; where: C_avg is the average confusion; C_max is the maximum confusion; ω_1, ω_2, ω_3 are the feature separation operator parameters.

[0088] For example, for the case of C_avg=0.4 and C_max=0.7, we can calculate ω_1=0.8·exp(2.5·0.4)+0.1=2.12, ω_2=0.6·0.7+0.3·0.4+0.1=0.64, and ω_3=0.4·(1-0.7)+0.1=0.22.

[0089] S424. Local Density Adaptive Parameter Calculation: Based on the parameter adjustment model and the local passenger flow density map D_crowd, calculate the feature separation operator parameters at each location (i,j): For each location (i,j), extract the density values ​​within its 5×5 neighborhood. Calculate the average confusion C_avg(i,j) and maximum confusion C_max(i,j) of this neighborhood. Based on the parameter dynamic adjustment model, calculate the feature separation operator parameters ω_1(i,j), ω_2(i,j), and ω_3(i,j) for this location. Generate a parameter spatial distribution map P_dist, containing three channels, each corresponding to one of the three parameters.

[0090] S425. Design of the feature separation operator kernel function: Based on the parameter space distribution map P_dist, design a feature separation operator kernel function D_op(i,j,ρ) = ω_1(i,j)·ρ² + ω_2(i,j)·ρ + ω_3(i,j) for each position (i,j); where: ω_1(i,j), ω_2(i,j), ω_3(i,j) are the feature separation operator parameters at position (i,j); ρ is the local density value; D_op(i,j,ρ) is the output value of the feature separation operator. For example, for the location (960, 540), assuming its parameters are ω_1(960, 540) = 2.12, ω_2(960, 540) = 0.64, ω_3(960, 540) = 0.22, when the density value ρ at this location is 0.7, the feature separation operator output is D_op(960, 540, 0.7) = 2.12·0.7² + 0.64·0.7 + 0.22 = 1.5.

[0091] S426. Feature Separation Parameter Set Generation: Integrate the kernel function set K_func of the separation operator and the parameter space distribution map P_dist to construct a complete feature separation parameter set D_params, containing the separation intensity, direction, and smoothing factor for each position: Separation intensity S_sep(i,j): Represents the intensity of feature separation, proportional to D_op(i,j,ρ). Separation direction Dir_sep(i,j): Represents the direction of feature separation, consistent with the direction of the density gradient. Smooth factor Smooth_sep(i,j): Controls the smoothness of feature separation, inversely proportional to the magnitude of the density gradient. The feature separation parameter set D_params = {S_sep, Dir_sep, Smooth_sep} is used for subsequent feature separation enhancement.

[0092] S43. High-Density Region Feature Enhancement: Apply the feature separation parameter set D_params to the enhanced flow feature map F_flow to enhance feature contrast and suppress background interference in high-density regions: F_density(x,y,c) = F_flow(x,y,c) + S_sep(x,y)·(F_flow(x,y,c) - F_blur(x,y,c)); where: F_flow is the enhanced flow feature map; S_sep is the separation intensity; F_blur is the feature map blurred along the Dir_sep direction using a Gaussian filter; F_density is the density-enhanced feature map. For high-density regions (ρ>0.7), the separation intensity S_sep is larger (>1.5), and the feature contrast is significantly enhanced; for low-density regions (ρ<0.3), the separation intensity is smaller (<0.5), and the features remain basically unchanged.

[0093] S44. Feature Channel Attention Mechanism Implementation: S441. Channel-Density Correlation Analysis: Analyze the correlation between each feature channel and passenger flow density: R_corr(c) = |Corr(F_density(:,:,c), D_crowd)|; where: F_density(:,:,c) is the c-th channel of the density enhancement feature map; D_crowd is the passenger flow density map; Corr() is the Pearson correlation coefficient; R_corr(c) is the correlation value of the c-th channel, with a value range of [0,1].

[0094] S442. Channel Importance Score: Based on the channel correlation vector R_corr, calculate the importance score for each feature channel: I_chan(c) = R_corr(c) + λ·Var(F_density(:,:,c)) / (max_c Var(F_density(:,:,c))); where: R_corr(c) is the correlation value of the c-th channel; Var(F_density(:,:,c)) is the variance of the c-th channel; λ=0.3 is the balance coefficient; I_chan(c) is the importance score of the c-th channel.

[0095] S443. Density-aware channel weight generation: Combining the passenger flow density map D_crowd and the channel importance vector I_chan, a density-aware channel weight calculation function w_j = σ(I_chan(j)·(1+α·D_norm)) is constructed; where: σ(x)=1 / (1+exp(-x)) is the sigmoid activation function; I_chan(j) is the importance score of the j-th channel; D_norm is the normalized passenger flow density; α=1.5 is the adjustment coefficient; and w_j is the weight of the j-th channel.

[0096] For example, for an importance score I_chan(j)=0.8 and a normalized density D_norm=0.6, the channel weight w_j=σ(0.8·(1+1.5·0.6))=σ(1.52)=0.82.

[0097] S444. Spatial Adaptive Attention Map Construction: Based on the spatial distribution of the passenger flow density map D_crowd, generate the attention weights for spatial locations: A_spatial(x,y) = σ(γ·D_crowd(x,y) - δ); where: σ is the sigmoid activation function; D_crowd(x,y) is the passenger flow density at location (x,y); γ=3.0 is the gain coefficient; δ=0.5 is the threshold offset; A_spatial(x,y) is the spatial attention weight at location (x,y).

[0098] S445. Channel Attention Feature Enhancement: Apply the channel weight vector W_chan to each channel of the density enhancement feature map F_density: F_chan_att(x,y,c) = F_density(x,y,c)·w_c; where: F_density is the density enhancement feature map; w_c is the weight of the c-th channel; F_chan_att is the channel enhancement feature map.

[0099] S446. Spatial-channel attention fusion: Integrate the spatial attention map A_spatial and the channel-enhanced feature map F_attention(x, y, c) = F_chan_att(x, y, c)·(1 + A_spatial(x, y)); where: F_chan_att is the channel-enhanced feature map; A_spatial is the spatial attention map; F_attention is the attention-enhanced feature map.

[0100] In this way, the areas with high passenger flow density and the feature channels important for passenger flow detection are strengthened, improving the feature expression ability in crowded environments.

[0101] S45. Sparse computation optimization: S451. Passenger flow density threshold analysis: Determine the demarcation thresholds for high density, medium density, and low density through an adaptive threshold algorithm: Calculate the histogram of the passenger flow density map D_crowd, and statistically analyze the frequency distribution of different density values. Use the Otsu algorithm to determine two optimal thresholds T_low and T_high, and divide the density into three levels. In this embodiment, T_low = 0.35 and T_high = 0.65 are calculated.

[0102] S452. Key area identification: According to the density thresholds T_low and T_high, divide the image area into three categories: High-density key area R_key: D_crowd(x, y) ≥ T_high; Medium-density area R_mid: T_low ≤ D_crowd(x, y) < T_high; Low-density area R_simple: D_crowd(x, y) < T_low. Form a processing area map R_proc, and the value of each pixel represents the area type it belongs to.

[0103] S453. Computation complexity allocation: According to the processing area map R_proc, allocate computing resources and processing complexity to each area: High-density key area R_key: Apply complete feature separation and enhancement processing, with a computation complexity of 100%. Medium-density area R_mid: Apply simplified feature processing, with a computation complexity of 60%. Low-density area R_simple: Apply the lowest-complexity processing, with a computation complexity of 30%. Generate a computation complexity map C_comp, representing the computing resource allocation at each position.

[0104] S454. Sparse Processing Kernel Function Design: Design corresponding feature processing kernel functions for regions with different processing complexities: High complexity region: Use the complete D_op operator D_op_high(i,j,ρ) = ω_1(i,j)·ρ² + ω_2(i,j)·ρ + ω_3(i,j); Medium complexity region: Use the simplified D_op_mid operator D_op_mid(i,j,ρ) = ω_1_mid(i,j)·ρ + ω_3_mid(i,j); where ω_1_mid(i,j) = 0.7·(ω_1(i,j)·ρ_avg + ω_2(i,j)), ω_3_mid(i,j) = ω_3(i,j), and ρ_avg is the average density of the region; Low complexity region: Use the D_op_simple operator D_op_simple(i,j,ρ) = ω_simple·ρ + 0.1; where ω_simple is a fixed value of 0.5; forming a multi-complexity kernel function set K_sparse = {D_op_high, D_op_mid, D_op_simple}.

[0105] S455. Region Adaptive Feature Processing: Read the attention-enhanced feature map F_attention, and apply feature processing of corresponding complexity to different regions based on the processing region map R_proc and the multi-complexity kernel function set K_sparse: F_region(x,y,c) = F_attention(x,y,c)·(1+D_op_high(x,y,D_crowd(x,y))), if (x,y)∈R_key; F_attention(x,y,c)·(1+D_op_mid(x,y,D_crowd(x,y))), if (x,y)∈R_mid; F_attention(x,y,c)·(1+D_op_simple(x,y,D_crowd(x,y))), if (x,y)∈R_simple. Where: F_attention is the attention enhancement feature map; D_op_high, D_op_mid, and D_op_simple are processing kernel functions of different complexities; D_crowd is the passenger flow density map; and F_region is the region processing feature map.

[0106] S456. Boundary Consistency Preservation: Apply smooth transition processing at the boundaries of regions with different processing complexities: F_sparse(x,y,c) = (1-μ(x,y))·F_region(x,y,c) + μ(x,y)·Avg_neighbor(F_region,x,y,c); where: μ(x,y) is the boundary smoothing coefficient, which is larger at the region boundary and 0 inside the region; Avg_neighbor is the neighborhood average; F_sparse is the final optimized feature map.

[0107] Through sparse computation optimization, high-density regions are fully processed, while low-density regions are simplified in computation, reducing the overall computational load by about 45% while maintaining detection accuracy.

[0108] S5. Multi-region Passenger Flow Detection and Localization S51. Multi-scale Target Candidate Box Generation: Based on the optimized feature map F_sparse, an improved region proposal network is used to generate a multi-scale target candidate box set: A Feature Pyramid Network (FPN) is constructed, containing 5 scale levels, corresponding to feature map sizes of {1 / 8, 1 / 16, 1 / 32, 1 / 64, 1 / 128} of the original size. Features are extracted using 3×3 convolutions at each scale level, and then the target location and target score are predicted using two parallel 1×1 convolutions. For each feature map location, three different anchor boxes are preset, generating approximately 10,000 initial candidate boxes. Non-maximum suppression (NMS) is applied, retaining the top 2000 candidate boxes by score, forming a multi-scale target candidate box set B_proposals. Each candidate box contains location (x, y, w, h), score, and level information.

[0109] S52. Optimization of Distortion-Aware Target Representation: S521. Candidate Box Distortion Evaluation: Calculate the degree of distortion affecting each candidate box E_dist(b) = (1 / A_b)·∑∑|D_tensor(x,y)|; where: b is the candidate box index; D_tensor(x,y) is the distortion characteristic tensor at position (x,y) within the candidate box; A_b is the area of ​​the candidate box; E_dist(b) is the distortion evaluation index of candidate box b.

[0110] S522. Representation Selection Strategy: Based on the distortion evaluation index E_dist, select the most suitable representation for each candidate box: When E_dist(b) < 0.1, use a standard rectangle. When 0.1 ≤ E_dist(b) < 0.25, use a rotated rectangle. When E_dist(b) ≥ 0.25, use an ellipse or polygon.

[0111] The representation mapping table F_repr is obtained, with each candidate box corresponding to a representation.

[0112] S523. Rectangular Representation Optimization: For candidate boxes represented as standard rectangles, adjust their position and size parameters: B_rect(b) = [x_b+Δx_b, y_b+Δy_b, w_b·(1+Δw_b), h_b·(1+Δh_b)]; where: [x_b,y_b,w_b,h_b] are the original rectangle parameters; Δx_b,Δy_b,Δw_b,Δh_b are the adjustment amounts calculated based on the distortion characteristic tensor; B_rect(b) are the optimized rectangle parameters.

[0113] S524. Rotation Rectangle Representation Optimization: For candidate boxes represented as rotation rectangles, calculate the principal direction deflection angle θ caused by distortion: θ(b) = atan2(D_tangential_y(b), D_tangential_x(b)); where: D_tangential_x(b) and D_tangential_y(b) are the tangential distortion components at the center of the candidate box; θ(b) is the rotation angle.

[0114] Generate an optimized rotated rectangle with position, size, and rotation angle parameters: B_rot(b) = [x_b+Δx_b, y_b+Δy_b, w_b·(1+Δw_b), h_b·(1+Δh_b), θ(b)]; where: [x_b,y_b,w_b,h_b] are the original rectangle parameters; Δx_b,Δy_b,Δw_b,Δh_b are the position and size adjustments; θ(b) is the rotation angle; and B_rot(b) are the optimized rotated rectangle parameters.

[0115] S525. Elliptical Representation Optimization: For candidate boxes represented as ellipses, calculate the lengths of the major and minor axes and the rotation angle: a(b) = max(w_b·(1+Δw_b), h_b·(1+Δh_b)) / 2 b(b) = min(w_b·(1+Δw_b), h_b·(1+Δh_b)) / 2 θ_e(b) = θ(b) + (π / 2)·I(h_b·(1+Δh_b)>w_b·(1+Δw_b)) Where: w_b, h_b are the original width and height; Δw_b, Δh_b are the size adjustment amounts; a(b), b(b) are the lengths of the major and minor axes, respectively; θ_e(b) is the ellipse rotation angle; I() is an indicator function, which is 1 when the condition in parentheses is true, and 0 otherwise. The parameters for generating the ellipse representation are: B_ellip(b) = [x_b+Δx_b, y_b+Δy_b, a(b), b(b), θ_e(b)].

[0116] S526. Polygon Representation Generation: For candidate bounding boxes in severely distorted regions, multiple control points are generated on the box boundaries through adaptive sampling to form a polygon representation that fits the distorted contour: 12 control points are uniformly sampled on the standard rectangular boundary. A distortion characteristic tensor is applied to each control point for transformation to obtain the deformed control point position. These control points are connected to form a 12-sided polygon representation. The polygon representation parameters are generated as follows: B_poly(b) = [[x1,y1], [x2,y2],..., [x12,y12]], where [xi,yi] are the coordinates of the i-th control point.

[0117] S527. Integration of multiple representations: Integrate the optimized rectangle set B_rect, the optimized rotated box set B_rot, the optimized ellipse set B_ellip, and the optimized polygon set B_poly to form a unified optimized candidate box B_refined. Each candidate box adopts the representation form most suitable for its distortion degree.

[0118] S53. Feature Integration and Target Classification: Extract and integrate the features corresponding to the optimized candidate box B_refined region in the optimized feature map F_sparse: Use the corresponding feature extraction method according to different representation forms: Standard rectangle: use ROI pooling; Rotated rectangle: use rotated ROI pooling; Ellipse and polygon: use mask-guided ROI pooling; Integrate features through fully connected layers to obtain a fixed-dimensional feature vector; Use two parallel fully connected layers to predict the target category (passenger / non-passenger) and bounding box regression parameters respectively to obtain the target score map S_obj; For each optimized candidate box B_refined(b), its target score S_obj(b) represents the probability that the candidate box contains a passenger, with a value range of [0,1].

[0119] S54. Non-maximum suppression and precise target localization: Apply the non-maximum suppression algorithm to the target score map S_obj and the optimized candidate boxes B_refined: Sort all candidate boxes in descending order according to the target score S_obj; select the candidate box with the highest score and add it to the final detection result; calculate the intersection-union ratio (IoU) of the candidate box with all remaining candidate boxes; remove candidate boxes with IoU greater than the threshold of 0.5; repeat the above steps until no candidate boxes remain or the maximum number of detections (100) is reached; obtain the final passenger flow detection result D_result, which includes the precise location, pose, and confidence of each detected passenger.

[0120] In this embodiment, the detection result D_result contains 83 passengers, of which 32 are in the center area of ​​the image, 29 are in the middle area, and 22 are in the edge area.

[0121] S6. Passenger Flow Statistics and Density Analysis S61. Multi-Area Passenger Flow Statistics: Based on the passenger flow detection results D_result, the carriage space is divided into multiple functional areas: Entrance / Exit Area: The area within 1.5 meters at each end of the carriage; Seating Area: The area where fixed seats are located; Standing Area: The open space in the middle of the carriage; Handrail Area: The area near the handrails; The number of passengers detected in each area is counted to obtain the area passenger flow statistics table T_count:

[0122] S62. Temporal Smoothing Processing: Apply the temporal smoothing algorithm T_smooth(r,t) = α·T_count(r,t) + (1-α)·T_smooth(r,t-1) to the regional passenger flow statistics table for multiple consecutive frames; where: T_count(r,t) is the passenger flow statistics of region r at time t; T_smooth(r,t-1) is the smoothed passenger flow statistics of region r at time t-1; α=0.3 is the smoothing coefficient; T_smooth(r,t) is the smoothed passenger flow statistics of region r at time t.

[0123] For example, for the entrance / exit area, assuming the smoothing value of the previous frame is T_smooth(entrance / exit area, t-1) = 16 and the detection value of the current frame is T_count(entrance / exit area, t) = 18, then the smoothed value T_smooth(entrance / exit area, t) = 0.3 × 18 + (1 - 0.3) × 16 = 16.6, which is rounded to 17 people.

[0124] By smoothing the time domain, detection fluctuations are reduced and statistical stability is improved, resulting in a smoothed passenger flow statistics table T_smooth:

[0125] S63. Passenger Flow Density Distribution Analysis: Based on the smoothed passenger flow statistics table T_smooth and the physical dimensions of the carriage space, the passenger flow density distribution of each area is calculated: The carriage space is divided into a 10×5 grid, with each grid cell corresponding to approximately 0.7×0.7 meters of physical space. The number of passengers in each grid cell is counted based on the detected passenger locations. A continuous passenger flow density distribution map D_distribution is generated using bilinear interpolation. The maximum value of the passenger flow density distribution map D_distribution is 4.2 people / m², occurring in the standing area in the middle of the carriage; the minimum value is 0 people / m², occurring in the equipment area at the edge of the carriage.

[0126] S64. Passenger Flow Statistics Report Generation: Integrates the smoothed passenger flow statistics table T_smooth and the passenger flow density distribution map D_distribution to generate a passenger flow statistics report R_report: Numerical Statistics Section: Includes the number, density, and percentage of passengers in each area. Trend Analysis Section: Compares the difference between the current frame and the historical average. Visualization Charts Section: Includes a density heatmap, a regional distribution pie chart, and a time-series change curve. Crowding Assessment Section: Determines the crowding level of each area based on the density values.

[0127] By constructing a radial mapping function, calculating an adaptive receptive field, adjusting the receptive field coefficient, and shifting the deformable convolution kernel, a larger receptive field (approximately 9×9) is used for edge regions, while the central region maintains a standard receptive field (3×3), enabling feature extraction to adapt to scale changes caused by fisheye distortion. This replaces the traditional discrete region segmentation method, using a continuous function to achieve adaptive feature extraction. The receptive field size is dynamically adjusted based on the distance from the pixel to the image center, adapting to feature scale changes in different regions of the fisheye image.

[0128] Spatial adaptive transformation is achieved by using the feature transformation parameter matrix to accurately correct the effects of distortion. This includes distortion characteristic tensor generation, local affine transformation estimation, deformation field interpolation, and feature reconstruction calculation. This enables feature reconstruction that accurately adapts to actual lens distortion, reducing the degree of feature deformation in the edge region by 65.3%.

[0129] Sparse computation optimization was employed, applying processing methods of varying complexity to regions of different densities. These methods included adjusting feature obfuscation levels, feature separation operators, channel weight calculation, feature separation enhancement, and sparse computation optimization. This resolved the feature obfuscation problem in crowded environments, improving detection accuracy in high-density regions by 18.7% while reducing computational load by 45%.

[0130] To verify the effectiveness of the proposed method, its performance was compared with three other methods on the same test dataset: Baseline method: Standard Faster R-CNN directly applied to the original fisheye image. Method A: Traditional workflow of distortion correction followed by detection. Method B: Fisheye target detection method based on region-specific feature extraction.

[0131]

[0132] Performance in crowded scenarios (density > 3 people / m²):

[0133] The comparison results show that the method of this invention achieves the highest detection accuracy in all regions, with significant advantages in edge regions and crowded scenarios. In edge regions, it improves accuracy by 28.4 percentage points compared to the baseline method; in crowded scenarios, the average accuracy is improved by 20.6 percentage points. Meanwhile, the method of this invention maintains lower computational complexity and outperforms methods A and B in real-time performance.

[0134] The multi-region passenger flow detection method for carriages based on fisheye distortion adaptive correction provided in this embodiment effectively solves the problems of inconsistent feature scales in different regions of fisheye images and feature confusion in crowded environments through three core steps: construction of continuous radial mapping function, distortion pattern perception and feature reconstruction, and feature separation enhancement based on density adaptation.

[0135] Compared to existing technologies, this invention has the following advantages: It replaces traditional discrete region partitioning with a continuous radial mapping function, solving the discontinuity problem at region boundaries. Through distortion mode learning and feature transformation parameter matrices, it accurately adapts to actual lens distortion characteristics. It introduces a density-adaptive feature separation operator and a sparse computation optimization strategy, effectively solving the feature confusion problem in congested environments. It significantly improves passenger flow detection accuracy in carriage edge areas while maintaining good real-time performance.

[0136] The method of this invention can be widely applied to passenger flow monitoring systems for public transportation vehicles such as subways, buses, and high-speed trains, providing technical support for intelligent traffic management.

[0137] By constructing a continuous radial mapping function and adaptively extracting features at continuous feature scales, it provides adaptive feature extraction capabilities for regions with different degrees of distortion. It can process the entire image using only one set of parameters, and its computational efficiency is 42.5% higher than that of multi-region independent feature extractors. At the same time, it achieves seamless feature representation.

[0138] Distortion pattern perception and feature reconstruction reduced the degree of feature deformation in edge regions by 65.3%, improved radial distortion correction rate by 33.8%, and improved tangential distortion correction rate by 28.5%. Particularly in non-standard environments (such as tilted camera mounting or lens defects), the edge region detection accuracy was improved by 17.3% compared to traditional methods. The bidirectional feature flow mechanism further enhanced information exchange between different regions, making the reconstructed features more robust and improving feature consistency between regions by 31.2%.

[0139] Through density-adaptive feature separation enhancement, the confusion level C(0.8) = 0.72 at a density value ρ = 0.8 (high density) is significantly higher than C(0.2) = 0.15 at ρ = 0.2 (low density), confirming the nonlinear relationship between density and confusion level. After applying the feature separation operator, the detection accuracy in high-density areas (>3 people / m²) improved by 18.7%, and the false detection rate of adjacent passengers decreased by 23.5%. The channel attention mechanism adaptively adjusts the weights of each channel based on the correlation between feature channels and passenger flow density, further improving the sensitivity to dense passenger flow. Experimental results show that in crowded scenarios (more than 4 people / m²), the false negative rate of this method is reduced by 26.8% and the false positive rate is reduced by 19.3% compared to traditional methods.

[0140] In this embodiment, the system automatically calculates density thresholds T_low=0.35 and T_high=0.65, accurately distinguishing regions with different densities. In the computationally intensive feature separation enhancement stage, sparse computation optimization reduces the overall computational load by 45%, meeting real-time processing requirements (40% increase in frame rate). Smooth boundary transition processing ensures feature continuity between regions of different processing complexities, avoiding feature fragmentation at region boundaries. Experimental results show that this method maintains detection accuracy (average reduction of only 0.5%) while reducing processing speed from 92ms to 78ms, a 15.2% improvement compared to traditional methods, with a more significant advantage on embedded devices with limited computing resources.

[0141] Based on distortion characteristics, parameters for each representation are optimized, such as calculating the optimal angle θ of the rotated rectangle, the major and minor axes of the ellipse, and the positions of the control points of the polygon. Experimental results show that in high-distortion regions (E_dist>0.25), using the optimized elliptical or polygonal representation improves the detection intersection-union ratio (IoU) by 24.3% and the localization accuracy by 19.7% compared to the standard rectangle. Particularly for passenger detection in edge regions, this method significantly reduces false positives caused by bounding box overlap, improving the separation accuracy in multi-target overlapping scenes by 21.5%. In edge regions of crowded environments, the average detection accuracy (AP) of this method reaches 0.773, which is 34.2 percentage points higher than the standard method's 0.431, truly reflecting the actual shape of passengers in fisheye images.

[0142] It should be noted that the various specific technical features described in the above embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, the present invention will not describe the various possible combinations separately.

Claims

1. A method for detecting passenger flow in multiple areas of a train carriage based on adaptive fisheye distortion correction, characterized in that, include: Acquire raw images from a fisheye camera inside the carriage, preprocess them, and generate preprocessed images; A pre-constructed radial mapping function is used to extract features from the preprocessed image to form a preliminary feature map; the preliminary feature map is then transformed using a pre-constructed feature transformation parameter matrix to form a distortion-adaptive feature representation. The passenger flow density distribution of each area of ​​the carriage is extracted from the preprocessed image, and the feature representation adapted to distortion is optimized to obtain an enhanced feature representation. Based on this, passenger flow target detection and precise localization are performed to generate passenger flow detection results containing passenger location information.

2. The method according to claim 1, characterized in that, Generating a preliminary feature map includes: determining the image center point based on the intrinsic parameter matrix and distortion coefficients of the fisheye camera; establishing a radial distance calculation model and constructing a continuous radial mapping function in conjunction with the carriage environment; determining the optimal parameter values ​​by analyzing the characteristics of the carriage fisheye image; generating a radial mapping parameter set; constructing a receptive field deformation convolutional layer based on the radial mapping function and the radial mapping parameter set; calculating the receptive field size at each pixel location to obtain a receptive field distribution map; and using the receptive field distribution map-guided deformation convolution to extract features from the preprocessed image, generating a preliminary feature map that adapts to the feature scale changes of different regions.

3. The method according to claim 2, characterized in that, The construction of a continuous radial mapping function includes: reading the intrinsic parameter matrix and distortion coefficients of the fisheye camera, and calculating the theoretical distortion curve of the standard fisheye lens; selecting multiple representative samples from a pre-stored car body fisheye sample library, extracting the curvature variation law of the straight structure in each sample, and obtaining the actual distortion sampling point set; comparing the theoretical distortion curve with the actual distortion sampling point set, and calculating the difference vector between the two at different radial distances using a radial distance calculation model; setting the initial parameter vector of the radial mapping function and optimizing it in combination with the difference vector to obtain the optimized parameter vector, and constructing the continuous radial mapping function accordingly.

4. The method according to claim 2, characterized in that, The adaptive receptive field includes: deformable convolutional kernels, which determine the sampling position of the convolutional kernel at each pixel based on the receptive field size mapping, and generate the convolutional kernel sampling offset; and adaptive convolutional layers, which apply the convolutional kernel sampling offset to implement adaptive convolution operations at each position, so that the receptive field size mapping directly guides the feature extraction process; wherein, by reading the radial distance field and adjustment coefficient mapping table, the receptive field size of each pixel is calculated to generate the receptive field size mapping of the entire image.

5. The method according to claim 1, characterized in that, The formation of an enhanced flow feature map includes: based on a preprocessed image and a radial mapping parameter set, learning the actual distortion characteristics of the camera through a distortion mode estimation network to obtain a distortion characteristic tensor; calculating the transformation parameter matrix at each location in the feature space based on the distortion characteristic tensor to form a feature transformation parameter matrix set, and performing a spatial adaptive transformation on the preliminary feature map accordingly to achieve distortion correction in the feature space, resulting in a distortion-adapted feature map; and establishing a feature flow channel on the distortion-adapted feature map to allow feature information to flow freely between different scales and regions and to share features, resulting in an enhanced flow feature map.

6. The method according to claim 5, characterized in that, The calculation of the feature transformation parameter matrix includes: calculating the set of distorted registration points affected by distortion based on the distortion characteristic tensor; dividing the feature map space into multiple grid cells, each cell corresponding to a region on the feature map, forming a feature grid; for each grid cell in the feature grid, estimating the local affine transformation matrix based on the standard registration point set and the corresponding distorted registration point set within its region; calculating the transformation parameters for each position on the feature map based on the local affine transformation matrix of the grid cell, forming a deformation field; and integrating the deformation field and the feature distribution information of the preliminary feature map to generate a set of feature transformation parameter matrices.

7. The method according to claim 5, characterized in that, To achieve distortion correction in the feature space through spatial adaptive transformation, the process includes: dividing the feature space into high-transformation, medium-transformation, and low-transformation regions based on the transformation intensity in the feature transformation parameter matrix set, forming a region partitioning map; constructing adaptive sampling features based on the region partitioning map and the feature transformation parameter matrix set, and performing spatial adaptive transformation on the feature transformation parameter matrix to achieve distortion correction, obtaining a reconstructed feature map and smoothing it to obtain an optimized reconstructed feature map; combining the optimized reconstructed feature map with the preliminary feature map through a weighted fusion mechanism to retain effective information, correct the distortion effect, and generate a distortion-adaptive feature map.

Images