Automatic railway fastener tightness degree detection method based on depth estimation

Abstract

The invention discloses a depth estimation-based automatic railway fastener tightness detection method, which comprises the following steps of: acquiring a fastener region gray scale and a sub-pixel depth map by using a binocular line laser fusion sensor, and generating a reference three-dimensional point cloud with an absolute scale through temperature, vibration and IMU (Inertial Measurement Unit) joint correction; fitting a dynamic rail top reference surface by using a rail head web to extract an overall sinking and floating drift distance; performing lightweight instance segmentation to obtain elastic strip, bolt and rail head masks and slicing point clouds; fitting a quadric surface to extract feature points, calculating relative elevation, and performing drift compensation to obtain self-deformation quantity of the fastener; and finally, inputting the deformation quantity into a physical-residual cascade model to judge the tightness state, and binding the deformation quantity with a mileage stake mark to output a detection result. According to the invention, the real relative looseness of the fastener can be separated from the rail reference drift, so that the accuracy and reliability of tightness detection are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of railway infrastructure detection, in particular to an automatic detection method for railway fastener tightness based on depth estimation. BACKGROUND

[0002] As a key component of track structure, the fastener system is responsible for firmly fixing the steel rail on the sleeper and providing necessary elastic buffer. The tightness of the elastic strip and bolt directly determines the size of the clamping pressure. Once the actual pre-tightening force drops and is not found in time, it is easy to induce rail creep, gauge change and even train derailment. Therefore, online monitoring of the fastener tightness has always been the core demand of the railway operation and maintenance department. At present, the field still mainly relies on manual inspection or torque wrench sampling and retightening, which is low in efficiency, long in sky window occupation time and high in missed detection rate. In recent years, with the development of machine vision, a fastener missing and breaking recognition method based on two-dimensional images has appeared, but the tightness belongs to millimeter-level three-dimensional deformation, which cannot be quantified by RGB images only. Therefore, the patent application with publication number CN118711015A proposes: using a monocular camera to shoot a two-dimensional image of the track, outputting an absolute depth map through a ZoeDepth derivative network, and then using YOLOv8 to segment the elastic strip and bolt area, and finally taking the difference DWAD between the average depth dc of the elastic strip and the average depth db of the bolt as the tightness criterion. This method can realize three-dimensional measurement without laser radar, and has the advantages of low hardware cost and convenient deployment.

[0003] However, it is found in line tests that the above-mentioned scheme has systematic misjudgment caused by reference drift: when the track bed is frozen and expanded, the sleeper is empty and lifted, or the overall settlement is caused by temperature stress, the whole set of fasteners will sink or rise together with the sleeper by 0.5-2mm. At this time, dc and db increase or decrease together, DWAD can remain unchanged, but the actual elastic strip pre-tightening force has decreased or even failed; on the contrary, if the overall track is lifted, DWAD may exceed the threshold and be misjudged as loose. Since the patent document does not test the rigidity assumption of the fastener-sleeper-track bed, nor does it introduce the track head elevation or the relative constraint of adjacent fasteners, it cannot distinguish between fastener loosening and overall track deformation, resulting in uncontrollable false negative and false positive rates, which seriously limits the popularization and application of this method on short sky window and long line of ordinary speed and high speed railways. Therefore, a new depth estimation scheme is needed to compensate for the overall deformation of the track online, which can separate the real relative loosening of the fastener from the track reference drift while retaining the low-cost advantage of the monocular camera, thereby significantly improving the accuracy and reliability of the tightness detection. SUMMARY

[0004] In order to solve the above technical problems, the present application provides an automatic detection method for railway fastener tightness based on depth estimation.

[0005] To achieve the above object, the technical scheme adopted by the present application is as follows:

[0006] The present application discloses an automatic detection method for railway fastener tightness based on depth estimation, comprising the following steps:

[0007] Step 1: The binocular line laser fusion sensor fixed to the trolley synchronously collects the fastener region gray scale image and sub-pixel depth image, and generates the reference three-dimensional point cloud with absolute scale after temperature, vibration and IMU joint correction;

[0008] Step 2: The dynamic rail top reference surface is fitted in the reference three-dimensional point cloud with the rail head web as the reference, and the overall sinking and floating drift of the fastener system is extracted;

[0009] Step 3: The gray scale image is input into the lightweight instance segmentation network to obtain the elastic strip mask, bolt mask and rail head mask, and the elastic strip slice point cloud, bolt slice point cloud and rail head slice point cloud are obtained by masking the three-dimensional point cloud;

[0010] Step 4: The elastic strip slice point cloud and the bolt slice point cloud are respectively fitted with a quadratic surface, the elastic strip feature points and the bolt feature points are extracted, and the elastic strip relative height and the bolt relative height are calculated relative to the average height of the rail head slice point cloud;

[0011] Step 5: The difference between the elastic strip relative height and the bolt relative height is obtained to obtain the fastener relative deformation difference, and the overall sinking and floating drift is compensated to generate the fastener self deformation;

[0012] Step 6: The fastener self deformation is input into the physical residual cascade discriminant model to output the fastener tightness category, and the fastener tightness detection result is formed by binding with the milepost number.

[0013] Further, the step 1 comprises:

[0014] The raw gray scale image and the raw sub-pixel depth image output by the binocular line laser fusion sensor rigidly fixed to the rail inspection trolley beam, the IMU six-degree-of-freedom acceleration and angular velocity sequence, the temperature sensor ambient temperature value, and the vibration sensor high-frequency amplitude signal are input into a temperature vibration IMU joint correction module. The temperature sensor ambient temperature value is used to compensate the raw sub-pixel depth image at the pixel level to eliminate the systematic depth deviation caused by the change of the wavelength of the laser with temperature. The IMU six-degree-of-freedom acceleration and angular velocity sequence is used to estimate the instantaneous attitude change of the rail inspection trolley beam in the spatial coordinate system through a Kalman filter and convert the instantaneous attitude change into a three-dimensional attitude correction vector of each pixel of the raw sub-pixel depth image. The vibration sensor high-frequency amplitude signal is used to extract the vibration component consistent with the inherent frequency of the rail inspection trolley through fast Fourier transform and perform synchronous inverse superposition compensation on the depth value of each pixel of the raw sub-pixel depth image. Finally, the sub-pixel depth image compensated by temperature drift, attitude correction, and vibration inverse superposition is fused with the raw gray scale image at the pixel level to generate a reference three-dimensional point cloud with an absolute millimeter-level scale, and the national height datum of the starting positioning point of the rail inspection trolley is taken as the origin to assign an absolute height to the whole coordinate system of the reference three-dimensional point cloud.

[0015] Further, the step 2 comprises:

[0016] In the reference three-dimensional point cloud, a local point cloud cluster is intercepted along the train running direction with a 0.5m sliding window. RANSAC plane fitting is performed on each local point cloud cluster to identify a rail head web point cloud subset, and the normal vector of the rail head web point cloud subset is taken as a rigid reference. A dynamic rail top reference plane equation that changes continuously with the mileage is obtained by weighted least squares fitting, wherein the weight is determined by the reciprocal of the distance from the rail head web point cloud subset to the fitted plane.

[0017] The Z-direction offset of the dynamic rail top reference plane equation at the current sliding window center mileage is subtracted from the Z-direction offset of the dynamic rail top reference plane equation at the previous sliding window center mileage to obtain the overall sinking and floating drift of the fastener system, and the overall sinking and floating drift of the fastener system is written into a drift amount lookup table corresponding to the mileage pile number with a resolution of 0.1mm, so that the overall sinking and floating drift of the fastener system can be excluded within an accuracy of 0.05mm when calculating the subsequent fastener self-deformation amount through table lookup.

[0018] Further, the step 3 comprises:

[0019] Input the gray image into the lightweight instance segmentation network completed with knowledge distillation, which takes MobileNet-V3 as the backbone, SOLOv2 as the instance segmentation head, and uses trackside sun shadow, ballast occlusion, and rust noise as augmented samples in the training stage, so that the network outputs elastic strip mask, bolt mask, and rail head mask corresponding to the pixels of the gray image;

[0020] Perform 8-connected domain noise removal and morphological closure on the elastic strip mask, bolt mask, and rail head mask respectively to obtain the elastic strip mask, bolt mask, and rail head mask with single-pixel closed edges;

[0021] Using the elastic strip mask, bolt mask, and rail head mask as Boolean indexes, perform mask-driven slicing on the reference three-dimensional point cloud to obtain the elastic strip slice point cloud, bolt slice point cloud, and rail head slice point cloud, which only retain the elastic strip point set, bolt point set, and rail head point set respectively. Write the elastic strip slice point cloud, bolt slice point cloud, and rail head slice point cloud in PLY format into memory for subsequent quadratic surface fitting calls.

[0022] Further, the step 4 includes:

[0023] Perform RANSAC-based quadratic surface fitting on the elastic strip slice point cloud to extract the three-dimensional coordinates of the elastic strip root arc center as the elastic strip feature point using the quadratic surface equation as the model;

[0024] Perform quadratic surface fitting with the same parameters on the bolt slice point cloud to extract the three-dimensional coordinates of the bolt top surface center as the bolt feature point;

[0025] Calculate the arithmetic mean of the Z coordinates of all points in the rail head slice point cloud to obtain the average elevation of the rail head slice point cloud;

[0026] Take the average elevation of the rail head slice point cloud as the reference to calculate the difference between the Z coordinate of the elastic strip feature point and the average elevation of the rail head slice point cloud as the elastic strip relative elevation, and calculate the difference between the Z coordinate of the bolt feature point and the average elevation of the rail head slice point cloud as the bolt relative elevation.

[0027] Further, the step 5 includes:

[0028] Perform algebraic difference operation on the elastic strip relative elevation and the bolt relative elevation at the same mileage section to obtain the fastener relative deformation difference, and maintain the operation precision as floating-point double bytes;

[0029] Read the overall sinking and floating drift of the fastener system corresponding to the current mileage from the drift lookup table, and perform vector subtraction compensation on the fastener relative deformation difference using the overall sinking and floating drift of the fastener system;

[0030] A sign bit determination is performed on the fastener self deformation amount, and when the fastener self deformation amount is greater than or equal to zero, the original value is kept, and when the fastener self deformation amount is less than zero, it is forced to be zero, so as to eliminate the physically meaningless negative value;

[0031] The fastener self deformation amount after the sign determination is written into the real-time memory queue, and is converted into a 16-bit unsigned integer with a quantization step of 0.01 mm, for reading by the physical residual cascade discrimination model.

[0032] Further, the step 6 comprises:

[0033] The fastener self deformation amount after quantization is input into the physical residual cascade discrimination model, which is sequentially cascaded by a front-end physically interpretable layer and a rear-end residual network, wherein the physically interpretable layer is built-in with a spring strip bolt stiffness matrix, the stiffness matrix takes the fastener self deformation amount as input and outputs an initial class vector with a spring strip root theoretical displacement of 0.2 mm as a threshold value, when the initial class vector is greater than or equal to 0.2 mm, an initial loose label is assigned, and when the initial class vector is less than 0.2 mm, a normal initial label is assigned;

[0034] The rear-end residual network takes DenseNet-40 as the backbone and takes a double-channel tensor obtained by splicing the initial class vector and the fastener self deformation amount as input, extracts high-order error features through 4 dense blocks and 1 global average pooling layer, and outputs a correction score for the initial label;

[0035] The initial label is hard-voted according to the correction score, if the correction score is greater than zero, the initial label is kept, and if the correction score is less than zero, the initial label is reversed, to obtain a fastener tightness class containing only normal or loose;

[0036] The fastener tightness class is bound with the synchronously recorded mileage post number in a UTF-8 string, written into a SQLite database, and a fastener tightness detection result is formed.

[0037] The present application can effectively solve the technical problems of the existing monocular depth estimation method, such as the reference drift misjudgment caused by the overall sinking and floating of the track, the difficulty in distinguishing the fastener self loosening and the track deformation, and the insufficient detection accuracy, and has the following beneficial effects:

[0038] The present application can effectively eliminate the reference drift misjudgment caused by the overall sinking and floating of the track, and realize accurate identification of the fastener self loosening state. By establishing a dynamic track top reference surface and introducing overall drift compensation, the stability and reliability of the detection are significantly improved; by using binocular line laser fusion depth perception and sub-millimeter level surface fitting, the measurement accuracy of the fastener deformation amount is improved; by combining the physical residual cascade discrimination model, high-precision intelligent judgment of the fastener tightness state is realized. In summary, the present application can separate the real fastener relative loosening from the track reference drift, thereby significantly improving the accuracy and reliability of the tightness detection. Attached Figure Description

[0039] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0040] In the attached diagram:

[0041] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0042] The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of the present invention will now be described with reference to the accompanying drawings.

[0043] like Figure 1 As shown, this invention discloses an automatic detection method for the tightness of railway fasteners based on depth estimation, comprising:

[0044] Step 1: The binocular laser fusion sensor fixed to the trolley synchronously acquires grayscale images and subpixel depth images of the fastener area. After joint correction by temperature, vibration and IMU, a reference 3D point cloud with absolute scale is generated.

[0045] Step 2: Fit a dynamic rail top reference surface within the reference 3D point cloud, using the rail head web as a reference, and extract the overall heave and drift of the fastener system;

[0046] Step 3: Input the grayscale image into the lightweight instance segmentation network to obtain the elastic bar mask, bolt mask, and rail head mask, and slice the reference 3D point cloud from the mask to obtain the elastic bar slice point cloud, bolt slice point cloud, and rail head slice point cloud;

[0047] Step 4: Fit quadratic surfaces to the point cloud of the elastic bar slice and the point cloud of the bolt slice respectively, extract the feature points of the elastic bar and the bolt, and calculate the relative elevation of the elastic bar and the relative elevation of the bolt relative to the average elevation of the rail head slice point cloud;

[0048] Step 5: Subtract the relative elevation of the elastic clip from the relative elevation of the bolt to obtain the relative deformation difference of the fastener, and then compensate for it with the overall sinking and floating displacement to generate the deformation of the fastener itself;

[0049] Step 6: Input the deformation of the fastener itself into the physical residual cascade discrimination model, output the fastener tightness category, and bind it with the mileage station number to form the fastener tightness detection result.

[0050] Specifically, step 1 includes:

[0051] The binocular laser fusion sensor is rigidly connected to the crossbeam of the track inspection trolley and outputs data in parallel according to a unified time base (hard synchronization signal).

[0052] original grayscale image (pixel coordinates );

[0053] original sub-pixel depth image (unit: mm, including system bias and noise) camera-laser coordinate extrinsic matrix determined by rigid installation relationship and one-time geometric calibration in the system integration stage of the binocular-laser fusion sensor ;

[0054] collect at the same time:

[0055] IMU six-degree-of-freedom sequence ;

[0056] temperature sensor ambient temperature value ;

[0057] vibration sensor high-frequency amplitude .

[0058] These signals are sent to the temperature vibration IMU joint correction module, compensated through three stages (temperature drift compensation→ attitude correction→ vibration reverse superposition compensation) and fused with the grayscale image pixel by pixel, and finally the pixel-level depth is converted into three-dimensional coordinates by using the camera intrinsic parameter, to obtain the benchmark three-dimensional point cloud with absolute millimeter level scale and taking the starting positioning point of the track inspection trolley as the origin of the scale .

[0059] Specifically:

[0060] First, pixel-level temperature drift compensation, the goal is to eliminate the systematic depth offset of the laser / depth sensor caused by the change of the ambient temperature.

[0061] Let the depth measured by the sensor at the reference temperature be , the model expresses the influence of temperature on depth in a linear / second-order approximation at the pixel level:

[0062]

[0063] and take

[0064]

[0065] where , is the pixel-level temperature-sensitive coefficient (stored in the form of a lookup table or a parameterized model inside the module), is the temperature sensor value incorporating the timestamp, this compensation is a pixel-by-pixel additive correction, and the output is the temperature-compensated depth map .

[0066] Second, IMU-based instantaneous pose estimation and pixel-level pose correction, which aims to eliminate the pixel depth geometric distortion caused by the instantaneous pose changes (pitch, roll, yaw) of the track inspection trolley beam, so that the depth in the vehicle reference coordinate system is consistent with the spatial coordinates.

[0067] Using the acceleration and angular velocity sequence of the IMU, an extended / universal Kalman filter or an inertial navigation filter is used to output the beam instantaneous pose quaternion / rotation matrix at a fixed frequency (synchronized with the depth map frame rate) and translation rate estimation (focusing on the pose, and the translation is constrained by the odometer / mileage pulse or additional positioning system). In this step, we only use the attitude change (relative to the sensor's transformation at the reference time) to calculate the correction.

[0068] Using the camera-laser joint intrinsic parameters (focal length , , principal point , ), each pixel depth is back-projected to a three-dimensional point in the sensor coordinate system:

[0069]

[0070] The instantaneous pose relative to the reference (such as the middle time of the frame or the sensor installation zero position) is , the translation is , and the pixel corresponding three-dimensional point under the reference attitude should be:

[0071]

[0072] Therefore, the pixel-level three-dimensional pose correction vector (difference vector with the original pixel point as the starting point) is

[0073]

[0074] Inside the module, the Kalman filter outputs , the above pixel-by-pixel calculation and conversion to equivalent depth correction (along the optical axis component) for subsequent depth correction.

[0075] Third, vibration anti-phase superposition compensation (frequency domain processing), which aims to suppress high-frequency depth jitter caused by the inherent frequency of the trolley or road periodic excitation, and reduce periodic depth errors caused by mechanical vibration.

[0076] Using the vibration sensor sampling signal Short-time Fast Fourier Transform (STFT / FFT) is performed to identify the significant spectral component consistent with the inherent frequency of the rail inspection trolley (or its close neighbors) Meanwhile, IMU / acceleration signals can also be incorporated into the frequency domain analysis to improve the robustness of the resonance component identification.

[0077] Assuming that this vibration component produces an approximately periodic offset in the depth measurement , the phase of which is consistent with the vibration sensor measurement This can be estimated by cross-correlation, and the inverse phase compensation is defined as:

[0078]

[0079] where is the pixel response coefficient (reflecting the effect of geometric magnification / reduction on different pixels), This term is calculated in the time domain in parallel with the temperature / attitude correction and is accumulated pixel by pixel as the vibration compensation.

[0080] Again, the three joint corrections are fused at the pixel level, and the final corrected depth of each pixel at time (t) is:

[0081]

[0082] where: is the temperature compensation, is the attitude correction along the optical axis component, is the vibration inverse phase compensation.

[0083] Each term here is calculated pixel by pixel to ensure the one-to-one correspondence of pixel-level fusion requirements; all intermediate variables are time-stamped to ensure strict synchronization with the grayscale image

[0084] Finally, from the corrected depth map to the reference three-dimensional point cloud, the is back-projected into the three-dimensional points of the sensor reference system to obtain the point set in the sensor coordinates . Then, the sensor coordinate system is converted to the global coordinate system of the rail inspection trolley (rigid installation transformation ), and the national elevation datum of the starting positioning point of the rail inspection trolley is taken as the origin to assign absolute elevation:

[0085]

[0086] where represents the rigid transformation matrix from the global coordinate system of the rail inspection trolley to the national / line global coordinate system;

[0087] where the absolute elevation offset is performed on the component of all points:​

[0088]

[0089] wherein represents the relative elevation component of the three-dimensional point in the global coordinate system of the track inspection trolley, represents the national elevation datum value corresponding to the starting positioning point of the track inspection trolley

[0090] the final output of the reference three-dimensional point cloud represents:

[0091]

[0092] with a unit of mm, the point cloud satisfies: one-to-one correspondence between pixels and three-dimensional points, absolute scale and millimeter-level precision traceability (guaranteed by temperature / attitude / vibration compensation chain and vehicle starting national elevation datum), and serves as the only scale traceability reference for subsequent steps (track top reference surface fitting, mask slicing, quadratic surface fitting, etc.).

[0093] In summary, through the three parts of pixel-level temperature drift compensation, IMU-Kalman attitude estimation and mapping to a pixel-level correction vector, and vibration frequency domain inverse compensation, the systematic deviation and dynamic interference in the depth map are eliminated; the corrected depth is projected back through the camera intrinsic parameters and the national elevation datum of the starting positioning point of the track inspection trolley in the vehicle global coordinate system to complete the absolute elevation assignment, and the output is a reference three-dimensional point cloud with absolute scale, which provides a unique scale traceability and spatiotemporal consistent data basis for subsequent steps (dynamic track top reference surface fitting, mask slicing, quadratic surface fitting, etc.).

[0094] Specifically, step 2 includes:

[0095] In the reference three-dimensional point cloud , a local point cloud cluster is intercepted along the train running direction with a fixed length of 0.5m sliding window. For each local point cloud cluster, first use RANSAC plane fitting to identify and extract the rail head web point cloud subset, and use the normal vector of the subset as the rigid reference direction (local rigid reference of the track), and then use weighted least squares method to fit to get the dynamic track top reference surface equation which changes with mileage. The Z-direction offset difference of the track top reference surface at the center of adjacent sliding windows is taken as the overall heave drift of the fastener system at this mileage. The drift is written into a drift lookup table corresponding to the mileage stake number with a resolution of 0.1mm, and the 0.05mm accuracy in subsequent compensation is ensured through interpolation and quantization strategy.

[0096] 1. Definition of sliding window and local point cloud interception

[0097] The train running direction is taken as The axis (in the global coordinate system output from step 1) is relative to the reference 3D point cloud. Continuous sampling by mileage. The length is defined as... A sliding window, with the center of the window centered at a step size (s) (usually less than or equal to 1). It is moving along the entire line.

[0098] For the center mileage The Use a window to capture a local point cloud cluster:

[0099]

[0100] 2. RANSAC plane fitting to identify a subset of the railhead web point cloud.

[0101] The aim is to robustly identify the set of points belonging to the rail head and web (i.e., a subset that is normally stable and has good planar approximation) in a local point cloud containing complex echoes from the rail head, rail head web, ballast, fasteners, etc.

[0102] right Using the RANSAC planar model (planar implicit form) Iterative sampling is performed, with the number of iterations and threshold configured as required (e.g., 200 iterations and a distance threshold on the order of 0.05 mm for reference), to identify the set of interior points. (RANSAC inliers).

[0103] Will This serves as an initial candidate set for the point cloud subset of the railhead web. To ensure geometric semantic consistency, a further constraint is imposed: the normal vector distribution of the interior point set must form an angle with the prior railhead normal direction that is less than a threshold. (e.g., 5°); if the threshold is exceeded, the window is marked as unreliable and subsequent fitting is skipped (or an extension check is triggered).

[0104] Record the initial normal vector of this subset. (Normalized) and used as a rigid reference direction. The RANSAC output also returns the vertical residual from each point to the fitted plane. .

[0105] 3. Weighted least squares fitting based on a subset of the rail head web (to obtain the dynamic rail top reference surface)

[0106] Objective: In Under the constraints, the local orbital top datum is precisely fitted using the weighted least squares method, making the fitting more robust to noise and local outliers, and ensuring continuity with mileage. An explicit functional form in local coordinates is adopted (so that the data can be directly obtained). Offset):

[0107]

[0108] where the parameter vector is described in the first window.

[0109] The weighted least squares is:

[0110]

[0111] The weight is determined by the reciprocal of the distance from the subset of railhead web point cloud to the fitted plane, so in the iterative implementation we have:

[0112]

[0113] where is the absolute vertical residual of point to the initial fitted plane (RANSAC output or the residual of last iteration), is a small constant to prevent division by zero (e.g. ).

[0114] Matrix solution (closed form):

[0115] Let

[0116]

[0117] Then

[0118]

[0119] After solving, the parameter is converted to the plane normal and is consistent with is checked; if the normal deviation is beyond the threshold (indicating that the fitting is unstable), the window is marked as abnormal and is interpolated / smoothed by the neighboring windows.

[0120] 4. Continuity and smoothness of the dynamic rail top reference surface

[0121] To ensure the continuity along the mileage and suppress the influence of measurement noise on the extraction of drift, the parameter sequence obtained along the mileage is one-dimensional smoothed (weighted moving average or one-dimensional Kalman filter can be used). After smoothing, the reference surface parameter at the mileage is obtained, which represents the local dynamic rail top reference surface equation .

[0122] It should be noted that smoothing only acts on the parameter sequence level and does not change the principle of fitting weight within a single window; when there are too many abnormal neighboring windows, weighted interpolation or longer baseline window recalculation can be used to restore continuity. ​

[0123] 5. Calculation of the total float of the fastener system

[0124] Definition: At the center of the sliding window, the height offset of the dynamic track top reference surface from the nominal height is denoted by the parameter

[0125]

[0126]

[0127]

[0128] The sign convention is: positive value indicates relative uplift, negative value indicates relative settlement.

[0129] To avoid ringing caused by high-frequency noise, a small-bandwidth low-pass filter (e.g. based on first-order IIR or small-window weighted average) can be applied after calculation, but the filter design must ensure sufficient response to sudden changes (such as sudden settlement of the track bed). 6. Construction of the drift amount lookup table and accuracy guarantee

[0130] The center of each sliding window is paired with the corresponding drift amount to form a key-value pair, which is written into the drift amount lookup table (indexed by milepost number or continuous meter-level coordinate):

[0131]

[0132]

[0133] , which is quantized and stored at a resolution of 0.1mm (i.e. rounding to the nearest 0.1mm step).

[0134] When making compensation for the deformation of the fastener itself later, the final compensation error can be guaranteed not to exceed 0.05mm by looking up the table and combining linear interpolation, the implementation method is as follows:

[0135] ​​​​​​​​​​​​​​​​​​Table resolution is 0.1mm (to meet storage and transmission efficiency);

[0136] When reading, the quantized value of the target mileage is linearly interpolated from the quantized values of the two adjacent centers , in the table. The upper bound of the quantization error of the interpolation result is 0.05mm, thereby ensuring the compensation accuracy:

[0137]

[0138] wherein represents a 0.1mm quantization function, is the linear weight of the mileage between the two centers. Linear interpolation limits the maximum error caused by quantization to half the step size, 0.1 / 2=0.05mm.

[0139] The table entry also carries a confidence or uncertainty field (for example ), so that the downstream compensation can decide whether to adopt or trigger redundant detection according to the uncertainty.

[0140] To ensure that the rigidity assumption with the rail head web as the reference is established within the local window, the following consistency / quality criterion needs to be performed for each window:

[0141] If the normal angle of adjacent windows exceeds the threshold , it is considered that the rail cross-section has a sudden change or the fitting is unreliable, and the window is marked as abnormal and re-fitted by neighborhood interpolation or extended window;

[0142] If the standard deviation of the weighted fitting residual exceeds the threshold (indicating that the point cloud is strongly polluted by ballast or fasteners), the abnormal points are down-weighted or removed and then re-fitted, and if necessary, the window length is expanded;

[0143] The number of effective rail head web points used for fitting within the window should not be less than (for example, 50 points), otherwise the adjacent window is discarded or merged;

[0144] If consecutive windows are marked as abnormal, the system should trigger a fitting strategy with a longer baseline to reconstruct the dynamic reference surface.

[0145] 7. Result output format and interface agreement

[0146] Drift amount lookup table: the table structure is (mileage , quantized drift , uncertainty , normal consistency flag, timestamp). Storage unit: mm, quantization step 0.1mm, UTF-8 encoding bound with mileage string.

[0147] Read and interpolate by request mileage in subsequent step call And in the compensation formula, the whole amount of heave drift is removed with 0.05mm precision.

[0148] Compensation formula:

[0149]

[0150] 8. Uncertainty propagation

[0151] The parameter covariance obtained by single window fitting can be derived from the weighted least squares normal equation:

[0152]

[0153] The standard deviation of the difference can be further obtained from the covariance , and the difference is propagated by the difference to obtain . The table records for downstream criterion use. Interpolation and quantization introduce a maximum system error of 0.05mm (quantization half step), which together with the measurement uncertainty constitutes the final compensation uncertainty evaluation.

[0154] In summary, on the reference three-dimensional point cloud, a local point cloud is intercepted with a 0.5m sliding window, and the RANSAC is used to identify and extract the rail head web point cloud subset, and then the normal of the subset is taken as the local rigid reference, and the weighted least squares fitting is used to obtain the window level dynamic rail top reference surface parameter . The offset difference of the reference surface in the direction at the center of the adjacent window is taken as the overall heave drift of the fastener system, and is written into the drift amount lookup table with 0.1mm resolution; through linear interpolation, the final error is ensured to be not more than 0.05mm when looking up and compensating. In order to ensure robustness, normal stability, residual distribution, minimum point number and other criteria are introduced; for abnormal windows, neighborhood interpolation or long baseline re-fitting strategy is adopted. The output drift amount lookup table contains uncertainty and quality flag, which is the only dimensional traceability data source for removing the overall heave drift of the track in subsequent steps.

[0155] Specifically, step 3 includes:

[0156] 1. Input and pre-data

[0157] Step 3 takes the output of steps 1-2:

[0158] Reference three-dimensional point cloud , which is corrected and aligned to the dynamic rail top reference surface coordinate system by step 2; the gray image synchronously collected, through the camera-laser coordinate external parameter matrix Realize the spatial mapping of image pixels and point cloud points; Dynamic rail top reference provides local reference, making mask segmentation focus only on the area above the rail top and fastener, reducing ballast interference.

[0159] 2. Grayscale image input lightweight instance segmentation network

[0160] Network structure: Lightweight instance segmentation network takes MobileNet-V3 as the backbone and SOLOv2 as the instance segmentation head.

[0161] Knowledge distillation: In the training stage, the student network is distilled by the teacher model (such as ResNet-101+SOLOv2), which retains the semantic segmentation ability while controlling the inference delay within 15ms.

[0162] Data augmentation: Training samples include trackside sunlight shadows, ballast local occlusion, metal rust noise, etc. to improve robustness.

[0163] Network output:

[0164]

[0165] Among them:

[0166] : Elastic strip mask;

[0167] : Bolt mask;

[0168] : Rail head mask;

[0169] , : Image height and width.

[0170] 3. Mask post-processing: denoising and morphological closing

[0171] To ensure that the mask boundary is closed and morphologically consistent, perform the following operations on each mask ( ):

[0172] 8-Connected domain denoising: Remove isolated noise connected domains with pixel area less than the threshold (typically less than 1% of the total area):

[0173]

[0174] Morphological closing operation: The structure element takes a radius (2-3px), eliminates edge notches and forms a single-pixel closed loop boundary:

[0175]

[0176] where, are dilation and erosion operators respectively.

[0177] After processing, a set of masks with continuous edges and no isolated noise is obtained:

[0178]

[0179] 4. Mask-driven reference 3D point cloud slices

[0180] According to the extrinsic matrix obtained in step 1 (from the laser coordinate system to the camera coordinate system ) and the camera intrinsic matrix , the pixel projection mapping of the point cloud and the image is established:

[0181]

[0182] where is the pixel coordinate of the point cloud point in the grayscale image.

[0183] The mask slice is realized by this indexing relationship:

[0184] If the projected pixel of the point satisfies , the point is retained into the slice point cloud of category .

[0185] Formal definition:

[0186]

[0187] 5. Slice point cloud generation and formatted storage

[0188] Three categories of mask-driven slice point clouds are obtained:

[0189] Bullet slice point cloud : containing only bullet surface points;

[0190] Bolt slice point cloud : containing only bolt top and peripheral points;

[0191] Rail head slice point cloud : containing only rail head metal surface points;

[0192] Each category of point cloud is re-aligned and cropped in the dynamic rail top reference plane coordinate system (fitted by step 2);

[0193] ​Write into the memory buffer area in PLY format (including three-dimensional coordinates, reflectivity, mask category label), for subsequent S4 quadratic surface fitting and deformation variable estimation call.

[0194] In summary, the MobileNet-V3+SOLOv2 network optimized by knowledge distillation is used for instance segmentation of grayscale images, generating high-robustness elastic strip, bolt, and rail head masks; after 8-connected domain denoising and morphological closing processing, the mask boundary is smooth and closed; using the camera-laser coordinate correspondence established in step 1, the mask-driven slicing is performed on the reference three-dimensional point cloud output in step 2; output three types of sliced point clouds 、 、 in PLY format for subsequent surface fitting and fastener tightness calculation in step 4; the whole process is completed within 0.05mm spatial accuracy and 30ms real-time, ensuring traceability and accuracy stability of subsequent calculations.

[0195] Specifically, step 4 includes:

[0196] This step realizes the extraction of quantifiable geometric feature parameters (elastic strip feature points, bolt feature points, and rail head average elevation) from the three-dimensional point cloud after mask slicing, providing high-precision input for subsequent fastener deformation variable calculation.

[0197] 1. Input data and spatial reference

[0198] The input data of step 4 comes from:

[0199] Elastic strip sliced point cloud

[0200] Bolt sliced point cloud

[0201] Rail head sliced point cloud

[0202] The three types of point clouds are in the dynamic rail top reference plane coordinate system defined in step 2, with absolute millimeter-level elevation meaning in the coordinate (Z), which has been aligned with the starting elevation reference of the rail inspection trolley.

[0203] 2. Definition of quadratic surface fitting model

[0204] To obtain the local morphological features of the elastic strip and bolt, RANSAC (Random Sample Consensus) quadratic surface fitting is performed on and respectively, and the quadratic surface mathematical model is as follows:

[0205]

[0206] where the parameter vector is:

[0207]

[0208] Model physical meaning:

[0209] Coefficient , , : describes the surface curvature;

[0210] Coefficient , : describes the surface tilt direction;

[0211] Coefficient : represents the overall height offset.

[0212] 3. RANSAC fitting process

[0213] (1) Set parameters

[0214] Number of iterations ;

[0215] Inlier distance threshold .

[0216] (2) Algorithm steps

[0217] Randomly select the minimum sample set from ( Indicates clip or bolt);

[0218] Solve the least squares equation To get the initial fitting coefficient;

[0219] Calculate the perpendicular distance from all points to the fitted surface:

[0220]

[0221] Determine the inlier set ;

[0222] Iterative update until the number of inliers is maximized;

[0223] Re-solve the least squares with the final inlier set .

[0224] 4. Feature point extraction

[0225] (1) Elastic strip feature point (elastic strip root arc center)

[0226] The quadratic surface fitted by the elastic strip slice point cloud usually presents a concave arc structure.

[0227] Obtain the local extreme point coordinates by taking the partial derivative of the surface to zero:

[0228]

[0229] The coordinates of the characteristic points of the spring strip in the xy plane are obtained:

[0230]

[0231] The characteristic point elevation is obtained by substituting the fitting equation:

[0232]

[0233] The three-dimensional coordinates of the characteristic points of the spring strip are obtained:

[0234]

[0235] (2) Bolt characteristic points (center of bolt top surface)

[0236] The bolt slice point cloud usually has a convex flat top feature, and the extreme point of the quadratic surface is taken as the center coordinate:

[0237]

[0238] The solving method is the same as that of the spring strip characteristic points, only the corresponding bolt fitting surface parameters.

[0239] 5. Calculation of rail head average elevation

[0240] The arithmetic mean of the coordinates of all points of the rail head slice point cloud is defined as the rail head average elevation:

[0241]

[0242] It serves as the absolute reference surface for subsequent elevation calculation.

[0243] 6. Relative elevation calculation

[0244] The relative elevations of the spring strip and the bolt are defined as the difference between their respective characteristic points and the rail head average elevation:

[0245]

[0246]

[0247] Wherein:

[0248] : Spring strip relative elevation;

[0249] : Bolt relative elevation;

[0250] Both are in millimeters with a resolution of 0.01 mm.

[0251] ​The calculation result is written in the register in floating-point double byte format for subsequent S5 fastener relative deformation difference calculation.

[0252] In summary, under the reference three-dimensional point cloud coordinate system, the RANSAC-based quadratic surface fitting is performed on the spring strip and bolt slice point cloud respectively; the local extreme points are obtained by analytical derivation, and the spring strip root arc center and bolt top surface center are extracted; the rail head slice point cloud is taken The average elevation of the rail head is obtained by coordinate averaging; the relative elevation difference of the spring strip and bolt is calculated based on the average elevation of the rail head as the unified reference surface; the result is written into the floating-point register to form the core input for subsequent fastener relative deformation difference calculation. This step abstracts the complex fastener geometry into precise and operable deformation variables under the premise of ensuring millimeter-level absolute scale consistency, ensuring the structural interpretability of the entire detection system and the physical consistency of the calculation chain.

[0253] Specifically, step 5 includes:

[0254] The core goal of this step is to separate the local geometric deformation of the fastener (spring strip-bolt relative displacement) from the overall rail heave in the unified coordinate system of millimeter-level absolute scale, thereby obtaining the fastener self-deformation variable that truly reflects the tightness of the fastener itself.

[0255] 1. Input data and operating environment

[0256] The input of step 5 comes from the previous steps:

[0257] Output by step 4:

[0258] Spring strip relative elevation ;

[0259] Bolt relative elevation .

[0260] Generated by step 2 and stored in the drift lookup table:

[0261] Overall fastener system heave drift .

[0262] All the above parameters are in the reference three-dimensional point cloud coordinate system after temperature-vibration-IMU correction, with a dimension of millimeters (mm) and a precision better than 0.05 mm.

[0263] 2. Fastener relative deformation difference calculation

[0264] Within the same mileage section, the relative displacement of the spring strip feature points and the bolt feature points along the vertical direction represents the instantaneous elastic deformation of the fastener assembly system, and the elevation algebraic difference between the two is defined as the fastener relative deformation difference:

[0265]

[0266] If , it means that the bolt is higher than the spring strip, the pre-tightening force decreases, and the fastener tends to be "loose";

[0267] If , it means that the relative position of the spring strip and the bolt remains stable.

[0268] The calculation results are stored in double-byte floating-point format (Float16) to ensure that the quantization error is less than 0.01 mm within the range of ±10 mm.

[0269] 3. Overall sinking and floating drift compensation

[0270] Considering the overall lifting phenomenon caused by the frost heaving and settlement of sleepers or ballast, the influence of systematic drift on the judgment result must be eliminated. By calling the drift amount lookup table established in step 2, the corresponding overall sinking and floating drift amount is read with the current milepost number as the index , and the compensation is performed in a vector subtraction manner to obtain the fastener's own deformation amount:

[0271]

[0272] Among them:

[0273] : Local relative deformation of the fastener;

[0274] : Overall sinking and floating drift of the track;

[0275] : Fastener's own deformation amount after compensation (a true indicator of pre-tightening force change).

[0276] 4. Physical validity verification and sign constraint

[0277] Due to the possibility of local negative offset caused by noise and fitting residual, the sign bit of the result is determined to maintain physical consistency:

[0278]

[0279] This operation ensures that the fastener's own deformation amount is always non-negative, avoiding non-physical states such as "negative loosening" or "reverse compression".

[0280] 5. Data quantization and cache writing

[0281] To balance storage efficiency and resolution, a fixed-point quantization strategy is adopted:

[0282]

[0283] Among them:

[0284] Quantization step is 0.01mm;

[0285] The result is represented as a 16-bit unsigned integer (UInt16);

[0286] The data range covers 0-655.35mm, which can meet the full range of fastener deformation conditions.

[0287] The quantized data is written to the real-time memory queue, and the current milestone number and timestamp are marked synchronously.

[0288] Based on the relative elevation output in step 4, the local deformation of the fastener is obtained through algebraic difference operation; combined with the drift amount lookup table compensation of the whole track, the local and global separation is realized; the sign determination and quantization coding are performed on the result, and the millimeter-level precision of the fastener itself is output. The deformation variable ensures that the physical meaning is clear, the numerical value is traceable, and it has real-time and standardized characteristics for direct connection with subsequent intelligent discrimination models.

[0289] Specifically, step 6 includes:

[0290] This step realizes the conversion from quantized geometric deformation variable to semantic level "fastener loosening state", which is the intelligent decision-making core link of the entire detection process.

[0291] 1. Input data and running environment

[0292] The input of step 6 includes:

[0293] The fastener itself deformation variable (quantized value) output by step 5:

[0294]

[0295] Unit: mm, synchronized recording of milestone number .

[0296] Model running environment:

[0297] The front end uses a physically interpretable layer (Physical Layer);

[0298] The back end uses a residual network module (Residual Network, ResNet-like);

[0299] The two are sequentially cascaded to form a physical-residual cascade discrimination model (referred to as PRC model).

[0300] 2. Physical interpretable layer

[0301] (1) Elastic strip-bolt stiffness matrix

[0302] This layer explicitly models the mechanical relationship of the fastener system as a linear stiffness matrix Vertical response of the clip-bolt system is described as:

[0303]

[0304] where:

[0305] : clip's own stiffness;

[0306] : bolt's own stiffness;

[0307] : coupling stiffness of clip and bolt.

[0308] (2) Initial class vector calculation

[0309] Input is the fastener's own deformation variable The theoretical clip displacement is calculated according to the linear response relationship:

[0310]

[0311] where is the nominal clamping force of the fastener.

[0312] Define the initial class vector of the physical layer:

[0313]

[0314] And give the initial label according to the threshold (0.2mm):

[0315]

[0316] This threshold corresponds to the upper limit of the theoretical relaxation displacement of the clip root, which is derived from experimental calibration.

[0317] 3. Residual network correction layer

[0318] (1) Input tensor construction

[0319] The initial class vector output by the physical interpretable layer is spliced with the fastener's own deformation variable into a two-dimensional tensor:

[0320]

[0321] (2) Network structure

[0322] DenseNet-40 is used as the backbone, which contains:

[0323] 4 dense blocks (Dense Block);

[0324] 1 global average pooling layer;

[0325] 1 full connection correction head.

[0326] Output is correction score , whose physical meaning is: whether the predicted initial label has systematic error.

[0327] 4. Class correction and hard voting decision

[0328] According to the sign of the correction score , hard voting is performed:

[0329]

[0330] That is:

[0331] If the correction score is positive, maintain the physical layer decision;

[0332] If the correction score is negative, it means that the network thinks that the physical threshold decision is too high, and the label should be reversed.

[0333] The final output only contains two types of fastener tightness classes: "normal" or "loose".

[0334] In summary, step 6 completes the final mapping from millimeter-level deformation to structural safety semantics, realizing automatic fastener tightness discrimination of geometric-mechanical-intelligent three-domain fusion.

[0335] Finally, it should be pointed out that: the above only for the preferred embodiments of the present application, and not for limiting the present application, although the foregoing embodiments of the present application are described in detail, for those skilled in the art, it still can be modified, or part of the technical features of the equivalent replacement of the technical solutions recorded in the foregoing embodiments. Any modification, equivalent replacement, improvement, etc. within the spirit and principles of the present application shall be included in the scope of protection of the claims of the present application.

Claims

1. An automatic detection method for the tightness of railway fasteners based on depth estimation, characterized in that, include: Step 1: The binocular laser fusion sensor fixed to the trolley synchronously acquires grayscale images and subpixel depth images of the fastener area. After joint correction by temperature, vibration and IMU, a reference 3D point cloud with absolute scale is generated. Step 2: Fit a dynamic rail top reference surface within the reference 3D point cloud, using the rail head web as a reference, and extract the overall heave and drift of the fastener system; Step 3: Input the grayscale image into the lightweight instance segmentation network to obtain the elastic bar mask, bolt mask, and rail head mask, and slice the reference 3D point cloud from the mask to obtain the elastic bar slice point cloud, bolt slice point cloud, and rail head slice point cloud; Step 4: Fit quadratic surfaces to the point cloud of the elastic bar slice and the point cloud of the bolt slice respectively, extract the feature points of the elastic bar and the bolt, and calculate the relative elevation of the elastic bar and the relative elevation of the bolt relative to the average elevation of the rail head slice point cloud; Step 5: Subtract the relative elevation of the elastic clip from the relative elevation of the bolt to obtain the relative deformation difference of the fastener, and then compensate for it with the overall sinking and floating displacement to generate the deformation of the fastener itself; Step 6: Input the deformation of the fastener itself into the physical residual cascade discrimination model, output the fastener tightness category, and bind it with the mileage station number to form the fastener tightness detection result.

2. The automatic detection method for railway fastener tightness based on depth estimation according to claim 1, characterized in that, Step 1 includes: The original grayscale image and original subpixel depth image output by the binocular laser fusion sensor rigidly connected to the crossbeam of the track inspection trolley are combined with the synchronously acquired IMU six-DOF acceleration and angular velocity sequences, ambient temperature values ​​from the temperature sensor, and high-frequency amplitude signals from the vibration sensor and input into the temperature-vibration IMU joint correction module. First, the ambient temperature values ​​from the temperature sensor are used to perform pixel-level temperature drift compensation on the original subpixel depth image to eliminate the systematic depth shift caused by the change in laser wavelength with temperature. Then, the IMU six-DOF acceleration and angular velocity sequences are used to estimate the instantaneous attitude change of the track inspection trolley crossbeam in the spatial coordinate system in real time through a Kalman filter. The instantaneous attitude change is converted into a three-dimensional attitude correction vector for each pixel of the original sub-pixel depth image. Simultaneously, the vibration component consistent with the natural frequency of the track inspection trolley is extracted by the high-frequency amplitude signal of the vibration sensor through fast Fourier transform, and synchronous inverse superposition compensation is performed on the depth value of each pixel of the original sub-pixel depth image. Finally, the sub-pixel depth image after temperature drift compensation, attitude correction, and vibration inverse superposition compensation is fused with the original grayscale image at the pixel level to generate a reference three-dimensional point cloud with absolute millimeter scale. The absolute elevation scale of the reference three-dimensional point cloud is then assigned to the entire coordinate system using the national elevation datum of the track inspection trolley's starting positioning point as the origin.

3. The automatic detection method for railway fastener tightness based on depth estimation according to claim 2, characterized in that, Step 2 includes: Local point cloud clusters are extracted from the reference 3D point cloud along the direction of train operation using a 0.5m sliding window. RANSAC plane fitting is performed on each local point cloud cluster to identify the rail head web point cloud subset. The normal vector of the rail head web point cloud subset is used as a rigid reference. The dynamic rail top reference surface equation that changes continuously with mileage is obtained by fitting using the weighted least squares method. The weight is determined by the reciprocal of the distance from the rail head web point cloud subset to the fitting plane. The Z-direction offset of the dynamic rail top reference surface equation at the current sliding window center mileage is subtracted from the Z-direction offset of the dynamic rail top reference surface equation at the previous sliding window center mileage to obtain the overall heave and drift of the fastener system. The overall heave and drift of the fastener system is then written into a drift lookup table corresponding to each mileage station with a resolution of 0.1mm. This allows the overall heave and drift of the fastener system to be eliminated within an accuracy of 0.05mm when calculating the deformation of the fastener itself in subsequent calculations by looking up the table.

4. The automatic detection method for railway fastener tightness based on depth estimation according to claim 3, characterized in that, Step 3 includes: The grayscale image is input into a lightweight instance segmentation network that has undergone knowledge distillation. The lightweight instance segmentation network uses MobileNet-V3 as the backbone and SOLOv2 as the instance segmentation head. During the training phase, it uses trackside sunlight shadows, ballast occlusion, and rust noise as augmented samples, so that the network outputs a spring bar mask, bolt mask, and track head mask that correspond one-to-one with the pixels of the grayscale image. Perform 8-connected domain denoising and morphological closure on the elastic bar mask, bolt mask, and rail head mask respectively to obtain the elastic bar mask, bolt mask, and rail head mask with single-pixel closed edges; Using the elastic bar mask, bolt mask, and rail head mask as Boolean indices, the baseline 3D point cloud is sliced ​​using a mask-driven method to obtain elastic bar slice point clouds that retain only the elastic bar point set, bolt slice point clouds that retain only the bolt point set, and rail head slice point clouds that retain only the rail head point set. The elastic bar slice point clouds, bolt slice point clouds, and rail head slice point clouds are then written into memory in PLY format for subsequent quadratic surface fitting.

5. The automatic detection method for railway fastener tightness based on depth estimation according to claim 4, characterized in that, Step 4 includes: Perform RANSAC-based quadratic surface fitting on the point cloud of the elastic bar slice. Using the quadratic surface equation as a model, extract the three-dimensional coordinates of the center of the circular arc at the root of the elastic bar as the feature points of the elastic bar. Perform quadratic surface fitting with the same parameters on the bolt slice point cloud, and extract the three-dimensional coordinates of the center of the top surface of the bolt as the bolt feature point; The average elevation of the track head slice point cloud is obtained by arithmetically averaging the Z coordinates of all points in the cloud computing of the track head slice point. Using the average elevation of the rail head slice point cloud as a benchmark, the difference between the Z coordinate of the elastic clip feature point and the average elevation of the rail head slice point cloud is calculated as the relative elevation of the elastic clip, and the difference between the Z coordinate of the bolt feature point and the average elevation of the rail head slice point cloud is calculated as the relative elevation of the bolt.

6. The automatic detection method for railway fastener tightness based on depth estimation according to claim 5, characterized in that, Step 5 includes: The relative elevation of the elastic bar and the relative elevation of the bolt are calculated by performing an algebraic difference operation on the same mileage section to obtain the relative deformation difference of the fastener, and the calculation accuracy is maintained at floating-point double-byte. The drift amount lookup table is called to read the overall sinking and floating drift amount of the fastener system corresponding to the current mileage, and the relative deformation difference of the fastener is compensated by vector subtraction using the overall sinking and floating drift amount of the fastener system. Perform sign bit determination on the deformation of the fastener itself. When the deformation of the fastener itself is greater than or equal to zero, keep the original value. When the deformation of the fastener itself is less than zero, force it to be set to zero in order to eliminate physically meaningless negative values. After the sign determination is completed, the deformation of the fastener itself is written into the real-time memory queue and converted into a 16-bit unsigned integer with a quantization step size of 0.01mm, for the physical residual cascade discrimination model to read.

7. The automatic detection method for railway fastener tightness based on depth estimation according to claim 6, characterized in that, Step 6 includes: The quantized deformation of the fastener itself is input into the physical-residual cascaded discrimination model. The physical residual cascaded discrimination model is composed of a front-end physical interpretable layer and a back-end residual network cascaded sequentially. The physical interpretable layer has a built-in elastic bolt stiffness matrix. The stiffness matrix takes the deformation of the fastener itself as input and outputs an initial category vector with the theoretical displacement of the elastic bolt root of 0.2mm as the threshold. When the initial category vector is greater than or equal to 0.2mm, a loose initial label is assigned. When the initial category vector is less than 0.2mm, a normal initial label is assigned. The back-end residual network uses DenseNet-40 as the backbone and a two-channel tensor obtained by concatenating the initial class vector with the deformation of the fastener itself as input. It extracts high-order error features through four dense blocks and one global average pooling layer, and outputs the corrected score of the initial label. The initial label is hard-voted based on the correction score. If the correction score is greater than zero, the initial label is kept. If the correction score is less than zero, the initial label is reversed, resulting in a fastener tightness category that only includes normal or loose fasteners. The fastener tightness category is bound to the synchronously recorded mileage marker number using UTF-8 strings and written to the SQLite database to generate the fastener tightness test result.

Images