An automatic detection method for railway fastener tightness based on depth estimation

Abstract

The application discloses an automatic detection method for the tightness of railway fasteners based on depth estimation, comprising the following steps: a binocular line laser fusion sensor is used to obtain the grayscale and sub-pixel depth map of the fastener area, and a reference three-dimensional point cloud with an absolute scale is generated through joint correction of temperature, vibration and IMU; the whole sinking and floating drift is extracted by fitting a dynamic rail head reference surface with a rail web; the spring bar, bolt and rail head masks are obtained through lightweight instance segmentation and the point cloud is sliced; the characteristic points are extracted by fitting a quadratic surface, the relative elevation is calculated and the fastener self deformation is obtained through drift compensation; finally, the deformation is input into a physical-residual cascade model to determine the tightness state, and the detection result is output by binding with the milepost number. The application can separate the real fastener relative looseness from the track reference drift, thereby significantly improving the accuracy and reliability of the tightness detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of railway infrastructure detection, in particular to an automatic detection method for railway fastener tightness based on depth estimation. BACKGROUND

[0002] As a key component of track structure, the fastener system is responsible for firmly fixing the steel rail on the sleeper and providing necessary elastic buffer. The tightness of the elastic strip and bolt directly determines the size of the clamping pressure. Once the actual pre-tightening force drops and is not found in time, it is easy to induce rail creep, gauge change and even train derailment. Therefore, online monitoring of the fastener tightness has always been the core demand of the railway operation and maintenance department. At present, the field still mainly relies on manual inspection or torque wrench sampling and retightening, which is low in efficiency, long in sky window occupation time and high in missed detection rate. In recent years, with the development of machine vision, a fastener missing and breaking recognition method based on two-dimensional images has appeared, but the tightness belongs to millimeter-level three-dimensional deformation, which cannot be quantified by RGB images only. Therefore, the patent application with publication number CN118711015A proposes: using a monocular camera to shoot a two-dimensional image of the track, outputting an absolute depth map through a ZoeDepth derivative network, and then using YOLOv8 to segment the elastic strip and bolt area, and finally taking the difference DWAD between the average depth dc of the elastic strip and the average depth db of the bolt as the tightness criterion. This method can realize three-dimensional measurement without laser radar, and has the advantages of low hardware cost and convenient deployment.

[0003] However, it is found in line tests that the above-mentioned scheme has systematic misjudgment caused by reference drift: when the track bed is frozen and expanded, the sleeper is empty and lifted, or the overall settlement is caused by temperature stress, the whole set of fasteners will sink or rise together with the sleeper by 0.5-2mm. At this time, dc and db increase or decrease together, DWAD can remain unchanged, but the actual elastic strip pre-tightening force has decreased or even failed; on the contrary, if the overall track is lifted, DWAD may exceed the threshold and be misjudged as loose. Since the patent document does not test the rigidity assumption of the fastener-sleeper-track bed, nor does it introduce the track head elevation or the relative constraint of adjacent fasteners, it cannot distinguish between fastener loosening and overall track deformation, resulting in uncontrollable false negative and false positive rates, which seriously limits the popularization and application of this method on short sky window and long line of ordinary speed and high speed railways. Therefore, a new depth estimation scheme is needed to compensate for the overall deformation of the track online, which can separate the real relative loosening of the fastener from the track reference drift while retaining the low-cost advantage of the monocular camera, thereby significantly improving the accuracy and reliability of the tightness detection. SUMMARY

[0004] In order to solve the above technical problems, the present application provides an automatic detection method for railway fastener tightness based on depth estimation.

[0005] To achieve the above object, the technical scheme adopted by the present application is as follows:

[0006] The present application discloses an automatic detection method for railway fastener tightness based on depth estimation, comprising the following steps:

[0007] Step 1: The binocular line laser fusion sensor fixed to the trolley synchronously collects the fastener region gray scale image and sub-pixel depth image, and generates the reference three-dimensional point cloud with absolute scale after temperature, vibration and IMU joint correction;

[0008] Step 2: The dynamic rail top reference surface is fitted in the reference three-dimensional point cloud with the rail head web as the reference, and the overall sinking and floating drift of the fastener system is extracted;

[0009] Step 3: The gray scale image is input into the lightweight instance segmentation network to obtain the elastic strip mask, bolt mask and rail head mask, and the elastic strip slice point cloud, bolt slice point cloud and rail head slice point cloud are obtained by masking the three-dimensional point cloud;

[0010] Step 4: The elastic strip slice point cloud and the bolt slice point cloud are respectively fitted with a quadratic surface, the elastic strip feature points and the bolt feature points are extracted, and the elastic strip relative height and the bolt relative height are calculated relative to the average height of the rail head slice point cloud;

[0011] Step 5: The difference between the elastic strip relative height and the bolt relative height is obtained to obtain the fastener relative deformation difference, and the overall sinking and floating drift is compensated to generate the fastener self deformation;

[0012] Step 6: The fastener self deformation is input into the physical residual cascade discriminant model to output the fastener tightness category, and the fastener tightness detection result is formed by binding with the milepost number.

[0013] Further, the step 1 comprises:

[0014] The raw gray scale image and the raw sub-pixel depth image output by the binocular line laser fusion sensor rigidly fixed to the rail inspection trolley beam are input into a temperature vibration IMU joint correction module, the temperature sensor environment temperature value is used to compensate the raw sub-pixel depth image at the pixel level to eliminate the systematic depth deviation caused by the change of the wavelength of the laser with the temperature, the six-degree-of-freedom acceleration and angular velocity sequence of the IMU is used to estimate the instantaneous attitude change of the rail inspection trolley beam in the space coordinate system through the Kalman filter and convert the instantaneous attitude change into a three-dimensional attitude correction vector of each pixel of the raw sub-pixel depth image, and the high-frequency amplitude signal of the vibration sensor is used to extract the vibration component consistent with the inherent frequency of the rail inspection trolley through the fast Fourier transform and perform synchronous inverse superposition compensation on the depth value of each pixel of the raw sub-pixel depth image, and finally the sub-pixel depth image after temperature drift compensation, attitude correction and vibration inverse superposition compensation is fused with the raw gray scale image at the pixel level one by one to generate a reference three-dimensional point cloud with an absolute millimeter level scale, and the national height datum of the starting positioning point of the rail inspection trolley is taken as the origin to assign an absolute height to the whole coordinate system of the reference three-dimensional point cloud.

[0015] Further, the step 2 comprises:

[0016] In the reference three-dimensional point cloud, a local point cloud cluster is intercepted along the train running direction with a 0.5m sliding window, RANSAC plane fitting is performed on each local point cloud cluster to identify a rail head web point cloud subset, and the normal vector of the rail head web point cloud subset is taken as a rigid reference to obtain a dynamic rail top reference plane equation that changes continuously with the mileage by weighted least squares fitting, wherein the weight is determined by the reciprocal of the distance from the rail head web point cloud subset to the fitting plane;

[0017] The Z-direction offset of the dynamic rail top reference plane equation at the current sliding window center mileage is subtracted from the Z-direction offset of the dynamic rail top reference plane equation at the previous sliding window center mileage to obtain the overall sinking and floating drift of the fastener system, and the overall sinking and floating drift of the fastener system is written into a drift amount lookup table corresponding to the mileage pile number with a resolution of 0.1mm, so that the overall sinking and floating drift of the fastener system is excluded within 0.05mm accuracy when calculating the subsequent fastener self-deformation amount through table lookup.

[0018] Further, the step 3 comprises:

[0019] Input the gray image into the lightweight instance segmentation network completed with knowledge distillation, which takes MobileNet-V3 as the backbone, SOLOv2 as the instance segmentation head, and uses trackside sun shadow, ballast occlusion, and rust noise as augmented samples in the training stage, so that the network outputs elastic strip mask, bolt mask, and rail head mask corresponding to the pixels of the gray image;

[0020] Perform 8-connected domain noise removal and morphological closure on the elastic strip mask, bolt mask, and rail head mask respectively to obtain the elastic strip mask, bolt mask, and rail head mask with single-pixel closed edges;

[0021] Using the elastic strip mask, bolt mask, and rail head mask as Boolean indexes, perform mask-driven slicing on the reference three-dimensional point cloud to obtain the elastic strip slice point cloud, bolt slice point cloud, and rail head slice point cloud, which only retain the elastic strip point set, bolt point set, and rail head point set respectively. Write the elastic strip slice point cloud, bolt slice point cloud, and rail head slice point cloud in PLY format into memory for subsequent quadratic surface fitting calls.

[0022] Further, the step 4 includes:

[0023] Perform RANSAC-based quadratic surface fitting on the elastic strip slice point cloud to extract the three-dimensional coordinates of the elastic strip root arc center as the elastic strip feature point using the quadratic surface equation as the model;

[0024] Perform quadratic surface fitting with the same parameters on the bolt slice point cloud to extract the three-dimensional coordinates of the bolt top surface center as the bolt feature point;

[0025] Calculate the arithmetic mean of the Z coordinates of all points in the rail head slice point cloud to obtain the average elevation of the rail head slice point cloud;

[0026] Take the average elevation of the rail head slice point cloud as the reference to calculate the difference between the Z coordinate of the elastic strip feature point and the average elevation of the rail head slice point cloud as the elastic strip relative elevation, and calculate the difference between the Z coordinate of the bolt feature point and the average elevation of the rail head slice point cloud as the bolt relative elevation.

[0027] Further, the step 5 includes:

[0028] Perform algebraic difference operation on the elastic strip relative elevation and the bolt relative elevation at the same mileage section to obtain the fastener relative deformation difference, and maintain the operation precision as floating-point double bytes;

[0029] Read the overall sinking and floating drift of the fastener system corresponding to the current mileage from the drift lookup table, and perform vector subtraction compensation on the fastener relative deformation difference using the overall sinking and floating drift of the fastener system;

[0030] A sign bit determination is performed on the fastener self deformation amount, and when the fastener self deformation amount is greater than or equal to zero, the original value is kept, and when the fastener self deformation amount is less than zero, it is forced to be zero, so as to eliminate the physically meaningless negative value;

[0031] The fastener self deformation amount after the sign determination is written into the real-time memory queue, and is converted into a 16-bit unsigned integer with a quantization step of 0.01 mm, for reading by the physical residual cascade discrimination model.

[0032] Further, the step 6 comprises:

[0033] The fastener self deformation amount after quantization is input into the physical residual cascade discrimination model, which is sequentially cascaded by a front-end physically interpretable layer and a rear-end residual network, wherein the physically interpretable layer is built-in with a spring strip bolt stiffness matrix, the stiffness matrix takes the fastener self deformation amount as input and outputs an initial class vector with a spring strip root theoretical displacement of 0.2 mm as a threshold value, when the initial class vector is greater than or equal to 0.2 mm, an initial loose label is assigned, and when the initial class vector is less than 0.2 mm, a normal initial label is assigned;

[0034] The rear-end residual network takes DenseNet-40 as the backbone and takes a double-channel tensor obtained by splicing the initial class vector and the fastener self deformation amount as input, extracts high-order error features through 4 dense blocks and 1 global average pooling layer, and outputs a correction score for the initial label;

[0035] The initial label is hard-voted according to the correction score, if the correction score is greater than zero, the initial label is kept, and if the correction score is less than zero, the initial label is reversed, to obtain a fastener tightness class containing only normal or loose;

[0036] The fastener tightness class is bound with the synchronously recorded mileage post number in a UTF-8 string, written into a SQLite database, and a fastener tightness detection result is formed.

[0037] The present application can effectively solve the technical problems of the existing monocular depth estimation method, such as the reference drift misjudgment caused by the overall sinking and floating of the track, the difficulty in distinguishing the fastener self loosening and the track deformation, and the insufficient detection accuracy, and has the following beneficial effects:

[0038] The present application can effectively eliminate the reference drift misjudgment caused by the overall sinking and floating of the track, and realize accurate identification of the fastener self loosening state. By establishing a dynamic track top reference surface and introducing overall drift compensation, the stability and reliability of the detection are significantly improved; by using binocular line laser fusion depth perception and sub-millimeter level surface fitting, the measurement accuracy of the fastener deformation amount is improved; by combining the physical residual cascade discrimination model, high-precision intelligent judgment of the fastener tightness state is realized. In summary, the present application can separate the real fastener relative loosening from the track reference drift, thereby significantly improving the accuracy and reliability of the tightness detection. Attached Figure Description

[0039] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0040] In the attached diagram:

[0041] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0042] The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of the present invention will now be described with reference to the accompanying drawings.

[0043] like Figure 1 As shown, this invention discloses an automatic detection method for the tightness of railway fasteners based on depth estimation, comprising:

[0044] Step 1: The binocular laser fusion sensor fixed to the trolley synchronously acquires grayscale images and subpixel depth images of the fastener area. After joint correction by temperature, vibration and IMU, a reference 3D point cloud with absolute scale is generated.

[0045] Step 2: Fit a dynamic rail top reference surface within the reference 3D point cloud, using the rail head web as a reference, and extract the overall heave and drift of the fastener system;

[0046] Step 3: Input the grayscale image into the lightweight instance segmentation network to obtain the elastic bar mask, bolt mask, and rail head mask, and slice the reference 3D point cloud from the mask to obtain the elastic bar slice point cloud, bolt slice point cloud, and rail head slice point cloud;

[0047] Step 4: Fit quadratic surfaces to the point cloud of the elastic bar slice and the point cloud of the bolt slice respectively, extract the feature points of the elastic bar and the bolt, and calculate the relative elevation of the elastic bar and the relative elevation of the bolt relative to the average elevation of the rail head slice point cloud;

[0048] Step 5: Subtract the relative elevation of the elastic clip from the relative elevation of the bolt to obtain the relative deformation difference of the fastener, and then compensate for it with the overall sinking and floating displacement to generate the deformation of the fastener itself;

[0049] Step 6: Input the deformation of the fastener itself into the physical residual cascade discrimination model, output the fastener tightness category, and bind it with the mileage station number to form the fastener tightness detection result.

[0050] Specifically, step 1 includes:

[0051] The binocular laser fusion sensor is rigidly connected to the crossbeam of the track inspection trolley and outputs data in parallel according to a unified time base (hard synchronization signal).

[0052] original grayscale image (pixel coordinates );

[0053] original sub-pixel depth image (unit: mm, including system bias and noise)

[0054] camera-laser coordinate extrinsic matrix determined by rigid installation relationship and one-time geometric calibration in the system integration stage of the binocular-line laser fusion sensor ;

[0055] collect at the same time:

[0056] IMU six-degree-of-freedom sequence ;

[0057] temperature sensor ambient temperature value ;

[0058] vibration sensor high-frequency amplitude .

[0059] These signals are sent to the temperature vibration IMU joint correction module, compensated through three stages (temperature drift compensation→ attitude correction→ vibration reverse superposition compensation), and fused with the grayscale image pixel by pixel. Finally, the pixel-level depth is converted into three-dimensional coordinates by using the camera intrinsic parameter, and the benchmark three-dimensional point cloud with absolute millimeter level scale and taking the starting positioning point of the track inspection trolley as the origin of the scale is obtained .

[0060] Specifically:

[0061] First, pixel-level temperature drift compensation, the goal is to eliminate the systematic depth offset of the laser / depth sensor caused by the change of the ambient temperature.

[0062] Let the depth measured by the sensor at the reference temperature be , and the model expresses the influence of temperature on depth in a linear / second-order approximation at the pixel level:

[0063]

[0064] and take

[0065]

[0066] where , is the pixel-level temperature-sensitive coefficient (stored in the form of a lookup table or a parameterized model inside the module), is the temperature sensor value with timestamp, and this compensation is a pixel-by-pixel additive correction. The output is the temperature-compensated depth map .

[0067] Secondly, IMU-based instantaneous pose estimation and pixel-level pose correction, which aims to eliminate the pixel depth geometric distortion caused by the instantaneous pose changes (pitch, roll, yaw) of the track inspection trolley beam, so that the depth in the vehicle reference coordinate system is consistent with the space coordinates.

[0068] Using the acceleration and angular velocity sequence of the IMU, an extended / universal Kalman filter or an inertial navigation filter is used to output the beam instantaneous pose quaternion / rotation matrix at a fixed frequency (synchronized with the depth map frame rate) And translation rate estimation (focus on attitude, translation is constrained by odometer / odometer pulse or additional positioning system). In this step, we only use the attitude change (relative transformation of the sensor at the reference time) to calculate the correction.

[0069] Use the camera-laser joint intrinsic parameters (focal length , , principal point , ) to project each pixel depth back to a three-dimensional point in the sensor coordinate system:

[0070]

[0071] The instantaneous pose relative to the reference (such as the middle time of the frame or the sensor installation zero position) is , the translation is , and the pixel corresponding three-dimensional point under the reference attitude should be:

[0072]

[0073] Therefore, the pixel-level three-dimensional pose correction vector (difference vector with the original pixel point as the starting point) is

[0074]

[0075] Inside the module, the Kalman filter outputs , , the above is calculated pixel by pixel and converted into an equivalent depth correction amount (along the optical axis component) for subsequent depth correction.

[0076] Thirdly, vibration anti-phase superposition compensation (frequency domain processing), which aims to suppress high-frequency depth jitter caused by the inherent frequency of the trolley or periodic excitation of the road surface, and reduce periodic depth errors caused by mechanical vibration.

[0077] The vibration sensor sampling signal Short-time Fourier transform (STFT / FFT) is performed to identify the significant spectral component consistent with the inherent frequency of the rail inspection trolley (or its neighbors) Meanwhile, IMU / acceleration signals can also be incorporated into the frequency domain analysis to improve the robustness of the resonance component identification.

[0078] Assuming that this vibration component produces an approximately periodic offset in the depth measurement , the phase of which is consistent with the vibration sensor measurement This can be estimated by cross-correlation, and the inverse phase compensation is defined as:

[0079]

[0080] where is the pixel response coefficient (reflecting the effect of geometric magnification / reduction on different pixels), This term is calculated in the time domain in parallel with the temperature / attitude correction, and is accumulated pixel by pixel as the vibration compensation.

[0081] Again, the three joint corrections are fused at the pixel level, and the final corrected depth of each pixel at time (t) is:

[0082]

[0083] where: is the temperature compensation, is the attitude correction along the optical axis component, is the vibration inverse phase compensation.

[0084] Each of these terms is calculated pixel by pixel to ensure the one-to-one correspondence of the pixel-level fusion requirements; all intermediate variables are time-stamped to ensure strict synchronization with the grayscale image .

[0085] Finally, from the corrected depth map to the reference three-dimensional point cloud, the is back-projected into the three-dimensional points of the sensor reference system to obtain the point set in the sensor coordinates . Then, the sensor coordinate system is converted to the global coordinate system of the rail inspection trolley (rigid installation transformation ), and the national elevation datum of the starting positioning point of the rail inspection trolley is taken as the origin to assign absolute elevation:

[0086]

[0087] where represents the rigid transformation matrix from the global coordinate system of the rail inspection trolley to the national / line global coordinate system;

[0088] where the absolute elevation offset is performed on the component of all points:

[0089]

[0090] wherein represents the relative elevation component of the three-dimensional point in the global coordinate system of the track inspection trolley, represents the national elevation datum value corresponding to the starting positioning point of the track inspection trolley

[0091] the final output of the reference three-dimensional point cloud represents:

[0092]

[0093] with a unit of mm, the point cloud satisfies: one-to-one correspondence between pixels and three-dimensional points, absolute scale and millimeter-level precision traceability (guaranteed by the temperature / attitude / vibration compensation chain and the starting national elevation datum of the vehicle), and serves as the only scale traceability reference for subsequent steps (track top reference surface fitting, mask slicing, quadratic surface fitting, etc.).

[0094] In summary, through the three parts of pixel-level temperature drift compensation, IMU-Kalman attitude estimation and mapping to a pixel-level correction vector, and vibration frequency domain inverse compensation, the systematic deviation and dynamic interference in the depth map are eliminated; the corrected depth is projected back through the camera intrinsic parameters and the national elevation datum of the starting positioning point of the track inspection trolley in the global coordinate system of the vehicle to complete the absolute elevation assignment, and the output is a reference three-dimensional point cloud with absolute scale, which provides a unique scale traceability and spatiotemporal consistent data basis for subsequent steps (dynamic track top reference surface fitting, mask slicing, quadratic surface fitting, etc.).

[0095] Specifically, step 2 includes:

[0096] In the reference three-dimensional point cloud , a local point cloud cluster is intercepted along the train running direction with a fixed length of 0.5m sliding window. For each local point cloud cluster, first use RANSAC plane fitting to identify and extract the rail head web point cloud subset, and use the normal vector of the subset as the rigid reference direction (local rigid reference of the track), and then use weighted least squares method to fit to get the dynamic track top reference surface equation which changes with mileage. The Z-direction offset difference of the track top reference surface at the center of adjacent sliding windows is taken as the overall heave drift of the fastener system at this mileage. The drift is written into a drift lookup table corresponding to the mileage stake number with a resolution of 0.1mm, and the 0.05mm accuracy in subsequent compensation is ensured through interpolation and quantization strategy.

[0097] 1. Definition of sliding window and local point cloud interception

[0098] The train running direction is taken as The axis (in the global coordinate system output from step 1) is relative to the reference 3D point cloud. Continuous sampling by mileage. The length is defined as... A sliding window, with the center of the window centered at a step size (s) (usually less than or equal to 1). It is moving along the entire line.

[0099] For the center mileage The Use a window to capture a local point cloud cluster:

[0100]

[0101] 2. RANSAC plane fitting to identify a subset of the railhead web point cloud.

[0102] The aim is to robustly identify the set of points belonging to the rail head and web (i.e., a subset that is normally stable and has good planar approximation) in a local point cloud containing complex echoes from the rail head, rail head web, ballast, fasteners, etc.

[0103] right Using the RANSAC planar model (planar implicit form) Iterative sampling is performed, with the number of iterations and threshold configured as required (e.g., 200 iterations and a distance threshold on the order of 0.05 mm for reference), to identify the set of interior points. (RANSAC inliers).

[0104] Will This serves as an initial candidate set for the point cloud subset of the railhead web. To ensure geometric semantic consistency, a further constraint is imposed: the normal vector distribution of the interior point set must form an angle with the prior railhead normal direction that is less than a threshold. (e.g., 5°); if the threshold is exceeded, the window is marked as unreliable and subsequent fitting is skipped (or an extension check is triggered).

[0105] Record the initial normal vector of this subset. (Normalized) and used as a rigid reference direction. The RANSAC output also returns the vertical residual from each point to the fitted plane. .

[0106] 3. Weighted least squares fitting based on a subset of the rail head web (to obtain the dynamic rail top reference surface)

[0107] Objective: In Under the constraints, the local orbital top datum is precisely fitted using the weighted least squares method, making the fitting more robust to noise and local outliers, and ensuring continuity with mileage. An explicit functional form in local coordinates is adopted (so that the data can be directly obtained). Offset):

[0108]

[0109] where the parameter vector is described in the first window.

[0110] The weighted least squares is:

[0111]

[0112] The weight is determined by the reciprocal of the distance from the rail head web point cloud subset to the fitted plane, so in the iterative implementation, we use:

[0113]

[0114] where is the absolute vertical residual of point to the initial fitted plane (RANSAC output or last iteration residual), is a small constant to prevent division by zero (for example ).

[0115] Matrix solution (closed form):

[0116] Let

[0117]

[0118] Then

[0119]

[0120] After solving, the parameters are converted into the plane normal and are consistent with are checked; if the normal deviation is beyond the threshold (indicating that the fitting is unstable), the window is marked as abnormal and is modified by interpolation / smoothing with the neighboring windows.

[0121] 4. Continuity and smoothing of the dynamic rail top reference surface

[0122] To ensure the continuity along the mileage and suppress the impact of measurement noise on the extraction of drift, the parameter sequence obtained along the mileage is one-dimensional smoothed (weighted moving average or one-dimensional Kalman filter can be used). After smoothing, the reference surface parameters at the mileage are obtained, representing the local dynamic rail top reference surface equation .

[0123] It should be noted that smoothing only acts on the parameter sequence level and does not change the principle of fitting weight within a single window; when there are too many abnormal neighboring windows, weighted interpolation or longer baseline window recalculation can be used to restore continuity. ​

[0124] 5. Calculation of the total float of the fastener system

[0125] Definition: At the center of the sliding window, the height offset of the dynamic track top reference surface from the datum is denoted by the parameter

[0126]

[0127]

[0128]

[0129]

[0130] To avoid ringing caused by high-frequency noise, a low-pass filter with small bandwidth (e.g. based on a first-order IIR or small-window weighted average) can be applied after the calculation, but the filter design must ensure that it still has sufficient response to sudden changes (such as sudden settlement of the roadbed). 6. Construction of the drift lookup table and accuracy guarantee

[0131]

[0132]

[0133]

[0134]

[0135]

[0136] ​​​​​​​​​​​​​​​​​​​​​​Table resolution is 0.1mm (to meet storage and transmission efficiency);

[0137] When reading, the target mileage is used to find the two adjacent centers in the table , The quantization value of the two adjacent centers is linearly interpolated, and the upper bound of the quantization error of the interpolation result is 0.05mm, thereby ensuring the compensation accuracy:

[0138]

[0139] wherein represents a 0.1mm quantization function, is a linear weight of the mileage between the two centers. The linear interpolation limits the maximum error caused by quantization to half the step size 0.1 / 2=0.05mm.

[0140] The table entry also carries a confidence or uncertainty field (for example ), so that the downstream compensation decides whether to adopt or trigger redundant detection according to the uncertainty.

[0141] To ensure that the rigidity assumption with the rail head web as the reference is established within the local window, the following consistency / quality criterion needs to be performed for each window:

[0142] If the normal angle of adjacent windows exceeds the threshold , it is considered that the rail cross-section has a sudden change or the fitting is unreliable, and the window is marked as abnormal and re-fitted by neighborhood interpolation or extended window;

[0143] If the standard deviation of the weighted fitting residual exceeds the threshold (indicating that the point cloud is strongly polluted by ballast or fasteners), the abnormal points are down-weighted or removed and then re-fitted, and if necessary, the window length is expanded;

[0144] The number of effective rail head web points used for fitting within the window should not be less than (for example, 50 points), otherwise the adjacent window is discarded or merged;

[0145] If consecutive windows are marked as abnormal, the system should trigger a fitting strategy with a longer baseline to reconstruct the dynamic reference surface.

[0146] 7. Result output format and interface agreement

[0147] Drift lookup table: the table structure is (mileage , quantized drift , uncertainty , normal consistency flag, timestamp). Storage unit: mm, quantization step 0.1mm, UTF-8 encoding bound with mileage string.

[0148] Read and interpolate by request mileage in subsequent step call And in the compensation formula, the whole amount of heave drift is removed with 0.05mm precision.

[0149] Compensation formula:

[0150]

[0151] 8. Uncertainty propagation

[0152] The parameter covariance obtained by single window fitting can be derived from the weighted least squares normal equation:

[0153]

[0154] The standard deviation of the difference can be further obtained from the covariance , and the difference is propagated to obtain . The table records for downstream criterion use. Interpolation and quantization introduce a maximum system error of 0.05mm (quantization half-step), which together with the measurement uncertainty constitutes the final compensation uncertainty evaluation.

[0155] In summary, on the reference three-dimensional point cloud, a local point cloud is intercepted with a 0.5m sliding window, and the RANSAC is used to identify and extract a rail head web point cloud subset, and then the normal of the subset is taken as the local rigid reference, and the weighted least squares fitting is used to obtain the window level dynamic rail top reference surface parameter . The offset difference of the reference surface in the direction at the center of adjacent windows is taken as the overall heave drift of the fastener system, and is written into the drift amount lookup table with 0.1mm resolution; through linear interpolation, the final error is ensured to be not more than 0.05mm when looking up and compensating. In order to ensure robustness, normal stability, residual distribution, minimum point number and other criteria are introduced; for abnormal windows, neighborhood interpolation or long baseline re-fitting strategy is adopted. The output drift amount lookup table contains uncertainty and quality flag, which is the only dimensional traceability data source for removing the overall heave drift of the track in subsequent steps.

[0156] Specifically, step 3 includes:

[0157] 1. Input and pre-data

[0158] Step 3 takes the output of steps 1-2:

[0159] Reference three-dimensional point cloud , which is corrected and aligned to the dynamic rail top reference surface coordinate system by step 2; the gray image synchronously collected, through the camera-laser coordinate external parameter matrix Realize the spatial mapping of image pixels and point cloud points; Dynamic rail top reference provides local reference, making mask segmentation focus only on the area above the rail top and fastener, reducing ballast interference.

[0160] 2. Grayscale image input lightweight instance segmentation network

[0161] Network structure: Lightweight instance segmentation network takes MobileNet-V3 as the backbone and SOLOv2 as the instance segmentation head.

[0162] Knowledge distillation: In the training stage, the student network is distilled by the teacher model (such as ResNet-101+SOLOv2), which retains the semantic segmentation ability while controlling the inference delay within 15ms.

[0163] Data augmentation: Training samples include trackside sunlight shadows, ballast local occlusion, metal rust noise, etc. to improve robustness.

[0164] Network output:

[0165]

[0166] Among them:

[0167] : Elastic strip mask;

[0168] : Bolt mask;

[0169] : Rail head mask;

[0170] , : Image height and width.

[0171] 3. Post-processing of mask: denoising and morphological closing

[0172] To ensure that the mask boundary is closed and morphologically consistent, perform the following operations on each mask ( ):

[0173] 8-Connected domain denoising: Remove isolated noise connected domains with pixel area less than the threshold (usually less than 1% of the total area):

[0174]

[0175] Morphological closing operation: The structure element takes a radius (2-3px), eliminates edge notches and forms a single-pixel closed loop boundary:

[0176]

[0177] in, and These are the expansion and erosion operators, respectively.

[0178] After processing, a set of masks with continuous edges and no isolated noise is obtained:

[0179]

[0180] 4. Mask-driven baseline 3D point cloud slicing

[0181] Based on the extrinsic parameter matrix obtained in step 1 (From the laser coordinate system) To the camera coordinate system and camera intrinsic parameter matrix Establish a pixel projection mapping between the point cloud and the image:

[0182]

[0183] in For point cloud points Pixel coordinates in a grayscale image.

[0184] This is used as an index to implement mask slicing:

[0185] If point Projected pixels satisfy If so, then retain that point and include it in the category. Sliced ​​point clouds.

[0186] Formal definition:

[0187]

[0188] 5. Sliced ​​point cloud generation and formatted storage

[0189] Three types of mask-driven slice point clouds were obtained:

[0190] Elastic bar slice point cloud : Contains only points on the surface of the elastic bar;

[0191] Bolt slice point cloud Includes only the top and perimeter of the bolt;

[0192] Railhead slice point cloud : Contains only the metal surface of the rail head;

[0193] Each type of point cloud is realigned and clipped in the coordinate system of the dynamic orbital top reference plane (the plane fitted in step 2);

[0194] Write into the memory buffer area in PLY format (including three-dimensional coordinates, reflectivity, mask category label), for subsequent S4 quadratic surface fitting and deformation variable estimation call.

[0195] In summary, the MobileNet-V3+SOLOv2 network optimized by knowledge distillation is used for instance segmentation of grayscale images, generating high-robustness elastic strip, bolt, and rail head masks; after 8-connected domain denoising and morphological closing processing, the mask boundary is smooth and closed; using the camera-laser coordinate correspondence established in step 1, the mask-driven slicing is performed on the reference three-dimensional point cloud output in step 2; output three types of sliced point clouds 、 、 in PLY format for subsequent surface fitting and fastener tightness calculation in step 4; the whole process is completed within 0.05mm spatial accuracy and 30ms real-time, ensuring traceability and accuracy stability of subsequent calculations.

[0196] Specifically, step 4 includes:

[0197] This step realizes the extraction of quantifiable geometric feature parameters (elastic strip feature points, bolt feature points, and rail head average elevation) from the three-dimensional point cloud after mask slicing, providing high-precision input for subsequent fastener deformation variable calculation.

[0198] 1. Input data and spatial reference

[0199] The input data of step 4 comes from:

[0200] Elastic strip sliced point cloud

[0201] Bolt sliced point cloud

[0202] Rail head sliced point cloud

[0203] The three types of point clouds are in the dynamic rail top reference plane coordinate system defined in step 2, with absolute millimeter-level elevation meaning in the coordinate (Z), which has been aligned with the starting elevation reference of the rail inspection trolley.

[0204] 2. Definition of quadratic surface fitting model

[0205] To obtain the local morphological features of the elastic strip and bolt, RANSAC (Random Sample Consensus) quadratic surface fitting is performed on and respectively, and the quadratic surface mathematical model is as follows:

[0206]

[0207] where the parameter vector is:

[0208]

[0209] Model physical meaning:

[0210] Coefficient , , : describes the surface curvature;

[0211] Coefficient , : describes the surface tilt direction;

[0212] Coefficient : represents the overall height offset.

[0213] 3. RANSAC fitting process

[0214] (1) Set parameters

[0215] Number of iterations ;

[0216] Inlier distance threshold .

[0217] (2) Algorithm steps

[0218] Randomly select the minimum sample set from ( Indicates clip or bolt);

[0219] Solve the least squares equation To get the initial fitting coefficient;

[0220] Calculate the perpendicular distance from all points to the fitted surface:

[0221]

[0222] Determine the inlier set ;

[0223] Iterative update until the number of inliers is maximized;

[0224] Re-solve the least squares with the final inlier set .

[0225] 4. Feature point extraction

[0226] (1) Elastic strip feature point (elastic strip root arc center)

[0227] The quadratic surface fitted by the elastic strip slice point cloud usually presents a concave arc structure.

[0228] Obtain the local extreme point coordinates by taking the partial derivative of the surface to zero:

[0229]

[0230] The coordinates of the characteristic points of the spring strip in the xy plane are obtained:

[0231]

[0232] The characteristic point elevation is obtained by substituting the fitting equation:

[0233]

[0234] The three-dimensional coordinates of the characteristic points of the spring strip are obtained:

[0235]

[0236] (2) Bolt characteristic points (center of bolt top surface)

[0237] The bolt slice point cloud usually has a convex flat top feature, and the extreme point of the quadratic surface is taken as the center coordinate:

[0238]

[0239] The solving method is the same as that of the spring strip characteristic points, only the corresponding bolt fitting surface parameters.

[0240] 5. Calculation of average elevation of rail head

[0241] The arithmetic mean of the coordinates of all points of the rail head slice point cloud is taken as the average elevation of the rail head:

[0242]

[0243] It serves as the absolute reference surface for subsequent elevation calculation.

[0244] 6. Relative elevation calculation

[0245] The relative elevations of the spring strip and the bolt are defined as the difference between their respective characteristic points and the average elevation of the rail head:

[0246]

[0247]

[0248] Wherein:

[0249] : Relative elevation of spring strip;

[0250] : Relative elevation of bolt;

[0251] Both are in millimeters with a resolution of 0.01 mm.

[0252] ​The calculation result is written in the register in floating-point double byte format for subsequent S5 fastener relative deformation difference calculation.

[0253] In summary, under the reference three-dimensional point cloud coordinate system, the RANSAC-based quadratic surface fitting is performed on the spring strip and bolt slice point clouds respectively; the local extreme points are obtained by analytical derivation, and the spring strip root arc center and bolt top surface center are extracted; the rail head slice point cloud is taken The average elevation of the rail head is obtained by coordinate averaging; the relative elevation difference of the spring strip and bolt is calculated based on the average elevation of the rail head as the unified reference surface; the result is written into the floating-point register to form the core input for subsequent fastener relative deformation difference calculation. This step abstracts the complex fastener geometry into precise and operable deformation variables under the premise of ensuring millimeter-level absolute scale consistency, ensuring the structural interpretability of the entire detection system and the physical consistency of the calculation chain.

[0254] Specifically, step 5 includes:

[0255] The core goal of this step is to separate the local geometric deformation of the fastener (spring strip-bolt relative displacement) from the overall rail heave in the unified coordinate system of millimeter-level absolute scale, thereby obtaining the fastener self-deformation variable that truly reflects the tightness of the fastener itself.

[0256] 1. Input data and operating environment

[0257] The input of step 5 comes from the previous steps:

[0258] Output from step 4:

[0259] Spring strip relative elevation ;

[0260] Bolt relative elevation .

[0261] From step 2, generated and stored in the drift lookup table:

[0262] Overall fastener system heave drift .

[0263] All the above parameters are in the reference three-dimensional point cloud coordinate system after temperature-vibration-IMU correction, with a dimension of millimeters (mm) and a precision better than 0.05 mm.

[0264] 2. Fastener relative deformation difference calculation

[0265] Within the same mileage section, the relative displacement of the spring strip feature points and the bolt feature points along the vertical direction represents the instantaneous elastic deformation of the fastener assembly system, and the elevation algebraic difference between the two is defined as the fastener relative deformation difference:

[0266]

[0267] If , it means that the bolt is higher than the spring strip, the pre-tightening force decreases, and the fastener tends to be "loose";

[0268] If , it means that the relative position of the spring strip and the bolt remains stable.

[0269] The calculation results are stored in double-byte floating-point format (Float16) to ensure that the quantization error is less than 0.01 mm within the range of ±10 mm.

[0270] 3. Overall sinking and floating drift compensation

[0271] Considering the overall lifting phenomenon caused by frost heaving and settlement of sleepers or ballast, the influence of systematic drift on the judgment result must be eliminated. By calling the drift amount lookup table established in step 2, the corresponding overall sinking and floating drift amount is read with the current milepost number as the index , and the compensation is performed in a vector subtraction manner to obtain the fastener's own deformation amount:

[0272]

[0273] Among them:

[0274] : Local relative deformation of the fastener;

[0275] : Overall sinking and floating drift of the track;

[0276] : Fastener's own deformation amount after compensation (a true indicator of pre-tightening force change).

[0277] 4. Physical validity verification and sign constraint

[0278] Due to noise and fitting residuals, local negative offset may occur. To maintain physical consistency, the sign bit of the result is determined:

[0279]

[0280] This operation ensures that the fastener's own deformation amount is always non-negative, avoiding non-physical states such as "negative looseness" or "reverse compression".

[0281] 5. Data quantization and cache writing

[0282] To balance storage efficiency and resolution, a fixed-point quantization strategy is adopted:

[0283]

[0284] Among them:

[0285] Quantization step is 0.01mm;

[0286] The result is represented as a 16-bit unsigned integer (UInt16);

[0287] The data range covers 0-655.35mm, which can meet the full working condition range of the fastener deformation.

[0288] The quantized data is written to the real-time memory queue, and the current milestone number and timestamp are marked synchronously.

[0289] Based on the relative elevation output in step 4, the local deformation of the fastener is obtained through algebraic difference operation; combined with the drift amount lookup table compensation of the whole track, the local and global separation is realized; the sign determination and quantization coding are performed on the result, and the millimeter-level precision of the fastener itself deformation variable is output; ensure that the variable has clear physical meaning, traceable numerical value, and real-time and standardized characteristics of direct docking with subsequent intelligent discrimination model.

[0290] Specifically, step 6 includes:

[0291] This step realizes the conversion from quantized geometric deformation variable to semantic level "fastener loosening state", which is the intelligent decision-making core link of the whole detection process.

[0292] 1. Input data and running environment

[0293] The input of step 6 includes:

[0294] The fastener itself deformation variable (quantized value) output by step 5:

[0295]

[0296] Unit: mm, synchronized recording of milestone number .

[0297] Model running environment:

[0298] The front end uses the physical interpretable layer (Physical Layer);

[0299] The back end uses the residual network module (Residual Network, ResNet-like);

[0300] The two are sequentially cascaded to form a physical-residual cascade discrimination model (referred to as PRC model).

[0301] 2. Physical interpretable layer

[0302] (1) Spring-bolt stiffness matrix

[0303] This layer explicitly models the mechanical relationship of the fastener system as a linear stiffness matrix Vertical response of the clip-bolt system is described as:

[0304]

[0305] where:

[0306] : clip self-stiffness;

[0307] : bolt self-stiffness;

[0308] : coupling stiffness of clip and bolt.

[0309] (2) Initial class vector calculation

[0310] Input is the fastener self-deformation variable The theoretical clip displacement is calculated according to the linear response relationship:

[0311]

[0312] where is the nominal clamping force of the fastener.

[0313] Define the initial class vector of the physical layer:

[0314]

[0315] And give the initial label according to the threshold (0.2mm):

[0316]

[0317] This threshold corresponds to the upper limit of the theoretical relaxation displacement of the clip root, which is derived from experimental calibration.

[0318] 3. Residual network correction layer

[0319] (1) Input tensor construction

[0320] The initial class vector output by the physical interpretable layer and the fastener self-deformation variable are spliced into a two-dimensional tensor:

[0321]

[0322] (2) Network structure

[0323] DenseNet-40 is used as the backbone, which includes:

[0324] 4 dense blocks (Dense Block);

[0325] 1 global average pooling layer;

[0326] 1 full connection correction head.

[0327] Output is correction score , whose physical meaning is: whether the predicted initial label has systematic error.

[0328] 4. Class correction and hard voting decision

[0329] According to the sign of the correction score Hard voting is performed:

[0330]

[0331] That is:

[0332] If the correction score is positive, maintain the physical layer decision;

[0333] If the correction score is negative, it means that the network thinks that the physical threshold decision is too high, and the label should be reversed.

[0334] The final output only contains two types of fastener tightness classes: "normal" or "loose".

[0335] In summary, step 6 completes the final mapping from millimeter-level deformation to structural safety semantics, realizing automatic fastener tightness discrimination in the geometric-mechanical-intelligent three-domain fusion.

[0336] Finally, it should be pointed out that: the above only for the preferred embodiments of the present application, and not for limiting the present application, although the foregoing embodiments of the present application are described in detail, for those skilled in the art, it still can be modified, or part of the technical features of the equivalent replacement of the technical solutions recorded in the foregoing embodiments. Any modification, equivalent replacement, improvement, etc. within the spirit and principles of the present application shall be included in the scope of protection of the claims of the present application.

Claims

1. A method for automatically detecting the tightness of a railway fastener based on depth estimation, characterized by, Comprise: Step 1: binocular line laser fusion sensor fixed to the trolley synchronously collects fastener area gray image and sub-pixel depth image, and generates reference three-dimensional point cloud with absolute scale after temperature, vibration and IMU joint correction; The step 1 comprises: The original gray image and the original sub-pixel depth image output by the binocular line laser fusion sensor rigidly fixed to the rail inspection trolley beam are input into the temperature vibration IMU joint correction module with the synchronously collected IMU six-degree-of-freedom acceleration and angular velocity sequence, the temperature sensor environment temperature value and the vibration sensor high-frequency amplitude signal, first, the temperature sensor environment temperature value is used to compensate the temperature drift of the original sub-pixel depth image at the pixel level to eliminate the systematic depth deviation caused by the change of the wavelength of the laser with temperature, then the IMU six-degree-of-freedom acceleration and angular velocity sequence is used to estimate the instantaneous attitude change of the rail inspection trolley beam in the space coordinate system through the Kalman filter and convert the instantaneous attitude change into a three-dimensional attitude correction vector of each pixel of the original sub-pixel depth image, at the same time, the vibration sensor high-frequency amplitude signal is used to extract the vibration component consistent with the inherent frequency of the rail inspection trolley through fast Fourier transform and synchronously superimpose and compensate the depth value of each pixel of the original sub-pixel depth image in reverse, finally, the sub-pixel depth image compensated by temperature drift, attitude correction and vibration reverse superimposition is fused with the original gray image at the pixel level one by one to generate the reference three-dimensional point cloud with absolute millimeter level scale, and the national height reference of the starting positioning point of the rail inspection trolley is taken as the origin to assign absolute height to the whole coordinate system of the reference three-dimensional point cloud; Step 2: fitting a dynamic rail top reference surface in the reference three-dimensional point cloud with the rail head web as a reference to extract the overall sinking and floating drift of the fastener system; Step 3: inputting the gray image into the lightweight instance segmentation network to obtain the elastic strip mask, bolt mask and rail head mask, and obtaining the elastic strip slice point cloud, bolt slice point cloud and rail head slice point cloud by slicing the mask reference three-dimensional point cloud; Step 4: fitting a quadratic surface to the elastic strip slice point cloud and the bolt slice point cloud respectively to extract the elastic strip feature points and bolt feature points, and calculating the elastic strip relative height and bolt relative height relative to the average height of the rail head slice point cloud; Step 5: subtracting the elastic strip relative height from the bolt relative height to obtain the fastener relative deformation difference, and then compensating with the overall sinking and floating drift to generate the fastener self-deformation; Step 6: inputting the fastener self-deformation into the physical residual cascade discriminant model to output the fastener tightness category, and binding it with the milepost number to form the fastener tightness detection result.

2. The automatic detection method of the tightness of railway fastenings based on depth estimation according to claim 1, characterized in that, The step 2 comprises: In the reference three-dimensional point cloud, a local point cloud cluster is intercepted along the train running direction with a 0.5m sliding window, RANSAC plane fitting is performed on each local point cloud cluster to identify a rail head web point cloud subset, and the normal vector of the rail head web point cloud subset is taken as a rigid reference to fit a dynamic rail top reference surface equation that changes continuously with the mileage by using the weighted least squares method, wherein the weight is determined by the reciprocal of the distance from the rail head web point cloud subset to the fitted plane; The Z-direction offset of the dynamic rail top datum plane equation at the center milestone of the current sliding window is subtracted from the Z-direction offset of the dynamic rail top datum plane equation at the center milestone of the previous sliding window to obtain the overall fastener system subsidence and drift, and the overall fastener system subsidence and drift is written into a drift amount lookup table corresponding to the mileage pile number at a resolution of 0.1 mm, so that the overall fastener system subsidence and drift is excluded from the subsequent fastener self-deformation amount calculation through table lookup within an accuracy of 0.05 mm.

3. The method according to claim 2, wherein, The step 3 comprises: inputting the gray image into the lightweight instance segmentation network completed by knowledge distillation, the lightweight instance segmentation network taking MobileNet-V3 as a backbone and SOLOv2 as an instance segmentation head, and using trackside sunshine shadow, ballast occlusion and rust noise as augmented samples in the training stage to make the network output elastic strip masks, bolt masks and rail head masks corresponding to pixels of the gray image; performing 8-connected domain noise removal and morphological closing on the elastic strip masks, bolt masks and rail head masks respectively to obtain the elastic strip masks, bolt masks and rail head masks with single-pixel closed edges; mask-driven slicing is performed on the reference three-dimensional point cloud by taking the elastic strip masks, bolt masks and rail head masks as Boolean indexes to obtain elastic strip slice point cloud, bolt slice point cloud and rail head slice point cloud which only retain elastic strip point sets, bolt point sets and rail head point sets respectively, and the elastic strip slice point cloud, bolt slice point cloud and rail head slice point cloud are written into the memory in PLY format for subsequent quadratic surface fitting.

4. The automatic detection method of the tightness of railway fastenings based on depth estimation according to claim 3, characterized in that, The step 4 comprises: performing RANSAC-based quadratic surface fitting on the elastic strip slice point cloud to extract the three-dimensional coordinates of the center of the elastic strip root arc as elastic strip feature points by taking the quadratic surface equation as a model; performing quadratic surface fitting with the same parameters on the bolt slice point cloud to extract the three-dimensional coordinates of the center of the bolt top surface as bolt feature points; calculating the arithmetic mean of the Z coordinates of all points of the rail head slice point cloud to obtain the average elevation of the rail head slice point cloud; taking the average elevation of the rail head slice point cloud as a reference, the difference between the Z coordinate of the elastic strip feature point and the average elevation of the rail head slice point cloud is calculated as the relative elevation of the elastic strip, and the difference between the Z coordinate of the bolt feature point and the average elevation of the rail head slice point cloud is calculated as the relative elevation of the bolt.

5. The method of claim 4, wherein the method further comprises: The step 5 comprises: performing algebraic difference operation on the elastic strip relative elevation and the bolt relative elevation at the same mileage section to obtain the fastener relative deformation difference, and keeping the operation accuracy as floating-point double bytes; calling the drift amount lookup table to read the overall fastener system subsidence and drift corresponding to the current mileage, and performing vector subtraction compensation on the fastener relative deformation difference by taking the overall fastener system subsidence and drift as a reference; performing sign bit judgment on the fastener self-deformation amount, and keeping the original value when the fastener self-deformation amount is greater than or equal to zero, and forcibly setting it to zero when the fastener self-deformation amount is less than zero, so as to exclude physically meaningless negative values; writing the fastener self-deformation amount after completing the sign judgment into the real-time memory queue, and converting it into a 16-bit unsigned integer with a quantization step of 0.01 mm for the physical residual cascade discrimination model to read.

6. The automatic detection method of the tightness of railway fastenings based on depth estimation according to claim 5, characterized in that, The step 6 comprises: The fastener self deformation quantity after completing quantization is input into a physical-residual cascade discriminant model, the physical-residual cascade discriminant model is sequentially cascaded by a front-end physical interpretable layer and a rear-end residual network, the physical interpretable layer is built-in with a spring strip bolt stiffness matrix, the stiffness matrix takes the fastener self deformation quantity as input and outputs an initial category vector with a threshold value of a spring strip root theoretical displacement of 0.2 mm, when the initial category vector is greater than or equal to 0.2 mm, a loose initial label is assigned, when the initial category vector is less than 0.2 mm, a normal initial label is assigned; the rear-end residual network takes a double-channel tensor obtained by splicing the initial category vector and the fastener self deformation quantity as input, extracts high-order error features through four dense blocks and one global average pooling layer, and outputs a correction score of the initial label; the initial label is hard-voted according to the correction score, if the correction score is greater than zero, the initial label is kept, if the correction score is less than zero, the initial label is reversed, and a fastener tightness category containing only normal or loose is obtained; the fastener tightness category and the simultaneously recorded milepost number are bound as a UTF-8 string, written into an SQLite database, and a fastener tightness detection result is formed.

Images