An online real-time perception method and system based on multi-sensor fusion

Abstract

The application relates to the technical field of rail transit vehicle safety monitoring, and provides a kind of on-line real-time perception method and system based on multi-sensor fusion adhesion, which acquires track surface image, environmental parameters and running state parameters in real time, identifies track surface medium type through image processing, selects corresponding adhesion model from a pre-established track surface adhesion model database, inputs real-time parameters into the model to calculate wheel-rail adhesion coefficient, wherein the adhesion model constructs friction model parameters as a coupling function containing medium type and multiple factors and introduces cross-coupling terms, and the undetermined coefficients in the coupling function are obtained by regression after independent feature extraction of each factor by grouping convolutional neural network fusion. While retaining the physical interpretability of the friction model, the application realizes high-precision online perception of adhesion coefficient under multiple working conditions.

Description

Technical Field

[0001] This application relates to the field of rail transit vehicle safety monitoring technology, specifically to an adhesive online real-time sensing method and system based on multi-sensor fusion. Background Technology

[0002] Wheel-rail adhesion is a critical factor affecting train traction, braking performance, and operational safety. When a train is running, a third medium such as water, oil, ice, snow, frost, rust, or leaves often exists between the wheel and rail contact interfaces. The presence of these third media significantly reduces the adhesion coefficient between the wheel and rail, leading to insufficient usable adhesion and potentially causing safety accidents such as prolonged braking distances.

[0003] Currently, there are several main methods for obtaining the wheel-rail adhesion state: One approach is the estimation method based on vehicle dynamics models. This type of method indirectly estimates the adhesion coefficient by establishing a train dynamics model and utilizing measurable vehicle state variables. However, this method is highly dependent on the accuracy of the dynamics model and lacks the ability to perceive the specific type, coverage state, and environmental temperature and humidity of the third medium on the rail surface, making it difficult to guarantee estimation accuracy when rail surface conditions change drastically.

[0004] The second method is based on visual inspection of the rail surface. This type of method acquires images of the rail surface using an onboard camera and uses image processing or deep learning techniques to identify the type of third medium present on the rail surface, thereby qualitatively determining the adhesion state. However, relying solely on visual means can only obtain information about the type of medium, and cannot provide quantitative adhesion coefficient values, nor can it reflect the influence of multiple factors such as temperature and humidity on adhesion characteristics. Summary of the Invention

[0005] To help solve the above-mentioned technical problems, this application provides a method and system for real-time online adhesion sensing based on multi-sensor fusion.

[0006] In a first aspect, this application provides a method for real-time online adhesion sensing based on multi-sensor fusion, which includes the following steps: Real-time acquisition of track surface images, environmental parameters, and operating status parameters during train operation; The track surface image is processed to identify the track surface medium type; Based on the identified rail surface medium type, select the corresponding adhesion model from the pre-established rail surface adhesion model database; The environmental parameters and operating status parameters collected in real time are input into the selected adhesion model to calculate the wheel-rail adhesion coefficient. The adhesion model in the rail surface adhesion model database is a parametric friction model. The parametric friction model expresses the adhesion coefficient as a function of the slip ratio and includes a first parameter, a second parameter, and a third parameter that control the amplitude and shape of the adhesion coefficient. The first parameter, the second parameter, and the third parameter are respectively constructed as coupling functions that include the medium type and multiple influencing factors. The coupling function includes at least a cross-coupling term describing the interaction between different environmental factors and a cross-coupling term describing the interaction between wheel-rail contact state factors. The coupling function has coefficients to be determined. The coefficients to be determined in the coupling function are obtained by fitting pre-trained grouped convolutional neural networks based on bench test data. The grouped convolutional neural network adopts a multi-path parallel grouping structure, in which each group performs independent one-dimensional convolutional feature extraction on different categories of input features, and then performs feature fusion and regression to output the coefficients to be determined.

[0007] The step of image processing of the rail surface image to identify the rail surface medium type specifically includes: The track surface image is deblurred and its size is normalized to obtain a normalized image; A semantic segmentation network is used to extract the rail tread region from the normalized image; The extracted image corresponding to the rail tread area is input into a media classification network, which outputs the rail surface media type. The loss function of the semantic segmentation network is a joint loss function of weighted cross-entropy loss and DiceLoss, expressed as: N is the total number of pixels involved in the loss calculation, K is the number of semantic segmentation categories, u is the pixel index, k is the semantic category index, and y is the number of semantic segmentation categories. uk p indicates whether the u-th pixel truly belongs to the k-th class. uk Let represent the probability that the u-th pixel belongs to the k-th class. The unbalanced weights for the number of pixels in the k-th class are... The weights are the weight coefficients for the weighted cross-entropy loss. For numerically stable terms, Let L be the class importance weight of the k-th class in the Dice loss, and L be the value of the joint loss function. The media classification network uses ResNet as the feature extraction network and is trained using transfer learning and metric learning; during training, anchor samples I... a Similar Samples I p and outlier sample I n The feature extraction network with shared weights is input to obtain feature vectors z. a z p and z nThe following triplet loss function is used to constrain the distance between samples of the same class to be less than the distance between samples of different classes: ; In the formula, m is the preset inter-class distance interval, and L′ is the triple loss function value; The media classification network outputs a media feature vector z=[z1,z2,…,z…]. 128 ] T And the probability of each medium category P=[p1,p2,…,p K ], where p k The probability that the image corresponding to the rail tread area belongs to the k-th type of medium is represented by the category corresponding to the highest probability. The category corresponding to the highest probability is taken as the rail surface medium identification result.

[0008] The parameterized friction model is expressed as: , In the formula, The adhesion coefficient is... Where c1 is the slip ratio, c2 is the first parameter, c3 is the second parameter, M is the medium type, T is the ambient temperature, H is the relative humidity, C is the medium coverage, Q is the axle load, and v is the train speed. The first parameter c1, the second parameter c2, and the third parameter c3 are respectively constructed as multi-factor coupling functions related to the medium type M, ambient temperature T, relative humidity H, medium coverage C, axle load Q, and train speed v: , Among them, c pq,M For the coefficients to be determined that depend on the medium type M, c pq,M In the table, the subscripts p and q are the row and column identifiers of the coefficient matrix, TH is the temperature-humidity cross-coupling term, Hv is the humidity-velocity cross-coupling term, and CQ is the medium coverage-axle weight cross-coupling term.

[0009] Using the parametric friction model, the rate of increase of the adhesion coefficient when the slip ratio approaches zero is... Determine the slip ratio corresponding to the peak adhesion coefficient. for c1, c2 and c3 are all greater than 0 and c1c2>c3.

[0010] The coefficient c to be determined in the multi-factor coupling function pq , MThe two-stage fitting method yields the following: First, the adhesion coefficient-slip ratio curves under various working conditions are obtained through orthogonal bench tests, and c1, c2, and c3 corresponding to a single working condition are fitted. Then, using the working condition characteristic parameters as input and c1, c2, and c3 corresponding to the working condition as fitting targets, the coefficients to be determined in the coupling function are obtained through regression using the grouped convolutional neural network. pq , M ; The grouped convolutional neural network is constructed in the following manner: The first group of input track surface medium LAB color features, LBP texture features and medium coverage are processed by at least two layers of one-dimensional convolution and global average pooling to obtain the first feature vector. The second group inputs ambient temperature and relative humidity, and after at least two layers of one-dimensional convolution and global average pooling, a second feature vector is obtained. The third group inputs the train's running speed, angular velocity, and axle load, and after at least two layers of one-dimensional convolution and global average pooling, a third feature vector is obtained. The first feature vector, the second feature vector, and the third feature vector are concatenated to form a fused feature, which is then passed through a fully connected layer to output the coefficient c to be determined. pq,M .

[0011] In the grouped convolutional neural network, each convolutional layer is followed by a ReLU activation function and a max-pooling layer, wherein the ReLU activation function is expressed as: , In the formula, i is the index of the input feature, j is the index of the neuron, and x i For the i-th input feature, b represents the connection weights between neurons. j For the bias term, y j This is the output of the neuron.

[0012] The method for establishing the orthogonal bench test is as follows: The medium type M is used as the outer layer group. For each medium, five orthogonal factors are set: medium coverage, ambient temperature, relative humidity, wheel-rail rolling speed and axle load. Each factor is set with multiple levels. The experiment is arranged according to the orthogonal table to obtain the adhesion coefficient data under each working condition. For water, oil, rust, and leaf media, the temperature level is set in the upper zero range; for ice, snow, and frost media, the temperature level is set separately in the lower zero range.

[0013] The calculation of the wheel-rail adhesion coefficient includes: Based on the train's running speed v and axle angular velocity Calculate the slip ratio using the wheel's effective rolling radius R. The calculation formula is: The calculated slip ratio The ambient temperature and relative humidity collected in real time are input into the selected adhesion model to calculate the estimated value of the adhesion coefficient.

[0014] It also includes sending the calculated adhesion coefficient, slip ratio, and identified rail surface medium type to the cab display terminal, and providing adhesion status display, adhesion alarm, and driving suggestion functions.

[0015] Secondly, this application provides an adhesion online real-time sensing system based on multi-sensor fusion, comprising: The multi-sensor data acquisition module includes a temperature sensor, a humidity sensor, a speed sensor, a Doppler radar, and an on-board camera, which are used to acquire ambient temperature, ambient humidity, axle speed, train speed, and track surface images, respectively. Adhesion sensing module, including board, processing module and host; The board is used to receive various signals collected by the multi-sensor data acquisition module; The processing module is used to segment, extract, and normalize the received track surface image, and to preprocess the train operation data. The host is used to identify the rail surface medium based on the processed rail surface image, and call the adhesion model corresponding to the identified rail surface medium type from the pre-established rail surface adhesion model database. The host inputs the pre-processed ambient temperature, ambient humidity, train speed and axle speed into the adhesion model, calculates the adhesion coefficient and slip ratio, and outputs the adhesion coefficient, the slip ratio and the rail surface medium type. The adhesion models in the track surface adhesion model database and the method for calculating the adhesion coefficient adopt the method described in any one of the first aspects.

[0016] In summary, compared with existing technologies, this application solves the problem of large calculation errors in traditional fixed-parameter models under multiple working conditions by constructing friction model parameters as coupled functions containing multiple factors and introducing cross-coupling terms, thereby improving the accuracy of adhesion coefficient calculation. Furthermore, by using a grouped convolutional neural network to independently extract features from each factor and then fusing them with regression model parameters, it overcomes the reliability deficiencies of existing technologies while considering model interpretability. In addition, it integrates visual recognition with multi-source sensing, realizing the transition from medium discrimination to adhesion coefficient estimation, providing a basis for safe train operation. Attached Figure Description

[0017] Figure 1 This is a block diagram illustrating the principle of the multi-sensor fusion adhesive online real-time perception model of this application; Figure 2This is a schematic diagram of the grouped convolutional neural network structure of this application; Figure 3 This is a schematic diagram illustrating the overall architecture and workflow of the multi-sensor fusion adhesive online real-time sensing system of this application; Figure 4 This is a block diagram of the multi-sensor fusion adhesive online real-time sensing system of this application. Detailed Implementation

[0018] The present application will be further described below with reference to the accompanying drawings. The structure and principle of the present application are very clear to those skilled in the art. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

[0019] To facilitate understanding of the technical solution of this application, the English abbreviations and their corresponding full names and Chinese meanings appearing in the text are explained as follows: LAB: LAB color space; LBP: Local Binary Pattern; ReLU: Rectified Linear Unit, the linear rectified activation function; DeepLabv3+: DeepLabv3+ semantic segmentation network; ResNet: Residual Network; ResNet50: 50-layer Residual Network; ImageNet: ImageNet image dataset; ASPP: Atrous Spatial Pyramid Pooling; Softmax: Softmax normalization function; DiceLoss: Dice loss function; Burckhardt: Burckhardt friction model; RGB: Red Green Blue, a color model consisting of the three primary colors of red, green, and blue.

[0020] This application provides a method and system for online real-time adhesion sensing based on multi-sensor fusion. Addressing the risk of low wheel-rail adhesion caused by third media such as water, ice, snow, frost, oil, leaves, and rust on the rail surface during train operation, this method combines onboard multi-sensor synchronous data acquisition, deep learning image recognition, and a multi-factor coupled adhesion model to achieve online real-time sensing of wheel-rail adhesion status. This provides data support for safe train operation and scheduling in complex climatic environments.

[0021] I. The adhesion online real-time sensing method based on multi-sensor fusion proposed in this application.

[0022] This method is divided into two parts: offline database construction and online sensing. In the offline stage, a rail surface medium sample database and a rail surface adhesion model database are constructed based on bench tests and deep learning algorithms. In the online stage, data is collected in real time through on-board sensors to complete medium identification, model matching and adhesion coefficient calculation.

[0023] Figure 1 This is a block diagram illustrating the principle of the multi-sensor fusion adhesive online real-time perception model of this application.

[0024] exist Figure 1 The system is divided into two parts: a multi-sensor data acquisition system and a rail surface adhesion database for different media and environments. This demonstrates the data logic relationship between offline model construction and online real-time perception. In the multi-sensor data acquisition system on the left, rail surface images captured by cameras are extracted from the rail surface area and combined with a sample library containing dry rail, wet rail, oil, ice, snow, frost, rust, and leaves to determine the rail surface medium. Temperature and humidity sensors output temperature and humidity data, while speed sensors and Doppler radar output train speed and axle speed data. These three types of real-time data serve as inputs for online perception. The rail surface adhesion database on the right is based on the Burckhardt friction model. Basic adhesion data is obtained through orthogonal tests on rail surface adhesion test benches under various media, and supplemented with data from real-vehicle tests under typical working conditions. The rail surface adhesion model is constructed by fitting a grouped convolutional neural network, forming a rail surface adhesion model database covering different temperature and humidity environments and different axle speeds and train speeds for various media. Finally, real-time perception of rail surface adhesion is achieved by combining the real-time sensor data.

[0025] The specific implementation steps are as follows: 1. Real-time acquisition of track surface images, environmental parameters, and operational status parameters during train operation. Specific data acquisition targets include: Rail surface images: RGB images of the rail surface in front of the wheel-rail contact point are continuously acquired using an onboard camera; Environmental parameters: Ambient temperature and relative humidity are collected using temperature and humidity sensors, respectively; Operating status parameters: Axle speeds are collected via speed sensors, and train speed is collected via Doppler radar. All collected data is synchronously transmitted to the subsequent processing unit.

[0026] 2. Perform image processing on the acquired track surface images to identify the track surface medium type, specifically including: The original track surface image is affected by factors such as train vibration, lighting changes, and obstruction by debris. Standardization preprocessing is performed first: a convolutional neural network is used to deblur the initial image, suppress motion blur and random noise, improve image clarity and feature recognizability, and the deblurred image is uniformly scaled to a pixel size of 512×512×3 to ensure that the input size of subsequent networks is consistent.

[0027] A semantic segmentation network based on the DeepLabv3+ architecture is used to extract the effective area of ​​the rail tread from the normalized image and remove background interference such as sleepers, fasteners, and track bed.

[0028] Using ResNet50 as the encoder backbone network and ImageNet pre-trained weights for initialization, multi-scale deep features of the image are extracted. Multi-scale feature extraction involves using a hollow spatial pyramid pooling module to extract multi-scale semantic features under different receptive fields. Finally, a 1×1 convolution and Softmax function are used to output four segmentation results: rail tread, rail non-tread, sleeper fasteners, and ballast background. Only the rail tread region is retained as input for subsequent medium classification. To address the class imbalance problem where the pixel area of ​​the rail tread is much smaller than that of the background, a joint loss function of weighted cross-entropy loss and DiceLoss is adopted, the expression of which is: , N is the total number of pixels involved in the loss calculation, K is the number of semantic segmentation categories, u is the pixel index, k is the semantic category index, and y uk p indicates whether the u-th pixel truly belongs to the k-th class. uk Let represent the probability that the u-th pixel belongs to the k-th class. The unbalanced weights for the number of pixels in the k-th class are... The weights are the weight coefficients for the weighted cross-entropy loss. For numerically stable terms, Let L be the class importance weight of the k-th class in the Dice loss, and L be the value of the joint loss function.

[0029] After scaling the segmented rail tread area image to 224×224×3, it is input into the medium classification network, which outputs the rail surface medium type. This classification network uses ResNet as the feature extraction backbone and combines transfer learning and metric learning for joint training to improve the ability to distinguish media with similar appearances, such as water film, oil film, ice, and frost.

[0030] Using a ResNet50 pre-trained on the ImageNet dataset as the source domain feature extraction network, the network retains the general visual features such as edges, colors, textures, and shapes learned by its convolutional layers and residual modules. The network is fine-tuned for the target domain using a self-built orbital surface medium sample database. After passing through the backbone network and global average pooling, a 2048-dimensional deep feature vector is obtained. This vector is then reduced to 512 and 128 dimensions respectively through two fully connected layers, finally outputting a 128-dimensional medium feature vector. z=[z1,z2,…,z 128 ] T ; Simultaneously output the predicted probability for each media category: P=[p1,p2,…,p K ]; In the formula p K Let be the probability that the input image belongs to the k-th type of medium. Take the category corresponding to the highest probability as the final track surface medium recognition result.

[0031] Based on transfer learning, metric learning is introduced, and a triplet network architecture with shared weights is used during training: Input contains anchor point sample I a Similar positive samples I p Compared with negative samples I n All three are input into the same feature extraction network with the same set of weights, and each outputs a feature vector z. a z p and z n ; The feature space distribution is constrained by a triplet loss function, which ensures that the distance between samples of the same class is less than the distance between samples of different classes. The expression for the loss function is: ; In the formula, m is the preset inter-class distance interval, and L′ is the triple loss function value; The media classification network outputs a media feature vector z=[z1,z2,…,z…]. 128 ] T And the probability of each medium category P=[p1,p2,…,p K ], where p k The probability that the image corresponding to the rail tread area belongs to the k-th type of medium is represented by the value of 'k'. The category corresponding to the highest probability is taken as the rail surface medium recognition result. During training, samples with larger distances among samples of the same type are preferentially selected, so that the network focuses on learning rail surface states that are easily confused.

[0032] The sample database used to train the segmentation and classification networks covered eight typical track surface conditions: dry, water film, oil stains, ice, snow, frost, rust, and leaves. By changing the medium thickness, coverage, ambient temperature, relative humidity, and camera shooting angle, 5000 valid track surface images were obtained through multiple shots.

[0033] 3. Pre-construction of the track surface adhesion model database.

[0034] Based on the identified rail surface medium type, a corresponding adhesion model is selected from a pre-established rail surface adhesion model database. The adhesion models in the rail surface adhesion model database are parametric friction models, constructed through bench orthogonal experiments and grouped convolutional neural network fitting.

[0035] The parametric friction model expresses the adhesion coefficient as a function of the slip ratio and includes a first parameter, a second parameter, and a third parameter that control the magnitude and shape of the adhesion coefficient. The model expression is as follows: , In the formula, The viscosity coefficient is 1. Where c is the slip ratio, c1 is the first parameter, c2 is the second parameter, c3 is the third parameter, M is the medium type, T is the ambient temperature, H is the relative humidity, C is the medium coverage, Q is the axle load, and v is the train speed.

[0036] The first, second, and third parameters are each constructed as coupling functions that include the medium type and multiple influencing factors. These coupling functions include at least cross-coupling terms describing the interactions between different environmental factors and cross-coupling terms describing the interactions between wheel-rail contact state factors, specifically in the following form: , Among them, c pq,M For the coefficients to be determined that depend on the medium type M, c pq,M In the table, the subscripts p and q are the row and column identifiers of the coefficient matrix, TH is the temperature-humidity cross-coupling term, Hv is the humidity-velocity cross-coupling term, and CQ is the medium coverage-axle weight cross-coupling term.

[0037] Cross-coupling terms are divided into two categories: Cross-coupling terms between environmental factors: TH: Temperature-humidity cross term, characterizing the coupled effect of changes in both temperature and humidity on adhesion; Hv: Humidity-speed cross term, characterizing the coupling effect of humidity and train speed. Increased humidity promotes water film formation and pollutant hygroscopicity, while increased speed affects the ability of water film to be drawn into the contact area and the speed of media removal. The two together affect the adhesion properties. Cross-coupling terms between wheel-rail contact condition factors: CQ: Medium Coverage-Axle Load Cross Term, which characterizes the coupling effect between medium coverage and axle load. Changes in axle load will change the compaction or removal effect of the wheel on the rail surface medium, thereby changing the degree of adhesion of the medium.

[0038] To ensure that the adhesion curve conforms to the physical laws of wheel-rail contact, namely, that the adhesion coefficient increases positively in the small slip ratio range and that the peak point has a positive real solution, the following parameter constraints must be met: c1, c2, and c3 are all greater than 0, and c1c2>c3.

[0039] Based on the above parameterized model, two key characteristic quantities of the adhesion curve can be derived: When the slip ratio approaches zero, the initial rate of increase of the adhesion coefficient is: , The slip ratio corresponding to the peak adhesion coefficient (peak slip ratio) is: .

[0040] The following describes the orthogonal test design of the test bench in the embodiments of this application.

[0041] The orthogonal test bench used in this application embodiment employs a looping wheel-rail adhesion test bench. Its basic rotary wheel-rail adhesion simulation main structure adopts the rotary wheel-rail adhesion simulation test device disclosed in Chinese invention patent application CN113740251A. This existing device fully discloses the overall structure and working principle of the circular track, the rotating arm revolution mechanism, the wheel rotation axle system, the vehicle speed motor, the wheel speed motor, the torque acquisition unit, and the water spraying medium application unit. It can realize basic vehicle speed and wheel speed control and adhesion force calculation. Those skilled in the art can construct the basic mechanical structure of the test bench based on this published document.

[0042] To cover the orthogonal testing requirements of multiple media types, wide temperature and humidity ranges, and controllable axle load, this application adds a high and low temperature environment control unit, a multi-media quantitative application unit, and an axle load pressure adjustment unit to the aforementioned basic device. These are all centrally managed by a measurement and control system, which can accurately control and collect all operating parameters, including media type M, ambient temperature T, relative humidity H, media coverage C, axle load Q, and operating speed v. The specific implementation methods for each parameter are as follows: Ambient temperature T: The entire basic test setup is placed in a high and low temperature chamber, which can achieve continuous temperature adjustment from -30℃ to 50℃. The chamber is set with an upper temperature range for water, oil, rust, and leaf media, and a lower temperature range for ice, snow, and frost media, to ensure the stability of the media during the test. Temperature data is collected in real time by the temperature measurement unit inside the chamber.

[0043] Relative humidity H: The high and low temperature chamber is equipped with a digitally controlled humidifier, which can accurately regulate the relative humidity of the environment within a set range. Once the set humidity is reached, humidification will automatically stop and maintain a constant humidity. The humidity data is collected in real time by the digital display module that comes with the humidification unit.

[0044] Medium type M and medium coverage C: Based on the existing sprinkler system, a metered water inlet and a rotatable, speed-adjustable snow sprayer are added; liquid media such as water and oil are applied by spraying through the metered water inlet, while ice, snow, and frost media are generated by the snow sprayer in conjunction with the low-temperature environment, and solid media such as rust and leaves are applied by uniform paving, thereby determining the rail surface medium type M; the rail surface medium coverage C is quantitatively controlled by adjusting the medium flow rate, snow generation rate, and paving amount.

[0045] Axle load Q: A pressure regulating valve and a pressure acquisition unit are set in the wheel loading mechanism. Different levels of axle load are applied by adjusting the pressure regulating valve. Pressure data is acquired in real time and fed back to the measurement and control system to achieve precise control and acquisition of axle load Q.

[0046] Operating speed v, slip ratio With viscosity coefficient The scheme of using a vehicle speed motor to control the revolution speed and a wheel speed motor to control the wheel rotation speed in the basic device is adopted. The train running speed and wheel axle angular velocity are simulated respectively, and the slip ratio is calculated by combining the effective rolling radius of the wheel. The adhesion coefficient under the corresponding working condition is calculated by collecting wheel circumference torque using a torque meter and combining it with real-time axle load data. .

[0047] The coefficients to be determined in the coupling function were obtained by fitting the bench test data. The test was conducted using the hierarchical orthogonal test method, based on the above-mentioned extended bypass wheel-rail adhesion test bench.

[0048] The complex environment bypass wheel-rail adhesion test bench of this application embodiment includes core components such as a rotating arm, wheels, a circular track, a vehicle speed motor, a wheel speed motor, an end-driven pump, a circular uniformly distributed nozzle, a water tank, a cylinder piston rod, a pressure sensor, and a torque meter. The rotating arm's revolution simulates the train's running speed, the wheel's rotation simulates the wheel axle speed, the cylinder applies a set axle load, the nozzle quantitatively sprays the target medium onto the rail surface, and the pressure sensor and torque meter respectively collect corresponding mechanical data for calculating the adhesion coefficient.

[0049] During the experiment, medium type M was used as the outer group, and independent orthogonal experiments were conducted for each medium type: water, oil, rust, leaves, ice, snow, and frost. Within each group, five orthogonal factors were set up: medium coverage, ambient temperature, relative humidity, wheel-rail rolling speed, and axle load. Each factor had four levels, using L... 16 (4 5 Orthogonal arrays are used to arrange experiments.

[0050] Temperature levels are zoned according to the physical properties of different media: Water, oil, rust, and leaf media: Temperature levels are set to 5℃, 15℃, 25℃, and 35℃ (above zero) to match the temperature range in which the media can stably exist. Ice, snow, and frost media: Temperature levels are individually set to -20℃, -15℃, -10℃, and -5℃ (sub-zero range) to ensure the stability of the media during the test.

[0051] The specific values ​​of each factor in the water, oil, rust, and leaf groups are shown in Table 1 below:

[0052] Taking the water, oil, rust, and leaf media as an example, L... 16 (4 5 The 16 test conditions arranged by the orthogonal array are shown in Table 2 below:

[0053] In Table 2, numbers 1 to 4 correspond to levels 1 to 4 of each factor. The ice, snow, and frost media groups are arranged using orthogonal tables with the same structure, except that the temperature levels are replaced with the corresponding levels in the sub-zero range.

[0054] Considering the differences between bench tests and actual line operating environments, three representative operating conditions—low adhesion, medium adhesion, and high adhesion—were selected from the bench database for real-vehicle line tests. The model coefficients fitted by the bench were corrected and supplemented using real-vehicle test data, ultimately obtaining an adhesion coefficient dataset that fits the actual operating scenario.

[0055] The undetermined coefficient c in the multi-factor coupling function pq,M The following was obtained through a two-stage fitting process: The first stage is single-condition fitting for bench testing: For each set of determined working conditions (medium type M, medium coverage C, ambient temperature T, relative humidity H, operating speed v, axle load Q), adhesion coefficient data across the entire slip ratio range are collected on the test bench to obtain the μ-λ adhesion curve for that working condition. Nonlinear fitting is then used to obtain the corresponding c1, c2, and c3 parameter values. The second stage is multi-factor regression using neural networks: Features such as T, H, C, Q, and v from all orthogonal test conditions are used as input, and the corresponding working condition fitting parameters c1, c2, and c3 are used as fitting targets. A grouped convolutional neural network is trained, and finally, all undetermined coefficients c in the multi-factor coupling function are obtained through regression. pq,M .

[0056] The grouped convolutional neural network adopts a multi-path parallel grouping structure. Each group performs independent one-dimensional convolutional feature extraction on different categories of input features, and then performs feature fusion and regression to output the coefficients to be determined.

[0057] Figure 2 This is a schematic diagram of the grouped convolutional neural network structure of this application.

[0058] exist Figure 2 In this paper, the grouped convolutional neural network adopts a three-way parallel grouped convolutional architecture, which is divided into an input region and an output region, containing three independent grouped convolutional processing branches. The three branches correspond to three types of input features with different physical meanings. The first input corresponds to the LAB color feature, LBP texture feature and medium coverage of the rail surface medium. The second input corresponds to the environmental parameters of ambient temperature and relative humidity. The third input corresponds to the train running speed, angular velocity and axle load running status parameters. Each group of input features is independently extracted by the convolutional layer in its own group. The convolutional operations of each group do not cross or interfere with each other. Finally, the feature vectors of the corresponding groups are output for subsequent feature fusion and regression calculation of the coefficients to be determined in the adhesion model.

[0059] The network is configured with three parallel groups, each processing different categories of input features: The first group (which may be the rail surface medium feature group in this embodiment): Input: LAB color features, LBP texture features, and media coverage of the track surface medium; Structure: Two layers of one-dimensional convolution and global average pooling; the first convolutional layer has 32 channels and a kernel size of 5, and the second convolutional layer has 64 channels and a kernel size of 3; Output: 64-dimensional first feature vector.

[0060] The second group (which may be an environmental feature group in this embodiment): Input: Ambient temperature, relative humidity; Structure: Two layers of one-dimensional convolution and global average pooling; the number of channels in the two convolutional layers are 16 and 32 respectively, and the kernel size is 3 for both layers; Output: 32-dimensional second feature vector.

[0061] The third group (which may be the vehicle operation characteristic group in this embodiment): Input: Train speed, angular velocity, axle load; Structure: Two layers of one-dimensional convolution and global average pooling; the number of channels in the two convolutional layers are 16 and 32 respectively, and the kernel size is 3 for both layers; Output: 32-dimensional third eigenvector.

[0062] The feature vectors from the three group outputs are concatenated to obtain a 128-dimensional fused feature vector. This fused feature vector is then passed through two fully connected layers (64-dimensional and 32-dimensional) to finally regress and output all the undetermined coefficients c of the adhesion model under the corresponding working condition. pq,M .

[0063] Each convolutional layer is followed by a ReLU activation function and a max pooling layer. The ReLU activation function expression is: In the formula, i is the index of the input feature, j is the index of the neuron, and x i For the i-th input feature, b represents the connection weights between neurons. j For the bias term, y j This is the output of the neuron.

[0064] The max-pooling layer kernel size is 2, which is used for downsampling to compress the feature dimension.

[0065] By training and fitting all orthogonal experimental data for each type of medium, a complete set of coefficients corresponding to each medium is obtained. The medium type is then bound to the corresponding adhesion model and stored to form a track surface adhesion model database, which can be directly called during online sensing.

[0066] 4. Real-time calculation of wheel-rail adhesion coefficient.

[0067] Input the real-time collected environmental parameters and operating status parameters into the selected adhesion model to calculate the wheel-rail adhesion coefficient. The specific steps are as follows: Based on the train's running speed v and axle angular velocity Calculate the slip ratio using the wheel's effective rolling radius R. The calculation formula is: In the formula, the axle angular velocity The effective rolling radius R of the wheel, obtained from the speed sensor, is the vehicle's preset calibration value.

[0068] Adhesion coefficient calculation: Substitute the calculated slip ratio, as well as the real-time collected ambient temperature, relative humidity, train speed, and axle load, into the selected adhesion model to calculate the estimated wheel-rail adhesion coefficient under the current working conditions.

[0069] 5. Results output and application.

[0070] The calculated adhesion coefficient, slip ratio, and identified rail surface medium type are sent to the driver's cab display terminal, providing adhesion status display, adhesion alarm, and driving suggestion functions. At the same time, the data can be uploaded to the train operation dispatching system to support train operation dispatching management in complex environments.

[0071] II. An online real-time adhesion sensing system based on multi-sensor fusion.

[0072] This system is used to implement the above-mentioned online real-time adhesion sensing method. It consists of three parts: a multi-sensor data acquisition module, an adhesion sensing module, and a cab display terminal. The data flow is as follows: multiple sensors synchronously collect raw data to the adhesion sensing module to complete signal reception, preprocessing, medium identification and adhesion calculation, and output the sensing results to the cab terminal.

[0073] Figure 3 This is a schematic diagram illustrating the overall architecture and workflow of the multi-sensor fusion adhesive online real-time sensing system of this application.

[0074] exist Figure 3 In the system, from left to right, there are three functional levels: multi-sensor data acquisition, online real-time adhesion perception, and cab decision-making. The left-hand multi-sensor data acquisition level shows the bogie layout and equipment illustrations of four types of acquisition devices: onboard cameras, temperature and humidity probes, axle speed sensors, and Doppler radar. It is responsible for collecting raw sensing data such as track surface images, ambient temperature and humidity, axle speed, and vehicle speed, and transmitting them to the intermediate processing unit. The middle online real-time adhesion perception level uses boards and a host as hardware carriers. It completes the data reception, image recognition and medium judgment, model matching, and real-time adhesion calculation in the order of data flow, and finally outputs the adhesion coefficient estimate, track surface medium type, and slip ratio results. The right-hand cab decision-making level corresponds to the cab display terminal, which can display the adhesion coefficient, slip ratio, and track surface medium type information in real time, and integrates three functions: adhesion display, adhesion alarm, and driving suggestions, providing decision support for train operation and control.

[0075] Figure 4 This is a block diagram of the multi-sensor fusion adhesive online real-time sensing system of this application.

[0076] exist Figure 4 The perception system is divided into four parts: multi-sensor data acquisition, input signals, adhesion perception module, and output signals, clearly presenting the system's modular composition and data flow logic. The multi-sensor data acquisition side on the left includes a temperature sensor, humidity sensor, speed sensor, Doppler radar, on-board camera, and rail surface adhesion database, corresponding to the temperature, humidity, axle speed, train speed, rail surface medium image, and rail surface adhesion patterns of various media in the input signals, respectively. The adhesion perception module in the middle is the core processing unit of the system, internally set up with three levels of processing units: boards, processing modules, and host. The boards are responsible for receiving various input signals, the processing modules complete rail surface image segmentation, extraction, normalization processing, and train operation data processing, and the host realizes rail surface medium identification, selection of corresponding adhesion model families, and calculation of adhesion parameters. The rightmost part is the output signal section, where the system finally outputs three types of perception results: adhesion coefficient, slip ratio, and rail surface medium.

[0077] 1. Multi-sensor data acquisition module.

[0078] The multi-sensor data acquisition module is deployed on the train bogie and under the car body, and includes temperature sensors, humidity sensors, speed sensors, Doppler radar, and onboard cameras, which are used to collect ambient temperature, ambient humidity, axle speed, train speed, and track surface images, respectively. The onboard camera is a high-definition industrial RGB camera, which is placed in a relatively stable position on the bogie and shoots towards the rail surface area in front of the wheel-rail contact point to continuously acquire RGB images of the rail surface. Temperature and humidity sensors are installed at the bottom of the vehicle body near the rail surface, and are used to collect the ambient temperature and relative humidity of the train's operating environment in real time. Speed ​​sensors are installed at the ends of each axle to collect the speed of each axle end; The Doppler radar is installed at the bottom of the vehicle body and uses non-contact measurement to obtain the train's speed.

[0079] The main data input signals required for image recognition and adhesion estimation in this application system are detailed in Table 3 below.

[0080]

[0081] 2. Adhesion sensing module.

[0082] The adhesion sensing module is the core computing unit of the system, including boards, processing modules and host.

[0083] The board is used to receive various signals collected by the multi-sensor data acquisition module, including image signals, temperature and humidity signals, rotational speed pulse signals and radar speed signals, and transmit the data to the processing module.

[0084] The processing module is used to segment, extract, and normalize the received track surface images, preprocess the train operation data, and calculate the real-time slip rate based on the vehicle speed and axle speed.

[0085] The host is used to identify the rail surface medium based on the processed rail surface image, and call the adhesion model corresponding to the identified rail surface medium type from the pre-established rail surface adhesion model database. The pre-processed ambient temperature, ambient humidity, train speed and axle speed are input into the adhesion model to calculate the adhesion coefficient and slip ratio, and output the adhesion coefficient, slip ratio and rail surface medium type.

[0086] The adhesion models and adhesion coefficient calculation methods in the rail surface adhesion model database adopt the content described in the above-mentioned method section.

[0087] 3. Driver's cab display terminal.

[0088] The driver's cab display terminal is deployed in the driver's control room, receives the sensing results output by the main unit, and has three functions: Adhesion status display: Real-time display of current wheel-rail adhesion coefficient, slip ratio, and rail surface medium type; Adhesion alarm function: Triggers an alarm when the adhesion state is abnormal; Driving suggestion function: Provides corresponding driving operation suggestions based on the current adhesion status.

[0089] The technical scope of this application is not limited to the contents of the above specification. Those skilled in the art can make various modifications and variations to the above embodiments without departing from the technical concept of this application, and all such modifications and variations should fall within the scope of this application.

Claims

1. A real-time online adhesion sensing method based on multi-sensor fusion, characterized in that, Includes the following steps: Real-time acquisition of track surface images, environmental parameters, and operating status parameters during train operation; The track surface image is processed to identify the track surface medium type; Based on the identified rail surface medium type, select the corresponding adhesion model from the pre-established rail surface adhesion model database; The environmental parameters and operating status parameters collected in real time are input into the selected adhesion model to calculate the wheel-rail adhesion coefficient. The adhesion model in the rail surface adhesion model database is a parametric friction model. The parametric friction model expresses the adhesion coefficient as a function of the slip ratio and includes a first parameter, a second parameter, and a third parameter that control the amplitude and shape of the adhesion coefficient. The first parameter, the second parameter, and the third parameter are respectively constructed as coupling functions that include the medium type and multiple influencing factors. The coupling function includes at least a cross-coupling term describing the interaction between different environmental factors and a cross-coupling term describing the interaction between wheel-rail contact state factors. The coupling function has coefficients to be determined. The coefficients to be determined in the coupling function are obtained by fitting pre-trained grouped convolutional neural networks based on bench test data. The grouped convolutional neural network adopts a multi-path parallel grouping structure, in which each group performs independent one-dimensional convolutional feature extraction on different categories of input features, and then performs feature fusion and regression to output the coefficients to be determined.

2. The method according to claim 1, characterized in that, The step of image processing of the rail surface image to identify the rail surface medium type specifically includes: The track surface image is deblurred and its size is normalized to obtain a normalized image; A semantic segmentation network is used to extract the rail tread region from the normalized image; The extracted image corresponding to the rail tread area is input into a media classification network, which outputs the rail surface media type. The loss function of the semantic segmentation network is a joint loss function of weighted cross-entropy loss and DiceLoss, expressed as: N is the total number of pixels involved in the loss calculation, K is the number of semantic segmentation categories, u is the pixel index, k is the semantic category index, and y is the number of semantic segmentation categories. uk p indicates whether the u-th pixel truly belongs to the k-th class. uk Let represent the probability that the u-th pixel belongs to the k-th class. The unbalanced weights for the number of pixels in the k-th class are... The weights are the weight coefficients for the weighted cross-entropy loss. For numerically stable terms, Let L be the class importance weight of the k-th class in the Dice loss, and L be the value of the joint loss function. The media classification network uses ResNet as the feature extraction network and is trained using transfer learning and metric learning; during training, anchor samples I... a Similar Samples I p and outlier sample I n The feature extraction network with shared weights is input to obtain feature vectors z. a z p and z n The following triplet loss function is used to constrain the distance between samples of the same class to be less than the distance between samples of different classes: ； In the formula, m is the preset inter-class distance interval, and L′ is the triple loss function value; The media classification network outputs a media feature vector z=[z1,z2,…,z…]. 128 ] T And the probability of each medium category P=[p1,p2,…,p K ], where p k The probability that the image corresponding to the rail tread area belongs to the k-th type of medium is represented by the category corresponding to the highest probability. The category corresponding to the highest probability is taken as the rail surface medium identification result.

3. The method according to claim 1, characterized in that, The parameterized friction model is expressed as: ， In the formula, The adhesion coefficient is... Where c1 is the slip ratio, c2 is the first parameter, c3 is the second parameter, M is the medium type, T is the ambient temperature, H is the relative humidity, C is the medium coverage, Q is the axle load, and v is the train speed. The first parameter c1, the second parameter c2, and the third parameter c3 are respectively constructed as multi-factor coupling functions related to the medium type M, ambient temperature T, relative humidity H, medium coverage C, axle load Q, and train speed v: ， Among them, c pq,M For the coefficients to be determined that depend on the medium type M, c pq,M In the table, the subscripts p and q are the row and column identifiers of the coefficient matrix, TH is the temperature-humidity cross-coupling term, Hv is the humidity-velocity cross-coupling term, and CQ is the medium coverage-axle weight cross-coupling term.

4. The method according to claim 3, characterized in that, Using the parametric friction model, the rate of increase of the adhesion coefficient when the slip ratio approaches zero is... Determine the slip ratio corresponding to the peak adhesion coefficient. for c1, c2 and c3 are all greater than 0 and c1c2>c3.

5. The method according to claim 3, characterized in that, The coefficient c to be determined in the multi-factor coupling function pq , M The two-stage fitting method yields the following: First, the adhesion coefficient-slip ratio curves under various working conditions are obtained through orthogonal bench tests, and c1, c2, and c3 corresponding to a single working condition are fitted. Then, using the working condition characteristic parameters as input and c1, c2, and c3 corresponding to the working condition as fitting targets, the coefficients to be determined in the coupling function are obtained through regression using the grouped convolutional neural network. pq,M ; The grouped convolutional neural network is constructed in the following manner: The first group of input track surface medium LAB color features, LBP texture features and medium coverage are processed by at least two layers of one-dimensional convolution and global average pooling to obtain the first feature vector. The second group inputs ambient temperature and relative humidity, and after at least two layers of one-dimensional convolution and global average pooling, a second feature vector is obtained. The third group inputs the train's running speed, angular velocity, and axle load, and after at least two layers of one-dimensional convolution and global average pooling, a third feature vector is obtained. The first feature vector, the second feature vector, and the third feature vector are concatenated to form a fused feature, which is then passed through a fully connected layer to output the coefficient c to be determined. pq,M .

6. The method according to claim 5, characterized in that, In the grouped convolutional neural network, each convolutional layer is followed by a ReLU activation function and a max-pooling layer, wherein the ReLU activation function is expressed as: ， In the formula, i is the index of the input feature, j is the index of the neuron, and x i For the i-th input feature, b represents the connection weights between neurons. j For the bias term, y j This is the output of the neuron.

7. The method according to claim 5, characterized in that, The method for establishing the orthogonal bench test is as follows: The medium type M is used as the outer layer group. For each medium, five orthogonal factors are set: medium coverage, ambient temperature, relative humidity, wheel-rail rolling speed and axle load. Each factor is set with multiple levels. The experiment is arranged according to the orthogonal table to obtain the adhesion coefficient data under each working condition. For water, oil, rust, and leaf media, the temperature level is set in the upper zero range; for ice, snow, and frost media, the temperature level is set separately in the lower zero range.

8. The method according to claim 1, characterized in that, The calculation of the wheel-rail adhesion coefficient includes: Based on the train's running speed v and axle angular velocity Calculate the slip ratio using the wheel's effective rolling radius R. The calculation formula is: The calculated slip ratio The ambient temperature and relative humidity collected in real time are input into the selected adhesion model to calculate the estimated value of the adhesion coefficient.

9. The method according to any one of claims 1 to 8, characterized in that, Also includes: The calculated adhesion coefficient, slip ratio, and identified rail surface medium type are sent to the cab display terminal, which provides adhesion status display, adhesion alarm, and driving suggestion functions.

10. A real-time online adhesion sensing system based on multi-sensor fusion, characterized in that, include: The multi-sensor data acquisition module includes a temperature sensor, a humidity sensor, a speed sensor, a Doppler radar, and an on-board camera, which are used to acquire ambient temperature, ambient humidity, axle speed, train speed, and track surface images, respectively. Adhesion sensing module, including board, processing module and host; The board is used to receive various signals collected by the multi-sensor data acquisition module; The processing module is used to segment, extract, and normalize the received track surface image, and to preprocess the train operation data. The host is used to identify the rail surface medium based on the processed rail surface image, and call the adhesion model corresponding to the identified rail surface medium type from the pre-established rail surface adhesion model database. The host inputs the pre-processed ambient temperature, ambient humidity, train speed and axle speed into the adhesion model, calculates the adhesion coefficient and slip ratio, and outputs the adhesion coefficient, the slip ratio and the rail surface medium type. The adhesion model in the rail surface adhesion model database and the method for calculating the adhesion coefficient are adopted according to any one of claims 1 to 9.

Images