Wheel-rail lateral displacement measuring method and apparatus for rail vehicle

Abstract

The present invention provides a wheel-rail lateral displacement measuring method and apparatus for a rail vehicle. The measuring method comprises: acquiring an original vehicle wheel-rail contact image set, and marking a plurality of pieces of key point information; inputting the image set with a plurality of key point information labels into a key point prediction network to obtain a predicted position and a predicted displacement deviation of each key point in contact with a vehicle wheel rail, and a predicted distance between two specified key points; measuring a real distance between the two specified key points by a displacement sensor, and obtaining a distance deviation between the two specified key points; fusing the distance deviation, a predicted key point coordinate position on the wheel rail and a corresponding predicted displacement deviation to perform training so as to obtain an optimized key point prediction network; and inputting a wheel-rail contact video into the optimized key point prediction network to obtain coordinate values of key points in wheel-rail contact images and a wheel-rail lateral relative displacement time-history curve. The measuring method provided in the present invention can improve the speed and accuracy of wheel-rail lateral displacement measurement, without setting a target.

Description

A method and device for measuring lateral displacement of rail vehicle wheels and rails Technical Field

[0001] The present invention relates to the technical field of rail vehicle wheel-rail lateral displacement detection technology, and in particular to a rail vehicle wheel-rail lateral displacement measurement method and measurement device based on key point detection. Background Art

[0002] To ensure the operational safety of rail transit vehicles under extreme conditions, such as earthquakes and crosswinds, scholars at home and abroad have conducted extensive research on derailment phenomena, derailment principles, and derailment criteria. However, accurate criteria for assessing train derailment remain lacking. The current derailment safety assessment standards used in my country are primarily based on indicators such as the derailment coefficient and wheelweight reduction rate derived from wheel-rail forces. Wheel-rail forces are primarily measured by dynamometric wheelsets. These wheelsets measure the forces acting between the wheels and rails using strain gauges placed on the wheelsets. These wheels then calculate the wheel-rail forces to determine derailment evaluation indicators such as the derailment coefficient. Finally, train safety assessments are conducted based on these calculations. However, due to the drawbacks of wheel-rail force measurement devices such as dynamometric wheelsets, such as these, such as difficult maintenance, complex calibration, and high cost, their widespread adoption has been hindered.

[0003] The relative position of the wheel and rail is the most intuitive indicator of whether a train's wheels have derailed. The wheel flange is embedded in the rail. When the wheel's lateral displacement is excessive and the wheel flange reaches the top of the rail, the likelihood of derailment increases significantly. Given that wheel-rail contact images at high speeds often feature dynamic, complex backgrounds and exhibit motion blur, traditional image processing algorithms are prone to performance degradation when processing such wheel-rail contact images, resulting in low processing efficiency and poor recognition accuracy. Furthermore, it is difficult to install targets at the wheel-rail contact location.

[0004] Therefore, there is an urgent need for a method and device for measuring the lateral displacement of the wheel and rail of a railway vehicle, which does not require setting a target and can greatly improve the speed and accuracy of the wheel and rail lateral displacement measurement.

[0005] Summary of the Invention

[0006] The purpose of the present invention is to provide a method and a device for measuring the lateral displacement of a rail vehicle wheel and rail, aiming to solve the technical problems of low detection efficiency and low accuracy of traditional measurement methods and measurement devices.

[0007] To achieve the above objectives, in a first aspect, the present invention provides a method for measuring the lateral displacement of a rail vehicle wheel / rail, the method comprising the following steps:

[0008] S1: Obtain an original vehicle wheel-rail contact image set and mark multiple key point information of the original vehicle wheel-rail contact image set;

[0009] S2: Input the vehicle wheel-rail contact image set with multiple key point information labels into the key point prediction network to obtain the predicted position and predicted displacement deviation of each key point of the vehicle wheel-rail contact, and calculate the predicted distance between two specified key points based on the predicted position of the key point;

[0010] S3: measuring the actual distance between the two designated key points by a displacement sensor, and subtracting the predicted distance from the actual distance to obtain a distance deviation between the two designated key points;

[0011] S4: fusing the distance deviation between the two designated key points, the predicted position of the key point coordinates on the wheel / rail, and the corresponding predicted displacement deviation to correct and train the key point prediction network to obtain an optimized key point prediction network;

[0012] S5: Decompose the wheel-rail contact video obtained by real-time shooting into wheel-rail contact images, and input the images into the optimized key point prediction network to obtain the coordinate value of each key point in the wheel-rail contact image, and obtain the wheel-rail lateral relative displacement time history curve.

[0013] As a further optimization of the above scheme, in step S5, when obtaining the wheel-rail lateral relative displacement time-history curve, the conversion relationship between the pixel coordinate system and the world coordinate system is first calibrated by the actual distance between the two specified key points measured by the displacement sensor, and then the wheel-rail lateral relative displacement time-history curve is obtained by combining the mathematical geometric relationship between the selected key points on the wheel and rail with the scale conversion coefficient between the pixel coordinate system and the world coordinate system.

[0014] As a further optimization of the above solution, in step S1, when obtaining the original vehicle wheel-rail contact image set, each vehicle wheel-rail contact image needs to be preprocessed so that the front of the vehicle wheel-rail contact has a preset proportion in the corresponding image.

[0015] As a further optimization of the above scheme, in step S1, the multiple key point information of the original vehicle wheel-rail contact image set is marked, including the marking of the multiple key point position coordinate information of all images in the vehicle wheel-rail contact image set, and the marking of the distance information between two specified key points, constituting the original data set of the key point prediction network.

[0016] As a further optimization of the above solution, multiple key points of the vehicle wheel-rail contact image include the center point of the outer edge of the wheel, the edge point of the bottom of the wheel, the wheel-rail contact edge point, the center point of the bogie axle box outside the wheel, and a fixed point on the car body;

[0017] The two designated key points are the center point of the bogie axle box outside the wheel and a fixed point on the carriage.

[0018] As a further optimization of the above scheme, the key point prediction network includes a size modification network and a prediction network of fusion offset connected in sequence.

[0019] The size modification network is used to compress the size of the input image to a preset ratio before transmitting it to the prediction network of the fusion offset;

[0020] The prediction network of the fused offset includes an encoder, a decoder, a confidence branch, a first offset branch and a second offset branch;

[0021] The confidence branch, the first offset branch, and the second offset branch are respectively provided with corresponding loss functions to constrain the training network to obtain corresponding outputs.

[0022] As a further optimization of the above scheme, the resizing network contains three convolutional layers, each with a convolution kernel size of 3×3, a convolution stride of 2, and an expansion edge width of 1;

[0023] After each convolution layer, the output data is processed using the Prelu nonlinear activation function and batch normalization method.

[0024] As a further optimization of the above scheme, the encoder includes four downsampling modules and two size preservation modules. The downsampling modules are used to obtain low-resolution feature maps (such as edges, textures, etc.) to reduce computational complexity, and the size preservation modules are used to ensure that the input and output feature maps have the same size.

[0025] The decoder includes four upsampling modules, which are used to restore low-resolution feature maps to high-resolution images and extract high-level object position features;

[0026] The first and third layers of the downsampling module, the upsampling module, and the size retention module all use a 1×1 convolution kernel, and the second layer uses a 3×3 convolution kernel for feature extraction. After each convolution layer, a prelu activation function and a normalization function are added to process the output data.

[0027] As a further optimization of the above scheme, before downsampling, the feature maps at different scales will be copied, and the information will be passed to the deeper layers of the network through the skip layer to provide the network with the ability to understand global information.

[0028] As a further optimization of the above scheme, the confidence branch adopts cross entropy loss as its loss function L confidence , as shown in the following formula:

[0029] Among them, Ge Indicates the cells where key points exist, G n Indicates a cell where no keypoint exists, g c Represents the predicted value of the output cell confidence, g c * Represents the true value of the output cell confidence; if there is a key point in the cell, the true value is 1, otherwise it is 0; γ e and γ n Indicates whether there is a weighted coefficient of loss, g represents a non-empty set G e and G n An element in , used to traverse the non-empty set G e and G n All elements in .

[0030] As a further optimization of the above scheme, the first offset branch uses the mean square error as its loss function L offset , the mean square loss is calculated for the horizontal and vertical coordinates of the predicted key points, as shown in the following formula:

[0031] Among them, N e Represents the number of grids with key points, (g x , g y ) represent the predicted values ​​of the (x, y) coordinates of the key points in the cells where the key points exist, (g x *, g y *) represent the true values ​​of the (x, y) coordinates of the key points in the cells where the key points exist; for grids where no key points exist, since there are no true values ​​of the coordinates, they are ignored when calculating the offset loss.

[0032] As a further optimization of the above scheme, the second offset branch uses the deviation between the true distance and the predicted distance between two specified key points as its loss function L disp The two designated key points are the center point P4 of the bogie axle box outside the wheel and a fixed point P5 on the carriage, as shown in the following formula: disp =|X*-X|

[0033] Wherein, X represents the actual distance between the two designated key points, and X* represents the predicted distance between the two designated key points.

[0034] As a further optimization of the above scheme, the total loss function L of the key point prediction network is total Equal to the confidence loss function L confidence , the first offset branch loss function L offset and the second offset branch loss function L disp The weighted sum of L is shown in the following formula: total =αLconfidence +βL offset +γL disp

[0035] Among them, α, β, and γ are the weighted coefficients corresponding to the three loss functions respectively.

[0036] As a further optimization of the above solution, the wheel-rail lateral relative displacement time history curve output is based on the geometric relationship between three key points; specifically,

[0037] The coordinate positions of the three key points are the center point of the outer edge of the wheel P1 (x1, y1), the bottom edge point of the wheel P2 (x2, y2), and the wheel-rail contact edge point P3 (x3, y3). The wheel-rail lateral displacement in the pixel coordinate system can be calculated by connecting the center point of the outer edge of the wheel P1 and the bottom edge point of the wheel P2 and calculating the distance from the wheel-rail contact edge point P3 to the straight line P1P2. Finally, the actual wheel-rail lateral displacement l can be calculated by combining the scale conversion coefficient SF between the pixel coordinate system and the world coordinate system. r , as shown in the following formula: r =l s -l i =l H ×SF-l i

[0038] in, B0=-1, l s is the absolute wheel-rail lateral displacement, l r is the relative displacement of wheel and rail in lateral direction, i.e. the actual displacement of wheel and rail in lateral direction, l i is the initial displacement value, l H is the absolute lateral displacement of the wheel / rail in the pixel coordinate system.

[0039] As a further optimization of the above solution, the steps for obtaining the scale conversion coefficient SF between the pixel coordinate system and the world coordinate system are as follows:

[0040] The object size l of the known structure at the wheel-rail contact plane to be measured world The corresponding pixel size l pixel The ratio of is used to determine the conversion coefficient SF1 between the pixel coordinate system and the world coordinate system, SF1 = l world / l pixel ;

[0041] The vertical distance d measured by the sensor world The corresponding pixel size d pixel The ratio of d to determine the conversion coefficient SF2 between the pixel coordinate system and the world coordinate system, SF2 = d world / d pixel ;

[0042] Then the scale conversion coefficient SF between the pixel coordinate system and the world coordinate system = αSF1 + βSF2, where α and β are weight values ​​of the two conversion coefficients respectively.

[0043] In a second aspect, the present invention further provides a rail vehicle wheel-rail lateral displacement measuring device, comprising an image acquisition device, a lighting device, a processor and a memory deployed with the optimized key point prediction network provided in the first aspect;

[0044] The image acquisition device is communicatively connected to the processor, is used to be installed on the bottom of the rail vehicle, and is capable of photographing the lateral change process of the wheel and rail from a frontal perspective to collect wheel-rail contact images in real time;

[0045] The processor is used to execute the optimized key point prediction network deployed therein and output the wheel-rail lateral relative displacement in real time;

[0046] The memory is communicatively connected to the processor and is used to store the output result of the processor;

[0047] The lighting device is configured to be disposed adjacent to the wheel-rail contact area to illuminate the wheel-rail interaction area.

[0048] Since the present invention adopts the above technical solution, the beneficial effects of this application are:

[0049] The present invention provides a method for measuring the lateral displacement of a railway vehicle wheel and rail, which obtains an original vehicle wheel and rail contact image set and marks multiple key point information of the original vehicle wheel and rail contact image set; the vehicle wheel and rail contact image set with multiple key point information labels is input into a key point prediction network to obtain the predicted position and predicted displacement deviation of each key point of the vehicle wheel and rail contact, and the predicted distance between two specified key points is calculated based on the predicted position of the key point; the real distance between the two specified key points is measured by a displacement sensor, and the predicted distance is subtracted from the real distance to obtain the distance deviation of the two specified key points; the distance deviation between the two specified key points, the predicted position of the key point coordinates on the wheel and rail, and the corresponding predicted displacement deviation are fused and corrected to train the key point prediction network to obtain an optimized key point prediction network; the wheel and rail contact video obtained by real-time shooting is decomposed into wheel and rail contact images, and the images are input into the optimized key point prediction network to obtain the coordinate value of each key point in the wheel and rail contact image, and a wheel and rail lateral relative displacement time history curve is obtained; the present invention adopts a method based on key points The key point prediction network measures the lateral displacement of the wheel and rail of a rail vehicle. Specifically, the key point prediction network will generate corresponding key point probability heat maps and offset loss predictions for multiple key points selected on the wheel and rail, respectively, which represent the predicted position and predicted displacement deviation of the corresponding key points; and obtain the predicted distance between two specified key points, and then measure the actual distance between the two specified key points through a displacement sensor, and then obtain the distance deviation of the two specified key points, and fuse the obtained key point displacement deviation and distance deviation to perform correction training on the key point prediction network, so as to obtain an optimized key point prediction network, and then input the obtained real-time wheel-rail contact video into the optimized key point prediction network, output the coordinate value corresponding to each key point, and obtain the wheel-rail lateral relative displacement time history curve. Since the wheel-rail lateral relative displacement is determined by the coordinate prediction regression of a few wheel-rail key points in the entire prediction process, the speed of the network prediction measurement can be improved. In addition, in the present invention, the key point prediction network is corrected and trained by the distance deviation, which can improve the accuracy of the network prediction measurement.

[0050] In certain preferred embodiments, when obtaining the wheel-rail lateral relative displacement time history curve, the conversion relationship between the pixel coordinate system and the world coordinate system is first calibrated using the actual distance between two designated key points measured by the displacement sensor. Then, the wheel-rail lateral relative displacement time history curve is obtained by combining the mathematical geometric relationship between the selected key points on the wheel and rail with the scale conversion coefficient between the pixel coordinate system and the world coordinate system, thereby achieving target-free measurement and solving the problem of significant performance loss of traditional image processing algorithms in dynamic and complex background problems.

[0051] In addition, by simply inputting the corrected and trained optimized key point prediction network into the processor and deploying image acquisition equipment, real-time processing of wheel-rail lateral displacement can be achieved, providing technical support for real-time monitoring and timely warning of vehicle operation safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0053] FIG1 is a schematic flow chart of a method for measuring lateral displacement of a rail vehicle wheel / rail disclosed in the present invention;

[0054] FIG2 is a schematic diagram of the definition of key points of wheel-rail relative displacement disclosed in the present invention;

[0055] FIG3 is a schematic diagram of the framework of the rail vehicle wheel-rail lateral displacement measurement method based on key point detection disclosed in the present invention (training phase);

[0056] FIG4 is a schematic diagram of the framework of the rail vehicle wheel-rail lateral displacement measurement method based on key point detection disclosed in the present invention (prediction stage);

[0057] FIG5 is a schematic diagram of wheel-rail displacement conversion disclosed in the present invention;

[0058] FIG6 is a structural block diagram of a rail vehicle wheel-rail lateral displacement measuring device disclosed in the present invention.

[0059] Reference numerals:

[0060] 31. Image acquisition device; 32. Lighting device; 33. Processor; 34. Memory; 35. Power supply; 36. Wheel-rail contact image.

[0061] The realization of the objectives, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0062] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of the present invention.

[0063] It should be noted that all directional indications (such as up, down, etc.) in the embodiments of the present invention are only used to explain the relative position relationship, movement status, etc. between the various components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0064] In addition, the terms "first," "second," and so on, used in this disclosure are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, features specified as "first" or "second" may explicitly or implicitly include at least one of these features.

[0065] Moreover, the technical solutions between the various embodiments of the present invention may be combined with each other, but this must be based on the fact that ordinary technicians in this field can implement them. When the combination of technical solutions is mutually contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0066] Example 1:

[0067] 1-5 , the present invention provides a method for measuring the lateral displacement of a rail vehicle wheel / rail, the method comprising the following steps:

[0068] S1: Obtain an original vehicle wheel-rail contact image set and mark multiple key point information of the original vehicle wheel-rail contact image set;

[0069] Specifically, the original vehicle wheel-rail contact image set includes multiple frontal images of vehicle wheel-rail contact; in this embodiment, the train model motion image measured under earthquake in the Central South University earthquake downtrain system is used as an example to illustrate the specific implementation process of the rail vehicle wheel-rail lateral displacement measurement method provided by the present invention. A type of motion camera is installed directly in front of the train model wheel, so that the camera shooting picture places the wheel-rail interaction area at the center of the camera shooting picture. The camera records the video of the wheel-rail interaction during the operation of the train model, and exports it into a series of pictures according to the video shooting frame rate; the wheel-rail interaction under multiple different working conditions is recorded to form an original vehicle wheel-rail contact image set;

[0070] Each image in the original vehicle wheel-rail contact image set is then preprocessed. Specifically, the front image of the vehicle wheel-rail contact is cropped to the target area of ​​the image to minimize the proportion of the background in the image while including the movement range of the wheel-rail area in the picture.

[0071] In this embodiment, to improve the model training speed and avoid the impact of the complex wheel-rail contact background on the model training stability, the captured image is first cropped in the ROI area, and the original 1920×1080 image is uniformly cropped to a size of 384×896. The principle of image cropping should take into account the movement range of the wheel-rail contact area in the picture, while minimizing the proportion of the background part within the image range.

[0072] Then, we define key points. Specifically, based on the fixed geometric shapes of wheels and rails, and taking into account the difficulty of labeling datasets and the convenience of calculating wheel-rail lateral displacement, we select key points of the vehicle wheel-rail contact image, including the center point P1 of the outer edge of the wheel, the edge point P2 of the bottom of the wheel, the wheel-rail contact edge point P3, the center point P4 of the bogie axle box outside the wheel, and a fixed point P5 on the carriage.

[0073] A dataset is then created based on the five defined key points. The five key points in the original image are annotated to form a dataset, which is then divided into a training set and a test set. Specifically, the dataset annotations include the key point position coordinate information for all images in the vehicle wheel-rail contact image set, as well as the distance information between two selected specified key points. In this embodiment, the distance information between the center point of the bogie axle box on the outer side of the wheel and a fixed point on the car is selected. Therefore, the input to the key point prediction network is the preprocessed wheel-rail contact image RGB data and the annotations of the five key point position coordinates, as well as the annotations of the distance between the two key points on the bogie and the car.

[0074] S2: Input the vehicle wheel-rail contact image set with multiple key point information labels into the key point prediction network to obtain the predicted position and predicted displacement deviation of each key point of the vehicle wheel-rail contact, and calculate the predicted distance between two specified key points based on the predicted position of the key point;

[0075] The obtained training set, specifically, including the pre-processed vehicle wheel-rail contact image and data labels, is input into the key point prediction network as network input and supervision information, respectively. The key point prediction network outputs five key point heat maps of the wheel-rail contact image, the pixel coordinates of the five predicted key points, and the predicted distances between two key points on the bogie and the car;

[0076] S3: measuring the actual distance between the two designated key points by a displacement sensor, and subtracting the predicted distance from the actual distance to obtain a distance deviation between the two designated key points;

[0077] S4: The distance deviation between the two designated key points, the predicted position of the key point coordinates on the wheel-rail, and the corresponding predicted displacement deviation are fused and corrected to train the key point prediction network to obtain an optimized key point prediction network; the fusion correction training process is shown in FIG3 .

[0078] The key point prediction network includes a size modification network and a prediction network for fusion offsets. After multiple training tests, if the total loss function drops to a threshold and remains basically stable, the training can be terminated and the training model can be saved, thereby obtaining an optimized key point prediction network.

[0079] S5: Decomposing the wheel-rail contact video obtained by real-time shooting into a wheel-rail contact image, and inputting the image into the optimized key point prediction network to obtain the coordinate value of each key point in the wheel-rail contact image, and obtaining a wheel-rail lateral relative displacement time history curve;

[0080] To obtain the wheel-rail lateral relative displacement time history curve, the conversion relationship between the pixel coordinate system and the world coordinate system is first calibrated using the actual distance between two designated key points measured by the displacement sensor. Then, the wheel-rail lateral relative displacement time history curve is obtained by combining the mathematical geometric relationship between the selected key points on the wheel and rail with the scale conversion coefficient between the pixel coordinate system and the world coordinate system. This enables target-free measurement and solves the problem of significant performance loss of traditional image processing algorithms in dynamic and complex backgrounds.

[0081] The present invention uses a key point prediction network to measure the lateral displacement of the wheel and rail of a rail vehicle. Specifically, the key point prediction network will generate corresponding key point probability heat maps and offset loss predictions for multiple key points selected on the wheel and rail, respectively, which represent the predicted position and predicted displacement deviation of the corresponding key points; and obtain the predicted distance between two specified key points, and then measure the actual distance between the two specified key points through a displacement sensor, and then obtain the distance deviation of the two specified key points, and fuse the obtained key point displacement deviation and distance deviation to perform correction training on the key point prediction network, so as to obtain an optimized key point prediction network, and then input the obtained real-time wheel-rail contact video into the optimized key point prediction network, output the coordinate value corresponding to each key point, and obtain the wheel-rail lateral relative displacement time history curve. Since the wheel-rail lateral relative displacement is determined by the coordinate prediction regression of a few wheel-rail key points in the entire prediction process, the speed of the network prediction measurement can be improved. In addition, in the present invention, the key point prediction network is corrected and trained by the distance deviation, which can improve the accuracy of the network prediction measurement.

[0082] In addition, by simply inputting the corrected and trained optimized key point prediction network into the processor and deploying image acquisition equipment, real-time processing of wheel-rail lateral displacement can be achieved, providing technical support for real-time monitoring and timely warning of vehicle operation safety.

[0083] As a preferred embodiment, the key point prediction network includes a size modification network and a prediction network for fusion offsets connected in sequence.

[0084] The size modification network is used to compress the size of the input image to a preset ratio before transmitting it to the prediction network of the fusion offset;

[0085] The prediction network of the fused offset includes an encoder, a decoder, a confidence branch, a first offset branch and a second offset branch;

[0086] The confidence branch is used to predict whether there are key points in the image; the first offset branch and the second offset branch are used to accurately locate the key points predicted by the confidence branch, thereby regressing and outputting the accurate position coordinates of the wheel-rail key points;

[0087] The confidence branch, the first offset branch, and the second offset branch are respectively provided with corresponding loss functions to constrain the training network to obtain corresponding outputs;

[0088] Specifically, the resizing network contains three convolutional layers, each with a convolution kernel size of 3×3, a convolution stride of 2, and an expansion edge width of 1. After each convolution layer, the output data is processed using the Prelu nonlinear activation function and batch normalization method.

[0089] Before the image is input into the prediction network of the fusion offset, the image features are preliminarily extracted through the size modification network, and the size of the input image is compressed to 1 / 8 of the original image to save network memory and computing time.

[0090] As a preferred embodiment, the encoder includes 4 downsampling modules and 2 size preservation modules. The downsampling modules perform multiple downsampling operations to reduce the resolution and obtain some low-resolution features (edges, textures, etc.) to reduce the complexity of calculation. The size preservation module is used to ensure that the input and output feature maps have the same size.

[0091] The decoder includes four upsampling modules, which are used to restore low-resolution feature maps to high-resolution images and extract high-level object location features. Before downsampling, feature maps at different scales are copied and passed to deeper layers of the network through skip layers to improve the network's ability to understand global information.

[0092] The first and third layers of the downsampling module, the upsampling module, and the size preservation module all use a 1×1 convolution kernel, and the second layer uses a 3×3 convolution kernel for feature extraction. After each convolution layer, a prelu activation function and a normalization function are added to process the output data, thereby achieving dimensionality reduction, dimensionality increase, and preservation of feature channels, and reducing computer memory and network model parameters.

[0093] As a preferred embodiment, in the process of training and optimizing the key point prediction network, corresponding loss functions are designed for different branches of the network. Specifically,

[0094] The confidence branch uses cross entropy loss as its loss function L confidence , as shown in the following formula:

[0095] Among them, G e Indicates the cells where key points exist, G n Indicates a cell where no keypoint exists, g c Represents the predicted value of the output cell confidence, g c * Represents the true value of the output cell confidence; if there is a key point in the cell, the true value is 1, otherwise it is 0; γ e and γ n Indicates whether there is a weighted coefficient of loss, g represents a non-empty set G e and G n An element in , used to traverse the non-empty set G e and G n All elements in

[0096] The first offset branch uses the mean square error as its loss function L offset , the mean square loss is calculated for the horizontal and vertical coordinates of the predicted key points, as shown in the following formula:

[0097] Among them, N e Represents the number of grids with key points, (g x , g y ) represent the predicted values ​​of the (x, y) coordinates of the key points in the cells where the key points exist, (g x *, g y *) represent the true values ​​of the (x, y) coordinates of the key points in the cells where the key points exist; for grids without key points, since there are no true values ​​of the coordinates, they are ignored when calculating the offset loss;

[0098] The second offset branch uses the deviation between the true distance and the predicted distance between two specified key points as its loss function Ldisp , as shown in the following formula: disp =|X*-X|

[0099] Wherein, X represents the actual distance between the two designated key points (i.e., the actual distance between the two key points on the bogie and the car measured by the displacement sensor), and X* represents the predicted distance between the two designated key points (i.e., the predicted distance between the two key points on the bogie and the car);

[0100] Since the measurement method provided by the present invention obtains the wheel-rail lateral displacement through the positional relationship between different key points, it is not comprehensive to use only the deviation of the predicted position of a single key point as the loss function of the offset branch. Therefore, the deviation between the distance between the key points measured by the displacement sensor and the distance between the two points obtained by network prediction is also used as part of the offset branch loss function. This can greatly reduce the error of the key point position prediction by the detection network, making the prediction of the key point position more accurate.

[0101] Based on the loss functions of the above branches, the total loss function L of the key point prediction network is total Equal to the confidence loss function L confidence , the first offset branch loss function L offset and the second offset branch loss function L disp The weighted sum of L is shown in the following formula: total =αL confidence +βL offset +γL disp

[0102] Among them, α, β, and γ are the weighted coefficients corresponding to the three loss functions respectively;

[0103] In this embodiment, based on the wheel-rail contact image dataset, the resolution of the input image is resized from the original 384×896 to 48×112, and then input into the training network. The batch size (batch_size) of the training data is set to 1, the training period (epoch) is set to 300, the momentum factor (momentum) is set to 0.9, the weight decay factor (weight_decay) is set to 0.00001, and the model learning rate (learning_rate) is set to 2×10 -6 After multiple model training and testing, the confidence loss L confidence Weighting coefficient γ e and γ n Set to 3 and 1 respectively, the total loss function L totalThe weighting coefficients α, β, and γ are all set to 1, and the model can reach the optimal training state. Finally, after training, the loss function basically reaches a stable value. At this time, the training can be terminated and the trained model can be saved.

[0104] The prediction process is shown in Figure 4. First, a set of wheel-rail contact images obtained under a new operating condition without key point annotations is input. The image features are initially extracted through the size modification network, and the size of the input image is compressed to 1 / 8 of the original image. The trained fusion offset prediction network is input and the pixel coordinates of the key points are output. Then, based on the scale conversion coefficient between the pixel coordinate system and the world coordinate system and the geometric relationship between the selected key points, the pixel coordinates of the key points are converted to world coordinates through the displacement calculation module, thereby obtaining the time history curve of the lateral relative displacement of the wheel and rail in the world coordinate system.

[0105] As a preferred embodiment, the wheel-rail lateral relative displacement time history curve output is based on the geometric relationship between three key points; specifically,

[0106] Referring to Figures 2 and 5, the coordinate positions of the three key points are the center point of the outer edge of the wheel P1 (x1, y1), the bottom edge point of the wheel P2 (x2, y2), and the wheel-rail contact edge point P3 (x3, y3). By connecting the center point of the outer edge of the wheel P1 and the bottom edge point of the wheel P2 and calculating the distance from the wheel-rail contact edge point P3 to the straight line P1P2, the wheel-rail lateral displacement in the pixel coordinate system can be calculated. Finally, the actual wheel-rail lateral displacement l can be calculated by combining the scale conversion coefficient SF between the pixel coordinate system and the world coordinate system. r , as shown in the following formula: r =l s -l i =l H ×SF-l i

[0107] in, B0=-1, l s is the absolute wheel-rail lateral displacement, l r is the relative displacement of wheel and rail in lateral direction, i.e. the actual displacement of wheel and rail in lateral direction, l i is the initial displacement value, l H is the absolute lateral displacement of the wheel / rail in the pixel coordinate system.

[0108] As a preferred embodiment, the scale conversion coefficient between the pixel coordinate system and the world coordinate system needs to determine the positional relationship between the shooting camera and the object to be measured. Specifically, the scale conversion coefficient is calculated by calculating the scale conversion coefficient between the pixel coordinate system and the world coordinate system during the measurement of the displacement of the one-dimensional plane object to simplify the conversion relationship between the pixel coordinate system and the world coordinate system; specifically,

[0109] The steps for obtaining the scale conversion coefficient SF between the pixel coordinate system and the world coordinate system are as follows:

[0110] The object size l of the known structure at the wheel-rail contact plane to be measured world The corresponding pixel size l pixel The ratio of is used to determine the conversion coefficient SF1 between the pixel coordinate system and the world coordinate system, SF1 = l world / l pixel ;

[0111] The vertical distance d measured by the sensor world The corresponding pixel size d pixel The ratio of d to determine the conversion coefficient SF2 between the pixel coordinate system and the world coordinate system, SF2 = d world / d pixel ;

[0112] Then the scale conversion coefficient SF between the pixel coordinate system and the world coordinate system = αSF1 + βSF2, where α and β are weight values ​​of the two conversion coefficients respectively.

[0113] Example 2:

[0114] 6 , the present invention further provides a rail vehicle wheel-rail lateral displacement measuring device, comprising an image acquisition device 31 , an illumination device 32 , a processor 33 deployed with the above-mentioned optimized key point prediction network, and a memory 34 ;

[0115] The image acquisition device 31 is in communication with the processor 33 and is mounted on the bottom of the rail vehicle body and is capable of capturing the lateral change process of the wheel and rail from a frontal perspective to collect the wheel-rail contact image 36 in real time;

[0116] The processor 33 is used to execute the optimized key point prediction network deployed therein and output the wheel-rail lateral relative displacement in real time;

[0117] The memory 34 is in communication with the processor 33 and is used to store the output result of the processor 33;

[0118] The lighting device 32 is configured to be positioned adjacent to the wheel-rail contact area to illuminate the wheel-rail interaction area. In this embodiment, the processor 33 and memory 34 are integrated into a computer, and a power supply 35 is also included to power the measuring device. The image acquisition device 31 is a monocular vision camera that captures wheel-rail contact images 36 in real time. The captured images are transmitted to the processor 33 in the computer in real time. The processor 33 executes an optimized key point prediction network, outputs the wheel-rail lateral relative displacement in real time, and stores the measurement results in the memory 34.

[0119] When the processor 33 receives a new image, due to the operation optimization of the key point prediction network, the time for outputting the complete process of the wheel-rail lateral relative displacement in the current image is less than the wheel-rail lateral displacement over-limit action time threshold, thereby enabling real-time monitoring and timely warning of the safety of rail transit vehicle operation.

[0120] The above are only preferred embodiments of the present invention and are not intended to limit the patent scope of the present invention. All equivalent structural transformations made using the contents of the present description and drawings under the inventive concept of the present invention, or direct / indirect applications in other related technical fields, are included in the patent protection scope of the present invention.

Claims

1. A method for measuring the lateral displacement of a rail vehicle wheel rail, characterized in that: The steps include: S1: Obtain an original vehicle wheel-rail contact image set and mark multiple key point information of the original vehicle wheel-rail contact image set; S2: Input the vehicle wheel-rail contact image set with multiple key point information labels into the key point prediction network to obtain the predicted position, predicted displacement deviation, and predicted distance between two specified key points of each key point of the vehicle wheel-rail contact; S3: measuring the real distance between the two designated key points by a displacement sensor to obtain a distance deviation between the two designated key points; S4: fusing the distance deviation between the two designated key points, the predicted position of the key point coordinates on the wheel / rail, and the corresponding predicted displacement deviation to correct and train the key point prediction network to obtain an optimized key point prediction network; S5: Decompose the wheel-rail contact video obtained by real-time shooting into wheel-rail contact images, and input the images into the optimized key point prediction network to obtain the coordinate value of each key point in the wheel-rail contact image, and obtain the wheel-rail lateral relative displacement time history curve.

2. A method for measuring the lateral displacement of a railway vehicle wheel rail according to claim 1, characterized in that: In step S5, when obtaining the wheel-rail lateral relative displacement time-history curve, the conversion relationship between the pixel coordinate system and the world coordinate system is first calibrated using the actual distance between two specified key points measured by the displacement sensor. Then, the wheel-rail lateral relative displacement time-history curve is obtained by combining the mathematical geometric relationship between the selected key points on the wheel and rail with the scale conversion coefficient between the pixel coordinate system and the world coordinate system.

3. A method for measuring the lateral displacement of a railway vehicle wheel rail according to claim 1 or 2, characterized in that: In step S1, the multiple key point information of the marked original vehicle wheel-rail contact image set, including the marking of the multiple key point position coordinate information of all images in the vehicle wheel-rail contact image set, and the marking of the distance information between two specified key points, constitute the original data set of the key point prediction network.

4. A method for measuring the lateral displacement of a rail vehicle wheel rail according to claim 3, characterized in that: The key points of the vehicle wheel-rail contact image include the center point of the outer edge of the wheel, the edge point of the bottom of the wheel, the wheel-rail contact edge point, the center point of the bogie axle box outside the wheel, and a fixed point on the car body; The two designated key points are the center point of the bogie axle box outside the wheel and a fixed point on the carriage.

5. A method for measuring the lateral displacement of a railway vehicle wheel rail according to claim 1 or 2, characterized in that: The key point prediction network includes a size modification network and a prediction network for fusion offsets connected in sequence. Network, The size modification network is used to compress the size of the input image to a preset ratio before transmitting it to the prediction network of the fusion offset; The prediction network of the fused offset includes an encoder, a decoder, a confidence branch, a first offset branch and a second offset branch; The confidence branch, the first offset branch, and the second offset branch are respectively provided with corresponding loss functions to constrain the training network to obtain corresponding outputs.

6. A method for measuring lateral displacement of a railway vehicle wheel rail according to claim 5, characterized in that: The resizing network consists of three convolutional layers, each with a convolution kernel size of 3×3, a convolution stride of 2, and an expansion edge width of 1. After each convolution layer, the output data is processed using the Prelu nonlinear activation function and batch normalization method.

7. The method for measuring the lateral displacement of a railway vehicle wheel and rail according to claim 5, characterized in that: The encoder includes four downsampling modules and two size-preserving modules. The downsampling modules are used to obtain low-resolution feature maps, and the size-preserving modules are used to ensure that the input and output feature maps have the same size. The decoder includes four upsampling modules for restoring low-resolution feature maps to high-resolution images and extracting high-level object position features; The first and third layers of the downsampling module, the upsampling module, and the size retention module all use a 1×1 convolution kernel, and the second layer uses a 3×3 convolution kernel for feature extraction. After each convolution layer, a prelu activation function and a normalization function are added to process the output data.

8. The method for measuring the lateral displacement of a railway vehicle wheel and rail according to claim 5, characterized in that: The confidence branch uses cross entropy loss as its loss function L confidence , as shown in the following formula: Among them, G e Indicates the cells where key points exist, G n Indicates a cell where no keypoint exists, g c Represents the predicted value of the output cell confidence, g c * Represents the true value of the output cell confidence; if there is a key point in the cell, the true value is 1, otherwise it is 0; γ e and γ n Indicates whether there is a weighted coefficient of loss; g represents a non-empty set G e and G n An element in , used to traverse the non-empty set G e and G n All elements in white.

9. The method for measuring the lateral displacement of a railway vehicle wheel and rail according to claim 5, characterized in that: The first offset branch uses the mean square error as its loss function L offset , the mean square loss is calculated for the horizontal and vertical coordinates of the predicted key points, as shown in the following formula: Among them, G e Indicates the cells where key points exist, N e Represents the number of grids with key points, (g x , g y ) represent the predicted values of the (x, y) coordinates of the key points in the cells where the key points exist, (g x *, g y *) represent the true value of the (x, y) coordinates of the key points in the cells where the key points exist; for grids without key points, they are ignored when calculating the offset loss; The second offset branch uses the deviation between the true distance and the predicted distance between two specified key points as its loss function L disp , as shown in the following formula: L disp =|X*-X| Wherein, X represents the actual distance between the two designated key points, and X* represents the predicted distance between the two designated key points.

10. A rail vehicle wheel-rail lateral displacement measuring device, characterized in that: The apparatus comprises an image acquisition device, a lighting device, a processor and a memory configured to implement the method for measuring the lateral displacement of a rail vehicle wheel / rail according to any one of claims 1 to 9; The image acquisition device is communicatively connected to the processor, is used to be installed on the bottom of the rail vehicle, and is capable of photographing the lateral change process of the wheel and rail from a frontal perspective to collect wheel-rail contact images in real time; The processor is used to execute the optimized key point prediction network deployed therein and output the wheel-rail lateral relative displacement in real time; The memory is communicatively connected to the processor and is used to store the output result of the processor; The lighting device is configured to be disposed adjacent to the wheel-rail contact area to illuminate the wheel-rail interaction area.

Images