A method and device for measuring displacement of seamless turnout structure based on computer vision

Abstract

The present application relates to a method and device for measuring the displacement of a seamless turnout structure based on computer vision. The optical flow tracking method is adopted and combined with SYOLO and an improved UNet algorithm, and computer vision is used to predict the displacement of the seamless turnout structure. The detection of the displacement of the seamless turnout structure is achieved by defining four optical flow points on a video containing the displacement of the seamless turnout structure. This scheme transforms the detection problem of the displacement of the seamless turnout structure into the form of pixel difference in the detection image. By observing the trend of the pixel difference, the displacement of the seamless turnout structure can be responded to more accurately and intuitively; and in the prediction, two global to local methods are adopted. Through this attention method, the network can extract more useful information, avoiding the common image resizing steps in traditional methods, thereby completely retaining the original information of the image, improving the accuracy of the prediction results, and providing a more reliable and efficient technical means for the displacement measurement of the seamless turnout structure.

Description

Technical Field

[0001] The present application relates to the field of seamless turnout structure displacement measurement, and in particular to a method and device for seamless turnout structure displacement measurement based on computer vision. Background Art

[0002] Turnouts are essential track infrastructure in the railway and subway industries, and are responsible for the important task of changing tracks during normal operation. Ensuring the hunting stability and operation safety of trains is a basic requirement in the railway transportation process. The displacement of the seamless turnout structure is one of the important factors determining train safety. During the train travel, the hunting movement of the vehicle will cause relative displacement between the wheels and the turnout structure, and a large relative displacement will cause a train derailment accident. The displacement of the seamless turnout structure can intuitively respond to the positional relationship between the wheels and the turnout structure. Online monitoring of the displacement of the seamless turnout structure provides a better basis for ensuring the safety of train operation, thereby reducing the risk of derailment. Existing methods include laser detection methods, thermal imaging detection methods, and traditional image processing methods.

[0003] The laser detection method is developed based on a laser diode module, a two-dimensional PSD module, an intensity filter, and a bandpass interference filter. The laser diode module is installed far away from the track, and the PSD receives the radiation of the laser beam. The displacement is calculated by detecting the center of gravity position of the beam focus in the PSD area. This method mainly detects vertical displacement, but requires the assistance of many laser sensors and receivers, and the PSD module needs to be fixed on the track. Since it requires the assistance of multiple PSD modules, this method is costly and cannot achieve online detection of wheel-rail displacement. The thermal imaging detection method uses infrared radiation from the target object or scene for detection and imaging purposes; the thermal imaging camera is installed at the bottom of the vehicle body, and the relationship between the relative movement of the wheel and the track can be observed by observing the local temperature recorded by the thermal imaging camera. However, since the thermal imaging camera is far away from the wheel-rail contact surface, the position of the wheel-rail contact point cannot be correctly estimated when the train passes through a curved track; the traditional image processing method relies on edge detection and Hough transform operations to extract image features. By applying traditional image processing methods to obtain the required information, the effect of identifying specific targets can be achieved. In order to detect the displacement of the seamless turnout structure, a CCD camera is used to detect the lateral displacement of the wheel and rail through Canny edge detection and Hough transform. The measurement system consists of two stereo cameras, one on each side of the wheelset and two LED light sources. A frame that connects the camera to an artificial light source is designed and fixed to the bogie, which enables real-time capture and detection of wheel-rail displacement images. However, this traditional image detection method relies heavily on the setting of internal parameters. When the external environment changes, the parameters need to be adjusted, the variables are too large, and the practical application value is not high. It can be seen that in the research direction of seamless turnout structure displacement, the detection method based on computer vision is less used, but this method has advantages in real-time monitoring and ensuring detection accuracy. This requires the determination of an effective computer vision detection method that can measure and respond to changes in the distance between the wheel and the seamless turnout structure in real time, thereby ensuring the stability of the hunting movement and operational safety. Summary of the invention

[0004] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a method and device for measuring the displacement of a seamless turnout structure based on computer vision. The optical flow tracking method used is to use computer vision to predict the displacement of the seamless turnout structure, and is combined with SYOLO and an improved UNet algorithm. The detection of the displacement of the seamless turnout structure is achieved by defining four optical flow points on a video containing the displacement of the seamless turnout structure. The scheme can convert the detection problem of the displacement of the seamless turnout structure into the form of pixel difference in the detection image, and by observing the trend of the pixel difference, it can respond to the displacement of the seamless turnout structure more accurately and intuitively.

[0005] This application first discloses a method for measuring the displacement of a seamless turnout structure based on computer vision, and the specific steps are as follows:

[0006] Step 1, ROI detection based on SYOLO model; use CCD camera to capture the image of the contact between the turnout structure and the roller; input the complete turnout contact image captured frame by frame into the SYOLO model for training; wherein, four types of regions of interest (ROIs) are set: the lowest point area 1 of the wheel flange, the lowest point area 2 of the top of the track close to the wheel flange, the midpoint area 3 of the chamfered corner of the tread, and the lowest point area 4 of the top of the track close to the tread; after the training is completed, the video to be predicted is input into the trained SYOLO model for prediction; the size of the prediction box is adjusted through preprocessing to adapt to the optical flow point detection; the turnout contact image of the prediction box with a preset size is output; the image of the region of interest in the prediction box is classified and saved; the prediction boxes are area 1, area 2, area 3 and area 4;

[0007] Step 2: Data enhancement: The classified and saved images are used for data enhancement. For each image in the data set, the brightness, contrast, and saturation of the image are randomly changed, and random noise is added. At the same time, considering the corresponding relationship between the image and the label in the model training set in relative position, the image and the label are randomly flipped in the same direction.

[0008] Step 3: Detect optical flow points based on the IUNet network model; mark the optical flow points for each type of image, and the four corresponding optical flow points are: the lowest point of the wheel rim P1, the lowest point of the top of the track close to the wheel rim P2, the midpoint of the chamfered corner of the tread P3, and the lowest point of the top of the track close to the tread P4; the marked and enhanced data are input into IUNet for training according to the original category, and four trained IUNet models are obtained;

[0009] Step 4: Calculate the position of the optical flow point; the SYOLO model and the four trained IUNet models are integrated into the optical flow tracking method for prediction; local optical flow point detection is performed on each region of interest (ROI) in the SYOLO prediction frame, and for each different category, different IUNet model weights are called according to The function finds the position of the optical flow point:

[0010]

[0011] in, ≥1, indicating the probability amplification factor, represents the pixel value in the feature map of the previous layer, Represents a pixel at a certain position; the global coordinates of each optical flow point , the global coordinates of each prediction box and the local optical flow point coordinates of each region of interest (ROI) get:

[0012]

[0013] in, represents the pixel coordinate on the horizontal axis, and Represents the pixel coordinates on the vertical axis; by combining the global coordinates of the prediction box and the local coordinates within each ROI, the exact position of the optical flow point on the entire map is calculated;

[0014] Step 5: Measure the displacement of the seamless turnout structure; calculate the pixel difference on both sides of the turnout contact area in the image:

[0015]

[0016] in, Represents the number of frames, that is, the total number of video frames; Represents the difference in the horizontal axis pixels on one side of the tread in the image, that is, the pixel position change of the optical flow point on one side of the tread in each frame of the video sequence; It represents the difference of horizontal axis pixels on one side of the rim in the image, that is, the pixel position change of the optical flow point on one side of the rim in each frame of the video sequence; Indicates category The pixel value of the optical flow point on the horizontal axis; category It is the index of the optical flow point, which distinguishes different optical flow points;

[0017]

[0018] in, and are two sets of vectors containing the pixel differences of the corresponding tread side and rim side in all frames;

[0019] By measuring the horizontal pixel changes of the optical flow points in the video frame, the displacement value of the turnout structure is obtained and output; when the displacement value is greater than the preset threshold, an alarm is issued.

[0020] Preferably, the specific structure of the SYOLO network model is:

[0021] It includes three consecutive downsampling parts, replacing the single downsampling structure in the original YOLO model;

[0022] 1) The first downsampling part consists of a depthwise separable convolution module, a batch normalization module, an activation function module, and three bottleneck layer modules;

[0023] 2) The second downsampling part consists of five bottleneck layer modules;

[0024] 3) The third downsampling part consists of three bottleneck layer modules;

[0025] Among them, depthwise convolution is performed in the depthwise separable convolution module, followed by point-wise convolution; batch normalization is applied after the convolution operation to speed up the model training process and reduce internal covariate shift by normalizing the input of the layer;

[0026] Activation function module The activation function is defined as:

[0027]

[0028] in, Represents the pixel value in the feature map of the previous layer. Each pixel contains information about a specific spatial position and channel.

[0029] The bottleneck layer module first converts the feature map of each channel of the input into a higher dimensional feature space through point-by-point convolution, then reduces the computational cost through depth-wise separable convolution, and finally adds the outputs of point-by-point convolution and depth-wise separable convolution through linear projection to obtain the final output feature map.

[0030] Preferably, the label is a heat map centered on the marker, showing the location of the object that the model needs to detect;

[0031]

[0032] Create a 2D grid coordinate matrix of the same size as the image matrix, with and Represent the horizontal and vertical coordinates of the grid points respectively; and Indicates the coordinates of the marked point; Represents the standard deviation that controls the range of the Gaussian distribution.

[0033] Preferably, the specific structure of the IUNet network model is:

[0034] The convolution module consists of a 3×3 convolutional layer, a 1×1 convolutional layer, a 3×3 convolutional layer, and a 1×1 convolutional layer, each followed by batch normalization and Activation function, 1×1 convolutional layer followed by Dropout layer;

[0035] Downsampling module, through 3×3 convolution, batch normalization and The activation function halves the size of the feature map;

[0036] The upsampling module doubles the size of the feature map through bilinear interpolation and reduces the number of channels through 1×1 convolution;

[0037] Feature fusion module, which concatenates and fuses the encoder low-level feature map with the upsampled feature map;

[0038] The output module outputs a single channel feature map through a 3×3 convolution.

[0039] Preferably, The activation function is:

[0040]

[0041] in, Represents pixel value;

[0042] The activation function is:

[0043]

[0044] in, Represents the pixel value in the feature map.

[0045] On the other hand, the present application also provides a seamless turnout structure displacement measurement device based on computer vision, the system comprising:

[0046] at least one processor; and a memory in communication with the at least one processor; wherein,

[0047] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the above-mentioned computer vision-based seamless turnout structure displacement measurement method.

[0048] This application proposes a method and device for measuring the displacement of a seamless turnout structure based on computer vision. The optical flow tracking method used is to use computer vision to predict the displacement of the seamless turnout structure, and is combined with SYOLO and an improved UNet algorithm. The detection of the displacement of the seamless turnout structure is achieved by defining four optical flow points on a video containing the displacement of the seamless turnout structure. This scheme can transform the detection problem of the displacement of the seamless turnout structure into the form of detecting pixel differences in the image. By observing the trend of the pixel difference, the displacement of the seamless turnout structure can be responded to more accurately and intuitively; and in the prediction, two global to local methods are used. This attention method enables the network to extract more useful information, avoids the resizing operation of the original image, and retains the complete information of the image, making the prediction result more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 A schematic diagram of the relative displacement between the wheel and the turnout structure;

[0050] Figure 2Schematic diagram of four types of regions of interest (ROIs);

[0051] Figure 3 The output prediction box is adjusted to a fixed size Schematic diagram of

[0052] Figure 4 Schematic diagram of four marked optical flow points;

[0053] Figure 5 This is a schematic diagram of the structure of the UNet model;

[0054] Figure 6 This is a schematic diagram of the structure of the IUNet model;

[0055] Figure 7 Schematic diagram of the pixel difference on both sides of the turnout contact area in the image. DETAILED DESCRIPTION

[0056] The detailed description set forth below in conjunction with the accompanying drawings is intended to be a description of various exemplary embodiments of the present invention, and is not intended to represent the only embodiment that can practice the present invention. For the purpose of providing a thorough understanding of the present invention, the detailed description includes specific details. However, it is apparent to those skilled in the art that the present invention can be practiced without these specific details. In some instances, well-known structures and components are shown in block diagram form to avoid blurring the concept of the present invention.

[0057] like Figure 1 As shown in the figure, during the running of the train, the hunting motion of the vehicle will produce relative displacement between the wheel and the turnout structure, and a large relative displacement will cause a train derailment accident. The displacement of the seamless turnout structure can intuitively respond to the positional relationship between the wheel and the turnout structure. Online monitoring of the displacement of the seamless turnout structure provides a better basis for ensuring the safe operation of the train, thereby reducing the risk of derailment.

[0058] In order to detect the displacement of seamless turnout structure, a complete video of the turnout contact area is first required; although a complete video is available, it contains a lot of irrelevant information. In order to focus on the turnout contact area, the video data needs to be screened; in order to improve the accuracy and speed of detection, a simplified ROI detection model based on the YOLO model is proposed.

[0059] 1. ROI detection based on SYOLO model

[0060] The complete turnout contact images captured frame by frame are input into the network for training; four types of ROIs are defined; Figure 2As shown in the figure, the lowest point area of ​​the wheel rim (area 1), the lowest point area of ​​the top of the track close to the wheel rim (area 2), the midpoint area of ​​the chamfered corner of the tread (area 3), and the lowest point area of ​​the top of the track close to the tread (area 4); after the training is completed, the SYOLO model trained on the video to be predicted is used for prediction. In order to ensure that the image in the prediction box area can be correctly input into the network for optical flow point detection, the size of the prediction box is adjusted through preprocessing. Figure 3 As shown, the size of the original output prediction box is , by limiting its size, the output prediction box is adjusted to a fixed size The final output of the network is a turnout contact image containing the predicted box. All images in the predicted box are classified and saved.

[0061] YOLO is a popular real-time object detection system that solves the object detection task as a regression problem. The YOLO network model is mainly divided into four parts: input, backbone, neck and head.

[0062] Input part: In this part, the input image is preprocessed to ensure that the image can be correctly input into the network;

[0063] Backbone: In this part, features are extracted from the input image and converted into a multi-layer feature map. This is the most computationally intensive part of the network and is responsible for extracting useful features of the image;

[0064] Neck part: In this part, the extracted shallow graphic features and deep semantic features are fused and processed to enhance the feature expression ability and receptive field of the model;

[0065] Head part: In this part, the prediction box is output to the original captured image based on three feature maps of different sizes to detect objects of different sizes;

[0066] Model optimization: Since the working conditions of the turnout contact video are relatively simple, the backbone of the YOLO model was improved. Specifically, the three downsampling parts in the original YOLO model were replaced with the following three downsampling parts;

[0067] The first downsampling part consists of a depthwise separable convolution module, a batch normalization module, an activation function module, and three bottleneck layer modules;

[0068] The second downsampling part consists of five bottleneck layer modules;

[0069] The third downsampling part consists of three bottleneck layer modules;

[0070] Depthwise Separable Convolution Module This is a convolution operation optimization technique that first performs depthwise convolution and then pointwise convolution. This structure can reduce the number of model parameters and computation while maintaining network performance;

[0071] The batch normalization module applies batch normalization after the convolution operation to speed up the model training process and reduce internal covariate shift by normalizing the input of the layer;

[0072] The activation function module is particularly useful in deep networks because it can provide non-monotonic activation characteristics while being computationally efficient; therefore, its design is particularly important;

[0073] The bottleneck layer module first converts the feature map of each input channel into a higher-dimensional feature space through point-by-point convolution, then reduces the computational cost through depth-wise separable convolution, and finally adds the outputs of point-by-point convolution and depth-wise separable convolution through linear projection to obtain the final output feature map; this structural optimization helps reduce the size and memory consumption of the model, while improving the performance of the model, making it more suitable for real-time detection tasks, such as online monitoring of seamless turnout structure displacement. With this improvement, the YOLO model can process image data more efficiently and improve detection speed and accuracy.

[0074] in, The activation function is defined as:

[0075] (1)

[0076] Represents the pixel value in the feature map of the previous layer. Each pixel contains information about a specific spatial position and channel. This function is used in different There are different expressions within the range of values, thus introducing nonlinearity. value, The functions degenerate to 0 and , while between -3 and +3 a smooth nonlinear transformation is provided.

[0077] The original YOLO model is optimized to create an improved model, commonly referred to as the SYOLO model. This optimization focuses on reducing model size and memory consumption while improving model performance. This model optimization is particularly important for real-time object detection applications because it allows the model to reduce the demand for computing resources while maintaining high accuracy. This is very valuable for systems that require fast responses, such as self-driving cars, video surveillance, and robotic navigation. With this optimization, the SYOLO model is able to achieve efficient object detection in resource-constrained environments.

[0078] 2. Data Augmentation

[0079] The SYOLO model outputs a complete image with a prediction box, and the image with the prediction box of each image is cropped and saved in a classified manner; each category of images is processed separately; during the actual operation of the train, the captured images will be affected by various factors such as weather, lighting, and camera quality; in order to weaken these effects and improve the generalization ability of the model, it is necessary to artificially perform data enhancement operations so that the model can obtain more robust features; in the specific operation, noise is randomly added to the image, and the contrast, saturation and brightness of the image are randomly changed; at the same time, considering the corresponding relationship between the images and labels in the relative positions of the model training set, the images and labels are randomly flipped in the same direction, where the labels are heat maps centered on the marked points, showing the target positions that the model needs to detect;

[0080] (2)

[0081] Create a 2D grid coordinate matrix of the same size as the image matrix, with and Represent the horizontal and vertical coordinates of the grid points respectively; and Indicates the coordinates of the marked point; Represents the standard deviation that controls the range of the Gaussian distribution. In data augmentation, a Gaussian distribution is used to determine the weight of each point in the heat map, which helps the model learn the location of the target more accurately.

[0082] 3. Local optical flow point detection based on IUNet

[0083] As a commonly used image segmentation model, UNet has been widely used in fields such as medical image processing. However, in practical applications, UNet still has some limitations, such as poor recognition of small objects in images. Therefore, we improved the UNet model and named it IUNet to improve the feature extraction ability of the network and find the defined local optical flow points. Figure 4 As shown, first, mark the optical flow points for each type of image. The four corresponding optical flow points are: the lowest point of the wheel rim (P1 in region 1), the lowest point of the top of the track close to the wheel rim (P2 in region 2), the midpoint of the chamfered corner of the tread (P3 in region 3), and the lowest point of the top of the track close to the tread (P4 in region 4). The marked and enhanced data are input into IUNet for training according to the original category, and four trained networks can be obtained.

[0084] like Figure 5As shown in the figure, in terms of the structure of the network model, the model consists of a convolution module, a downsampling module, an upsampling module and an output module, forming an encoder-decoder structure. After the image is input into the network, in the encoder part, the size of the feature map is reduced by stacking multiple layers of convolution and downsampling; in the decoder part, the upsampled feature map is concatenated with the corresponding encoder feature map on the channel, and the size of the feature map is gradually restored through multiple layers of convolution and upsampling; finally, the output of the network is a single-channel mask;

[0085] like Figure 6 As shown in the figure, compared with the original UNet model, the original two 3×3 convolution operations in the convolution module are modified to a combination of "one 3×3 convolution operation + one 1×1 convolution operation + one 3×3 convolution operation + one 1×1 convolution operation"; such operations significantly reduce the number of parameters in the model; batch normalization is added after each convolution operation to accelerate the convergence of the network, and a Dropout layer is added after two 1×1 convolution operations to prevent overfitting, and there is a dropout layer after each normalization layer. The activation function is used in conjunction with it;

[0086] in,

[0087] (3)

[0088] In the previous layer feature map, represents the pixel value; after each convolution module, an attention mechanism module is added, as follows: after each convolution module, the feature map of the output channel is converted into a single-channel weight matrix through a 1×1 convolution, and then regularization is added to ensure the stability of the mean and variance; the value of the weight is limited to between 0 and 1, using The function is calculated as a weight factor for each pixel of the feature map; these weights are multiplied with the feature map of the output channel to achieve enhancement or suppression of the feature map, which improves the network's ability to focus on the feature map and expands the model's receptive field;

[0089] The improved downsampling module uses 3×3 convolution operations, batch normalization operations and Activation function, which reduces the size of the feature map to half of the original size;

[0090] (4)

[0091] in, Represents the pixel value in the feature map;

[0092] The improved upsampling module expands the size of the feature map to twice its original size through bilinear interpolation and reduces the output channel to twice its original size through 1×1 convolution;

[0093] The feature fusion module concatenates and fuses the low-level feature maps of corresponding sizes in the encoder with the upsampled feature maps on the channel, and passes them to the subsequent convolutional blocks;

[0094] In the output module, a 3×3 convolution operation is used to output a Activated single channel feature map;

[0095] 4. Measurement of displacement of seamless turnout structure

[0096] After obtaining the weights of the four IUNet models, the SYOLO model and the four IUNet models were integrated into the optical flow tracking method for prediction; local optical flow point detection was performed on each region of interest (ROI) in the SYOLO prediction box, and for each different category, different IUNet model weights were called and predicted using heat map regression. The function finds the position of the optical flow point, where ≥1, indicating the probability amplification factor, represents the pixel value in the feature map of the previous layer, Represents a pixel at a certain location;

[0097] (5)

[0098] This enables the optical flow points in each region to be detected; the global coordinates of each optical flow point in the entire map, i.e. , the global coordinates of the upper left corner of each prediction box and the local optical flow point coordinates of each region of interest (ROI) get;

[0099] (6)

[0100] in, represents the pixel coordinate on the horizontal axis, and Represents the pixel coordinate on the vertical axis;

[0101] By combining the global coordinates of the predicted box and the local coordinates within each ROI, the exact position of the optical flow point on the entire map can be calculated. This method allows global information to be extracted from local features, thereby achieving accurate positioning of the optical flow point.

[0102] 5. Measurement of displacement of seamless turnout structure

[0103] like Figure 7 As shown in the figure, the displacement of the seamless turnout structure is measured; the pixel difference on both sides of the turnout contact area in the image is calculated:

[0104] (7)

[0105] in, represents the number of frames, i.e., the total number of video frames considered in the analysis; Represents the difference in the horizontal axis pixels on one side of the tread in the image; that is, the pixel position change of the optical flow point on one side of the tread in each frame of the video sequence; Represents the difference in horizontal axis pixels on one side of the rim in the image; that is, the pixel position change of the optical flow point on one side of the rim in each frame of the video sequence; Represents the pixel value of the optical flow point of category i on the horizontal axis; here i is the index of the optical flow point, which can distinguish different optical flow points;

[0106] (8)

[0107] in, and There are two sets of vectors, which contain the pixel differences of the corresponding tread side and flange side in all frames; by measuring the horizontal pixel changes of the optical flow points in the video frame, the displacement value of the turnout structure is obtained and output; when the displacement value is greater than the preset threshold, an alarm is issued; two curves reflecting the displacement changes of the seamless turnout structure can also be obtained; this method can provide important data support for the operation safety and stability of railway vehicles.

[0108] The present application also provides a seamless turnout structure displacement measurement device based on computer vision, the system comprising:

[0109] at least one processor; and a memory in communication with the at least one processor; wherein,

[0110] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the above-mentioned computer vision-based seamless turnout structure displacement measurement method.

[0111] In summary, this scheme is based on a method and device for measuring the displacement of a seamless turnout structure based on computer vision. The optical flow tracking method used in this scheme is to use computer vision to predict the displacement of a seamless turnout structure, and is combined with SYOLO and an improved UNet algorithm. The detection of the displacement of a seamless turnout structure is achieved by defining four optical flow points on a video containing the displacement of a seamless turnout structure. This scheme can transform the detection problem of the displacement of a seamless turnout structure into the form of pixel differences in the detection image. By observing the trend of pixel differences, the displacement of a seamless turnout structure can be responded to more accurately and intuitively; and in the prediction, two global to local methods are used. This attention method enables the network to extract more useful information, avoids the resizing operation of the original image, and retains the complete information of the image, making the prediction result more accurate.

[0112] The above is a detailed description of an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and cannot be considered to limit the scope of implementation of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. A method for measuring the displacement of a seamless turnout structure based on computer vision, the specific steps of which are as follows: Step 1: ROI detection based on SYOLO; use a CCD camera to capture the image of the turnout structure and the roller contacting each other; input the complete turnout contact image captured frame by frame into the optimized SYOLO network for training; wherein, Four types of regions of interest (ROIs) are set: the lowest point area of ​​the wheel flange 1, the lowest point area of ​​the rail top close to the wheel flange 2, the midpoint area of ​​the chamfered corner of the tread 3, and the lowest point area of ​​the rail top close to the tread 4; After training is complete, the video is fed into the network and predictions are made using the trained network; Adjust the size of the prediction box through preprocessing to adapt to optical flow point detection; output the prediction box turnout contact image with a preset size; classify and save the regions of interest in the prediction box; the prediction boxes are region 1, region 2, region 3 and region 4; Among them, the specific structure of the optimized SYOLO network model is: It includes three consecutive downsampling parts, replacing the single downsampling structure in the original YOLO model; 1) The first downsampling part consists of a depthwise separable convolution module, a batch normalization module, an activation function module, and three bottleneck layer modules; 2) The second downsampling part consists of five bottleneck layer modules; 3) The third downsampling part consists of three bottleneck layer modules; Among them, depthwise convolution is performed in the depthwise separable convolution module, followed by point-wise convolution; batch normalization is applied after the convolution operation to speed up the model training process; The f(x) activation function in the activation function module is defined as: Among them, x represents the pixel value in the feature map of the previous layer, and each pixel contains information about a specific spatial position and channel; The bottleneck layer module first converts the feature map of each channel of the input into a higher-dimensional feature space through point-by-point convolution, then reduces the computational cost through depth-wise separable convolution, and finally adds the outputs of point-by-point convolution and depth-wise separable convolution through linear projection to obtain the final output feature map; Step 2: Data enhancement: The classified and saved images are used for data enhancement operations. For each image in the data set, the brightness, contrast, and saturation of the image are randomly changed, random noise is added, and the image and label are randomly flipped in the same direction, taking into account the corresponding relationship between the image and label in the model training set in relative position. Step 3: Optical flow point detection based on the IUNet network model; mark the optical flow points for each type of image, and the four corresponding optical flow points are: the lowest point of the wheel rim P1, the lowest point of the top of the track close to the wheel rim P2, the midpoint of the tread chamfer corner P3, and the lowest point of the top of the track close to the tread P4; the marked and enhanced data are input into IUNet for training according to the original category, and four trained network models are obtained; Among them, the specific structure of the IUNet network model is: The convolution module includes a 3×3 convolution layer, a 1×1 convolution layer, a 3×3 convolution layer, and a 1×1 convolution layer. Each layer is followed by batch normalization and ReLU activation function, and the 1×1 convolution layer is followed by a Dropout layer. Downsampling layer, which halves the feature map size through 3×3 convolution, batch normalization, and LReLU activation function; The upsampling layer doubles the size of the feature map through bilinear interpolation and reduces the number of channels through 1×1 convolution; Feature fusion module, which concatenates and fuses the encoder low-level feature map with the upsampled feature map; The output layer outputs a single channel feature map through 3×3 convolution; Step 4, calculate the position of the optical flow point; the SYOLO model and the four trained IUNet models are integrated into the optical flow tracking method for prediction; local optical flow point detection is performed on each region of interest (ROI) in the SYOLO prediction frame, and for each different category, different IUNet model weights are called to find the position of the optical flow point according to the g(x) function; Among them, β≥1 represents the probability amplification factor, x represents the pixel value in the feature map of the previous layer, i and j represent the pixel points at a certain position; the global coordinates of each optical flow point, i.e., P i (u i ,v i ), the global coordinates B of each predicted box i (u i ,v i ) and the local optical flow point coordinates p of each region of interest (ROI) i (u i ,v i )get; P i (u i ,v i )=B i (u i ,v i )+p i (u i ,v i ) i=(1,2,3,4) Among them, u represents the pixel coordinate on the horizontal axis, and v represents the pixel coordinate on the vertical axis; by combining the global coordinates of the prediction box and the local coordinates within each ROI, the exact position of the optical flow point on the entire map can be calculated; Step 5: Measure the displacement of the seamless turnout structure; calculate the pixel difference on both sides of the turnout contact area in the image: Where f represents the number of frames, that is, the total number of video frames; S 1,f represents the difference of horizontal axis pixels on one side of the tread in the image, that is, the pixel position change of the optical flow point on one side of the tread in each frame of the video sequence; S 2,f represents the difference in the horizontal axis pixels on one side of the rim in the image, that is, the pixel position change of the optical flow point on one side of the rim in each frame of the video sequence; u Pi,f Represents the pixel value of the optical flow point of category i on the horizontal axis; here i is the index of the optical flow point, which can distinguish different optical flow points; Among them, S 1,f and S 2,f are two sets of vectors containing the pixel differences of the corresponding tread side and rim side in all frames; By measuring the horizontal pixel change of the optical flow point in the video frame, the displacement value of the turnout structure is obtained and output; when the displacement value is greater than the threshold, an alarm is issued.

2. A method for measuring the displacement of a seamless turnout structure based on computer vision according to claim 1, wherein: In step 2, the label is a heat map centered on the marker point, showing the location of the object that the model needs to detect; Create a two-dimensional grid coordinate matrix of the same size as the image matrix. The x and y in the matrix represent the horizontal and vertical coordinates of the grid points respectively. cal and cal represents the coordinates of the marked point; σ represents the standard deviation that controls the range of the Gaussian distribution.

3. A method for measuring the displacement of a seamless turnout structure based on computer vision according to claim 1, wherein: The ReLU activation function is: Where x represents the pixel value; The LReLU activation function is: Among them, x represents the pixel value in the feature map.

4. A seamless turnout structure displacement measurement device based on computer vision, characterized in that: The device comprises: at least one processor; and a memory in communication with the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the computer vision-based seamless turnout structure displacement measurement method as described in any one of claims 1-3.

Images