A rail longitudinal displacement monitoring method and system based on visual monitoring

Abstract

This invention discloses a method and system for monitoring longitudinal displacement of rails based on visual monitoring. The system includes a rail target, a camera, and a recognition host. The camera takes a picture of the rail target, and the recognition host performs image recognition based on the longitudinal displacement monitoring method to calculate the longitudinal displacement of the rail. The monitoring method includes: acquiring an initial target image before rail testing and selecting a ROI region of a preset size; dividing the ROI region into multiple sub-regions; acquiring a test target image after rail deformation and performing an optimal search to confirm the position of relevant sub-regions; calculating the pixel displacement of each sub-region based on the correspondence of each pixel in each relevant sub-region to obtain the pixel displacement of the entire ROI region; calibrating the pixel transformation relationship to calculate the actual rail displacement. This invention is based on a sub-image search algorithm for the target image, which, compared with existing pixel-level accuracy schemes, can improve accuracy to the sub-pixel level.

Description

Technical Field

[0001] This invention relates to a method for monitoring longitudinal displacement of rails based on visual monitoring, and also to a corresponding longitudinal displacement monitoring system for rails, belonging to the field of rail transit technology. Background Technology

[0002] Seamless railway tracks are susceptible to deformation due to ambient temperature. In the high temperatures of summer, rail bulging can easily occur, while in winter, rail breakage is more likely. This is detrimental to the safe operation of trains and can easily lead to safety accidents.

[0003] Currently, monitoring in the railway industry mainly relies on manual measurement. Non-contact observation is conducted using equipment such as collimators, and displacement is read manually. Alternatively, cameras are used to capture images of laser beams projected onto a scale for automatic displacement identification. However, these methods require significant manpower, readings are easily affected by installation location, and they only achieve pixel-level accuracy, resulting in low detection precision.

[0004] Therefore, an observation post is set up every 500 meters or less along the seamless line, and it is inspected at least once a month. This method has low measurement efficiency, long inspection distance, requires a lot of manpower and resources, has a long inspection cycle, and cannot achieve real-time monitoring and automatic early warning.

[0005] Chinese patent application No. 201910795825.X discloses a machine vision-based method for monitoring rail displacement, comprising: acquiring a left marker image captured by a left detection camera and a right marker image captured by a right detection camera; segmenting the left marker from the left marker image; segmenting the right marker from the right marker image; calculating the current spatial position of the marker using a binocular recognition algorithm; and calculating the displacement of the marker based on the current spatial position and the reference spatial position of the marker. This method enables non-contact measurement, achieves absolute displacement measurement through a binocular recognition algorithm, adapts to special environments such as tunnels and bridges, is unaffected by signal strength, and requires minimal manpower. Summary of the Invention

[0006] The primary technical problem to be solved by this invention is to provide a method for monitoring the longitudinal displacement of rails based on visual monitoring.

[0007] Another technical problem to be solved by the present invention is to provide a rail longitudinal displacement monitoring system based on visual monitoring.

[0008] To achieve the above-mentioned technical objectives, the present invention adopts the following technical solution:

[0009] According to a first aspect of the present invention, a method for monitoring longitudinal displacement of rails based on visual monitoring is provided, comprising the following steps:

[0010] Obtain an initial target image before rail testing, and select a ROI (region of interest) of a preset size in the initial target image;

[0011] The ROI region is divided into multiple sub-regions;

[0012] Obtain images of the test target after rail deformation;

[0013] An optimal search is performed in the test target image to confirm the location of the relevant sub-region;

[0014] Based on the correspondence of each pixel in each relevant sub-region, the pixel displacement of each sub-region is calculated respectively;

[0015] The pixel displacement of each sub-region is averaged to obtain the pixel displacement of the entire ROI region.

[0016] The pixel conversion relationship is calibrated based on the actual size of the rail target, and the actual displacement of the rail is calculated based on the pixel conversion relationship.

[0017] Preferably, the rail longitudinal displacement monitoring method further includes:

[0018] The actual displacement of different rails is obtained and recorded as Δ1, Δ2...Δn, where n is the rail number;

[0019] Obtain the reference displacement of the ground reference target and record it as Δ0;

[0020] Errors are eliminated based on the reference displacement of the ground reference target to obtain the relative displacements of the different rails as Δ1-Δ0, Δ2-Δ0...Δn-Δ0.

[0021] Preferably, the rail longitudinal displacement monitoring method further includes:

[0022] The actual displacement of the rail is uploaded to the remote business monitoring terminal for long-term monitoring of the rail.

[0023] Preferably, the optimal search method includes a full-pixel search method or a sub-pixel search method;

[0024] The full-pixel search method includes at least one or more of hill climbing, genetic algorithm, and human-computer interaction method; the sub-pixel search method includes at least one or more of surface fitting, gradient algorithm, and gray-level gradient iteration algorithm.

[0025] Preferably, the pixel displacement of each sub-region is calculated based on the correspondence between each pixel in each relevant sub-region, specifically including:

[0026] Using the zero-mean normalized cross-correlation criterion, assuming that the two images to be matched are f and g, the pixel correlation is calculated using the following formula to obtain the pixel correlation calculation results of all sub-regions.

[0027]

[0028] Among them, C zncc The correlation is represented by a value in the range [-1, 1], with a stronger correlation closer to 1; i represents the number of rows; j represents the number of columns; m represents the image width; f(x) i ,y j ) represents the graph f in x i ,y j The grayscale value at that location; g(x) represents the mean gray level of image f; i ′,y j ′) represents the image g at x i ′,y j The gray value at the ' position; This represents the mean gray level of image g.

[0029] Preferably, the ROI region is divided into n*n sub-regions, and the shape of the sub-regions includes at least circles and squares; where n is a positive integer determined based on the characteristics of the rail target.

[0030] According to a second aspect of the present invention, a rail longitudinal displacement monitoring system based on visual monitoring is provided, comprising:

[0031] Rail targets are set on the rails.

[0032] A camera is positioned on one side of the rail and facing the rail target to take pictures of the rail target.

[0033] An identification host is set on one side of the rail and communicates with the camera to receive images captured by the camera, and performs image recognition using the rail longitudinal displacement monitoring method as described in any one of claims 1 to 6 to calculate the actual displacement of the rail.

[0034] Preferably, the rail longitudinal displacement monitoring system further includes:

[0035] A ground reference target is placed below the rail to eliminate errors in the actual displacement of the rail.

[0036] Preferably, the rail longitudinal displacement monitoring system further includes:

[0037] A rechargeable battery, electrically connected to the camera and the recognition host, is used to power the camera and the recognition host.

[0038] A photovoltaic panel is electrically connected to the rechargeable battery for charging the rechargeable battery.

[0039] Preferably, the rail longitudinal displacement monitoring system further includes:

[0040] The remote business monitoring terminal is connected to the identification host to store the actual displacement of the rail and for data interaction.

[0041] Compared with the prior art, the present invention has the following technical effects:

[0042] 1. A sub-image search algorithm based on target images is implemented, which can improve the accuracy to the sub-pixel level compared with the existing pixel-level accuracy scheme, thereby realizing high-precision longitudinal displacement monitoring of rails.

[0043] 2. By setting up a ground reference target, monitoring errors caused by camera shake or movement can be eliminated, further improving monitoring accuracy.

[0044] 3. The integrated design of image acquisition, image analysis, automatic power supply, and automatic transmission enables automated longitudinal displacement monitoring of rails and uploads the monitoring results to a remote business monitoring terminal via a 4G communication module, providing data support for rail operation and maintenance. Attached Figure Description

[0045] Figure 1 A schematic diagram of a rail longitudinal displacement monitoring system based on visual monitoring provided in an embodiment of the present invention;

[0046] Figure 2 A schematic diagram illustrating the usage status of a rail longitudinal displacement monitoring system based on visual monitoring, provided for an embodiment of the present invention;

[0047] Figure 3 A flowchart illustrating a method for monitoring longitudinal displacement of rails based on visual monitoring, provided in an embodiment of the present invention;

[0048] Figure 4 This is a schematic diagram illustrating the selection of a ROI region of a preset size in an embodiment of the present invention;

[0049] Figure 5 This is a schematic diagram illustrating the division of the ROI region into multiple circular sub-regions in an embodiment of the present invention;

[0050] Figure 6This is a schematic diagram illustrating the division of the ROI region into multiple square sub-regions in an embodiment of the present invention;

[0051] Figure 7 This is a schematic diagram of a reference image before monitoring, as shown in an embodiment of the present invention.

[0052] Figure 8 This is a schematic diagram of the monitored test image in an embodiment of the present invention. Detailed Implementation

[0053] The technical content of the present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0054] like Figure 1 and Figure 2 As shown, this embodiment of the invention provides a visual monitoring-based rail longitudinal displacement monitoring system, comprising at least a rail target 1, a camera 2, a recognition host 3, a ground reference target 4, a power supply device 5, and a remote monitoring terminal 6. The rail target 1, camera 2, recognition host 3, ground reference target 4, and power supply device 5 are positioned near the rail 50 for longitudinal displacement monitoring. The remote monitoring terminal 6 is communicatively connected to the recognition host 3 to store the displacement data after rail monitoring, for data exchange and long-term monitoring.

[0055] Specifically, in one embodiment of the present invention, the rail target 1 can be in the form of speckle. It should be noted that speckle can be simply understood as a unique, irregular feature description of an object's surface; it is a general description and can be artificial speckle, such as that obtained by observation rulers, spraying, or drawing, or it can be a natural texture, such as the texture features of a rail. Preferably, the rail target 1 is positioned at the web of the rail, thereby facilitating image acquisition of the rail target 1. The position of the rail target 1 can also be adaptively adjusted as needed.

[0056] Camera 2 is positioned on one side of rail 50, facing rail target 1, for photographing rail target 1. For details, refer to... Figure 1 As shown, a column 10 is provided on one side of the rail 50, and the camera 2 is mounted on the column 10. The column 10 provides a stable mounting base for the camera 2 to ensure the installation accuracy of the camera 2. Preferably, in one embodiment of the present invention, the camera 2 can be a PTZ camera or multiple bullet cameras, each bullet camera being responsible for monitoring the corresponding rail.

[0057] Reference Figure 1As shown, the identification host 3 is also mounted on the column 10 and is communicatively connected to the camera 2 to receive images captured by the camera 2. Furthermore, the identification host 3 can perform image recognition based on the visual monitoring method for longitudinal displacement monitoring of rails provided by this invention (described in detail below), thereby calculating the actual displacement of the rail 50.

[0058] Reference Figure 1 As shown, the ground reference target 4 is set on the ground or a fixed object below the rail 50. The camera 2 is also used to take pictures of the ground reference target 4, thereby obtaining the reference displacement Δ0 of the ground reference target 4 through image recognition by the recognition host 3. After obtaining the actual displacement of different rails 50, it is recorded as Δ1, Δ2...Δn, where n is the rail number. Then, based on the reference displacement Δ0 of the ground reference target, the actual displacement of different rails can be error-eliminating to obtain the relative displacement of different rails as Δ1-Δ0, Δ2-Δ0...Δn-Δ0. Thus, the ground reference target 4 can be used for correction to prevent errors caused by camera shake due to train passing, wind, etc.

[0059] Reference Figure 1 As shown, the power supply device 5 includes a rechargeable battery and a photovoltaic panel. The rechargeable battery is electrically connected to both the camera 2 and the identification host 3 to power them. The photovoltaic panel is electrically connected to the rechargeable battery to convert solar energy into electrical energy, thereby charging the battery and ensuring the continuous operation of the rail longitudinal displacement monitoring system. This design is simple in structure and easy to install. It is understood that in other embodiments, this power supply device 5 can be replaced with other wired power supply facilities to meet power requirements.

[0060] The remote monitoring terminal 6 communicates with the identification host 3 via a 4G network (or other wireless connection methods, such as Wi-Fi, Bluetooth, or a direct wired connection). Once the identification host 3 obtains the actual displacement of the rail 50 through image recognition, it transmits the data to the remote monitoring terminal 6. The remote monitoring terminal 6 then stores the data for later retrieval or data interaction. Furthermore, it is understood that the remote monitoring terminal 6 can store continuous displacement data of the rail 50 for long-term monitoring, thereby understanding the continuous deformation of the rail 50 and facilitating maintenance or replacement.

[0061] The following details the process of image recognition performed by the recognition host 3 using the visual monitoring method for longitudinal displacement monitoring of rails.

[0062] Reference Figure 3As shown in the figure, the longitudinal displacement monitoring method for rails based on visual monitoring provided by this embodiment of the invention specifically includes steps S1 to S7:

[0063] S1: Obtain the initial target image before rail testing, and select a ROI region of a preset size in the initial target image.

[0064] Specifically, before the test, camera 2 takes a picture of the rail target 1 to obtain an initial target image. For example... Figure 4 As shown, after the recognition host 3 obtains the initial target image, it selects a ROI region of a preset size in the initial target image. The size of the ROI region can be determined according to the requirements for use in the next step of calculation.

[0065] It is understandable that if the rail 50 is being monitored for longitudinal displacement for the first time, the initial target image is the initial image of the rail 50 before deformation; if the rail 50 has been monitored for longitudinal displacement before, the initial target image is the test target image taken by camera 2 during the last monitoring of the rail 50 (i.e., the image taken in step S3 during the last monitoring).

[0066] S2: Divide the ROI region into multiple sub-regions.

[0067] Specifically, in one embodiment of the present invention, the ROI region is divided into n*n sub-regions, where n is a positive integer and is determined based on the characteristics of the rail target, and can be adjusted as needed. For example... Figure 5 and Figure 6 As shown, the shape of the sub-region includes at least circles and squares. In other embodiments, the sub-region may also be other shapes, such as hexagons, rhombuses, etc.

[0068] S3: Obtain images of the test target after the rail has deformed.

[0069] Specifically, after the rail 50 deforms over a period of time, camera 2 takes another picture of the rail target 1 to obtain a picture of the test target after the rail has deformed. The photo interval of camera 2 can be set automatically according to factors such as the material of the rail 50, the season, and the weather to meet the monitoring needs of different working conditions.

[0070] S4: Perform an optimal search within the test target image to identify the location of the relevant sub-region.

[0071] Specifically, in one embodiment of the present invention, the optimal search method includes a full-pixel search method or a sub-pixel search method. The full-pixel search method includes at least one or more of hill climbing, genetic algorithms, and human-computer interaction methods; the sub-pixel search method includes at least one or more of surface fitting, gradient algorithms, and gray-level gradient iterative algorithms. In practical applications, the surface fitting method is the fastest, and different search methods can be selected according to different situations.

[0072] It is understood that the search methods described above are all conventional algorithms in this field and will not be elaborated upon here.

[0073] S5: Calculate the pixel displacement of each sub-region based on the correspondence of each pixel in each relevant sub-region.

[0074] like Figure 7 and Figure 8 As shown, the basic principle of this monitoring method is as follows: Corresponding points in two images before and after displacement are matched, and the actual displacement is calculated based on the positional changes of the speckle field on the surface of the object before and after displacement. Specifically, to calculate the displacement of a point, a subset (a set of coordinates of points near point P) centered on the seed point P(x0,y0) is selected in the reference image (i.e., the speckle image without displacement). After deformation, a search is performed using a specific search method, and a certain type of correlation coefficient is compared between the subsets of images before and after displacement to find the corresponding point P′(x0+u0,y0+v0) after displacement, where u0 and v0 are the displacement values ​​in two directions of the two-dimensional image, respectively.

[0075] The reference subset is slid into the search area of ​​the deformed image, and the correlation coefficient at each location is calculated. Then, an optimization algorithm is used to search for the peak coordinates of the correlation coefficient distribution, completing the matching process. Once the extreme point corresponding to the correlation coefficient is found, the coordinates after strain can be determined. The difference between the center of the reference subset and the center of the target subset generates an in-plane displacement vector at point P.

[0076] Specifically, in one embodiment of the present invention, the zero-mean normalized cross-correlation criterion is adopted. Assuming that the two images to be matched are f and g, with a size of 2m*2m, the pixel correlation is calculated using the following formula to obtain the pixel correlation calculation results of all sub-regions.

[0077]

[0078] Among them, C zncc The correlation is represented by a value in the range [-1, 1], with a stronger correlation closer to 1; i represents the number of rows; j represents the number of columns; m represents the image width; f(x) i ,y j ) represents the graph f in xi ,y j The grayscale value at that location; g(x) represents the mean gray level of image f; i ′,y j ′) represents the image g at x i ′,y j The gray value at the ' position; This represents the mean gray level of image g.

[0079] S6: Average the pixel displacement of each sub-region to obtain the pixel displacement of the entire ROI region.

[0080] Specifically, after obtaining the pixel displacement of each sub-region, the pixel displacement of each sub-region can be sorted in ascending order. Therefore, the pixel displacement of the center position can be directly selected as the pixel displacement of the entire ROI region.

[0081] Alternatively, after obtaining the pixel displacement of each sub-region, the pixel displacement of each sub-region can be added together and then divided by the number of each sub-region to obtain the average pixel displacement, which can be used as the pixel displacement of the entire ROI region.

[0082] S7: Based on the actual size of the rail target, calibrate the pixel conversion relationship, and calculate the actual displacement of the rail based on the pixel conversion relationship.

[0083] Specifically, after obtaining the pixel displacement of the entire ROI region based on step S6, the pixel conversion relationship is calibrated according to the actual size of the rail target, that is: how much actual displacement one pixel represents. Thus, the actual displacement of the rail 50 can be calculated based on the pixel displacement of the entire ROI region.

[0084] S8: Eliminate errors in the actual displacement of different rails.

[0085] Specifically, after obtaining the actual displacement of different rails based on steps S1 to S7 above, they are recorded as Δ1, Δ2, ..., Δn, where n is the rail number. Furthermore, an image of the ground reference target 4 is captured by camera 2, and after image recognition is performed by the recognition host 3, the reference displacement of the ground reference target is calculated (the specific process is similar to steps S1 to S7 and will not be described in detail here), and recorded as Δ0.

[0086] Therefore, the reference displacement Δ0 based on the ground reference target can correct the actual displacement of different rails to obtain the relative displacement of different rails as Δ1-Δ0, Δ2-Δ0...Δn-Δ0, preventing errors caused by camera shake when trains pass by or when there is wind.

[0087] S9: Data upload to remote business monitoring terminal.

[0088] Specifically, after the displacement monitoring of rail 50 is completed, the actual displacement of rail 50 is uploaded to the remote business monitoring terminal 6 for long-term monitoring of rail 50 and subsequent data interaction.

[0089] In summary, the longitudinal displacement monitoring method and system for rails based on visual monitoring provided by the embodiments of the present invention have the following beneficial effects:

[0090] 1. A sub-image search algorithm based on target images is implemented, which can improve the accuracy to the sub-pixel level compared with the existing pixel-level accuracy scheme, thereby realizing high-precision longitudinal displacement monitoring of rails.

[0091] 2. By setting up a ground reference target, monitoring errors caused by camera shake or movement can be eliminated, further improving monitoring accuracy.

[0092] 3. The integrated design of image acquisition, image analysis, automatic power supply, and automatic transmission enables automated longitudinal displacement monitoring of rails and uploads the monitoring results to a remote business monitoring terminal via a 4G communication module, providing data support for rail operation and maintenance.

[0093] The foregoing has provided a detailed description of the visual monitoring-based longitudinal displacement monitoring method and system for rails provided by this invention. Any obvious modifications made by those skilled in the art without departing from the essential content of this invention will constitute an infringement of the patent rights of this invention and will incur corresponding legal liability.

Claims

1. A method of monitoring longitudinal displacement of a rail based on visual monitoring, characterized by Includes the following steps: Obtain an initial target image before rail testing, and select a ROI region of a preset size in the initial target image; The ROI region is divided into multiple sub-regions; Obtain images of the test target after rail deformation; An optimal search is performed in the test target image to confirm the location of the relevant sub-region; Based on the correspondence of each pixel in each relevant sub-region, the pixel displacement of each sub-region is calculated respectively; specifically, it includes: using the zero-mean normalized cross-correlation criterion, assuming that the two images for matching calculation are f and g, the pixel correlation is calculated using the following formula to obtain the pixel correlation calculation results of all sub-regions; Among them, C zncc The correlation is represented by a value in the range [-1, 1], with a stronger correlation closer to 1; i represents the number of rows; j represents the number of columns; m represents the image width; f(x) i ,y j ) represents the graph f in x i ,y j The grayscale value at that location; g(x) represents the mean gray level of image f; i ′,y j ′) represents the image g at x i ′,y j The gray value at the ' position; This represents the mean gray level of image g; The pixel displacement of each sub-region is averaged to obtain the pixel displacement of the entire ROI region. The pixel conversion relationship is calibrated based on the actual size of the rail target, and the actual displacement of the rail is calculated based on the pixel conversion relationship. The actual displacement of different rails is obtained and recorded as Δ1, Δ2, ..., Δn, where n is the rail number; Obtain the reference displacement of the ground reference target and record it as Δ0; Errors are eliminated in the actual displacement of different rails based on the reference displacement of the ground reference target to obtain the relative displacement of different rails as Δ1-Δ0, Δ2-Δ0......Δn-Δ0; The ground reference target is set on the ground or a fixed object below the rail. The camera is used to take pictures of the ground reference target, and the reference displacement Δ0 of the ground reference target is obtained through image recognition by the recognition host.

2. The rail longitudinal displacement monitoring method as described in claim 1, characterized in that... The optimal search method includes a full-pixel search method or a sub-pixel search method; The full-pixel search method includes any one or more of the following: hill climbing method, genetic algorithm, and human-computer exchange method; The subpixel search method includes any one or more of the following: surface fitting method, gradient algorithm, and gray-level gradient iteration algorithm.

3. The rail longitudinal displacement monitoring method as described in claim 1, characterized in that: The ROI region is divided into Each sub-region has a shape that includes at least circles and squares; where n is a positive integer determined based on the characteristics of the rail target.

4. The rail longitudinal displacement monitoring method as described in claim 1, characterized in that... Also includes: The actual displacement of the rail is uploaded to the remote business monitoring terminal for long-term monitoring of the rail.

5. A rail longitudinal displacement monitoring system based on visual monitoring, characterized in that... include: Rail targets are set on the rails. A camera is positioned on one side of the rail and facing the rail target to take pictures of the rail target. An identification host is set on one side of the rail and communicates with the camera to receive images captured by the camera, and performs image recognition using the rail longitudinal displacement monitoring method according to any one of claims 1 to 4 to calculate the actual displacement of the rail.

6. The rail longitudinal displacement monitoring system as described in claim 5, characterized in that... Also includes: A ground reference target is placed below the rail to eliminate errors in the actual displacement of the rail.

7. The rail longitudinal displacement monitoring system as described in claim 5, characterized in that... Also includes: A rechargeable battery, electrically connected to the camera and the recognition host, is used to power the camera and the recognition host. A photovoltaic panel is electrically connected to the rechargeable battery for charging the rechargeable battery.

8. The rail longitudinal displacement monitoring system as described in claim 7, characterized in that... Also includes: The remote business monitoring terminal is connected to the identification host to store the actual displacement of the rail and for data interaction.

Images