A method and device for monitoring track alignment based on binocular vision measurement

Abstract

This invention discloses a method and device for monitoring track alignment based on binocular vision measurement, comprising: fixing markers with coded images to each track segment; moving a binocular camera along the measurement track and taking pictures at different measurement points; preprocessing the captured image sequence and performing target detection to extract the coded images; calibrating the internal parameters of the binocular camera and the external parameters at each measurement point; obtaining the three-dimensional coordinates of each marker in the local binocular coordinate system based on the internal parameters of the binocular camera, and then converting them into the three-dimensional coordinates of each marker in the global coordinate system using the external parameters of the binocular camera; optimizing all the three-dimensional coordinates of the markers with unique labels in the global coordinate system using bundle adjustment to obtain the optimized three-dimensional coordinates of all markers in the global coordinate system, and then assembling them in order to obtain the three-dimensional alignment curve of the measurement track. This invention has high accuracy and is unaffected by environmental interference.

Description

Technical Field

[0001] This invention relates to computer vision technology, and more particularly to a method and device for monitoring track alignment based on binocular vision measurement. Background Technology

[0002] Under the influence of factors such as high-speed train rolling, starting and braking, and changes in external temperature, significant longitudinal stress is generated inside the track. This causes the rails to move longitudinally along the sleepers or track frame and along the top surface of the ballast bed. This phenomenon is called rail creep or longitudinal rail displacement. Longitudinal rail displacement leads to stress redistribution in seamless track rails and can even induce rail buckling, thus affecting the operational safety of high-speed trains. Therefore, accurate monitoring of the three-dimensional track alignment is of great significance for ensuring the safe operation of high-speed railways.

[0003] Traditional methods for measuring the three-dimensional alignment of railway tracks primarily employ electrical displacement sensors and accelerometers. These methods typically require the installation of displacement sensing devices, such as magnetostrictive displacement sensors, laser displacement sensors, and fiber optic sensors, on railway facilities like tracks or sleepers. Changes in track alignment are obtained by monitoring the displacement of these points. While these methods are simple to operate and fast in calculation, the reliability and accuracy of the sensing devices are compromised due to long-term exposure to track vibrations and adverse weather conditions. Summary of the Invention

[0004] To address the problems existing in the prior art, the purpose of this invention is to provide a more accurate method and device for monitoring track alignment based on binocular vision measurement.

[0005] To achieve the above-mentioned objectives, the present invention provides the following technical solution:

[0006] A method for monitoring track alignment based on binocular vision measurement includes the following steps:

[0007] S1. Divide the measurement track into several track segments, and fix markers with coded images at the waist of each track segment. The coded images on the markers of each track segment are different, and each coded image is a unique identifier for the track segment.

[0008] S2. Move the binocular camera with the mobile platform base along the measurement track and take pictures at different measurement points to obtain a sequence of photos containing markers;

[0009] S3. Use an adversarial neural network to preprocess the photo sequence to obtain a high-quality image sequence;

[0010] S4. For each photo in the photo sequence, use YOLOv5 to perform object detection and extract the encoded image region.

[0011] S5. Preprocess each extracted coded image region, and extract the coded image from each preprocessed coded image region using the edge extraction operator, and parse the unique label represented by each coded image;

[0012] S6. Calibrate the internal parameters of the binocular camera using the calibration plate;

[0013] S7. Solve the relative pose of the binocular camera by using epipolar geometry constraints to obtain the external parameters of the binocular camera at each measurement point.

[0014] S8. Locate the center point of each coded image in each photo, and reconstruct the coordinates of each coded image based on the located center point and the internal parameters of the binocular camera to obtain the three-dimensional coordinates of the marker corresponding to each coded image in the local binocular coordinate system.

[0015] S9. Convert the three-dimensional coordinates of each marker in the local binocular coordinate system into the three-dimensional coordinates of each marker in the global coordinate system using the external parameters of the binocular camera.

[0016] S10. Optimize all three-dimensional coordinates of the markers with the same unique label in the global coordinate system using bundle adjustment to obtain the optimized three-dimensional coordinates of all markers in the global coordinate system.

[0017] S11. Set the optimized three-dimensional coordinates of all markers in the global coordinate system according to the order of the track segments to obtain the three-dimensional linear curve of the measurement track.

[0018] Furthermore, the coded image consists of three concentric rings with a radius ratio of 5:12:20. The three concentric rings are divided into 15 equal parts to form 15-bit coding bands. Each coding band is a coding area. If the coding area is black, it represents the binary number 0. If it is white, it represents the binary number 1. Finally, a 15-bit binary number is formed as a unique identifier for each track segment.

[0019] Furthermore, step S3 specifically includes:

[0020] S301. Filter and denoise each photo in the photo sequence;

[0021] S302. Enhance the brightness and contrast of photos with low light or overexposure, and repair photos with strong light and reflection problems using image dehazing or dereflection algorithms.

[0022] S303. Using a trained adversarial neural network, perform image preprocessing on each photo to obtain high-quality photos. The adversarial neural network is trained using several low-quality images and corresponding high-quality image labels.

[0023] Furthermore, step S4 specifically includes:

[0024] S401. Perform noise reduction, lighting adjustment, and geometric correction on each photo in the photo sequence;

[0025] S402. Load the trained YOLOv5 model, perform object detection on each photo in the photo sequence, and extract the encoded image region and its confidence score; wherein, the YOLOv5 model is trained using images collected by the track at different scenes, different angles and different distances;

[0026] S403. Select and retain coded image regions with a confidence level greater than the preset threshold, and delete the others;

[0027] S404. If the detected coded image region deviates from the expected position of the track region, it is determined to be a false detection; if the number of detected coded image regions is less than the preset number, it is determined to be a missed detection.

[0028] S405. If a missed detection is found, adjust the position of the binocular camera or readjust the exposure parameters of the binocular camera.

[0029] Furthermore, step S5 specifically includes:

[0030] S501. Preprocess the extracted coded image region, including two-dimensional Gaussian filtering and binarization;

[0031] S502. The Canny operator is used to extract the edges of the preprocessed coded image region to obtain the coded image in the coded image region.

[0032] S503. Decode the encoded image and parse out the unique identifier represented by the encoded image.

[0033] Furthermore, step S6 specifically includes: calibrating the internal parameters of the binocular camera using a checkerboard calibration board.

[0034] Furthermore, step S8 specifically includes:

[0035] S801. Use Gaussian curve fitting method to locate the center point of each encoded image in subpixel, and obtain the coordinates of the center point of each encoded image on the two-dimensional image plane.

[0036] S802. Based on the internal parameters of the binocular camera, reconstruct the three-dimensional coordinates of the center point of each coded image on the two-dimensional image plane, and use them as the three-dimensional coordinates of each coded image in the local binocular coordinate system.

[0037] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described above.

[0038] A computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implements the above-described method.

[0039] A computer program product includes a computer program / instructions that, when executed by a processor, implement the above-described method.

[0040] Compared with the prior art, the beneficial effects of this invention are:

[0041] (1) This invention employs a 3D reconstruction method based on coded markers and close-range photogrammetry to accurately measure the 3D spatial coordinates of the markers. Furthermore, this invention uses bundle adjustment to optimize the coordinate reconstruction results of multiple measurement points, significantly improving the accuracy of the 3D reconstruction of the world coordinate system.

[0042] (2) The present invention simulates multi-camera group measurement by moving a set of binocular camera equipment at multiple measurement points, which significantly reduces the complexity of the equipment required for track measurement.

[0043] (3) The present invention adopts a non-contact measurement method, which eliminates the need to install complex vibration sensors and avoids additional equipment installation and maintenance costs.

[0044] (4) This invention has low requirements for adaptability to the background environment and has a wide range of applications. It can accurately identify under complex environmental conditions such as low light, strong light, rain and snow, thereby enhancing the reliability and accuracy of the system in various environments and having broad application prospects. Attached Figure Description

[0045] Figure 1 This is a flowchart illustrating the track alignment monitoring method based on binocular vision measurement provided by the present invention.

[0046] Figure 2 This is a schematic diagram of the track surface marker pasting provided by the present invention;

[0047] Figure 3 This is a schematic diagram of a binocular camera provided by the present invention;

[0048] Figure 4 This is a schematic diagram of the structure of the computer device provided by the present invention;

[0049] In the diagram, there is a measuring track (1), a sleeper (2), a marker (3), an industrial camera (4), a binocular camera mount (5), and a mobile platform (6). Detailed Implementation

[0050] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0051] Example 1

[0052] This invention provides a method for monitoring track alignment based on binocular vision measurement, such as... Figure 1 As shown, it includes the following steps:

[0053] S1. Divide the measurement track into several track segments and fix markers with coded images at the waist of each track segment.

[0054] Among them, such as Figure 2 As shown, the measurement track 1 is divided into several segments, which are set on sleepers 2. Each track segment has a different coded image on its marker 3, serving as a unique identifier for that segment. Specifically, the coded image consists of three concentric rings with radii in the ratio of 5:12:20. These three rings are divided into 15 equal parts, forming 15-bit coded bands. Each coded band is a coded area; a black coded area represents a binary 0, and a white coded area represents a binary 1, ultimately forming a 15-bit binary number as the unique identifier for each track segment. Because there are 15 coded bands, a maximum of 15 binary numbers can be read. Considering the permutation and combination method and the uniqueness of the encoding, and removing the cases where all coded bands are 0 or 1 (for robust identification and decoding of marker points), the capacity of this coded image is 2190, which can meet the needs of most camera group extrinsic calibration and key point measurement.

[0055] S2. Move the binocular camera with the mobile platform base along the measurement track and take pictures at different measurement points to obtain a sequence of photos containing markers.

[0056] Several measuring points are set up along the track, and the marker is photographed at each specified measuring point. A set of binocular cameras is used, such as... Figure 3 As shown, two industrial cameras 4 are supported on the mobile platform 6 by a binocular camera bracket 5. The binocular cameras simultaneously acquire left and right view images of the track to ensure coverage of all markers on the track.

[0057] S3. Use an adversarial neural network to preprocess the photo sequence to obtain a high-quality image sequence.

[0058] This step specifically includes:

[0059] S301. Filter and denoise each photo in the photo sequence. Specifically, Gaussian filtering, median filtering and other denoising algorithms can be used to remove noise caused by rain, snow or low light conditions.

[0060] S302. Enhance the brightness and contrast of photos with low light or overexposure, and repair photos with strong light and reflection problems using image dehazing or dereflection algorithms.

[0061] S303. Using a trained adversarial neural network (ANN), image preprocessing is performed on each photo to obtain high-quality photos. The AAN is trained using several low-quality images and corresponding high-quality image labels. Specifically, the AAN can be CycleGAN (suitable for environments without paired low-quality and high-quality images when training is not possible) or Pix2Pix (if corresponding high-quality images are available, Pix2Pix is ​​used for end-to-end image transformation), processing different low-quality images (e.g., low light, strong light, rain, snow, etc.). The generator is based on a U-Net or ResNet structure, incorporating skip connections to preserve details in the encoded point regions. The discriminator uses PatchGAN to evaluate the local quality of the generated images, ensuring enhanced realism and detail.

[0062] S4. For each photo in the photo sequence, use YOLOv5 to perform object detection and extract the encoded image region.

[0063] This step specifically includes:

[0064] S401. Denoise, adjust lighting, and perform geometric correction on each photo in the photo sequence to ensure that the image quality meets the requirements when inputting into the YOLOv5 model.

[0065] S402. Load the trained YOLOv5 model and perform object detection on each photo in the photo sequence, extracting the encoded image regions and their confidence scores. The YOLOv5 model was trained using images captured on the track under different scenes, angles, and distances, including daytime, nighttime, bright light, shadow, rain, and snow, at varying angles and distances (e.g., changes in camera mounting height and tilt angle). The performance of the trained model is then evaluated using a test set, with metrics including mAP (mean Average Precision), Recall, and Precision. Based on the evaluation results, data augmentation strategies and model hyperparameters (learning rate, batch size, etc.) are further adjusted to ensure the model can detect and accurately locate track encoding points. The photo size needs to be adjusted according to the default input size of YOLO v5 (640×640). The output should include the category, bounding box coordinates, and detection confidence score for each detected encoded image region.

[0066] S403. Select and retain coded image regions with a confidence level greater than a preset threshold (e.g., 0.7), and delete the rest;

[0067] S404. If the detected coded image region deviates from the expected position of the track region, it is determined to be a false detection; if the number of detected coded image regions is less than the preset number, it is determined to be a missed detection.

[0068] S405. If a missed detection is found, adjust the position of the binocular camera or readjust the exposure parameters of the binocular camera. For example, if a missed detection is caused by occlusion or viewing angle problems in certain areas, adjust the camera position, tilt angle, roll angle, etc.

[0069] S5. Preprocess each extracted coded image region, and extract the coded image from each preprocessed coded image region using the edge extraction operator, and parse the unique label represented by each coded image.

[0070] This step specifically includes:

[0071] S501. Preprocess the extracted coded image region, including two-dimensional Gaussian filtering and binarization; among which, edge extraction directly on the binarized image is easily affected by the binarization threshold, so binarization is only used for localization, while integer pixel edge extraction is performed on the two-dimensional Gaussian filtered image.

[0072]

[0073] In the formula, G(u,v) represents the Gaussian filter function, (u,v) represents the coordinates of the image, and σ is the standard deviation.

[0074] S502. The Canny operator is used to extract the edges of the preprocessed coded image region to obtain the coded image in the coded image region.

[0075] In integer edge extraction, edges refer to locations in an image where gray levels are discontinuous and have large gray-level gradients. Commonly used edge detection operators include Canny, Robert, Sobel, and Prewitt. This invention employs the Canny operator, which is insensitive to noise and can generate single-pixel edges, to perform convolution calculations on the image to enhance edges and obtain the magnitude and direction of the gray-level gradient.

[0076]

[0077] Where f(u,v) is the pixel gray level at coordinates (u,v). G is the convolution operator. x and G y H represents the grayscale gradient magnitude. x and H y They are respectively:

[0078]

[0079] gradient size And the direction θ is:

[0080]

[0081] The previous calculation typically yields edges spanning multiple pixels. Gradient direction non-maximum suppression (COPS) preserves the local maximum gradient while reducing other gradients to zero, ultimately identifying the sharpest grayscale gradient changes. Finally, a dual-threshold method is used: edge points with grayscale gradients greater than the higher threshold are designated as strong edges, while those with gradients less than the higher threshold but greater than the lower threshold are designated as weak edges, completing edge extraction. Edge extraction then yields the coded image within the coded image region.

[0082] S503. Decode the encoded image and parse out the unique identifier represented by the encoded image.

[0083] Specifically, decoding can be performed by following the correspondence between the encoded image and the unique identifier.

[0084] S6. Calibrate the internal parameters of the binocular camera using the calibration board.

[0085] Specifically, a checkerboard calibration board can be used to calibrate the internal parameters of a binocular camera.

[0086] S7. Solve the relative pose of the binocular camera by using epipolar geometric constraints to obtain the external parameters of the binocular camera at each measurement point.

[0087] At each measurement point, let S1 and S2 represent the imaging planes of any two consecutive shots, P be a marker point in space, and p1(u1,v1) and p2(u2,v2) be the images formed by point P. c1 and O c2 The positions of the optical centers of the two cameras are represented by P and O. c1 and O c2 The plane determined by the three points is the polar plane, O c1 and O c2 The line connecting the two points is the baseline, and the intersections e1 and e2 of the baseline with planes S1 and S2 are the poles. The intersections l1 and l2 of the polar plane with planes S1 and S2 are the epipolar lines. From the pinhole camera model, the transformation relationship between the pixel coordinate system and the camera coordinate system can be expressed as:

[0088]

[0089] Where K is the intrinsic parameter matrix of the camera; c x and c y The principal point of the image (the intersection of the optical axis and the imaging plane); f x f yThe equivalent focal length is the ratio of the focal length to the horizontal and vertical dimensions of a single pixel; f s The coordinate axis tilt parameter is related to the phase distance during imaging, the horizontal pixel size, and the true angle between the pixel arrangement on the sensor plane. Combining the above formula, we have:

[0090] s1p1=KP s2p2=K(RP+t)

[0091] Where s1 and s2 are the depths of point P in the two images, the above equation can be transformed into the following equation:

[0092] s2x2=s1Rx1+t

[0093] Where, x1 = K -1 p1,x2=K -1 p c .

[0094] Define the antisymmetric symbol ^, and let:

[0095]

[0096] Multiply the transformed equation on the left have:

[0097]

[0098] From the perpendicular relationship, the left side of the above equation equals zero, therefore we can obtain the equation for the epipolar constraint:

[0099]

[0100] Substituting p2 and p2 into the above equation yields another form of the epipolar constraint:

[0101]

[0102] Among them, t ∧ R is the essential matrix E. The fundamental matrix is ​​F. Therefore, solving for the relative pose of the camera can be transformed into solving for the essential matrix E or the fundamental matrix F. The camera pose is solved using 6-8 pairs of matched markers in two images. This yields the extrinsic parameters R and t.

[0103] S8. Locate the center point of each coded image in each photo, and reconstruct the coordinates of each coded image based on the located center point and the internal parameters of the stereo camera, to obtain the three-dimensional coordinates of the marker corresponding to each coded image in the local stereo coordinate system.

[0104] S801. Use Gaussian curve fitting to locate the center point of each coded image at the sub-pixel level, and obtain the coordinates of the center point of each coded image on the two-dimensional image plane.

[0105] In subpixel localization, each square represents a pixel. Let P4(u0,v0) be the grayscale gradient direction of the integer pixel edge. P1, P7, P2, P6, P3, and P5 are points 3, 2, and 1 pixels away from P4 in the gradient direction and the opposite direction, respectively. Their grayscale values ​​are obtained by bilinear interpolation of the grayscale values ​​of neighboring pixels and are denoted as I. i Calculate the grayscale difference between P2, P3, P4, P5, and P6 using the difference formula:

[0106]

[0107] The gray-level difference was calculated using a one-dimensional Gaussian curve fitting, with the following objective function:

[0108]

[0109] Taking the logarithm of both sides, and denoteing g = lny, we have:

[0110] g = ax 2 +bx+c

[0111] According to the square aperture sampling theorem, the pixel grayscale difference is:

[0112]

[0113] Where g(n) = lnD n (n = 1, 2, 3, 4, 5), with corresponding x values ​​of -2, 1, 0, 1, 2. Using the least squares method to solve for the three unknowns a, b, and c, and taking the extreme value of the previous equation, we obtain:

[0114]

[0115] in,

[0116]

[0117] Let δ = x be the offset of the sub-pixel boundary relative to the integer pixel boundary along the gradient direction θ. Then the sub-pixel boundary coordinates can be expressed as:

[0118]

[0119] u ′ 0,v ′ 0 represents subpixel boundary coordinates, and u0, v0 represent integer pixel boundary coordinates.

[0120] The calculated sub-pixel boundaries are used for fitting. Since the image of a circle after perspective projection is usually an ellipse, the objective function is generally chosen to be the general equation of an ellipse:

[0121] u2 +Auv+Bv 2 +Cu+Dv+E=0

[0122] There are five unknown coefficients A, B, C, D, and E. Therefore, the number of sub-pixel boundaries must be more than five. In general, the number of boundaries will be much greater than five. Therefore, the least squares method can be used to solve for these coefficients, and finally, the center coordinates (u) can be obtained. c ,v c )as follows:

[0123]

[0124] S802. Based on the internal parameters of the binocular camera, reconstruct the three-dimensional coordinates of the center point of each coded image on the two-dimensional image plane, and use them as the three-dimensional coordinates of each coded image in the local binocular coordinate system.

[0125] Specifically, the coded image's three-dimensional coordinates (X, Y, Z) in the local stereo coordinate system. C ,Y C Z C )for:

[0126]

[0127] In the formula, K is the camera's intrinsic parameter matrix, Z C This is the depth matrix.

[0128] S9. Convert the 3D coordinates of each marker in the local binocular coordinate system into the 3D coordinates of each marker in the global coordinate system using the external parameters of the binocular camera.

[0129] Specifically, the three-dimensional coordinates (X, Y, Z) of the marker in the global coordinate system. W ,Y W Z W )for:

[0130]

[0131] S10. Optimize all three-dimensional coordinates of the markers with the same unique label in the global coordinate system using the bundle adjustment method to obtain the optimized three-dimensional coordinates of all markers in the global coordinate system.

[0132] Based on the collinearity condition equation, the optimization objective of the bundle adjustment method is as follows:

[0133]

[0134] In the formula, The image coordinates of the center point of the image at the j-th measurement point are encoded as follows: The projected image coordinates, K, D, R, are the center points of the corresponding marker image. j ,t j These represent the camera intrinsic parameters, lens distortion, and extrinsic parameters at the j-th measurement point, respectively. This represents the three-dimensional spatial coordinates of the i-th marker at the j-th measurement point.

[0135] Adjustment processing is performed uniformly across the entire region to optimize and solve for camera intrinsic parameters, lens distortion, extrinsic parameters of camera coordinate systems at different measurement points relative to the world coordinate system, and spatial point coordinates.

[0136] S11. Set the optimized three-dimensional coordinates of all markers in the global coordinate system according to the order of the track segments to obtain the three-dimensional linear curve of the measurement track.

[0137] Example 2

[0138] Figure 4 This is a schematic diagram of the structure of a computer device provided by an embodiment of the present invention. The embodiment of the present invention provides services for the implementation of the method of the first embodiment of the present invention. Figure 4 As shown, the device may include: a memory 301 storing a computer-executable program; a processor 302 coupled to the memory 301; the processor 302 calls the computer-executable program stored in the memory 301 to perform the steps in the method described in Embodiment 1.

[0139] Memory 301 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory. The device may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, memory 301 may be used to read and write non-removable, non-volatile magnetic media (commonly referred to as a "hard disk drive"). A program / utility having a set (at least one) of program modules may be stored, for example, in memory 301. Such program modules include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. The computer-executable program of the program modules typically performs the functions and / or methods described in the embodiments of the present invention.

[0140] The processor 302 executes various functional applications and data processing by running programs stored in the memory 301, such as implementing the method provided in Embodiment 1 of the present invention.

[0141] The code of a computer executable program can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages.

[0142] Example 3

[0143] This invention provides a storage medium containing a computer-executable program, which, when executed by a computer processor, is used to perform the method of Embodiment 1.

[0144] The storage medium of embodiments of the present invention may be any combination of one or more computer-readable media. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0145] Of course, the computer-executable program in the storage medium provided in the embodiments of the present invention is not limited to the above-described method operations, but can also perform related operations in the methods provided in any embodiment of the present invention.

[0146] Example 4

[0147] This invention also provides a computer program product, such as an app on a mobile phone or tablet, or an installer on a computer. This product includes a computer program / instructions that, when executed by a processor, implement the method described in Embodiment 1. The code for the computer-executable program used to perform the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0148] It should be noted that in this paper, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0149] It should be understood that the embodiments and descriptions above are only the principles, main features and advantages of the present invention. Various changes and modifications can be made to the present invention without departing from the spirit and scope of the invention, and all such changes and modifications fall within the protection scope of the present invention.

Claims

1. A method for monitoring track alignment based on binocular vision measurement, characterized in that, Includes the following steps: S1. Divide the measurement track into several track segments, and fix markers with coded images at the waist of each track segment. The coded images on the markers of each track segment are different, and each coded image is a unique identifier for the track segment. S2. Move the binocular camera with the mobile platform base along the measurement track and take pictures at different measurement points to obtain a sequence of photos containing markers; S3. Use an adversarial neural network to preprocess the photo sequence to obtain a high-quality image sequence; S4. For each photo in the photo sequence, use YOLOv5 to perform object detection and extract the encoded image region. S5. Preprocess each extracted coded image region, and extract the coded image from each preprocessed coded image region using the edge extraction operator, and parse the unique label represented by each coded image; S6. Calibrate the internal parameters of the binocular camera using the calibration plate; S7. Solve the relative pose of the binocular camera by using epipolar geometry constraints to obtain the external parameters of the binocular camera at each measurement point. S8. Locate the center point of each coded image in each photo, and reconstruct the coordinates of each coded image based on the located center point and the internal parameters of the binocular camera to obtain the three-dimensional coordinates of the marker corresponding to each coded image in the local binocular coordinate system. S9. Convert the three-dimensional coordinates of each marker in the local binocular coordinate system into the three-dimensional coordinates of each marker in the global coordinate system using the external parameters of the binocular camera. S10. Optimize all three-dimensional coordinates of the markers with the same unique label in the global coordinate system using bundle adjustment to obtain the optimized three-dimensional coordinates of all markers in the global coordinate system. S11. Set the optimized three-dimensional coordinates of all markers in the global coordinate system according to the order of the track segments to obtain the three-dimensional linear curve of the measurement track.

2. The method for monitoring track alignment based on binocular vision measurement according to claim 1, characterized in that, The coded image consists of three concentric rings with a radius ratio of 5:12:

20. The three concentric rings are divided into 15 equal parts to form 15-bit coded bands. Each coded band is a coded area. If the coded area is black, it represents the binary number 0. If it is white, it represents the binary number 1. Finally, a 15-bit binary number is formed as a unique identifier for each track segment.

3. The method for monitoring track alignment based on binocular vision measurement according to claim 1, characterized in that, Step S3 specifically includes: S301. Filter and denoise each photo in the photo sequence; S302. Enhance the brightness and contrast of photos with low light or overexposure, and repair photos with strong light and reflection problems using image dehazing or dereflection algorithms. S303. Using a trained adversarial neural network, perform image preprocessing on each photo to obtain high-quality photos. The adversarial neural network is trained using several low-quality images and corresponding high-quality image labels.

4. The method for monitoring track alignment based on binocular vision measurement according to claim 1, characterized in that, Step S4 specifically includes: S401. Perform noise reduction, lighting adjustment, and geometric correction on each photo in the photo sequence; S402. Load the trained YOLOv5 model, perform object detection on each photo in the photo sequence, and extract the encoded image region and its confidence score; wherein, the YOLOv5 model is trained using images collected by the track under different scenes, different angles and different distances; S403. Select and retain coded image regions with a confidence level greater than the preset threshold, and delete the others; S404. If the detected coded image region deviates from the expected position of the track region, it is determined to be a false detection; if the number of detected coded image regions is less than the preset number, it is determined to be a missed detection. S405. If a missed detection is found, adjust the position of the binocular camera or readjust the exposure parameters of the binocular camera.

5. The method for monitoring track alignment based on binocular vision measurement according to claim 1, characterized in that, Step S5 specifically includes: S501. Preprocess the extracted coded image region, including two-dimensional Gaussian filtering and binarization; S502. The Canny operator is used to extract the edges of the preprocessed coded image region to obtain the coded image in the coded image region. S503. Decode the encoded image and parse out the unique identifier represented by the encoded image.

6. The method for monitoring track alignment based on binocular vision measurement according to claim 1, characterized in that, Step S6 specifically includes: calibrating the internal parameters of the binocular camera using a checkerboard calibration board.

7. The method for monitoring track alignment based on binocular vision measurement according to claim 1, characterized in that, Step S8 specifically includes: S801. Use Gaussian curve fitting method to locate the center point of each encoded image in subpixel, and obtain the coordinates of the center point of each encoded image on the two-dimensional image plane. S802. Based on the internal parameters of the binocular camera, reconstruct the three-dimensional coordinates of the center point of each coded image on the two-dimensional image plane, and use them as the three-dimensional coordinates of each coded image in the local binocular coordinate system.

8. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the method as described in any one of claims 1-7.

9. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, The computer program / instructions, when executed by a processor, implement the method of any one of claims 1-7.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the method of any one of claims 1-7.

Images