Method and system for resolving relative position of heavy-duty truck and railway slab

Abstract

The invention discloses a method and system for resolving the relative position of a heavy-duty truck and a railway slab, and the method comprises the steps: collecting an RGBD image, converting the collected RGB image into a gray-scale image, carrying out the edge detection of the image through an edge detection algorithm, and fusing depth information to determine an edge line of the railway slab. And eliminating recesses at the edge of the railway slab by using straight line detection, fitting left and right edge lines of the railway slab by using a least square method, and finally determining the relative position relationship between the heavy-duty truck and the railway slab. By accurately identifying the edge of the railway slab, the relative position of the heavy-duty truck and the railway slab can be quickly and accurately measured, the railway loading and unloading efficiency of the heavy-duty truck is improved, the personnel waste is reduced, and the cost is saved.

Description

Technical Field

[0001] The invention relates to the field of heavy-load vehicle state perception, and in particular to a method and system for calculating the relative position of a heavy-load truck and a railway flatbed. Background Art

[0002] With the rapid development of the economy and the significant improvement of people's living standards, the improvement of logistics and transportation efficiency has become one of the key needs of social development. Due to its great advantages in environmental protection, development of technical standards, efficiency improvement, progress of intelligent railways, service innovation and sustainable development, railway transportation plays an important role in the modern logistics system. Especially for heavy-duty truck railway transportation, large-scale and fast transportation is particularly important in this context. However, the railway loading and unloading of heavy-duty trucks has strict requirements on the parking position of the vehicle, especially the distance difference between the left and right sides of the vehicle and the edge of the railway flatbed, which is an important factor restricting the rapid loading and unloading of heavy-duty trucks by rail.

[0003] At present, the relative position measurement of heavy-duty trucks and railway flatbeds uses traditional methods. Before loading and unloading heavy-duty trucks, personnel need to be arranged to perform auxiliary work such as positioning and pulling wires in the center of the flatbed. During the railway loading and unloading process, the relative position relationship between the heavy-duty truck and the railway flatbed is mainly determined by visual inspection by railway staff and the driver is directed. After the railway staff determines that the heavy-duty truck almost meets the positioning requirements, special personnel are arranged to use a tape measure to measure the distance between the outer edge of the heavy-duty truck tire and the edge of the railway flatbed. According to the distance between the outer edge of the left and right tires of the heavy-duty truck and the edges of both sides of the flatbed, the vehicle is repeatedly adjusted until the center line of the vehicle is accurately aligned with the center line of the flatbed to meet the requirements of railway transportation. This method is not only time-consuming, but also requires more human resources, which leads to an increase in overall transportation costs. Summary of the invention

[0004] In view of the above problems, the present invention aims to provide a method for calculating the relative position of a heavy-duty truck and a railway flatbed, comprising the following steps: Use RGBD camera module to obtain image data containing information of heavy-duty trucks and railway flatbeds; Performing image processing on the image data to identify edge information of the railway slab; Detecting the edge line of the railway slab by using the depth information in the image; Perform straight line detection on the image to eliminate the depressions on the edge of the railway slab; Using the coordinates of the endpoints of the straight line, the left and right edge lines of the railway slab are fitted using the least squares method; Based on the processed edge information, the relative positions of the heavy-duty truck and the railway flatbed are determined.

[0005] Based on the above solution, obtaining the image data includes: The RGBD camera module is installed above the front of the vehicle and RGB images are collected. The centerline of the field of view of the RGBD camera module is aligned with the centerline of the vehicle after calibration. The geometric center point P of the vehicle is located on the centerline of the field of view. After calibration, the distance L from P to the center of the railway slab and the distance d from P to the centerline of the railway slab in the camera field of view have a relationship of L=K d, where K is the distance coefficient.

[0006] Based on the above solution, the image processing at least includes converting the color image into a grayscale image and performing edge detection on the grayscale image.

[0007] Based on the above solution, the edge detection uses the Canny algorithm, and the specific steps are: Use a Gaussian filter to smooth the image; Use the Sobel operator to calculate the gradient strength and direction at each pixel in the image; Perform non-maximum suppression; Double threshold detection uses two thresholds, a high threshold and a low threshold. Pixels above the high threshold are considered strong edges, while pixels below the low threshold are considered non-edges. Pixels between the two thresholds are considered edges only if they are connected to strong edge pixels. Edge tracking, which connects weak edge pixels to strong edge pixels through edge connection algorithm to form a continuous edge; Edges are refined and an edge image is output, where edge pixels are displayed in white and non-edge pixels are displayed in black.

[0008] Based on the above scheme, the relative position of the heavy-duty truck and the railway flatbed is determined by: According to the edge line image fitted by the least squares method, the expression of the middle line of the railway slab is obtained: Y = k + b ,in, Represents the horizontal coordinate in the image, with the origin at the upper left corner of the image and the direction to the right. Y It represents the longitudinal coordinate in the picture, with the direction downward. k and b are constants. The coordinates of the geometric center point of the calibrated special vehicle are , then the relative position of the special vehicle and the railway flatbed is determined by the angle between the center axis of the vehicle and the center axis of the railway flatbed The distance between the vehicle geometric center and the centerline of the railway slab Sure.

[0009] Based on the above solution, the RGBD camera module includes an RGB camera and a depth sensor, which can simultaneously capture RGB images and depth images.

[0010] On the basis of the above scheme, the straight line detection is Hough transform, which finds the longest and second longest straight lines, eliminates the edge concave parts, and finally obtains a smooth railway slab edge line.

[0011] Based on the above solution, using depth information to detect the edge line includes solving the derivative of the angle: ,in, Indicates depth, Indicates the angular resolution of the camera module.

[0012] The present invention also provides a system for calculating the relative position of a heavy-duty truck and a railway flatbed, comprising an RGBD camera module and an image processing module. The relative position is calculated using the method for calculating the relative position of a heavy-duty truck and a railway flatbed described in the above technical solution.

[0013] Compared with the prior art, the present invention has the following beneficial effects: by accurately identifying the edge of the railway, the relative position of the heavy-loaded truck and the railway flatbed can be quickly and accurately measured, the efficiency of railway loading and unloading of heavy-loaded trucks can be improved, and the waste of personnel can be reduced, saving costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a flow chart of the method for calculating the relative position of a heavy-duty truck and a railway flatbed of the present application; Figure 2 This is a schematic diagram of the installation position of the RGBD camera module of this application; Figure 3 is the RGB image actually captured by this application; Figure 4 It is the grayscale image converted from the RGB image in this application; Figure 5 This is the result of edge detection of the image using the Canny algorithm in this application; Figure 6 is a graphical illustration of the derivative of depth information with respect to angle in this application; Figure 7 is a schematic diagram of the left railroad slab edge line identified in this application; Figure 8 is a schematic diagram of the right railroad slab edge line identified in this application; Fig. 9 It is the result graph of line detection using Hough transform in this application; Fig.10 This is a schematic diagram of the left and right edge lines of a railway slab fitted using the least squares method in this application; DETAILED DESCRIPTION The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0015] The embodiment of the present invention provides a method for calculating the relative position of a heavy-duty truck and a railway flatbed, which can realize rapid and accurate measurement of the relative position of a heavy-duty truck and a railway flatbed. Figure 1 As shown, the specific steps are as follows: Step S1, install or firmly attach the RGBD (RGB-Depth) camera module above the front of the vehicle, such as Figure 2 As shown in the figure. On the one hand, this position is at a moderate height and is easy to install. The personnel do not need to use escalators, but only need to stand on the railway flatbed to complete the module installation; on the other hand, it can provide a larger field of view for the camera module. In addition, the height of the camera module is relatively close to the height of the personnel's field of view, making it easier to incorporate the driver's driving experience into the image processing algorithm.

[0016] The following reasonable assumptions are made for the vehicle equipped with the RGBD camera module: ensure that the centerline of the field of view of the RGBD camera module is aligned with the centerline of the vehicle; determine that the geometric center point P of the vehicle is located at the centerline of the field of view. After calibration, the linear relationship between the distance L from P to the center of the railway slab and the distance d from P to the centerline of the railway slab in the camera field of view is L=K d, where K is the distance coefficient; Step S2, using the RGBD camera module installed in step S1 to collect RGB images and convert them into grayscale images; like Figure 3 As shown, taking the actual captured image as an example, convert it into a grayscale image, such as Figure 4 shown.

[0017] Specifically, the specific method of converting to a grayscale image is: weighted average the values ​​of the three color channels RGB to generate a grayscale value, Gray=0.299R+0.587G+0.114B, where R represents the value of the red channel, G represents the value of the green channel, and B represents the value of the blue channel; Step S3, performing edge detection on the grayscale image generated in step S2, and outputting an edge image; Preferably, the Canny edge detection algorithm is used to Figure 4 The specific steps for edge detection of the grayscale image shown are as follows: Step S31, using a Gaussian filter to smooth the image and reduce image noise; Through step S31, high-frequency noise components in the image can be removed to provide a clear image for subsequent edge detection; Step S32, using the Sobel operator to calculate the gradient strength and direction at each pixel of the image; Specifically, the Sobel operator realizes edge detection by calculating the gradient of the image in the horizontal and vertical directions; Step S33, based on step S32, performing non-maximum suppression; Through step S33, the gradient strength is locally maximized, that is, in the neighborhood of an edge pixel, only the pixel point with the largest gradient strength will be retained, and the strength of other pixel points will be suppressed; Step S34, dual threshold detection; Specifically, two thresholds are used, a high threshold and a low threshold. Pixels above the high threshold are considered strong edges, while pixels below the low threshold are considered non-edges. Pixels between the two thresholds are considered edges only if they are connected to strong edge pixels. Step S35, edge tracking: based on step S34, the weak edge pixels are connected to the strong edge pixels by an edge connection algorithm to form a continuous edge; Step S36, based on step S35, edge refinement is performed and an edge image is output, wherein edge pixels are displayed in white and non-edge pixels are displayed in black. The edge detection result obtained is as follows: Figure 5 shown.

[0018] exist Figure 5 In the edge detection image shown, there is a lot of interference information, including the edge line of the platform and the inner edge line of the railway flat iron sheet.

[0019] In order to eliminate interference, the edge line of the railway slab is determined in step S4 using the depth information of the RGBD camera module. The specific method is to solve the derivative of the depth information with respect to the angle, that is: ,in, l i Indicates depth, δθ Indicates the angular resolution of the camera module, such as Figure 6 As shown in the figure, it can be seen that for the surface of the railway slab, the derivative is relatively smooth, but for the edge of the railway slab, a relatively large mutation will occur. Therefore, if the derivative exceeds the set threshold at a certain place, it is considered that the edge of the railway slab is near that place.

[0020] The edge line of the railway slab is accurately found by using the depth information of the RGBD camera module in step S4. The edge lines on the left and right sides of the railway slab are detected as follows: Figure 7 and Figure 8 shown.

[0021] Step S5, using Hough transform to eliminate the depression on the edge of the railway slab; according to Figure 7 and Figure 8From the edge detection results shown, we can see that the concavity and unevenness of the edge of the railway slab will affect the detection effect of the edge of the railway slab. Eliminating the edge concavity is very important for accurately fitting the edge of the slab.

[0022] Hough transform is a feature extraction technology commonly used in image processing and computer vision. It is highly robust to noise. The basic idea is to map the image space (usually a two-dimensional pixel coordinate system) to the parameter space, which is used to identify shapes such as straight lines, circles, and ellipses from the image. The present invention uses Hough transform to detect straight lines, then finds the longest and second longest straight lines, and removes the concave parts. The final effect is as follows: Fig. 9 shown.

[0023] Step S6, fitting the left and right edge lines of the railway slab using the least squares method; like Fig.10 As shown, there are two points at each end of the longest and second longest straight lines on the left and right sides. The coordinates of the four points on each side can be used to fit the edge lines of the railway slab on the left and right sides respectively in combination with the least squares method, and finally the left and right edge lines of the railway slab can be fitted; Step S7, determining the relative position relationship between the vehicle and the railway flatbed: According to the edge lines on both sides of the railway slab, the expression of the middle line of the railway slab can be determined, which is recorded as: ,in, X Represents the horizontal coordinate in the image, with the origin at the upper left corner of the image and the direction to the right. Y represents the longitudinal coordinate in the image, with the direction downward, k and b are constants. The coordinates of the geometric center point of the calibrated vehicle are P ( X p , Y p ), then the relative position of the vehicle and the railway plate can be determined by the following two parameters: (1) Angle between the center axis of the vehicle and the center axis of the railway slab: ; (2) Distance between the vehicle geometric center and the centerline of the railway slab: .

[0024] The present invention also includes a system for calculating the relative position of a heavy-duty truck and a railway flatbed, including an RGBD camera module and an image processing module. The relative position is calculated using the method for calculating the relative position of a heavy-duty truck and a railway flatbed described in the above embodiment.

[0025] According to the present invention, accurate identification of the railway edge is achieved through RGBD image information acquisition and image processing, which can realize rapid and accurate measurement of the relative position of heavy-load trucks and railway flatbeds, improve the efficiency of railway loading and unloading of heavy-load trucks, reduce manpower waste, and effectively save costs.

[0026] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0027] Although the above describes the specific implementation methods of the present invention, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A method for calculating the relative position between a heavy-duty truck and a railway flatbed, characterized in that: The following steps are involved: Use RGBD camera module to obtain image data containing information of heavy-duty trucks and railway flatbeds; Performing image processing on the image data to identify edge information of the railway slab; Detecting the edge line of the railway slab by using the depth information in the image; Perform straight line detection on the image to eliminate the depressions on the edge of the railway slab; Using the coordinates of the endpoints of the straight line, the left and right edge lines of the railway slab are fitted using the least squares method; Based on the processed edge information, the relative positions of the heavy-duty truck and the railway flatbed are determined.

2. A method for calculating the relative position between a heavy-duty truck and a railway flatbed according to claim 1, characterized in that: Using the RGBD camera module to obtain the image data specifically includes: The RGBD camera module is installed above the front of the vehicle and RGB images are collected. The centerline of the field of view of the RGBD camera module is aligned with the centerline of the vehicle after calibration. The geometric center point P of the vehicle is located on the centerline of the field of view. After calibration, the distance L from P to the center of the railway slab and the distance d from P to the centerline of the railway slab in the camera field of view have a relationship of L=K d, where K is the distance coefficient.

3. The method for calculating the relative position between a heavy-duty truck and a railway flatbed according to claim 1, characterized in that: The image processing at least includes converting the color image into a grayscale image and performing edge detection on the grayscale image.

4. A method for calculating the relative position between a heavy-duty truck and a railway flatbed according to claim 3, characterized in that: The edge detection uses the Canny algorithm, and the specific steps are: Use a Gaussian filter to smooth the image; Use the Sobel operator to calculate the gradient strength and direction at each pixel in the image; Perform non-maximum suppression; Double threshold detection uses two thresholds, a high threshold and a low threshold. Pixels above the high threshold are considered strong edges, while pixels below the low threshold are considered non-edges. Pixels between the two thresholds are considered edges only if they are connected to strong edge pixels. Edge tracking, which connects weak edge pixels to strong edge pixels through edge connection algorithm to form a continuous edge; Edges are refined and an edge image is output, where edge pixels are displayed in white and non-edge pixels are displayed in black.

5. The method for calculating the relative position between a heavy-duty truck and a railway flatbed according to claim 1, characterized in that: Determining the relative position of the heavy-duty truck and the railway flatbed includes: According to the edge line image fitted by the least squares method, the expression of the middle line of the railway slab is obtained: Y = k + b ,in, Represents the horizontal coordinate in the image, with the origin at the upper left corner of the image and the direction to the right. Y It represents the longitudinal coordinate in the picture, with the direction downward. k and b are constants. The coordinates of the geometric center point of the calibrated special vehicle are , then the relative position of the special vehicle and the railway flatbed is determined by the angle between the center axis of the vehicle and the center axis of the railway flatbed The distance between the vehicle geometric center and the centerline of the railway slab Sure.

6. The method for calculating the relative position between a heavy-duty truck and a railway flatbed according to claim 1, characterized in that: The RGBD camera module includes an RGB camera and a depth sensor, and can simultaneously capture RGB images and depth images.

7. The method for calculating the relative position between a heavy-duty truck and a railway flatbed according to claim 1, characterized in that: The straight line detection is Hough transformation, which finds the longest and second longest straight lines, eliminates the edge concave parts, and finally obtains a smooth railway slab edge line.

8. The method for calculating the relative position between a heavy-duty truck and a railway flatbed according to claim 1, characterized in that: Using depth information to detect the edge of a railway slab involves solving the derivative of the angle: ,in, Indicates depth, Indicates the angular resolution of the camera module.

9. A system for calculating the relative position of a heavy-duty truck and a railway flatbed, characterized in that: The system comprises an RGBD camera module and an image processing module, and the system uses the relative position calculation method of the heavy-duty truck and the railway flatbed described in any one of claims 1 to 8 to determine the relative position of the heavy-duty truck and the railway flatbed.

Images