Multi-information-fusion-based binocular ranging method and system for autonomous rail rapid transit

Abstract

Provided in the embodiments of the present application are a multi-information-fusion-based binocular ranging method and system for an autonomous rail rapid transit. Parameters of a binocular camera are calibrated A laser emitter and the binocular camera are properly mounted according to a safety early-warning distance and are corrected. A road image is input into a target detection algorithm, a target is output, and an image position of the target is determined. An SGBM stereo matching algorithm is called to calculate a parallax. An initial ranging distance of the target is calculated according to a focal length, an optical center distance and the parallax. Due to changes in a road environment and non-linear effects of camera imaging, when the target moves to a position at the safety early-warning distance, a reference baseline formed by two beams of linear structured light is used to calculate a ranging calibration coefficient at the current moment. The calibration coefficient is used to correct an initial ranging result of the target to obtain a final ranging result, thus improving the binocular ranging precision for an autonomous rail rapid transit.

Description

A binocular ranging method and system for smart rail trains based on multi-information fusion Technical Field

[0001] The present application relates to the technical field of safe assisted driving of smart rail trains, and specifically to a binocular ranging method and system for smart rail trains based on multi-information fusion. Background Art

[0002] Smart rail trains usually use mixed road rights, and the possibility of sudden incidents occurring along the route is higher. Compared with other road vehicles, they have the characteristics of large size, high field of view, and long braking distance. Therefore, it is necessary to accurately measure the relative distance of the target obstacle in front of the vehicle to help the driver avoid danger.

[0003] Compared to other ranging methods, binocular vision ranging offers advantages such as the ability to distinguish obstacle types, low cost, and wide application. Currently, traditional binocular vision ranging techniques and those assisted by laser crosshairs are commonly used. Existing binocular camera calibration is typically performed in the laboratory. As ranging environments change, laboratory calibration parameters may not be applicable in all environments. Furthermore, due to nonlinear effects such as the external environment and camera imaging, the accuracy and reliability of binocular ranging still needs to be improved.

[0004] Summary of the Invention

[0005] The embodiments of the present application provide a binocular ranging method and system for smart rail trains based on multi-information fusion.

[0006] According to a first aspect of an embodiment of the present application, a binocular ranging method for a smart rail train based on multi-information fusion is provided, comprising:

[0007] Obtain a left image, a right image, and internal and external parameters of the left and right cameras; the internal and external parameters of the left and right cameras include the coordinate system of the left image, the coordinate system of the right image, the optical center distance, and the focal length; the left image is an image of the optical center image taken by the left lens of the left camera; the right image is an image of the optical center image taken by the right lens of the left camera; the left and right cameras are installed on the front of the smart rail train;

[0008] Based on the internal and external parameters of the left and right cameras, the left image and the right image are adjusted into a left adjustment coordinate image and a right adjustment coordinate image respectively through a stereo correction algorithm; wherein the origins of the coordinates of the left adjustment coordinate image and the right adjustment coordinate image are the same as those of the left image and the right image respectively;

[0009] Based on the left-adjusted coordinate image and the right-adjusted coordinate image, a target to be measured is obtained through a target detection algorithm; the target to be measured is a target whose distance from the smart rail train needs to be measured;

[0010] Based on the left adjustment coordinate image, the right adjustment coordinate image, the target to be measured, and the internal and external parameters of the left and right cameras, an initial ranging distance is obtained; the initial ranging distance represents the distance from the target to be measured to the left and right cameras;

[0011] Obtaining a safety warning distance; the safety warning distance indicates a distance for determining whether correction is required;

[0012] Based on the focal length, initial ranging distance and safety warning distance, correction is performed to obtain the final ranging distance.

[0013] Optionally, performing correction based on the focal length, the initial ranging distance, and the safety warning distance to obtain the final ranging distance includes:

[0014] If the initial distance is greater than the safety warning distance, it is determined that the target is not within the safety warning distance, and the focal length is subtracted from the initial distance to obtain the final distance.

[0015] If the initial ranging distance is equal to the safety warning distance, determining that the target to be measured is located at the safety warning distance, and obtaining a first correction coefficient based on the safety warning distance, the characteristic distance, and the focal length;

[0016] The initial measuring distance is corrected by subtracting the first correction coefficient from the initial measuring distance and then subtracting the focal length to obtain the final measuring distance.

[0017] If the initial ranging distance is less than the safety warning distance, it is determined to be within the safety warning distance, and a second correction coefficient is obtained based on the initial ranging distance, the safety warning distance, and the focal length;

[0018] The initial measuring distance is corrected by subtracting the second correction coefficient from the initial measuring distance and then subtracting the focal length to obtain the final measuring distance.

[0019] Optionally, if the initial ranging distance is equal to the safety warning distance, determining that the target to be measured is located at the safety warning distance, and obtaining a first correction coefficient based on the safety warning distance, the characteristic distance, and the focal length include:

[0020] The cross intersection of the horizontal line structured light and the vertical line structured light is used as a feature point; the horizontal line structured light and the vertical line structured light are two line structured lights emitted by a horizontal laser emitter and a vertical laser emitter respectively; the horizontal laser emitter and the vertical laser emitter are respectively installed above and below the horizontal line perpendicular to the left and right cameras;

[0021] Based on the feature points, binocular ranging is performed to obtain a feature distance; the feature distance represents the distance between the feature point and the optical center of the lens of the left and right cameras;

[0022] A first correction coefficient is obtained based on the safety warning distance, the characteristic distance and the focal length.

[0023] Optionally, if the initial ranging distance is less than the safety warning distance, it is determined to be within the safety warning distance, and a second correction coefficient is obtained based on the initial ranging distance, the safety warning distance, and the focal length, including:

[0024] Based on the initial ranging distance, the safety warning distance and the focal length, determining a proportional relationship between the first correction coefficient and the second correction coefficient to obtain a correction proportional relationship;

[0025] Multiplying the first correction coefficient by the correction ratio to obtain a second correction coefficient;

[0026] The second correction coefficient is specifically obtained by the following formula: β = K β ×α,

[0027] Among them, the K β represents a correction proportional relationship, β represents the second correction coefficient, and α represents the first correction coefficient.

[0028] Optionally, obtaining the initial ranging distance based on the left adjusted coordinate image, the right adjusted coordinate image, the target to be measured, and the internal and external parameters of the left and right cameras includes:

[0029] Based on the left adjustment coordinate image and the right adjustment coordinate image, a disparity is obtained by a stereo matching algorithm; the disparity represents the difference between the positions of the same target to be measured in the left adjustment coordinate image and the right adjustment coordinate image;

[0030] An initial ranging distance is obtained based on the focal length, optical center distance and parallax.

[0031] Optionally, obtaining a first correction coefficient based on the safety warning distance, the characteristic distance, and the focal length includes:

[0032] The first correction coefficient is obtained by the following formula: α=LL s -f,

[0033] Wherein, α represents the first correction coefficient, L represents the safety warning distance, and L s represents, and f represents the focal length.

[0034] Optionally, determining a proportional relationship between the first correction coefficient and the second correction coefficient based on the initial ranging distance, the safety warning distance, and the focal length to obtain the correction proportional relationship includes:

[0035] The correction ratio relationship is specifically obtained by calculating using the following formula:

[0036] Among them, the K β represents the second correction coefficient, D0 represents the initial ranging distance, L represents the safety warning distance, and f represents the focal length.

[0037] Optionally, obtaining an initial ranging distance based on the focal length, optical center distance, and parallax includes:

[0038] The initial ranging distance is specifically obtained by calculating using the following formula:

[0039] Wherein, D represents the initial ranging distance, B represents the optical center distance, d represents the parallax, and f represents the focal length.

[0040] Optionally, the calculation formula of the initial ranging distance is derived as follows:

[0041] According to the principle of similar triangles, the equation is:

[0042] Wherein, D0 represents the initial distance, B represents the optical center distance, and x l is the coordinate of the target to be measured in the image plane of the left camera, x r is the coordinate of the target to be measured in the image plane of the right camera, P L is the imaging point of the target to be measured on the left lens of the left and right cameras, P is the imaging point of the target to be measured on the right lens of the left and right cameras, and f is the focal length;

[0043] It can be deduced that:

[0044] Among them, x l -x r It represents the imaging parallax of the target to be measured between the left and right cameras. The equation is:

[0045] Here, d represents the parallax.

[0046] According to a second aspect of the embodiment of the present application, a binocular ranging system for smart rail trains based on multi-information fusion is provided.

[0047] Based on the internal and external parameters of the left and right cameras, the left image and the right image are respectively adjusted to the left adjustment coordinate image and the right adjustment coordinate image through a stereo correction algorithm; wherein the origins of the coordinates of the left adjustment coordinate image and the right adjustment coordinate image are the same as those of the left image and the right image respectively.

[0048] This invention provides a binocular ranging method for smart rail trains based on multi-information fusion. By introducing line structured light with a preset safety warning distance, it enables continuous correction of ranging accuracy. Applying this binocular ranging method to road detection ahead of smart rail vehicles, this purely visual solution solves the problem of measuring the distance to targets ahead of smart rail vehicles. Compared to other ranging solutions, this method offers lower costs and can accurately distinguish the target type, whether it is a vehicle, pedestrian, or other obstacle.

[0049] Based on the existing binocular ranging method, the initial ranging results of the binocular camera are corrected by combining line structured light, which can significantly improve the accuracy of binocular ranging.

[0050] Whether to perform initial ranging correction is determined based on the safety warning distance of the smart rail train. When the target object is outside the safety warning distance, the ranging accuracy requirement is relatively low. The initial ranging result of the binocular camera can be used as the final ranging result. The initial ranging result of the binocular camera is only corrected when the target object is within the safety warning distance. This improves the ranging efficiency and accuracy while ensuring driving safety.

[0051] In summary, by overcoming the effects of nonlinearities such as the external environment and camera imaging on measurement accuracy, we added two lines of structured light, one horizontal and one vertical, to the road ahead based on the safety warning distance of the smart rail train. The distance between the intersection of the two lines of structured light and the smart rail train was used as the scale for binocular ranging, and the error correction coefficient for binocular ranging was calculated. This improved the accuracy of binocular ranging. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0053] FIG1 is a flow chart of a binocular ranging method for a smart rail train based on multi-information fusion provided by an embodiment of the present invention.

[0054] FIG2 is a specific flow chart of a binocular ranging method for a smart rail train based on multi-information fusion provided by an embodiment of the present invention.

[0055] FIG3 is a schematic diagram of the installation positions of two laser emitters and a binocular camera in a binocular ranging method for a smart rail train based on multi-information fusion provided by an embodiment of the present invention.

[0056] FIG4 is a schematic diagram showing the principle of initial ranging derivation of a binocular ranging method for a smart rail train based on multi-information fusion provided by an embodiment of the present invention.

[0057] FIG5 is a schematic diagram showing the principle of target correction in three stages in a binocular ranging method for a smart rail train based on multi-information fusion provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0058] In order to make the technical solutions and advantages of the embodiments of the present application more clearly understood, the exemplary embodiments of the present application are further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, and are not an exhaustive list of all the embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other unless they conflict.

[0059] Example 1

[0060] As shown in Figures 1 and 2, an embodiment of the present application provides a binocular ranging method for smart rail trains based on multi-information fusion.

[0061] S101: Obtain a left image, a right image, and internal and external parameters of the left and right cameras; the internal and external parameters of the left and right cameras include the coordinate system of the left image, the coordinate system of the right image, the optical center distance, and the focal length; the left image is an image of the optical center imaging taken by the left lens of the left and right cameras; the right image is an image of the optical center imaging taken by the right lens of the left and right cameras; the left and right cameras are installed on the front of the smart rail train.

[0062] The left image and the right image are images captured by two cameras, a left camera and a right camera, at the same time.

[0063] The left and right cameras are binocular cameras. The left side of the smart rail train, which is viewed from the rear to the front, is the left camera. The right side of the smart rail train, which is viewed from the rear to the front, is the right camera.

[0064] The optical center distance represents the horizontal distance between camera lenses.

[0065] The focal length is the focal length of the left camera and the right camera, and the focal length of the left camera is the same as the focal length of the right camera.

[0066] The internal and external parameters of the left and right cameras are the parameters of the calibrated left and right cameras, and the method for calibrating the left and right cameras is the Zhang Zhengyou calibration method.

[0067] Among them, the left and right cameras are reasonably installed on the front of the smart rail train, and are installed in combination with the structural dimensions of the front of the smart rail train. While ensuring that the optical axes of the left and right cameras are parallel to each other, the images captured by the cameras have a wide range and a small blind spot.

[0068] S102: Based on the internal and external parameters of the left and right cameras, the left image and the right image are adjusted into a left adjustment coordinate image and a right adjustment coordinate image respectively through a stereo correction algorithm; wherein the origins of the coordinates of the left adjustment coordinate image and the right adjustment coordinate image are the same as those of the left image and the right image respectively.

[0069] The camera coordinate system takes the optical axis of the camera as its origin, the front as the Z axis, the right as the X axis, and the bottom as the Y axis.

[0070] The two coordinate-adjusted imaging images are a left image and a right image captured by the left and right cameras, which are adjusted to have the same origin coordinates.

[0071] The two adjusted coordinate imaging images include a left adjusted coordinate imaging image and a right adjusted coordinate imaging image. The left adjusted coordinate imaging image is an image captured by the left camera of the left and right cameras after the coordinates are adjusted. The right adjusted coordinate imaging image is an image captured by the right camera of the left and right cameras after the coordinates are adjusted.

[0072] Among them, the stereo correction algorithm is called according to the calibrated left and right camera parameters and installation position relationship so that the plane origin coordinates of the two images captured by the left and right cameras are the same, and the projection surfaces are coplanar and parallel.

[0073] S103: Obtaining the target to be detected through a target detection algorithm based on the adjusted coordinate image;

[0074] Specifically, based on the left-adjusted coordinate image and the right-adjusted coordinate image, a target to be measured is obtained through a target detection algorithm; the target to be measured is a target whose distance from the smart rail train needs to be measured.

[0075] The target to be measured represents the target imaged in the adjusted coordinate image.

[0076] Among them, the road image input to the target detection algorithm is the adjusted coordinate image after image enhancement. The image enhancement algorithm is used to enhance the image features under unfavorable visual weather conditions, which is beneficial to reducing the measurement error caused by unclear feature points in binocular ranging.

[0077] Among them, the adjusted coordinate image of the road conditions taken in front of the smart rail train is input into the target detection algorithm, the target is output and the image position of the target is determined.

[0078] Among them, the target detection algorithm can adopt YOLOV5 and the like.

[0079] S104: Obtaining an initial ranging distance based on the left adjustment coordinate image, the right adjustment coordinate image, the target to be measured, and internal and external parameters of the left and right cameras; the initial ranging distance represents the distance from the target to be measured to the left and right cameras.

[0080] The initial ranging distance is the shortest distance between the projection point of the target on the plane where the line between the optical axis and the optical center is located and the line between the lens, calculated using the principle of similar triangles.

[0081] S105: Obtain safety warning distance;

[0082] The safety warning distance indicates a distance for determining whether correction is required.

[0083] Among them, whether to perform initial ranging correction is determined based on the safety warning distance of the smart rail train. When the target object is outside the safety warning distance, the ranging accuracy requirement is relatively low. The initial ranging result of the binocular camera can be used as the final ranging result. The initial ranging result of the binocular camera is only corrected when the target object is within the safety warning distance. This improves the ranging efficiency and accuracy while ensuring driving safety.

[0084] Among them, the safety warning distance is a set distance value determined based on the structure, running speed and braking distance of the smart rail train.

[0085] The safety warning distance is a cross point formed by two line structured lights.

[0086] S106: Calibrate based on the focal length, the initial ranging distance, and the safety warning distance to obtain a final ranging distance.

[0087] Optionally, performing correction based on the focal length, the initial ranging distance, and the safety warning distance to obtain the final ranging distance includes:

[0088] Among them, the ranging correction is determined based on the distance between the target to be measured and the body of the smart rail train, and judges whether the target to be measured is within the safety warning distance. If not, the initial ranging distance of the target to be measured is used as the final ranging result. If so, the first correction coefficient and the second correction coefficient are used to correct the initial ranging distance, and the corrected value is used as the final ranging result.

[0089] If the initial ranging distance is greater than the safety warning distance, it is determined that the target to be measured is not within the safety warning distance, and the focal length is subtracted from the initial ranging distance to obtain the final ranging distance.

[0090] If the initial distance is greater than the safety warning distance, the final distance is calculated as: D = D0 - f, where D represents the final distance, D0 represents the initial distance, and f represents the focal length.

[0091] If the initial ranging distance is equal to the safety warning distance, it is determined that the target to be measured is located at the safety warning distance, and a first correction coefficient is obtained based on the safety warning distance, the characteristic distance and the focal length.

[0092] The first correction coefficient is a correction coefficient of the target to be measured at the safety warning distance, and the calculated first correction coefficient varies with the external environment and the target object.

[0093] The initial ranging distance is corrected by subtracting the first correction coefficient from the initial ranging distance and then subtracting the focal length to obtain the final ranging distance.

[0094] If the initial ranging distance is equal to the safety warning distance, the final ranging distance is calculated as follows: D = D0 - f - α, where D represents the final ranging distance, D0 represents the initial ranging distance, f represents the focal length, and α represents the first correction coefficient.

[0095] If the initial ranging distance is less than the safety warning distance, it is determined to be within the safety warning distance, and a second correction coefficient is obtained based on the initial ranging distance, the safety warning distance and the focal length.

[0096] Among them, the second correction coefficient is the correction coefficient when the target is within the safety warning distance. As the distance between the target and the smart rail vehicle shortens, the first correction coefficient is corrected by linear proportion to calculate the second correction coefficient.

[0097] The initial measuring distance is corrected by subtracting the second correction coefficient from the initial measuring distance and then subtracting the focal length to obtain the final measuring distance.

[0098] If the initial ranging distance is less than the safety warning distance, the final ranging distance is calculated as follows: D = D0 - f - β, where D represents the final ranging distance, D0 represents the initial ranging distance, f represents the focal length, and β represents the second correction coefficient.

[0099] Through the above method, due to the introduction of line structured light with a preset safety warning distance, the ranging accuracy can be continuously corrected, thereby improving the binocular ranging accuracy of the smart rail train.

[0100] Optionally, if the initial ranging distance is equal to the safety warning distance, determining that the target to be measured is located at the safety warning distance, and obtaining a first correction coefficient based on the safety warning distance, the characteristic distance, and the focal length include:

[0101] The cross intersection formed by the horizontal line structured light and the vertical line structured light is used as the feature point; the horizontal line structured light and the vertical line structured light are two line structured lights emitted by the horizontal laser emitter and the vertical laser emitter respectively; the horizontal laser emitter and the vertical laser emitter are respectively installed above and below the horizontal line perpendicular to the left and right cameras.

[0102] As shown in Figure 5, to ensure that the target object can be scanned into the safety warning position, a transverse laser emitter must be installed above the front of the vehicle. The transverse line structured light emitted by the transverse laser emitter is projected obliquely onto the ground in front of the vehicle at a fixed vertical distance from the vehicle body, which is the safety warning distance (L). The longitudinal laser emitter is installed below the front of the vehicle. The longitudinal line structured light emitted by the longitudinal laser emitter is projected vertically onto the ground in front of the vehicle. The line structured light projected by the two lasers forms a cross-shaped reference standard on the ground, and the laser planes are perpendicular to each other. JU is the launch point of the transverse laser emitter, JD is the launch point of the longitudinal laser emitter, CL is the center point of the left camera lens, and CR is the center point of the right camera lens.

[0103] Based on the feature points, binocular ranging is performed to obtain a feature distance, which represents the distance between the feature point and the optical center of the lens of the left and right cameras.

[0104] The binocular distance measurement method for obtaining the characteristic distance based on the characteristic points is the same as the distance measurement method for obtaining the initial distance measurement, and both methods perform distance measurement based on the principle of similar triangles.

[0105] Among them, as shown in Figure 3, point S (the cross intersection formed by the horizontal line structured light and the vertical line structured light) is selected as the feature point, and the ranging principle shown in Figure 4 is selected to perform binocular ranging on point S to obtain the distance Ls from the point to the optical center of the lens, and calculate the first correction coefficient (α) mentioned above.

[0106] Among them, according to the principle of similar triangles:

[0107] A correction coefficient α is obtained based on the safety warning distance, the characteristic distance and the focal length.

[0108] Control two transverse laser emitters to emit two line structured lights respectively to obtain the target object position; the target object position is the position where the cross intersection of the transverse structured light emitted by the transverse laser emitter changes; the two transverse laser emitters are respectively installed above and below the horizontal line perpendicular to the left and right cameras.

[0109] The vertical distances between the two lateral laser emitters and the horizontal line between the left and right cameras are the same.

[0110] Among them, due to the nonlinear influence of road environment changes and camera imaging, when the target moves to the safety warning distance position, the reference benchmark composed of two line structured lights is used to calculate the ranging calibration coefficient at the current moment.

[0111] Optionally, if the initial ranging distance is less than the safety warning distance, it is determined to be within the safety warning distance, and a second correction coefficient is obtained based on the initial ranging distance, the safety warning distance, and the focal length, including:

[0112] Based on the initial ranging distance, the safety warning distance and the focal length, a proportional relationship between the first correction coefficient and the second correction coefficient is determined to obtain a correction proportional relationship.

[0113] The correction proportional relationship is obtained according to the proportional relationship between the initial ranging distance of the target to be measured at the current position and the safety warning distance.

[0114] The first correction coefficient is multiplied by the correction ratio to obtain a second correction coefficient.

[0115] The second correction coefficient is a coefficient that needs to be used to correct the distance after the target to be measured enters the safety warning distance.

[0116] The second correction coefficient is specifically obtained by the following formula: β = K β ×α,

[0117] Among them, the K β represents a correction proportional relationship, β represents the second correction coefficient, and α represents the first correction coefficient.

[0118] Optionally, obtaining the initial ranging distance based on the left adjusted coordinate image, the right adjusted coordinate image, the target to be measured, and the internal and external parameters of the left and right cameras includes:

[0119] Based on the left adjustment coordinate image and the right adjustment coordinate image, a disparity is obtained through a stereo matching algorithm; the disparity represents the difference between the positions of the same target to be measured in the left adjustment coordinate image and the right adjustment coordinate image.

[0120] In this embodiment, the SGBM stereo matching algorithm is adopted.

[0121] An initial ranging distance is obtained based on the focal length, optical center distance and parallax.

[0122] Among them, the principle of similar triangles is used to calculate the distance from the projection point of the target on the plane where the line between the optical axis and the optical center is located to the line between the lenses to obtain the initial ranging distance.

[0123] Optionally, obtaining a first correction coefficient based on the safety warning distance, the characteristic distance, and the focal length includes:

[0124] The first correction coefficient is obtained by the following formula: α=LL s -f,

[0125] Wherein, α represents the first correction coefficient, L represents the safety warning distance, and L s represents, and f represents the focal length.

[0126] As shown in Figure 4, as the smart rail train and the target to be measured move, when the target to be measured just touches the horizontal line structured light at the safety warning distance, the horizontal line structured light in the frame image captured by the binocular camera occurs, and the initial ranging result is corrected at this time. As shown in Figure 3, based on the initial ranging distance between the intersection formed by the two line structured lights and the smart rail train, this distance value is compared with the actual set distance value of the intersection (safety warning distance) to calculate the first correction coefficient when the target is at the safety warning position.

[0127] By using the above method, the initial distance measurement result of the target at the safety warning distance is corrected with the first correction coefficient α to improve the measurement accuracy and obtain the final distance measurement result.

[0128] Optionally, if the proportional relationship between the first correction coefficient and the second correction coefficient is determined based on the initial ranging distance, the safety warning distance, and the focal length, the correction proportional relationship is obtained, including:

[0129] The correction ratio relationship is specifically obtained by calculating using the following formula:

[0130] Among them, the K β represents the second correction coefficient, D0 represents, L represents the safety warning distance, and f represents the focal length.

[0131] Optionally, obtaining an initial ranging distance based on the focal length, optical center distance, and parallax includes:

[0132] The initial ranging distance is specifically obtained by calculating using the following formula:

[0133] Wherein, D represents the initial ranging distance, B represents the horizontal distance between the left and right camera lenses, d represents the parallax, and f represents the focal length.

[0134] Optionally, the calculation formula of the initial ranging distance is derived as follows:

[0135] According to the principle of similar triangles, the equation is:

[0136] Wherein, D0 represents the initial distance, B represents the optical center distance, and x l is the coordinate of the target to be measured in the image plane of the left camera, x r is the coordinate of the target to be measured in the image plane of the right camera, P L is the imaging point of the target to be measured in the left lens of the left and right cameras, P is the imaging point of the target to be measured in the right lens of the left and right cameras, and f is the focal length.

[0137] The focal length is the distance from the optical center of the camera to the image plane.

[0138] It can be deduced that:

[0139] Among them, x l -x r It represents the imaging parallax of the target to be measured between the left and right cameras. The equation is:

[0140] Here, d represents the parallax.

[0141] The relationship between parallax, optical center distance, focal length, and initial ranging distance during the derivation process is shown in Figure 4, based on the principle of similar triangles. The vertical upward arrow points in the direction of the camera's optical axis, and the two horizontal lines are the horizontal projections of the image plane.

[0142] Optionally, the left and right cameras are calibrated based on the Zhang Zhengyou calibration method, and the internal and external parameters of the left and right cameras are obtained, including the coordinates of the optical centers of the left and right cameras, the distance between the optical centers, and the focal length;

[0143] Optionally, the left and right cameras can be properly installed on the front of the smart rail train;

[0144] Optionally, a horizontal laser emitter and a vertical laser emitter are installed on the front of the smart rail train, with the two laser emitters installed in an upper and lower arrangement between the left and right cameras;

[0145] Optionally, a stereo correction algorithm is called according to the calibrated parameters and installation position relationship of the left and right cameras so that the coordinates of the plane origins of the two images captured by the left and right cameras are the same and the projection planes are coplanar and parallel;

[0146] Optionally, the road image is input into a target detection algorithm, which identifies targets (vehicles, pedestrians, obstacles) on the road and determines the horizontal and vertical coordinates of the identified targets in the image coordinate system, where the horizontal and vertical coordinates are coordinates of the target imaging;

[0147] Optionally, call the SGBM stereo matching algorithm to perform stereo matching on the two frames of images of the target captured simultaneously by the binocular camera and calculate the disparity;

[0148] Optionally, the shortest distance between the projection point of the target on the plane where the line between the optical axis and the optical center lies and the line connecting the lens is calculated using the principle of similar triangles based on the focal length, the distance between the optical center and the lens;

[0149] Optionally, as the smart rail train and the target object move, when the target object just touches the horizontal line structured light at the safety warning distance, the horizontal line structured light occurs in the frame image captured by the binocular camera. At this time, the initial ranging result is corrected. The initial ranging result of the distance between the cross point formed by the two line structured lights and the smart rail train is compared with this distance value and the actual set distance value of the cross point (safety warning distance). The correction coefficient α of the target when it is at the safety warning position is calculated, and the initial ranging result of the target at the safety warning distance position is corrected with the correction coefficient α at this moment to improve the measurement accuracy and obtain the final ranging result.

[0150] Optionally, after the target enters the safety warning distance, the correction coefficient α is further corrected according to the proportional relationship between the initial ranging result of the target at the current position and the safety warning distance to obtain the correction coefficient β. The correction coefficient β is used as the distance correction coefficient of the target within the safety warning distance, and the distance of the target is corrected to improve the measurement accuracy and obtain the final ranging result.

[0151] Example 2

[0152] In the embodiment of the present application, based on the first embodiment, this embodiment provides a binocular ranging system for smart rail trains based on multi-information fusion, including:

[0153] Acquisition module: obtains the left image, right image, and internal and external parameters of the left and right cameras; the internal and external parameters of the left and right cameras include the coordinate system of the left image, the coordinate system of the right image, the optical center distance, and the focal length; the left image is the optical center imaging image captured by the left lens of the left camera; the right image is the optical center imaging image captured by the right lens of the left camera; the left and right cameras are installed on the front of the smart rail train; obtains the safety warning distance; the safety warning distance indicates the distance for determining whether correction is required;

[0154] Image adjustment module: Based on the internal and external parameters of the left and right cameras, the left image and the right image are adjusted into a left adjustment coordinate image and a right adjustment coordinate image respectively through a stereo correction algorithm; wherein the origin of the coordinates of the left adjustment coordinate image and the right adjustment coordinate image is the same as that of the left image and the right image respectively;

[0155] Target detection module: Based on the left-adjusted coordinate image and the right-adjusted coordinate image, the target to be measured is obtained through the target detection algorithm; the target to be measured is the target whose distance from the smart rail train needs to be measured;

[0156] Initial ranging distance calculation module: based on the left adjustment coordinate image, the right adjustment coordinate image, the target to be measured and the internal and external parameters of the left and right cameras, obtains the initial ranging distance; the initial ranging distance represents the distance from the target to the left and right cameras;

[0157] Correction module: Based on the focal length, initial ranging distance and safety warning distance, correction is performed to obtain the final ranging distance.

[0158] The specific implementation of each module can refer to the above method and will not be repeated here. The system has the same technical effect as the above method.

[0159] In the description of this application, it should be understood that the terms "front", "rear", "head", "tail", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, they cannot be understood as limitations on this application.

[0160] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Thus, a feature specified as "first" or "second" may explicitly or implicitly include one or more of the features. Throughout the description of this application, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0161] In this application, unless otherwise specified or limited, terms such as "mounted" and "connected" should be understood broadly. For example, "connected" can refer to direct connection or indirect connection through an intermediate medium, and can refer to internal communication between two components or interaction between two components. Those skilled in the art will understand the specific meanings of these terms in this application based on the specific circumstances.

[0162] Although some optional embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including some optional embodiments and all changes and modifications that fall within the scope of the present application.

[0163] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A binocular ranging method for smart rail trains based on multi-information fusion, characterized in that: include: Obtain a left image, a right image and internal and external parameters of the left and right cameras; the internal and external parameters of the left and right cameras include the coordinate system of the left image, the coordinate system of the right image, the optical center distance and the focal length; the left image is an image of the optical center imaging taken by the left lens of the left and right cameras; the right image is an image of the optical center imaging taken by the right lens of the left and right cameras; the left and right cameras are installed on the front of the smart rail train; Based on the internal and external parameters of the left and right cameras, the left image and the right image are adjusted into a left adjustment coordinate image and a right adjustment coordinate image respectively through a stereo correction algorithm; wherein the origins of the coordinates of the left adjustment coordinate image and the right adjustment coordinate image are the same as those of the left image and the right image respectively; Based on the left-adjusted coordinate image and the right-adjusted coordinate image, a target to be measured is obtained through a target detection algorithm; the target to be measured is a target whose distance from the smart rail train needs to be measured; Based on the left adjustment coordinate image, the right adjustment coordinate image, the target to be measured and the internal and external parameters of the left and right cameras, an initial ranging distance is obtained; the initial ranging distance represents the distance from the target to be measured to the left and right cameras; Obtaining a safety warning distance; the safety warning distance indicates a distance for determining whether correction is required; Based on the focal length, initial ranging distance and safety warning distance, correction is performed to obtain the final ranging distance.

2. The binocular ranging method for smart rail trains based on multi-information fusion according to claim 1 is characterized in that: The correction is performed based on the focal length, the initial ranging distance and the safety warning distance to obtain the final ranging distance, including: If the initial ranging distance is greater than the safety warning distance, it is determined that the target to be measured is not within the safety warning distance, and the focal length is subtracted from the initial ranging distance to obtain the final ranging distance; If the initial ranging distance is equal to the safety warning distance, it is determined that the target to be measured is located at the safety warning distance, and a first correction coefficient is obtained based on the safety warning distance, the characteristic distance and the focal length; The initial ranging distance is corrected by subtracting the difference between the initial ranging distance and the first correction coefficient and the focal length to obtain a final ranging distance; If the initial ranging distance is less than the safety warning distance, it is determined to be within the safety warning distance, and a second correction coefficient is obtained based on the initial ranging distance, the safety warning distance and the focal length; The initial ranging distance is corrected by subtracting the focal length from the difference between the initial ranging distance and the second correction coefficient to obtain the final ranging distance.

3. The binocular ranging method for smart rail trains based on multi-information fusion according to claim 2 is characterized in that: If the initial ranging distance is equal to the safety warning distance, it is determined that the target to be measured is located at the safety warning distance, and a first correction coefficient is obtained based on the safety warning distance, the characteristic distance and the focal length, including: The cross intersection formed by the horizontal line structured light and the vertical line structured light is taken as a feature point; the horizontal line structured light and the vertical line structured light are two line structured lights emitted by the horizontal laser emitter and the vertical laser emitter respectively; the horizontal laser emitter and the vertical laser emitter are respectively installed above and below the horizontal line perpendicular to the left and right cameras; Based on the feature points, binocular distance measurement is performed to obtain a feature distance; the feature distance represents the distance between the feature point and the optical center of the lens of the left and right cameras; A first correction coefficient is obtained based on the safety warning distance, the characteristic distance and the focal length.

4. The binocular ranging method for smart rail trains based on multi-information fusion according to claim 2 is characterized in that: If the initial ranging distance is less than the safety warning distance, it is determined that the distance is within the safety warning distance, and a second correction coefficient is obtained based on the initial ranging distance, the safety warning distance and the focal length, including: Based on the initial ranging distance, the safety warning distance and the focal length, determining the proportional relationship between the first correction coefficient and the second correction coefficient to obtain a correction proportional relationship; Multiplying the first correction coefficient by the correction ratio to obtain a second correction coefficient; The second correction coefficient is specifically obtained by calculating using the following formula: β＝K β ×α Among them, the K β represents a correction ratio relationship, β represents the second correction coefficient, and α represents the first correction coefficient.

5. The binocular ranging method for smart rail train based on multi-information fusion according to claim 1 is characterized in that: The initial ranging distance is obtained based on the left adjustment coordinate image, the right adjustment coordinate image, the target to be measured and the internal and external parameters of the left and right cameras, including: Based on the left adjustment coordinate image and the right adjustment coordinate image, a disparity is obtained by a stereo matching algorithm; the disparity represents the difference between the positions of the same target to be measured in the left adjustment coordinate image and the right adjustment coordinate image; Based on the focal length, optical center distance and parallax, an initial ranging distance is obtained.

6. The binocular ranging method for smart rail trains based on multi-information fusion according to claim 3 is characterized in that: The obtaining of a first correction coefficient based on the safety warning distance, the characteristic distance and the focal length comprises: The first correction coefficient is specifically obtained by calculating using the following formula: a=LL s -f, Wherein, α represents the first correction coefficient, L represents the safety warning distance, and L s represents, and f represents the focal length.

7. The binocular ranging method for smart rail trains based on multi-information fusion according to claim 4 is characterized in that: The method of determining the proportional relationship between the first correction coefficient and the second correction coefficient based on the initial ranging distance, the safety warning distance and the focal length to obtain the correction proportional relationship includes: The correction ratio relationship is specifically obtained by the following calculation formula: Among them, the K β represents the second correction coefficient, D0 represents the initial ranging distance, L represents the safety warning distance, and f represents the focal length.

8. The binocular ranging method for smart rail trains based on multi-information fusion according to claim 5 is characterized in that: The obtaining of the initial distance measurement based on the focal length, the optical center distance and the parallax comprises: The initial ranging distance is specifically obtained by calculating using the following formula: Wherein, D represents the initial ranging distance, B represents the optical center distance, d represents the parallax, and f represents the focal length.

9. The binocular ranging method for smart rail trains based on multi-information fusion according to claim 8 is characterized in that: The derivation process of the calculation formula of the initial ranging distance is: According to the principle of similar triangles, the equation is: Wherein, D0 represents the initial distance, B represents the optical center distance, and x l is the coordinate of the target to be measured in the image plane of the left camera, x r is the coordinate of the target to be measured in the image plane of the right camera, P L is the imaging point of the target to be measured in the left lens of the left and right cameras, P is the imaging point of the target to be measured in the right lens of the left and right cameras, and f is the focal length; It can be deduced that: Among them, x l -x r It represents the imaging parallax of the target to be measured between the left and right cameras. The equation is: Here, the d represents the parallax.

10. A binocular ranging system for smart rail trains based on multi-information fusion, characterized in that: include: Acquisition module: obtain the left image, right image and internal and external parameters of the left and right cameras; the internal and external parameters of the left and right cameras include the coordinate system of the left image, the coordinate system of the right image, the optical center distance and the focal length; the left image is the image of the optical center imaging taken by the left lens of the left and right cameras; the right image is the image of the optical center imaging taken by the right lens of the left and right cameras; the left and right cameras are installed on the front of the smart rail train; obtain the safety warning distance; the safety warning distance indicates the distance for judging whether correction is required; Image adjustment module: Based on the internal and external parameters of the left and right cameras, the left image and the right image are adjusted into a left adjustment coordinate image and a right adjustment coordinate image respectively through a stereo correction algorithm; wherein the origins of the coordinates of the left adjustment coordinate image and the right adjustment coordinate image are the same as those of the left image and the right image respectively; Target detection module: Based on the left-adjusted coordinate image and the right-adjusted coordinate image, a target to be measured is obtained through a target detection algorithm; the target to be measured is a target whose distance from the smart rail train needs to be measured; Initial ranging distance calculation module: based on the left adjustment coordinate image, the right adjustment coordinate image, the target to be measured and the internal and external parameters of the left and right cameras, the initial ranging distance is obtained; the initial ranging distance represents the distance from the target to be measured to the left and right cameras; Correction module: Based on the focal length, initial ranging distance and safety warning distance, correction is performed to obtain the final ranging distance.

Images