Vehicle leading device, vehicle leading system and vehicle leading method based on machine vision and interlocking signal

Abstract

The present invention relates to the field of shunting operations in railway yards, and specifically to a train leading device, a train leading system based on machine vision and interlocking signals, and a train leading method. The train leading system of the present invention includes a train leading device, a data service platform, and a mobile operation terminal. The present invention realizes automatic identification of routes, automatic detection of foreign objects, and precise measurement of distances by real-time shooting of the road conditions and signal status in front of the vehicle, and performs video recognition and analysis. The real-time video stream and analysis results are transmitted to the data service platform. The data service platform further verifies and confirms the video stream and analysis results in combination with interlocking signals such as the route and signal status of the interlocking system, and transmits the video stream, route and signal status and other information to the shunting team members and the driver through the mobile operation terminal, instructing the driver to drive the locomotive safely, thereby realizing automatic and precise control of the train leading process, while improving operation efficiency and reducing the labor intensity of shunting operators.

Description

Technical Field

[0001] The present invention relates to the field of railway yard shunting operations, and in particular to a train leading device, a train leading system based on machine vision and interlocking signals, and a train leading method. Background Art

[0002] Push shunting is the most basic shunting method in railway yards. This involves a shunting locomotive behind the moving train, using its power to propel the train forward. Because the shunting locomotive driver cannot clearly see the track or signals ahead during push shunting, existing operational organization and technical equipment require a lead car driver to follow the lead car. This driver confirms signal availability, road conditions, and the distances to the tenth, fifth, and third cars during the push. Regardless of wind or rain, extreme cold or heat, whenever shunting operations are underway, the lead car driver must pull the lead car forward, which is physically demanding. Furthermore, due to factors both personal and external, the lead car driver can easily fall from the locomotive, miss when getting on or off the train, or collide with trackside signals, overhead wire stanchions, high platforms, or be injured when the shunting vehicle collides with other vehicles at a level crossing, leading to personal injuries.

[0003] To free the lead locomotive from the burden of navigating the train, it's crucial to enable the shunting locomotive driver to automatically monitor the locomotive's forward environment, signal status, foreign object information, and the distance to the foreign object. Currently, a small number of studies have proposed technical and equipment-based approaches to free the lead locomotive from this burden. For example, the interlocking system's interlocking signal can be transmitted to the locomotive. While this signal can reflect signal status and track occupancy, it cannot detect the presence of a foreign object. Furthermore, the interlocking signal alone cannot determine the distance between the lead locomotive and the preceding vehicle or foreign object. Alternatively, there are video lead locomotive systems that transmit video of the locomotive's forward movement to the driver's cab. However, cameras can only replace human eyes, not brains, and still require human monitoring, which is labor-intensive, prone to oversight, and can lead to inaccurate distance judgments. Some researchers have attempted to use radar ranging systems in railway shunting operations in mining areas, providing effective reference conditions for operators to control locomotive speed. Trials have been conducted in dead-end line shunting operations, and the test results have shown that this system can improve the safety factor of shunting operations. However, the large size of the lidar equipment cannot meet the portability requirements of shunting operations for lead vehicles, resulting in a lack of feasibility of the system. In terms of visual ranging technology, existing research and experiments have mainly focused on road vehicles, especially in the field of autonomous driving. However, there is relatively little research on ranging for railway push shunting operations. Moreover, most existing visual ranging solutions only use a single ranging model for distance measurement, that is, ranging based on the retained vehicle target. This does not consider scenarios where the distance between the lead vehicle and the retained vehicle is too far or too close. There is also a lack of fusion methods based on multiple reference ranging models. Therefore, there is still room for improvement in target ranging accuracy and robustness. Summary of the Invention

[0004] In view of the above technical problems, the present invention proposes a vehicle leading device, a vehicle leading system based on machine vision and interlocking signals, and a vehicle leading method.

[0005] In order to solve the above technical problems, the specific technical solutions adopted by the present invention are summarized as follows:

[0006] A vehicle leader comprises a video acquisition module and a machine vision intelligent analysis module; the video acquisition module comprises a main camera and an auxiliary camera; when the video ranging vehicle leader is working, the optical center of the main camera is parallel to the ground, the optical center of the auxiliary camera is set at an angle to the ground, and the angle is adjustable; the machine vision intelligent analysis module comprises a distance measurement unit for the own vehicle and a retained vehicle based on the track spacing, a distance measurement unit for the own vehicle and a retained vehicle based on the anchor line vanishing point and the retained vehicle detection frame, and a distance measurement unit for the own vehicle coupler and the retained vehicle coupler; the video acquisition module is connected to the machine vision intelligent analysis module.

[0007] Furthermore, to address the varying distances between the ego vehicle and a parked vehicle or foreign object, the present invention proposes ranging methods for three different application scenarios: long distance, medium distance, and short distance. The long distance refers to a distance between the ego vehicle and the parked vehicle of 100-200 meters, the medium distance refers to a distance between the ego vehicle and the parked vehicle of 2-100 meters, and the short distance refers to a distance between the ego vehicle and the parked vehicle of less than 2 meters.

[0008] Furthermore, for long-distance application scenarios, the present invention proposes a method for measuring the distance between the vehicle and the remaining vehicle based on the track spacing using the vehicle leader. The method for measuring the distance between the vehicle and the remaining vehicle based on the track spacing is applied to the distance measurement unit based on the track spacing, specifically including the following steps:

[0009] Step 1. The machine vision intelligent analysis module receives and reads the video image captured by the main camera in the video acquisition module;

[0010] Step 2. The track anchor line detection unit in the machine vision intelligent analysis module detects the tracks in the video image, obtains pixel coordinate pairs of all identifiable tracks in the video image, and determines whether the track is a main track or a side track;

[0011] Step 3. If the track in step 2 is determined to be a main track, calculate the distance D between the vehicle and the reserved vehicle based on the width of the main track.

[0012] Furthermore, the distance D between the vehicle and the reserved vehicle is calculated based on the width of the main track as follows:

[0013]

[0014] Among them, L line is the true width of the main track, f x is the focal length of the main camera, L pixel The pixel width of the main track.

[0015] Furthermore, for medium-distance application scenarios, the present invention proposes a method for measuring the distance between the vehicle and the remaining vehicle based on the vanishing point of the anchor line and the remaining vehicle detection frame using the vehicle leader. The method for measuring the distance between the vehicle and the remaining vehicle based on the vanishing point of the anchor line and the remaining vehicle detection frame is applied to the distance measurement unit based on the vanishing point of the anchor line and the remaining vehicle detection frame, specifically including the following steps:

[0016] Step 1. The machine vision intelligent analysis module receives and reads the video image captured by the main camera in the video acquisition module;

[0017] Step 2. The track anchor line detection unit in the machine vision intelligent analysis module detects and analyzes the track anchor line in the video image to obtain the vanishing point coordinates of the track anchor line;

[0018] Step 3. Based on the vanishing point coordinates of the track anchor line in step 2, correct the attitude angle of the main camera to obtain the yaw angle and pitch angle of the main camera;

[0019] Step 4. The distance measurement unit between the ego vehicle and the reserved vehicle in the machine vision intelligent analysis module detects the reserved vehicle in the video image and measures the distance based on the midpoint of the bottom edge of the reserved vehicle detection frame to obtain the distance d between the ego vehicle and the reserved vehicle.

[0020] Step 5. The track detection unit in the machine vision intelligent analysis module detects the track and determines whether the track is a straight track. If the track is a straight track,

[0021] Based on the pitch angle of the main camera in step 3, the distance between the self-vehicle and the reserved vehicle in step 4 is corrected to obtain the corrected distance d1 between the self-vehicle and the reserved vehicle.

[0022] The corrected distance d1 between the ego vehicle and the reserved vehicle is corrected based on the yaw angle of the main camera in step 3 to obtain a second corrected distance D1 between the ego vehicle and the reserved vehicle.

[0023] If the track is not a straight track, the yaw angle of the main camera is set to zero, and the distance between the ego vehicle and the retained vehicle in step 4 is corrected based only on the pitch angle of the main camera in step 3 to obtain the corrected distance d1 between the ego vehicle and the retained vehicle.

[0024] Furthermore, the vanishing point coordinates of the track anchor line in step 2 are the intersection points of the track anchor lines detected in the video image, and the track anchor line is represented as a sequence of N points, that is, P = {(x0, y0), (x1, y1), ..., (x N―1 ,y N―1 )}, where the y coordinates of the track anchor points are fixed and uniformly sampled along the vertical axis of the video image. The slope m and intercept c of the best-fit anchor line are found using the least squares method, resulting in the anchor line fitting equation y = mx + c. The intersection coordinates (u, v) of the track anchor lines are further calculated:

[0025]

[0026] Among them, m1 and m2 are the slopes of the two track anchor lines, and c1 and c2 are the intercepts of the two track anchor lines.

[0027] Furthermore, the yaw angle of the main camera in step 3 of the method for measuring the distance between the vehicle and the remaining vehicle based on the anchor line vanishing point and the remaining vehicle detection frame is, Where γ is the yaw angle of the main camera, W is the width of the imaging plane, u1 is the horizontal coordinate of the vanishing point of the track anchor line in the presence of pitch and yaw angles, and f x is the equivalent focal length in the x-axis direction of the camera coordinate system, where the camera coordinate system takes the center of the main camera's optical axis as its origin, the right is the positive direction of the x-axis, the downward is the positive direction of the y-axis, and the forward is the positive direction of the z-axis; the pitch angle of the main camera is, Where θ is the pitch angle of the main camera, H is the height of the imaging plane, v1 is the vertical coordinate of the track vanishing point when there are pitch and yaw angles, and f y is the equivalent focal length in the y-axis direction of the camera coordinate system.

[0028] Furthermore, the distance d between the ego vehicle and the retained vehicle in step 4 of the method for measuring the distance between the ego vehicle and the retained vehicle based on the anchor line vanishing point and the retained vehicle detection frame is:

[0029]

[0030] Among them, μ is the angle formed by the main camera optical axis and the line connecting the center of the bottom edge of the retained car detection frame and the main camera optical center, and H c is the distance between the main camera and the ground.

[0031] Furthermore, the distance d1 between the ego vehicle and the retained vehicle after correction in step 5 of the method for measuring the distance between the ego vehicle and the retained vehicle based on the anchor line vanishing point and the retained vehicle detection frame is,

[0032]

[0033] Wherein, d1 is the distance between the self-vehicle and the reserved vehicle, which is corrected based on the pitch angle of the main camera; H c is the distance between the main camera and the ground; μ is the angle formed by the main camera's optical axis and the line connecting the center of the bottom edge of the retained vehicle detection frame and the main camera's optical center; θ is the main camera's pitch angle;

[0034] The distance D1 between the vehicle and the remaining vehicle after the secondary correction is:

[0035]

[0036] in, The angle formed by the line connecting the midpoint of the bottom edge of the retained vehicle detection frame and the optical center of the main camera and the optical axis in the vertical plane is calculated as follows:

[0037]

[0038] Among them, u c u is the horizontal coordinate of the center point of the bottom edge of the car detection frame retained in the camera imaging plane, o is the coordinate of the origin of the image coordinate system in the pixel coordinate system.

[0039] Furthermore, for close-range application scenarios, the present invention proposes a method for measuring the distance between the own vehicle coupler and the retained vehicle coupler using the vehicle leader. The method for measuring the distance between the own vehicle coupler and the retained vehicle coupler is applied to the unit for measuring the distance between the own vehicle coupler and the retained vehicle coupler, specifically including the following steps:

[0040] Step 1. The machine vision intelligent analysis module receives and reads the video image captured by the auxiliary camera in the video acquisition module;

[0041] Step 2: The track anchor line detection unit in the machine vision intelligent analysis module detects and analyzes the track anchor line in the video image to obtain the vanishing point coordinates of the track anchor line, and sets the region of interest based on the vanishing point coordinates of the track anchor line.

[0042] Step 3. The distance measurement unit between the own vehicle coupler and the reserved vehicle coupler in the machine vision intelligent analysis module detects the own vehicle coupler and the reserved vehicle coupler in the video image to identify the own vehicle coupler and the reserved vehicle coupler;

[0043] Step 4. The distance measurement unit of the self-vehicle coupler and the reserved vehicle coupler in the machine vision intelligent analysis module tracks the movement of the self-vehicle coupler and the reserved vehicle coupler and generates a coupler tracking ID, determines whether the self-vehicle coupler and the reserved vehicle coupler are located within the region of interest in step 2, and the coupler located within the region of interest is the reserved vehicle coupler, and records the tracking ID of the reserved vehicle coupler;

[0044] Step 5. Based on the retained vehicle coupler in step 4, the vehicle-to-vehicle coupler and the retained vehicle coupler-to-vehicle coupler distance measurement unit in the machine vision intelligent analysis module analyzes and calculates the distance D2 between the vehicle coupler and the retained vehicle coupler.

[0045] Furthermore, the vanishing point coordinates of the track anchor line in step 2 of the method for measuring the distance between the self-vehicle coupler and the reserved vehicle coupler are the intersection points of the track anchor lines detected in the video image, and the track anchor lines are represented as a sequence of N points, i.e., P = {(x0, y0), (x1, y1), ..., (x N―1 ,y N―1)}, where the y coordinates of the track anchor points are fixed and uniformly sampled along the vertical axis of the video image. The slope m and intercept c of the best-fit anchor line are found using the least squares method, resulting in the anchor line fitting equation y = mx + c. The intersection coordinates (u, v) of the track anchor lines are further calculated:

[0046]

[0047] Among them, m1 and m2 are the slopes of the two track anchor lines, and c1 and c2 are the intercepts of the two track anchor lines.

[0048] Furthermore, the distance D2 between the own vehicle coupler and the reserved vehicle coupler in step 5 of the method for measuring the distance between the own vehicle coupler and the reserved vehicle coupler is,

[0049]

[0050] Where l is the straight-line distance between the auxiliary camera and the reserved vehicle coupler, h is the height of the reserved vehicle coupler, w is the width of the reserved vehicle coupler, D0 is the horizontal distance between the auxiliary camera and the vehicle coupler, h0 is the vertical distance between the auxiliary camera and the vehicle coupler, l OB l is the straight-line distance from the upper right corner of the coupler detection frame in the world coordinate system to the origin of the world coordinate system. OC It is the straight-line distance from the lower left corner coordinate point of the coupler detection frame in the world coordinate system to the origin of the world coordinate system.

[0051] Furthermore, the present invention also proposes a vehicle leading system with machine vision and interlocking signals, including the vehicle leading device, a data service platform and a mobile operation terminal, wherein the vehicle leading device is connected to the data service platform via wireless communication, and the data service platform is connected to the mobile operation terminal via wireless communication.

[0052] Furthermore, the vehicle leader also includes a positioning module.

[0053] Furthermore, the data service platform includes a data information receiving module, a data information analyzing module and a data information sending module.

[0054] Furthermore, the data information receiving module is used to receive and save the real-time video stream collected by the video acquisition module in the vehicle leader, the positioning information collected by the positioning module, the ranging information and foreign object recognition information of the machine vision intelligent analysis module, and the interlocking signal of the interlocking system.

[0055] Furthermore, the data information analysis module is used to compare, verify and correct the interlocking signal of the interlocking system with the real-time video stream collected by the video acquisition module in the vehicle leader, the positioning information collected by the positioning module and the ranging information and foreign object recognition information analyzed by the machine vision intelligent analysis module to determine the route information and signal status ahead of the vehicle.

[0056] Furthermore, the data information sending module is used to send the real-time video stream, ranging information and foreign object identification information to the mobile operation terminal.

[0057] Furthermore, the mobile operation terminal includes a video receiving and display unit, a shunting signal receiving unit, a route signal receiving unit, a shunting plan display unit, a satellite positioning unit and an early warning unit.

[0058] Furthermore, the present invention also proposes a vehicle leading method of a vehicle leading system based on machine vision and interlocking signals, which specifically includes the following steps:

[0059] Step 1. During the shunting operation, before the pushing operation, the leading device is installed on the front of the vehicle;

[0060] Step 2. During the propulsion operation, the video odometry leader collects a real-time video stream in front of the vehicle through the video acquisition module, and the machine vision intelligent analysis module performs intelligent analysis on the collected real-time video stream to obtain an intelligent analysis result;

[0061] Step 3. The video ranging device sends the real-time video stream, the intelligent analysis results and the positioning information collected by the positioning module to the data service platform;

[0062] Step 4. The data information receiving module in the data service platform receives and stores the real-time video stream collected by the video acquisition module in the vehicle leader, the positioning information collected by the positioning module, the ranging information and foreign object recognition information collected by the machine vision intelligent analysis module, and the interlocking signal of the interlocking system;

[0063] Step 5. The data information analysis module compares and verifies the interlocking signal of the interlocking system with the real-time video stream collected by the video acquisition module in the vehicle leader, the positioning information collected by the positioning module, and the ranging information and foreign object recognition information analyzed by the machine vision intelligent analysis module, and corrects them to determine the route information and signal status ahead of the vehicle;

[0064] Step 6. The data information sending module sends the real-time video stream, the distance measurement information, and the foreign object identification information to the mobile operation terminal. If the video distance measurement device identifies a foreign object in front of the vehicle, the data service platform sends a warning message to the mobile operation terminal.

[0065] Step 7. The mobile operation terminal receives and displays the real-time video stream via the video receiving and display unit, receives shunting signaling via the shunting signaling receiving unit, receives route information and signal status as well as the distance measurement information via the route signal receiving unit, receives the shunting plan via the shunting plan display unit, and receives the warning information via the early warning unit;

[0066] Step 8. The shunting team instructs the driver to drive safely according to the information flow received by the mobile operation terminal in step 7.

[0067] Furthermore, the analysis result in step 2 of the vehicle leading method includes distance measurement information and foreign object identification information, and the distance measurement information includes the distance between the own vehicle and the retained vehicle, the distance between the own vehicle coupler and the retained vehicle coupler, and the distance between the own vehicle and the foreign object.

[0068] The present invention uses a vehicle-leading system based on machine vision and interlocking signals to capture real-time road conditions and signal status ahead of the vehicle. Using artificial intelligence technologies such as computer vision, the system performs video recognition and analysis, enabling automatic route identification, automatic detection of foreign objects, and precise measurement of the distance between the vehicle and a reserved vehicle or foreign objects. The system then transmits the real-time video stream and analysis results to a data service platform. The video stream and analysis results are further verified and confirmed using interlocking signals such as the route and signal status of the interlocking system. The video stream, route, and signal status information are then transmitted to the shunting crew and driver via a mobile operation terminal, allowing the driver to direct the locomotive. This allows for automatic and precise control of the vehicle-leading process, improving operational efficiency while reducing the workload of shunting personnel, enhancing the safety of shunting operations, and improving the practicality and safety of the entire vehicle-leading system. The present invention replaces traditional laser radar ranging with video distance measurement, thereby addressing the portability issue of vehicle-leading equipment. The present invention proposes different ranging methods for three scenarios: long distance, medium distance and close distance, so that the vehicle leading system of the present invention has a multi-scenario video ranging function, which solves the problem that a single video ranging cannot obtain the distances that are too close or too far to the retained vehicle or foreign objects, making the shunting operation more accurate. In addition, a method of dynamically estimating the camera attitude angle based on the coordinates of the anchor line vanishing point is adopted to improve the accuracy and robustness of video ranging in dynamic scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to achieve the above and other advantages and features of the present invention, the invention summarized above will be described in more detail below with reference to specific embodiments of the invention shown in the accompanying drawings. It should be understood that these drawings only illustrate typical embodiments of the invention and are not to be considered as limiting the scope of the invention. Through the use of the accompanying drawings, the invention will be described and explained in more detail; in the drawings:

[0070] Figure 1 This is a functional module diagram of a vehicle leading system based on machine vision and interlocking signals of the present invention;

[0071] Figure 2 This is a flowchart of a method for measuring the distance between a vehicle and a reserved vehicle based on track spacing according to the present invention;

[0072] Figure 3 This is a distance measurement model diagram of a method for measuring the distance between a vehicle and a reserved vehicle based on track spacing according to the present invention;

[0073] Figure 4 This is a flowchart of a method for measuring the distance between a vehicle and a retained vehicle based on an anchor line vanishing point and a retained vehicle detection frame according to the present invention;

[0074] Figure 5 This is a camera yaw angle calculation model diagram of the present invention;

[0075] Figure 6 This is a model diagram of the present invention based on the bottom edge position of the retained vehicle target detection frame;

[0076] Figure 7 This is a flowchart of a method for measuring the distance between a vehicle coupler and a reserved vehicle coupler according to the present invention;

[0077] Figure 8 This is a distance measurement model diagram based on the method for measuring the distance between the coupler of the own vehicle and the coupler of the retained vehicle of the present invention; DETAILED DESCRIPTION

[0078] The following description is intended to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are for example only, and those skilled in the art may conceive of other obvious variations. Directional terms such as "front," "back," "left," and "right" in the following description are not to be construed as limiting the present invention. The basic principles of the present invention defined in the following description may be applied to other embodiments, variations, improvements, equivalents, and other technical solutions that do not depart from the spirit and scope of the present invention.

[0079] The following is combined with Figure 1-8 The present invention provides a vehicle leading device, a vehicle leading system based on machine vision and interlocking signals, and a vehicle leading method, which are further described in detail:

[0080] As attached Figure 1 As shown, the present invention proposes a vehicle leading system based on machine vision and interlocking signals, including a vehicle leading device, a data service platform and a mobile operation terminal. The vehicle leading device is connected to the data service platform via wireless communication, and the data service platform is connected to the mobile operation terminal via wireless communication.

[0081] Further, such as Figure 1 As shown, a vehicle leader proposed in the present invention includes a video acquisition module and a machine vision intelligent analysis module; the video acquisition module includes a main camera and an auxiliary camera. When the video ranging vehicle leader is working, the optical center of the main camera is parallel to the ground, and the optical center of the auxiliary camera is set at an angle to the ground, and the angle is adjustable; the machine vision intelligent analysis module includes a distance measurement unit for the own vehicle and the retained vehicle based on the track spacing, a distance measurement unit for the own vehicle and the retained vehicle based on the anchor line vanishing point and the retained vehicle detection frame, and a distance measurement unit for the own vehicle coupler and the retained vehicle coupler; the video acquisition module is connected to the machine vision intelligent analysis module.

[0082] As a further preferred technical solution, the vehicle leader is magnetically mounted on the front side of the vehicle being pushed, and collects real-time video streams from the perspective of the head hook connector, and transmits the real-time video streams and the analysis results of the machine vision intelligent analysis module to the data service platform through wireless communication.

[0083] The installation angle of the main camera on the self-vehicle is such that its optical center is parallel to the ground. The main camera is mainly responsible for collecting the video stream of the scene directly in front of the self-vehicle during the pushing operation. The optical center of the auxiliary camera is set at an angle to the ground, and the angle is preferably 60°. The auxiliary camera is mainly responsible for collecting the video stream of the distance between the self-vehicle coupler and the retained vehicle coupler and the connection status scene between the self-vehicle coupler and the retained vehicle coupler during the pushing operation, so as to facilitate the operating personnel to better grasp the completion status of the connection operation between the self-vehicle coupler and the retained vehicle coupler.

[0084] The video acquisition module has a high-performance image sensor and supports 3D noise reduction, strong light suppression, and backlight compensation.

[0085] As a further preferred technical solution, the vehicle leader also includes a positioning module, which has a Beidou high-precision multi-frequency chip and a miniaturized omnidirectional helical antenna, and supports the intervention of a cors base station.

[0086] The machine vision intelligent analysis module uses machine vision algorithms such as video enhancement algorithm, single-stage detection algorithm, target classification algorithm, etc. to perform multi-target detection on the collected video through deep recognition of large-scale image network, thereby realizing automatic identification of the route, automatic detection of foreign objects and precise measurement of the distance between the vehicle and the retained vehicle or foreign objects, and accurately identifying information such as the route and signal status ahead, including the status of the track and turnout, the type and color of the signal light, the foreign objects ahead and the distance between the vehicle and the foreign objects ahead.

[0087] As a further preferred technical solution, the data service platform shown includes a data information receiving module, a data information analyzing module and a data information sending module.

[0088] The data information receiving module is used to receive and save the real-time video stream collected by the video acquisition module in the vehicle leader, the positioning information collected by the positioning module, the ranging information and foreign object recognition information of the machine vision intelligent analysis module, and the interlocking signal of the interlocking system; the interlocking signal includes information such as the station approach and signal status.

[0089] The data information analysis module is used to compare, verify and correct the interlocking signal of the interlocking system with the real-time video stream collected by the video acquisition module in the leader, the positioning information collected by the positioning module and the ranging information, route identification information and foreign object identification information analyzed by the machine vision intelligent analysis module, that is, combining the data sources such as video ranging, satellite positioning and business logic positioning, and adopting a multi-source fusion ranging and positioning method to achieve accurate ranging and positioning, and determine the route information and signal status ahead of the vehicle.

[0090] The data information sending module is used to send the real-time video stream, distance measurement information and foreign object identification information to the mobile operation terminal.

[0091] The data service platform can automatically associate and match the video acquisition device with the mobile operation terminal, and automatically send the video image in front of the corresponding route to the corresponding mobile operation terminal.

[0092] The data service platform supports multi-channel forwarding of video information, realizes multi-point remote real-time monitoring, and supports recording and query functions of video and other operation processes.

[0093] As a further preferred technical solution, the mobile operation terminal includes a video receiving and display unit, a shunting signaling receiving unit, a route signal receiving unit, a shunting plan receiving and display unit, a satellite positioning unit and an early warning unit.

[0094] The mobile operation terminal receives and displays the real-time video stream through the video receiving and display unit, receives shunting signaling through the shunting signaling receiving unit, receives route information and signal status as well as the distance measurement information through the route signal receiving unit, receives the shunting plan through the shunting plan display unit, and receives the warning information through the early warning unit.

[0095] The mobile operation terminal has functions such as video display, voice intercom, shunting signaling, route signal, shunting plan display and satellite positioning. It can not only display and push the video collection information in front of the operation to the shunting crew, but also meet the functional requirements of the existing horizontal shunting handheld station. At the same time, it provides functions such as shunting plan display and shunting crew safety protection warning. During the advancement process, the shunting master directs the driver to drive the locomotive based on the video and station signals.

[0096] Furthermore, the present invention proposes three different distance measurement methods for the three scenarios of long distance, medium distance and short distance, respectively, so that the vehicle leading system of the present invention has a multi-scenario video distance measurement function, the distance measurement is more accurate, and the problem that a single video distance measurement cannot obtain the distance to the retained vehicle or foreign objects that is too close or too far is solved, making the shunting operation more efficient. The long distance refers to the distance between the own vehicle and the retained vehicle being 100 meters to 200 meters, the medium distance refers to the distance between the own vehicle and the retained vehicle being 2 meters to 100 meters, and the short distance refers to the distance between the own vehicle and the retained vehicle being within 2 meters. The following is a further detailed description with reference to the accompanying drawings:

[0097] As attached Figure 2 As shown, for application scenarios where the distance from the vehicle is relatively far, the present invention proposes a method for measuring the distance between the vehicle and the remaining vehicle based on the track spacing. The measurement method is applied to the distance measurement unit between the vehicle and the remaining vehicle based on the track spacing, and specifically includes the following steps:

[0098] Step 1. The machine vision intelligent analysis module receives and reads the video image captured by the main camera in the video acquisition module;

[0099] Step 2. The track anchor line detection unit in the machine vision intelligent analysis module detects the tracks in the video image, obtains pixel coordinate pairs of all identifiable tracks in the video image, and determines whether the track is a main track or a side track;

[0100] Step 3. If the track in step 2 is determined to be a main track, calculate the distance D between the vehicle and the reserved vehicle based on the width of the main track.

[0101] For the long-distance application scenario, considering that the track width is a priori information that is relatively easy to obtain, the ranging robustness is better in the case of motion, and the factors such as the change of distance and camera attitude angle have little effect on the ranging method based on track width, the present invention proposes a target positioning model based on standard track width and track line as a reference, such as Figure 3 shown.

[0102] Based on the principle that the projection of the track line in the camera is similar to pinhole imaging, according to the pixel width of the track detected in the image, the calculation formula of the distance between the remaining vehicle in front and the vehicle can be derived through geometric knowledge. The actual width of the track L is known. line , the camera focal length f can be obtained through the internal and external parameters of the camera x , front track pixel width L pixel Therefore, the calculation formula for the distance D between the vehicle and the vehicle to be attached is:

[0103]

[0104] Furthermore, the step of determining the main track in step 2 of the method for measuring the distance between the vehicle and the reserved vehicle based on the track spacing is specifically as follows:

[0105] Step 1: Initialize and clear data: The code first resets the index and related data of the found main track and side track.

[0106] Step 2: Find all detected track starting points: Traverse the starting point array and use the classified pixel data to determine the category of each starting point, thereby collecting the starting points of all tracks.

[0107] Step 3: Find possible main tracks: Traverse all starting points and consider each pair of starting points as a possible track. For each pair of starting points, if they are both located at the bottom of the image (with the largest y coordinate), calculate the distance between them. If this distance is close to the set main track distance range, and the center of the pair of tracks is close to the center of the image, it is considered a possible main track pair. Store all track pairs that meet the conditions. Select the most central pair from the possible main tracks.

[0108] Step 4: Determine the main track: Traverse all possible main track pairs, find the pair whose center is closest to the center of the image, and mark it as the main track.

[0109] Step 5: Determine the side track: Continue to traverse all starting points, excluding the pair that is already the main track. For each pair of starting points, if they are both at the bottom of the image, determine whether they are the left track or the right track based on their position (left or right half of the image).

[0110] As attached Figure 4 As shown, for the application scenario of the distance from the self-vehicle, the present invention proposes a method for measuring the distance between the self-vehicle and the remaining vehicle based on the anchor line vanishing point and the remaining vehicle detection frame. The measurement method is applied to the self-vehicle and the remaining vehicle distance measurement unit based on the anchor line vanishing point and the remaining vehicle detection frame, and specifically includes the following steps:

[0111] Step 1. The machine vision intelligent analysis module receives and reads the video image captured by the main camera in the video acquisition module;

[0112] Step 2. The track anchor line detection unit in the machine vision intelligent analysis module detects and analyzes the track anchor line in the video image to obtain the vanishing point coordinates of the track anchor line;

[0113] Step 3. Based on the vanishing point coordinates of the track anchor line in step 2, correct the attitude angle of the main camera to obtain the yaw angle and pitch angle of the main camera;

[0114] Step 4. The distance measurement unit between the ego vehicle and the reserved vehicle in the machine vision intelligent analysis module detects the reserved vehicle in the video image and measures the distance based on the midpoint of the bottom edge of the reserved vehicle detection frame to obtain the distance d between the ego vehicle and the reserved vehicle.

[0115] Step 5. The track detection unit in the machine vision intelligent analysis module detects the track and determines whether the track is a straight track. If the track is a straight track,

[0116] Based on the pitch angle of the main camera in step 3, the distance between the self-vehicle and the reserved vehicle in step 4 is corrected to obtain the corrected distance d1 between the self-vehicle and the reserved vehicle.

[0117] The corrected distance d1 between the ego vehicle and the reserved vehicle is corrected based on the yaw angle of the main camera in step 3 to obtain a second corrected distance D1 between the ego vehicle and the reserved vehicle.

[0118] If the track is not a straight track, the yaw angle of the main camera is set to zero, and the distance between the ego vehicle and the retained vehicle in step 4 is corrected based only on the pitch angle of the main camera in step 3 to obtain the corrected distance d1 between the ego vehicle and the retained vehicle.

[0119] For the medium-distance application scenario, the present invention applies a distance measurement method for retaining the bottom edge position of the vehicle target detection frame based on the coordinates of the anchor line vanishing point to correct the camera posture angle.

[0120] In the field of visual ranging, changes in the camera's attitude angle can affect its accuracy. This is because at different attitude angles, the ratio between an object's actual distance and its projected distance in the camera changes, affecting the accuracy of distance measurements. The inevitable jolting of a locomotive during propulsion can cause the visual sensor's angle relative to the ground to dynamically change, altering the parameters of the camera's geometric model. If the ranging algorithm fails to promptly update these parameter changes, erroneous distance measurements can occur.

[0121] In a 3D world, track anchor lines are parallel. However, in camera images of the track, it can be observed that these lines intersect in the image. The intersection point is called a vanishing point, and the movement of the vanishing point reflects the change in the rotation matrix. Compared with traditional line segment detection and weighted voting methods, the end-to-end anchor line detection method simplifies the computational process and reduces the errors and complexity that may be introduced by intermediate steps.

[0122] The present invention adopts a camera attitude angle estimation method based on the anchor line vanishing point to update the camera external parameters in actual operation, so that the system can better adapt to environmental changes and ensure the stability and reliability of the ranging results.

[0123] In the method for measuring the distance between the vehicle and the reserved vehicle based on the anchor line vanishing point and the reserved vehicle detection frame, the vanishing point coordinates of the track anchor line in step 2 are the intersection points of the track anchor lines detected in the video image, and the track anchor lines are represented as a sequence of N points, i.e., P = {(x0, y0), (x1, y1), ..., (x N―1 ,y N―1 )}, where the y coordinates of the track anchor points are fixed and uniformly sampled along the vertical axis of the video image. The slope m and intercept c of the best-fit anchor line are found using the least squares method, resulting in the anchor line fitting equation y = mx + c. The intersection coordinates (u, v) of the track anchor lines are further calculated:

[0124]

[0125] Among them, m1 and m2 are the slopes of the two track anchor lines, and c1 and c2 are the intercepts of the two track anchor lines.

[0126] When the camera fixed on the locomotive has a pitch angle, the position of the vanishing point of the collected image will be offset laterally; if the on-board camera has a yaw angle, the position of the vanishing point of the road in the collected image will be offset longitudinally, as shown in the following figure. Figure 5 As shown in the figure, where W and H are the width and height of the imaging plane, respectively; β represents half of the camera's horizontal field of view; and γ and θ are the camera's yaw and pitch angles, respectively.

[0127] When γ = 0, the coordinates of the road vanishing point in the imaging plane are V(u0, v0); when γ ≠ 0, the coordinates of the road vanishing point are offset to point V′(u1, v1); the yaw and pitch angle calculation formulas of the vehicle-mounted camera are derived as follows:

[0128]

[0129]

[0130] Where γ is the yaw angle of the main camera, W is the width of the imaging plane, u1 is the horizontal coordinate of the vanishing point of the track anchor line in the presence of pitch and yaw angles, and f x is the equivalent focal length in the x-axis direction of the camera coordinate system, where the camera coordinate system takes the center of the main camera's optical axis as its origin, the right is the positive direction of the x-axis, the downward is the positive direction of the y-axis, and the forward is the positive direction of the z-axis; θ is the pitch angle of the main camera, H is the height of the imaging plane, v1 is the vertical coordinate of the vanishing point of the track in the presence of pitch and yaw angles, and f y is the equivalent focal length in the y-axis direction of the camera coordinate system.

[0131] The distance d between the ego vehicle and the retained vehicle in step 4 of the method for measuring the distance between the ego vehicle and the retained vehicle based on the anchor line vanishing point and the retained vehicle detection frame is,

[0132]

[0133] Among them, μ is the angle formed by the main camera optical axis and the line connecting the center of the bottom edge of the retained car detection frame and the main camera optical center, and H c is the distance between the main camera and the ground.

[0134] The distance measurement model based on the bottom edge position of the retained vehicle target detection frame is shown in the attached figure. Figure 6 As shown, the position of point C corresponds to the center C (u c ,v c ), at this time, according to the pitch angle estimated by the camera attitude angle based on the anchor line vanishing point, the corrected distance d1 between the propulsion vehicle and the vehicle to be connected is,

[0135]

[0136] Among them, H c where μ is the distance between the main camera and the ground; μ is the angle formed by the line connecting the optical axis of the main camera and the center of the bottom edge of the retained vehicle detection frame and the optical center of the main camera; θ is the pitch angle of the main camera.

[0137] As attached Figure 6As shown in Figure 2, the calculation of the distance between the ego vehicle and the reserved vehicle is affected not only by the pitch angle of the camera but also by the yaw angle of the camera. By correcting the calculation formula (6), the distance D1 between the ego vehicle and the reserved vehicle to be connected after secondary correction is obtained as follows:

[0138]

[0139] in, It represents the angle formed by the line connecting the midpoint of the lower edge of the target vehicle detection frame and the optical center of the vehicle-mounted camera and the optical axis in the vertical plane. The calculation formula is:

[0140]

[0141] Among them, u c u is the horizontal coordinate of the center point of the bottom edge of the car detection frame retained in the camera imaging plane, o is the coordinate of the origin of the image coordinate system in the pixel coordinate system.

[0142] However, in actual situations, when driving on a curve, the yaw angle of the on-board camera will change significantly, seriously affecting the ranging accuracy of the ranging model based on the retained vehicle detection frame position. At this time, it is necessary to stop the ranging model from correcting the yaw angle, that is, only consider the change in the camera's pitch angle and make corrections. Whether the yaw angle correction based on the ranging model based on the retained vehicle target detection frame position is set to 0 depends on the position of the horizontal coordinate of the vanishing point in the entire image. The change in the vanishing point determines whether the current road is a straight road or a curve. When the offset of the vanishing point is large, the current road is considered to be a curve, otherwise it is a straight road. The specific judgment basis is as follows:

[0143]

[0144] Where W is the pixel width of the image, u vp is the horizontal coordinate of the road vanishing point; when the conditions shown in formula (9) are met, only the pitch angle needs to be corrected, that is, the correction of the yaw angle based on the distance measurement model of the retained vehicle detection frame position is taken as 0.

[0145] like Figure 7 As shown, for the application scenario at a relatively close distance from the own vehicle, the present invention proposes a method for measuring the distance between the own vehicle coupler and the retained vehicle coupler. The measurement method is applied to a measurement unit based on the distance between the own vehicle coupler and the retained vehicle coupler, and specifically includes the following steps:

[0146] Step 1. The machine vision intelligent analysis module receives and reads the video image captured by the auxiliary camera in the video acquisition module;

[0147] Step 2: The track anchor line detection unit in the machine vision intelligent analysis module detects and analyzes the track anchor line in the video image to obtain the vanishing point coordinates of the track anchor line, and sets the region of interest based on the vanishing point coordinates of the track anchor line.

[0148] Step 3. The distance measurement unit between the own vehicle coupler and the reserved vehicle coupler in the machine vision intelligent analysis module detects the own vehicle coupler and the reserved vehicle coupler in the video image to identify the own vehicle coupler and the reserved vehicle coupler;

[0149] Step 4. The distance measurement unit of the self-vehicle coupler and the reserved vehicle coupler in the machine vision intelligent analysis module tracks the movement of the self-vehicle coupler and the reserved vehicle coupler and generates a coupler tracking ID, determines whether the self-vehicle coupler and the reserved vehicle coupler are located within the region of interest in step 2, and the coupler located within the region of interest is the reserved vehicle coupler, and records the tracking ID of the reserved vehicle coupler;

[0150] Step 5. Based on the retained vehicle coupler in step 4, the vehicle-to-vehicle coupler and the retained vehicle coupler-to-vehicle coupler distance measurement unit in the machine vision intelligent analysis module analyzes and calculates the distance D2 between the vehicle coupler and the retained vehicle coupler.

[0151] For close-range scenes, since the scene in front of the lens is completely filled with retained vehicles, the only available reference object is the coupler. Therefore, the present invention adopts a distance measurement method based on the coupler size.

[0152] The PnP problem involves calculating the specific positions of many 3D and 2D matching points in a perspective projection environment based on known camera internal parameters. PnP technology uses the 2D image coordinates of feature points in the image and the 3D coordinates of those points in the target coordinate system to determine the pose relationship between the camera and target coordinate systems. In the PnP algorithm, if n is 4 and the four feature points are coplanar, the unit orthogonality of the coordinate exchange matrix can be used to calculate a unique analytical solution.

[0153] The projection principle of the coupler on the camera is shown in the attached Figure 8 As shown. Figure 8 There are four coordinate systems:

[0154] World coordinate system O W X W Y W Z W , camera coordinate system O C X C Y C Z C , O ixyImage coordinate system and O puv Pixel coordinate system; A, B, C, and D represent the four corner points of the coupler detection frame, and a, b, c, and d are the projection points of the four corner points on the camera imaging plane.

[0155] As attached Figure 8 As shown, for ΔO c ab and ΔOAB are calculated according to the law of cosines:

[0156]

[0157] Similarly, for other triangles, we can get the following formula:

[0158]

[0159] Where h is the coupler height (AC) of the retained car, and w is the coupler width (CD) of the retained car. The geometric dimension requirements of couplers in my country are almost unchanged. Since the positions of the four points a, b, c, and d on the video image are known, the cosine angles α, β, γ, and θ are also known. The formula (10) and formula (11) can be used to calculate l OA 、l OB 、l OC 、l OD Therefore, the distance l between the auxiliary camera and the coupler of the reserved vehicle is,

[0160]

[0161] Among them, l OB l is the straight-line distance from the upper right corner of the coupler detection frame in the world coordinate system to the origin of the world coordinate system. OC It is the straight-line distance from the lower left corner coordinate point of the coupler detection frame in the world coordinate system to the origin of the world coordinate system.

[0162] Because there is a horizontal distance D0 between the auxiliary camera and the coupler in front of the ego vehicle, this distance needs to be subtracted when calculating the distance D2 between the actual ego vehicle coupler and the target retained vehicle coupler. The distance D2 between the two couplers is:

[0163]

[0164] Among them, h0 is the vertical distance between the auxiliary camera and the vehicle coupler.

[0165] The close-range ranging process includes the following steps: first, the detection model is used to detect the coupler and anchor line in the image; then, since the opposite vehicle coupler always appears near the track vanishing point in the perspective of the auxiliary camera during the push operation, the vanishing point coordinates are calculated based on the anchor line detection results. A region of interest is set based on the vanishing point to determine whether the coupler is located within this area to identify the opposite vehicle coupler; then, the movement of the distant coupler is tracked using the target tracking algorithm, and its position and tracking ID are updated; finally, the actual distance to the opposite vehicle coupler is calculated using a distance measurement method based on coupler size, and the distance measurement result is output.

[0166] Furthermore, the present invention also proposes a method for obtaining a vehicle using the vehicle obtaining system, which specifically includes the following steps:

[0167] Step 1. During the shunting operation, before the pushing operation, the leading device is installed on the front of the vehicle;

[0168] Step 2. During the propulsion operation, the video odometry leader collects a real-time video stream in front of the vehicle through the video acquisition module, and the machine vision intelligent analysis module performs intelligent analysis on the collected real-time video stream to obtain an analysis result;

[0169] Step 3. The video ranging device sends the real-time video stream, the analysis results and the positioning information collected by the positioning module to the data service platform;

[0170] Step 4. The data information receiving module in the data service platform receives and stores the real-time video stream collected by the video acquisition module in the vehicle leader, the positioning information collected by the positioning module, the ranging information and foreign object recognition information collected by the machine vision intelligent analysis module, and the interlocking signal of the interlocking system;

[0171] Step 5. The data information analysis module compares and verifies the interlocking signal of the interlocking system with the real-time video stream collected by the video acquisition module in the vehicle leader, the positioning information collected by the positioning module, and the ranging information and foreign object recognition information analyzed by the machine vision intelligent analysis module, and corrects them to determine the route information and signal status ahead of the vehicle;

[0172] Step 6. The data information sending module sends the real-time video stream, the distance measurement information, and the foreign object identification information to the mobile operation terminal. If the video distance measurement device identifies a foreign object in front of the vehicle, the data service platform sends a foreign object warning message to the mobile operation terminal.

[0173] Step 7. The mobile operation terminal receives and displays the real-time video stream via the video receiving and display unit, receives shunting signaling via the shunting signaling receiving unit, receives route information and signal status as well as the distance measurement information via the route signal receiving unit, receives the shunting plan via the shunting plan display unit, and receives the warning information via the early warning unit;

[0174] Step 8. The shunting team instructs the driver to drive safely according to the information flow received by the mobile operation terminal in step 7.

[0175] Furthermore, the analysis result in step 2 of the vehicle leading method includes distance measurement information and foreign object identification information, and the distance measurement information includes the distance between the own vehicle and the retained vehicle, the distance between the own vehicle coupler and the retained vehicle coupler, and the distance between the own vehicle and the foreign object.

[0176] Furthermore, the interlock signal in step 4 of the vehicle leading method at least includes route information and signal status information.

[0177] The beneficial technical effects achieved by the present invention are as follows: Through the vehicle-leading system based on machine vision and interlocking signals, the present invention achieves real-time capture of road conditions and signal status ahead of the vehicle. Using artificial intelligence technologies such as computer vision, the present invention performs video recognition and analysis, thereby achieving automatic route identification, automatic detection of foreign objects, and precise measurement of the distance between the vehicle and the remaining vehicle or foreign objects. The real-time video stream and analysis structure are transmitted to a data service platform. Combined with interlocking signals such as the route and signal status of the interlocking system, the video stream and analysis results are further verified and confirmed. The video stream, route, and signal status information are transmitted to the shunting team members and driver via a mobile operation terminal, directing the driver to operate the locomotive, thereby achieving automatic and precise control of the vehicle-leading process. While improving operational efficiency, the present invention reduces the labor intensity of shunting operators, enhances the safety of shunting operations, and improves the practicality and safety of the entire vehicle-leading system. The present invention replaces traditional laser radar ranging with video distance measurement, thereby solving the problem of vehicle-leading equipment portability. The present invention proposes different ranging methods for three scenarios: long distance, medium distance and close distance, so that the vehicle leading system of the present invention has a multi-scenario video ranging function, which solves the problem that a single video ranging cannot obtain the distances that are too close or too far to the retained vehicle or foreign objects, making the shunting operation more accurate. In addition, a method of dynamically estimating the camera attitude angle based on the coordinates of the anchor line vanishing point is adopted to improve the accuracy and robustness of video ranging in dynamic scenarios.

[0178] The above embodiments are preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Any person skilled in the art may make slight modifications without departing from the scope of the present invention. In other words, any equivalent modifications made in accordance with the present invention should be covered by the scope of the present invention.

Claims

1. A car guide device, characterized in that: The vehicle tracking device includes a video acquisition module and a machine vision intelligent analysis module; The video acquisition module includes a main camera and an auxiliary camera. When the vehicle guide device is working, the optical center of the main camera is parallel to the ground, and the optical center of the auxiliary camera is set at an angle to the ground, and the angle is adjustable. The machine vision intelligent analysis module includes a unit for measuring the distance between the vehicle and the reserved vehicle based on the vanishing point of the anchor line and the reserved vehicle detection frame, a unit for measuring the distance between the vehicle and the reserved vehicle based on the track spacing, and a unit for measuring the distance between the vehicle coupler and the reserved vehicle coupler; The video acquisition module is connected to the machine vision intelligent analysis module; The distance measurement unit between the vehicle and the remaining vehicle based on the anchor line vanishing point and the remaining vehicle detection frame includes a distance measurement method between the vehicle and the remaining vehicle based on the anchor line vanishing point and the remaining vehicle detection frame. The distance measurement method between the vehicle and the remaining vehicle based on the anchor line vanishing point and the remaining vehicle detection frame includes the following steps: Step 1. The machine vision intelligent analysis module receives and reads the video image captured by the main camera in the video acquisition module; Step 2. The track anchor line detection unit in the machine vision intelligent analysis module detects and analyzes the track anchor line in the video image to obtain the vanishing point coordinates of the track anchor line; Step 3. Based on the vanishing point coordinates of the track anchor line in step 2, correct the attitude angle of the main camera to obtain the yaw angle and pitch angle of the main camera; Step 4. The distance measurement unit between the ego vehicle and the reserved vehicle in the machine vision intelligent analysis module detects the reserved vehicle in the video image and measures the distance based on the midpoint of the bottom edge of the reserved vehicle detection frame to obtain the distance d between the ego vehicle and the reserved vehicle. Step 5. The track detection unit in the machine vision intelligent analysis module detects the track and determines whether the track is a straight track. If the track is a straight track, the distance between the ego vehicle and the retained vehicle in step 4 is corrected based on the pitch angle of the main camera in step 3 to obtain the corrected distance d1 between the ego vehicle and the retained vehicle, and the corrected distance d1 between the ego vehicle and the retained vehicle is corrected based on the yaw angle of the main camera in step 3 to obtain a second-corrected distance D1 between the ego vehicle and the retained vehicle. If the track is a non-straight track, the yaw angle of the main camera is set to zero, and the distance between the ego vehicle and the retained vehicle in step 4 is corrected only based on the pitch angle of the main camera in step 3 to obtain the corrected distance d1 between the ego vehicle and the retained vehicle. The vanishing point coordinates of the track anchor line in step 2 are the intersection points of the track anchor lines detected in the video image. The track anchor line is represented as a sequence of N points, i.e., P = {(x0, y0), (x1, y1), ..., (xN-1, yN-1)}, where the y coordinates of the track anchor line points are fixed and uniformly sampled along the vertical axis of the video image. The least squares method is used to find the slope m and intercept c of the best-fit anchor line, and the fitting equation of the anchor line y = mx + c is obtained. The intersection coordinates (u, v) of the track anchor line are further calculated: in, and is the slope of the two track anchor lines, and is the intercept of the two track anchor lines; The yaw angle and pitch angle of the main camera in step 3 are respectively, and the yaw angle of the main camera is, Among them, γ is the yaw angle of the main camera, W is the width of the imaging plane, is the horizontal coordinate of the vanishing point of the track anchor line when there are pitch angles and yaw angles, is the equivalent focal length along the x-axis of the camera coordinate system, where the origin of the camera coordinate system is the center of the optical axis of the main camera, the rightward direction is the positive direction of the x-axis, the downward direction is the positive direction of the y-axis, and the forward direction is the positive direction of the z-axis; The pitch angle of the main camera is, Among them, θ is the pitch angle of the main camera, H is the height of the imaging plane, is the vertical coordinate of the track vanishing point when there are pitch angles and yaw angles, is the equivalent focal length in the y-axis direction of the camera coordinate system; The distance d between the vehicle and the reserved vehicle in step 4 is, Among them, μ is the angle formed by the main camera optical axis and the line connecting the center of the bottom edge of the retained vehicle detection frame and the main camera optical center. is the distance between the main camera and the ground; The distance d between the self-vehicle and the remaining vehicle after correction in step 5 is, in, The distance between the ego vehicle and the reserved vehicle is corrected based on the pitch angle of the main camera; where μ is the angle between the main camera and the ground; μ is the angle between the main camera's optical axis and the line connecting the center of the bottom edge of the retained vehicle detection frame and the main camera's optical center; θ is the main camera's pitch angle; The distance between the vehicle and the remaining vehicle after the secondary correction for, ,in, The angle formed by the line connecting the midpoint of the bottom edge of the retained vehicle detection frame and the optical center of the main camera and the optical axis in the vertical plane is calculated as follows: in, is the horizontal coordinate of the center point of the bottom edge of the car detection frame retained in the camera imaging plane, is the coordinate of the origin of the image coordinate system in the pixel coordinate system.

2. The vehicle driver according to claim 1, characterized in that: The distance measurement unit for the vehicle and the remaining vehicle based on the track spacing includes a distance measurement method for the vehicle and the remaining vehicle based on the track spacing. The distance measurement method for the vehicle and the remaining vehicle based on the track spacing includes the following steps: Step 1. The machine vision intelligent analysis module receives and reads the video image captured by the main camera in the video acquisition module; Step 2. The track anchor line detection unit in the machine vision intelligent analysis module detects the tracks in the video image, obtains pixel coordinate pairs of all identifiable tracks in the video image, and determines whether the track is a main track or a side track; Step 3. If the track in step 2 is determined to be a main track, calculate the distance D between the vehicle and the reserved vehicle based on the width of the main track.

3. The vehicle driver according to claim 2, characterized in that: The distance D between the vehicle and the reserved vehicle is calculated based on the width of the main track. in, is the true width of the main track, is the equivalent focal length in the x-axis direction of the camera coordinate system, The pixel width of the main track.

4. The vehicle driver according to claim 1, characterized in that: The unit for measuring the distance between the own vehicle coupler and the reserved vehicle coupler includes a method for measuring the distance between the own vehicle coupler and the reserved vehicle coupler. The method for measuring the distance between the own vehicle coupler and the reserved vehicle coupler includes the following steps: Step 1. The machine vision intelligent analysis module receives and reads the video image captured by the auxiliary camera in the video acquisition module; Step 2. The track anchor line detection unit in the machine vision intelligent analysis module detects and analyzes the track anchor line in the video image to obtain the vanishing point coordinates of the track anchor line. Based on the vanishing point coordinates of the track anchor line, a region of interest is defined. Step 3. The own-vehicle coupler-retained-vehicle coupler distance measurement unit in the machine vision intelligent analysis module detects the own-vehicle coupler and the retained-vehicle coupler in the video image to identify the own-vehicle coupler and the retained-vehicle coupler. Step 4. The distance measurement unit of the self-vehicle coupler and the reserved vehicle coupler in the machine vision intelligent analysis module tracks the movement of the self-vehicle coupler and the reserved vehicle coupler and generates a coupler tracking ID, determines whether the self-vehicle coupler and the reserved vehicle coupler are located within the region of interest in step 2, and the coupler located within the region of interest is the reserved vehicle coupler, and records the tracking ID of the reserved vehicle coupler; Step 5. Based on the reserved vehicle coupler in step 4, the distance measurement unit between the vehicle coupler and the reserved vehicle coupler in the machine vision intelligent analysis module analyzes and calculates the distance between the vehicle coupler and the reserved vehicle coupler. .

5. The vehicle driver according to claim 4, characterized in that: The distance between the self-vehicle coupler and the reserved vehicle coupler in step 5 of the method for measuring the distance between the self-vehicle coupler and the reserved vehicle coupler is for, Where, l is the straight-line distance between the auxiliary camera and the coupler of the retained vehicle, h is the height of the coupler of the retained vehicle, and w is the width of the coupler of the retained vehicle. is the horizontal distance between the auxiliary camera and the vehicle coupler, is the vertical distance between the auxiliary camera and the vehicle coupler, It is the straight-line distance from the upper right corner coordinate point of the coupler detection frame in the world coordinate system to the origin of the world coordinate system. It is the straight-line distance from the lower left corner coordinate point of the coupler detection frame in the world coordinate system to the origin of the world coordinate system.

6. An intelligent vehicle leading system based on machine vision and interlocking signals, characterized by: It includes the vehicle lead device, data service platform and mobile operation terminal as described in claim 1, the vehicle lead device is connected to the data service platform through wireless communication, and the data service platform is connected to the mobile operation terminal through wireless communication.

7. The intelligent vehicle control system according to claim 6, characterized in that: The vehicle leader also includes a positioning module, and the data service platform includes a data information receiving module, a data information analysis module, and a data information sending module. The data information receiving module is used to receive and store the real-time video stream collected by the video acquisition module in the vehicle leader, the positioning information collected by the positioning module, the ranging information and foreign object recognition information collected by the machine vision intelligent analysis module, and the interlocking signal of the interlocking system; The data information analysis module is used to compare and verify the interlocking signal of the interlocking system with the real-time video stream collected by the video acquisition module in the vehicle leader, the positioning information collected by the positioning module, and the distance measurement information and foreign object recognition information analyzed by the machine vision intelligent analysis module, and to determine the route information and signal status of the vehicle ahead; The data information sending module is used to send the real-time video stream, distance measurement information and foreign object identification information to the mobile operation terminal.

8. The intelligent vehicle control system according to claim 7, characterized in that: The mobile operation terminal includes a video receiving and display unit, a shunting signal receiving unit, a route signal receiving unit, a shunting plan display unit, a satellite positioning unit and an early warning unit.

9. A method for collecting a vehicle based on the intelligent vehicle collecting system according to claim 8, characterized in that: The method comprises the following steps: step 1. during a shunting operation, before advancing the operation, installing the leading device on the front of the vehicle; Step 2. During the pushing operation, the vehicle leader collects a real-time video stream in front of the vehicle through the video acquisition module, and the machine vision intelligent analysis module performs intelligent analysis on the collected real-time video stream to obtain an analysis result; Step 3. The vehicle leader sends the real-time video stream, the analysis results, and the positioning information collected by the positioning module to the data service platform; Step 4. The data information receiving module in the data service platform receives and stores the real-time video stream collected by the video acquisition module in the vehicle leader, the positioning information collected by the positioning module, the ranging information and foreign object recognition information collected by the machine vision intelligent analysis module, and the interlocking signal of the interlocking system; Step 5. The data information analysis module compares and verifies the interlocking signal of the interlocking system with the real-time video stream collected by the video acquisition module in the vehicle leader, the positioning information collected by the positioning module, and the ranging information and foreign object recognition information analyzed by the machine vision intelligent analysis module, and corrects them to determine the route information and signal status ahead of the vehicle; Step 6. The data information sending module sends the real-time video stream, the distance measurement information, and the foreign object identification information to the mobile operation terminal. If the vehicle leader identifies a foreign object in front of the vehicle, the data service platform sends a foreign object warning message to the mobile operation terminal. Step 7. The mobile operation terminal receives and displays the real-time video stream via the video receiving and display unit, receives shunting signaling via the shunting signaling receiving unit, receives route information and signal status as well as the distance measurement information via the route signal receiving unit, receives the shunting plan via the shunting plan display unit, and receives the warning information via the early warning unit; Step 8. The shunting team instructs the driver to drive safely according to the information flow received by the mobile operation terminal in step 7.

10. The vehicle leading method according to claim 9, characterized in that: The analysis result in step 2 includes distance measurement information and foreign object identification information. The distance measurement information includes the distance between the own vehicle and the retained vehicle, the distance between the own vehicle coupler and the retained vehicle coupler, and the distance between the own vehicle and the foreign object.

Images