Transportation equipment cluster path tracking control method and device, equipment and storage medium

Abstract

This application relates to a path tracking control method, apparatus, device, and storage medium for a transportation equipment cluster. The path tracking control method for the transportation equipment cluster includes: calculating an initial path deviation based on road images collected by a lead transportation equipment in the cluster; optimizing the initial path deviation to obtain visual navigation parameters; determining the steering angle of the lead transportation equipment based on the visual navigation parameters, and mapping the steering angle of the lead transportation equipment to the steering angles of the other transportation equipment in the cluster; and controlling each transportation equipment in the cluster to perform a steering action according to the corresponding steering angle. This application eliminates the need to install visual sensors on other transportation equipment besides the lead transportation equipment, reducing hardware deployment costs. Furthermore, by using the steering angle of the lead transportation equipment to map the steering angles of other transportation equipment, it fundamentally avoids angle command conflicts caused by sensor error accumulation and differences in lighting conditions in multi-machine vision systems.

Description

Technical Field

[0001] This application relates to the field of railway construction beam transportation technology, specifically to a method, device, equipment, and storage medium for path tracking and control of a transportation equipment cluster. Background Technology

[0002] With the rapid development of large-scale infrastructure construction, scenarios such as railway beam transportation, bridge erection, and heavy component transportation have placed increasingly urgent demands on the automation and intelligence levels of heavy-duty transportation equipment. In these applications, multiple heavy-duty transport vehicles often need to form a cluster for collaborative operation to accurately handle oversized and overweight goods. To improve transportation efficiency, reduce the difficulty of manual operation, and ensure operational safety, cluster control of transport equipment based on automatic guidance technology has become a major trend in the industry. Especially under complex working conditions, achieving high-precision path tracking of clustered vehicles and kinematic synchronization between multiple vehicles is a key technical requirement for ensuring the successful completion of heavy-duty transportation tasks.

[0003] In related technologies, a common approach for multi-vehicle cluster collaborative control is the distributed perception control mode. In this mode, each vehicle in the cluster is independently equipped with a vision sensor or positioning module, calculating its own path deviation and performing steering control. However, this approach requires deploying high-precision sensors on each device, increasing system costs. Furthermore, the accumulation of sensor errors across multiple devices and differences in lighting conditions can easily lead to conflicts in the angle commands generated by each vehicle, making it difficult to guarantee the geometric consistency of the cluster's movement.

[0004] Therefore, it is necessary to design a new path tracking control method for transportation equipment clusters to overcome the above problems. Summary of the Invention

[0005] This application provides a method, apparatus, device, and storage medium for path tracking control of a transportation equipment cluster, which can solve the technical problems of high cost and difficulty in ensuring the geometric consistency of cluster movement in related technologies.

[0006] In a first aspect, embodiments of this application provide a method for path tracking and control of a transportation equipment cluster, the method comprising: The initial path deviation is calculated based on the road images collected by the navigation transport equipment in the cluster. The initial path deviation is optimized to obtain visual navigation parameters; The steering angle of the lead transport equipment is determined based on visual navigation parameters, and the steering angle of the lead transport equipment is mapped to the steering angle of the other transport equipment in the cluster. Each transport device in the control cluster performs a steering action according to its corresponding steering angle.

[0007] In conjunction with the first aspect, in one implementation, the calculation of the initial path deviation based on road images collected by the navigation transport equipment in the cluster includes: Extract candidate regions for guide lines from road images captured by navigation transport equipment; The slope and intercept of the guide line in the image coordinate system are calculated from the candidate regions of the guide line; Based on the camera's intrinsic parameters and the camera's installation geometry on the navigation transport equipment, the slope and intercept are converted into the initial path deviation of the actual driving path. The initial path deviation includes angular deviation and lateral displacement deviation.

[0008] In conjunction with the first aspect, in one implementation, extracting candidate regions for guide lines from road images acquired by the navigation transport equipment includes: A trained visual segmentation model is used to perform semantic segmentation on road images collected by navigation and transportation equipment to obtain preliminary segmentation regions. The initial segmented region is used as the center to expand outwards, and the set pixel values ​​are used as candidate regions for guide lines.

[0009] In conjunction with the first aspect, in one implementation, calculating the slope and intercept of the guide line in the image coordinate system from the candidate region of the guide line includes: The skeleton of the guide line candidate region is extracted and edge points are sampled to obtain the central skeleton and edge lines of the guide line. The guide line's central skeleton and edge lines are solved using a straight line fitting algorithm to obtain the guide line's slope and intercept in the image coordinate system.

[0010] In conjunction with the first aspect, in one implementation, the step of extracting the skeleton and sampling edge points of the candidate region for the guide line to obtain the central skeleton and edge lines of the guide line includes: Calculate the average pixel value within the candidate region of the guide line and the average pixel value within the background region; The dynamic segmentation threshold is calculated using the average pixel values ​​within the candidate region of the guide line and the average pixel values ​​within the background region. The candidate regions for the guide lines are binarized using a dynamic segmentation threshold to obtain the binarized regions. Extract the central skeleton of the guide line within the binarized region.

[0011] In conjunction with the first aspect, in one implementation, the optimization of the initial path deviation to obtain visual navigation parameters includes: Maintain a sliding window of length N, calculate the effective fitted rotation angle for each of the first N frames based on the initial path deviation of each frame, and calculate the moving average rotation angle based on the effective fitted rotation angle of the first N frames, where N is a positive integer greater than 1. Calculate the fitted rotation angle of the current frame based on the initial path deviation of the current frame, and calculate the absolute value of the deviation between the fitted rotation angle of the current frame and the moving average rotation angle of the previous N frames. If the absolute value of the deviation meets the preset conditions, it is determined that the current ground features are clear and the extraction results are continuous. The current frame fitted angle is used as the original measurement value input to the Kalman filter, and the smoothed visual navigation parameters are output. If the absolute value of the deviation does not meet the preset conditions, the current frame fitting value is determined to be abnormal. The moving average rotation angle is used as the original measurement value input to the Kalman filter, and the smoothed visual navigation parameters are output.

[0012] In conjunction with the first aspect, in one implementation, determining the steering angle of the lead transport device based on visual navigation parameters and mapping the steering angle of the lead transport device to the steering angles of the remaining transport devices in the cluster includes: Determine the steering angle of the pilot transport equipment based on visual navigation parameters; The instantaneous turning center is determined based on the turning angle of the pilot transport equipment; Based on extended Ackerman geometric constraints, the instantaneous turning center of the lead transport equipment is mapped to the entire cluster, and the turning angles of the other transport equipment in the cluster are calculated according to the coordinate positions of the other transport equipment relative to the lead transport equipment.

[0013] Secondly, embodiments of this application provide a path tracking control device for a transportation equipment cluster, the path tracking control device for the transportation equipment cluster comprising: The deviation calculation module is used to calculate the initial path deviation based on the road images collected by the navigation transport equipment in the cluster. The optimization module is used to optimize the initial path deviation and obtain visual navigation parameters; The calculation module is used to determine the turning angle of the lead transport equipment based on visual navigation parameters and map the turning angle of the lead transport equipment to the turning angle of the other transport equipment in the cluster. The control module is used to control each transport device in the cluster to perform steering actions according to the corresponding steering angle.

[0014] Thirdly, embodiments of this application provide a path tracking control device for a transportation equipment cluster. The path tracking control device for the transportation equipment cluster includes a processor, a memory, and a path tracking control program for the transportation equipment cluster stored in the memory and executable by the processor. When the path tracking control program for the transportation equipment cluster is executed by the processor, it implements the steps of the above-described path tracking control method for the transportation equipment cluster.

[0015] Fourthly, embodiments of this application provide a computer-readable storage medium storing a path tracking control program for a transportation equipment cluster, wherein when the path tracking control program for the transportation equipment cluster is executed by a processor, it implements the steps of the above-described path tracking control method for the transportation equipment cluster.

[0016] The beneficial effects of the technical solutions provided in this application include: The initial path deviation of the lead transport device can be calculated by collecting road images from the lead transport device in the cluster. Optimization of the initial path deviation yields visual navigation parameters. Based on these parameters, the steering angle of the lead transport device can be determined and mapped to the steering angles of the other transport devices in the cluster. This embodiment eliminates the need to install visual sensors on other transport devices besides the lead device, reducing hardware deployment costs. Furthermore, the steering angle of the lead transport device can be used to map the steering angles of other transport devices, fundamentally avoiding angle command conflicts caused by sensor error accumulation and lighting environment differences in multi-machine vision systems. This ensures the uniqueness and global stability of the cluster correction commands, solving the technical problems of high cost and difficulty in guaranteeing the geometric consistency of cluster movement in related technologies. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating an embodiment of the transportation equipment cluster path tracking control method of this application; Figure 2 This is a schematic diagram of path extraction provided in the embodiments of this application; Figure 3 This is a partial flowchart of the transportation equipment cluster path tracking and control method of this application; Figure 4 The following is a logic diagram for judging angle confidence based on moving average verification provided in the embodiments of this application; Figure 5 This is a diagram of a collaborative control architecture for a transportation equipment cluster provided in an embodiment of this application. Figure 6 A schematic diagram of the structure for deploying cameras in a navigation transport device provided in this application embodiment; Figure 7 A diagram showing the relative positions of various transportation devices provided in the embodiments of this application; Figure 8 This is a schematic diagram of the hardware structure of the path tracking control device for the transportation equipment cluster involved in the embodiments of this application. Detailed Implementation

[0018] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0019] In related technologies, traditional visual line-following techniques are mainly based on computer vision image processing methods (such as Canny, Sobel edge detection, or color segmentation). These methods have high positioning accuracy in ideal environments, but they are extremely sensitive to changes in lighting, shadow interference, and complex ground textures, and have weak generalization ability, making them difficult to adapt to complex open-air operation scenarios such as railway construction.

[0020] In recent years, deep learning technology has demonstrated significant robustness advantages in visual recognition. However, when applied to heavy-duty transportation equipment such as railway beam transport vehicles and self-propelled hydraulic flatbed trucks, the following problems still urgently need to be addressed: 1. The challenge of accuracy compensation for large inertial systems: Heavy-duty transport equipment has enormous physical inertia and dynamic response lag. Existing deep learning-based line-following methods often focus on image feature extraction, but they still lack absolute accuracy in edge localization. Small visual recognition deviations can accumulate and lead to significant path deviations during long-distance travel of heavy-duty equipment.

[0021] 2. Robustness Requirements for Cluster Collaboration: In the cluster operation mode of self-propelled hydraulic flatbed trucks, the obstruction between multiple devices, uneven lighting, and complex on-site environments (such as road surface water stains and construction dust) place extremely high demands on the continuity and stability of the navigation system. Existing algorithms are prone to false or missed detections of guide lines when dealing with such dynamic and complex backgrounds, leading to interruptions in cluster collaborative operations or safety hazards.

[0022] 3. Existing conventional visual line-following technologies mainly focus on the travel correction design of individual equipment, lacking consideration for the spatial geometric constraints when multiple vehicles are operating in a cluster. When modular vehicles perform clustered collaborative transportation, if each vehicle relies solely on an independent single-vehicle correction algorithm, differences in visual perception and uneven power response can easily lead to cumulative pose deviations in each workshop, thereby disrupting the overall geometric topology of the cluster and failing to meet the stringent requirements of synchronization and coordination for heavy-duty component transportation.

[0023] This application provides a method, apparatus, device, and storage medium for path tracking and control of a transportation equipment cluster, which can overcome the limitations of existing visual line-following technology in the application of heavy-duty transportation equipment.

[0024] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0025] In a first aspect, embodiments of this application provide a method for path tracking and control of a transportation equipment cluster.

[0026] In one embodiment, reference is made to Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the transportation equipment cluster path tracking control method of this application. Figure 1 As shown, the path tracking control method for transportation equipment clusters includes: S100: The initial path deviation is calculated based on the road images collected by the navigation transportation equipment in the cluster.

[0027] S200: Optimize the initial path deviation to obtain visual navigation parameters.

[0028] S300: Determines the steering angle of the lead transport equipment based on visual navigation parameters, and maps the steering angle of the lead transport equipment to the steering angle of the other transport equipment in the cluster.

[0029] S400: Controls each transport device in the cluster to perform a steering action according to the corresponding steering angle.

[0030] It should be understood that in this embodiment, the transportation equipment cluster contains multiple transportation devices. In this embodiment, only the front end of the lead transportation device needs to be equipped with a vision sensor; the other transportation devices do not require vision sensors. The road image acquired by the lead transportation device can not only calculate the steering angle of the lead transportation device but also map the steering angle of the lead transportation device to obtain the steering angles of the other transportation devices. The vision sensor in this embodiment uses a resolution of [resolution missing]. An industrial camera is mounted above the front bogie of the pilot transport equipment, with its optical axis perpendicular to the ground (see...). Figure 6 (As shown).

[0031] This embodiment uses road images collected by the lead transport device in the cluster to calculate the initial path deviation of the lead transport device. Optimization of the initial path deviation yields visual navigation parameters. Based on these parameters, the steering angle of the lead transport device can be determined and mapped to the steering angles of the other transport devices in the cluster. This embodiment eliminates the need to install visual sensors on other transport devices besides the lead device, reducing hardware deployment costs. Furthermore, the steering angle of the lead transport device can be used to map the steering angles of other transport devices, fundamentally avoiding angle command conflicts caused by sensor error accumulation and lighting environment differences in multi-machine vision systems. This ensures the uniqueness and global stability of the cluster correction commands, solving the technical problems of high cost and difficulty in guaranteeing the geometric consistency of cluster movement in related technologies.

[0032] Furthermore, in one embodiment, the initial path deviation calculated based on road images collected by the navigation transport equipment in the cluster may include: S101: Extract candidate regions for guide lines from road images collected by the navigation transport equipment.

[0033] S102: Calculate the slope and intercept of the guide line in the image coordinate system from the candidate region of the guide line.

[0034] S103: Based on the camera's intrinsic parameters and the camera's installation geometry on the navigation transport equipment, the slope and intercept are converted into the initial path deviation of the actual driving path. The initial path deviation includes angular deviation and lateral displacement deviation. In step S103, physical quantity conversion is performed, that is, based on the camera's intrinsic parameters and installation geometry, the parameters in the image coordinate system are converted into the angular deviation and lateral displacement deviation of the actual driving path.

[0035] Preferably, in the above embodiments, the step of extracting the guide line candidate region from the road image collected by the navigation transportation equipment includes: using a trained visual segmentation model to perform semantic segmentation on the road image collected by the navigation transportation equipment to obtain a preliminary segmentation region; and then expanding outward from the preliminary segmentation region by a set pixel value as the guide line candidate region.

[0036] In this embodiment, during real-time road inspection, a trained visual segmentation model is used to perform semantic segmentation on the real-time acquired road image, extracting a highly robust preliminary segmentation region from the complex background. This preliminary segmentation region is then expanded outward by 10 pixels (or other values) as a guide line candidate region, which is also a region of interest (ROI).

[0037] Furthermore, before real-time road observation, an automated annotation and visual segmentation model is trained. Specifically, a large model (such as the Segment Anything Model) is used to perform initial segmentation and annotation on the original road image to obtain a guide line mask (semi-automatic annotation). Then, the Zhang-Suen thinning algorithm is used to process the guide line mask in three iterations to extract the guide line skeleton with a width of one pixel. The least squares method is then used for line fitting to obtain high-precision initial corner labels (centerline skeleton extraction). Next, a convolutional neural network integrating a channel attention mechanism and a Feature Pyramid Network (FPN) is built. The model is trained using the image as input and the guide line segmentation map and corner vectors as training targets to obtain a trained visual segmentation model (model construction). In other words, a large model (such as the Segment Anything Model) is used in advance to automatically segment and annotate the collected road images to generate a road observation guide line mask dataset. Based on this dataset, a deep learning model integrating a channel attention mechanism and an FPN structure is trained to obtain a trained visual segmentation model.

[0038] Further, in step S102, calculating the slope and intercept of the guide line in the image coordinate system from the candidate region of the guide line may include: extracting the skeleton and sampling the edge points of the candidate region of the guide line to obtain the central skeleton and edge line of the guide line. Figure 2 (As shown in the image). Then, a straight line fitting algorithm is used to solve the central skeleton and edge lines of the guide line to obtain the slope and intercept of the guide line in the image coordinate system (this step is sub-pixel straight line fitting). In this embodiment, the straight line fitting algorithm used is, for example, the least squares method or the RANSAC algorithm.

[0039] See Figure 3 As shown, preferably, in the above embodiment, the step of extracting the skeleton and sampling the edge points of the candidate region of the guide line to obtain the central skeleton and edge line of the guide line may include: S1021: Calculate the average pixel value within the candidate region of the guide line and the average pixel value within the background region.

[0040] S1022: Calculate the dynamic segmentation threshold using the average pixel value within the candidate region of the guide line and the average pixel value within the background region.

[0041] S1023: Use dynamic segmentation threshold to binarize the candidate region of the guide line to obtain the binarized region.

[0042] S1024: Extract the central skeleton of the guide line within the binarized region.

[0043] In this embodiment, the pixel mean within the candidate region of the guide line in the road image is calculated. and the average pixel value of the background area Using the formula Get dynamic segmentation threshold The candidate regions for the guide lines are binarized; then the process is repeated in the binarized regions. The algorithm is refined to extract the central skeleton of the guide line. This embodiment calculates the dynamic threshold by calculating the pixel mean of the "guide line candidate area" and the "background area". Whether it is a bright ground under strong sunlight or a dim scene in tunnel construction, the algorithm can automatically adjust the "black and white" boundary, effectively solving the pain point of "light sensitivity".

[0044] Furthermore, in one embodiment, optimizing the initial path deviation to obtain visual navigation parameters may include inputting the initial path deviation as an observation variable into a Kalman filter and outputting smoothed visual navigation parameters. This embodiment introduces a Kalman filter algorithm to address mechanical vibrations and visual noise during the operation of heavy-duty transport equipment. The extracted angular deviation and lateral displacement deviation are used as observation variables to predict and update the line-following status in real time, outputting smoothed visual navigation parameters and eliminating the interference of high-frequency vibrations from heavy-duty equipment on the control system.

[0045] Furthermore, in some optional embodiments, the optimization of the initial path deviation to obtain visual navigation parameters may further include: S201: Maintain a sliding window of length N, calculate the effective fitted rotation angle of each of the first N frames based on the initial path deviation of each frame, and calculate the moving average rotation angle based on the effective fitted rotation angle of the first N frames, where N is a positive integer greater than 1.

[0046] S202: Calculate the fitted rotation angle of the current frame based on the initial path deviation of the current frame, and calculate the absolute value of the deviation between the fitted rotation angle of the current frame and the moving average rotation angle of the previous N frames.

[0047] S203: If the absolute value of the deviation meets the preset conditions, it is determined that the current ground features are clear and the extraction results are continuous. The current frame fitted angle is used as the original measurement value input to the Kalman filter, and the smoothed visual navigation parameters are output. If the absolute value of the deviation does not meet the preset conditions, it is determined that the current frame fitted value is abnormal. The moving average angle is used as the original measurement value input to the Kalman filter, and the smoothed visual navigation parameters are output.

[0048] See Figure 4 As shown, in this embodiment, a length of [length missing] is maintained. (This embodiment takes) The sliding window records the previous The frame is identified as having a valid fitted rotation angle, and its moving average rotation angle is calculated. The effective fitted rotation angle for each frame is calculated based on the initial path deviation, which includes angular deviation and lateral displacement deviation.

[0049] Geometric fit rotation angle for the current frame Perform the following logical judgment: Consistency determination: Calculate the fitted rotation angle for the current frame. Compared to the past Average frame rotation absolute value of deviation Among them, the fitted rotation angle of the current frame. Calculated based on the current initial path deviation.

[0050] Normal operating condition handling: If If the current ground features are clear and the algorithm extraction results are continuous, the geometric fitting angle of the current frame is used. The original measurements are input into subsequent Kalman filters and used to update the sliding window.

[0051] Abnormal operating condition compensation: If The system determines that the current environment contains abrupt interference caused by strong light reflection, broken guide wires, or severe contamination. In this case, the system automatically removes outlier values ​​from the current frame and forcibly switches to a moving average rotation angle. As a compensation output, to maintain the continuity of control commands and ensure that the overloaded cluster does not misdirect.

[0052] Finally, the determined rotation angle value and lateral displacement deviation are input into a Kalman filter to filter out visual jitter data caused by engine vibration of the beam transport vehicle and road surface undulation.

[0053] In order to prevent the fitting algorithm from failing when the guide line is broken or severely contaminated, this embodiment synchronously refers to the global features output by the deep learning model; when the residual of the straight line fitting exceeds the preset threshold, anomaly handling is triggered or the alternative orientation predicted by the model is adopted.

[0054] Furthermore, in one embodiment, determining the steering angle of the lead transport device based on visual navigation parameters and mapping the steering angle of the lead transport device to the steering angles of the remaining transport devices in the cluster may include: S301: Determine the turning angle of the pilot transport equipment based on visual navigation parameters.

[0055] S302: Determine the instantaneous turning center based on the turning angle of the pilot transport equipment.

[0056] S303: Based on extended Ackerman geometric constraints, the instantaneous turning center of the lead transport equipment is mapped to the entire cluster, and the turning angle of the remaining transport equipment in the cluster is calculated according to the coordinate position of the remaining transport equipment relative to the lead transport equipment.

[0057] This embodiment models a multi-transportation equipment cluster as a unified kinematic constraint system. The lead vehicle determines its trajectory based on filtered visual navigation parameters. The system maps the lead vehicle's steering center to the entire cluster according to the extended Ackerman angle principle. Based on the coordinate positions of the other transport equipment relative to the lead transport equipment, the theoretical deflection angle of each follower vehicle (i.e., the other transport equipment) relative to the unified steering center is calculated. A hydraulic steering system ensures geometric consistency of the multiple heavy-duty equipment on their spatial trajectories, guaranteeing synchronized line following and coordinated movement of the cluster under heavy-load conditions. This embodiment obtains the deviation distance (i.e., the pixel value within the camera when the lateral displacement deviates; this offset distance is the actual physical quantity converted from the camera's measurement) through visual line following calculation and utilizes a PID control algorithm to control the lead transport equipment. Required steering angle (Right now Figure 7 In When the vehicle body is biased to the left of the center line, the right steering angle is output; conversely, the left steering angle is output.

[0058] See Figure 5 As shown in the above embodiments, once the navigation transportation equipment... When the turning command is issued, the cluster system responds according to the instructions of each slave transport device ( Relative to the coordinate position of the pilot transport equipment, and based on extended Ackerman geometric constraints, calculate the independent deflection angles of all wheel sets within the entire cluster. The angle mapping process based on the extended Ackerman principle is as follows: Parameter definition: : No. The center coordinates of the first wheel set in the upper left corner of the transport equipment.

[0059] : Wheelset longitudinal spacing; : Lateral spacing of the wheelset.

[0060] Instantaneous turning center arrive The horizontal distance between points.

[0061] (1) Pilot transport equipment ( Wheelset rotation angle calculation: First, calculate the instantaneous distance to the center of rotation: .

[0062] for No. Wheelchair group: Left wheel corner : .

[0063] Right wheel angle : .

[0064] (2) From the transportation equipment ( Wheelset rotation angle calculation: Calculate the steering center arrive Vertical axis horizontal distance : .

[0065] for No. Wheelchair group: Left wheel corner : .

[0066] Right wheel angle : .

[0067] And so on, through the coordinates of each transportation device The system dynamically calculates the turning angle of all wheel sets in the cluster in real time. This scheme ensures that every wheel set of all transport equipment in the cluster points to the same instantaneous turning center. This eliminates the lateral slip stress during heavy-load transportation from a geometric and mechanical perspective.

[0068] The technical solution provided in this application has the following beneficial effects: (1) Compared with traditional manual annotation: The automatic annotation tool for large models realizes the initial generation of the traverse segmentation map, and only manual verification is required, which improves the annotation efficiency by more than 80%.

[0069] (2) Innovation in corner calculation: The corners of the wires are extracted from the segmentation map skeleton, which greatly improves the robustness against breakage and occlusion compared with traditional edge detection methods.

[0070] (3) Reliability of the two-stage verification mechanism: The sensitivity of the threshold to illumination changes in traditional methods is solved by adaptive ROI region verification, and the centerline orientation is compared with the model angle to ensure robust output results while maintaining high accuracy. (4) Extended Ackerman principle: Only a single visual perception of the lead vehicle is needed to accurately map the line-following trajectory into a unified steering constraint for the entire cluster. This achieves a technological leap from single-machine correction to multi-vehicle collaborative steering control, significantly improving the trajectory synchronization and kinematic consistency of heavy-duty clusters on complex paths.

[0071] This embodiment can solve the following technical problems: 1. Addressing the issue of insufficient edge localization accuracy in heavy-duty device navigation using deep learning: To address the stringent accuracy requirements for the driving trajectory of large-sized heavy-duty equipment such as beam transport vehicles and hydraulic flatbed trucks, this application optimizes the visual recognition algorithm logic to achieve sub-pixel-level edge positioning in complex construction environments, thereby reducing the deviation of the driving trajectory of heavy-duty vehicles caused by recognition errors.

[0072] 2. Improve the robustness of the algorithm in the complex and dynamic environment of railway construction: This addresses the issues of false positives and false negatives that traditional vision algorithms are prone to under conditions of strong light, low light, occlusion, and ground texture interference, ensuring the continuous and stable operation of cluster devices in extreme field operating environments.

[0073] 3. Solve the challenges of kinematic synchronization and trajectory tracking for heavy-duty transport clusters under single-point guidance mode: To address the characteristics of heavy-duty transport equipment, such as high physical inertia and delayed dynamic response, this application maps the visual pose of the lead vehicle to a unified instantaneous steering center constraint for the entire cluster in real time. This mechanism not only overcomes the tracking distortion caused by the slave equipment's inability to directly perceive the guide line, but also offsets the control time delay in multi-vehicle linkage through geometric hard constraints, ensuring that the entire cluster maintains strict topological consistency during dynamic line following.

[0074] 4. Construct a distributed control architecture of "single-point perception - global mapping" to improve the reliability and economy of cluster operations.

[0075] This application fundamentally avoids angle command conflicts caused by sensor error accumulation and lighting environment differences in multi-machine vision systems by centralizing the line-following perception module in the navigation device and using spatial pose algorithms to calculate the cooperative rotation angle of each slave device. This architecture reduces hardware deployment costs while ensuring the uniqueness and global stability of cluster correction commands, and solves the non-cooperative force problem that easily occurs in heavy-load clusters under complex curvature paths.

[0076] Secondly, embodiments of this application also provide a path tracking and control device for a transportation equipment cluster.

[0077] In one embodiment, the path tracking control device for a transportation equipment cluster includes: a deviation calculation module, which calculates an initial path deviation based on road images collected by the lead transportation equipment in the cluster; an optimization module, which optimizes the initial path deviation to obtain visual navigation parameters; a calculation module, which determines the steering angle of the lead transportation equipment based on the visual navigation parameters and maps the steering angle of the lead transportation equipment to the steering angles of the other transportation equipment in the cluster; and a control module, which controls each transportation equipment in the cluster to perform a steering action according to the corresponding steering angle.

[0078] Further, in one embodiment, the deviation calculation module includes: an extraction module for extracting candidate regions of guide lines from road images acquired by the navigation transport equipment; a calculation module for calculating the slope and intercept of the guide line in the image coordinate system from the candidate regions of the guide line; and a conversion module for converting the slope and intercept into the initial path deviation of the actual driving path based on the camera's intrinsic parameters and the camera's installation geometry on the navigation transport equipment, wherein the initial path deviation includes angular deviation and lateral displacement deviation.

[0079] Furthermore, in one embodiment, the extraction module is used to perform semantic segmentation on the road image collected by the navigation transportation equipment using a trained visual segmentation model to obtain a preliminary segmentation region; and to expand outward from the preliminary segmentation region by a set pixel value as a candidate region for guide lines.

[0080] Furthermore, in one embodiment, the calculation module is used to extract the skeleton and sample the edge points of the candidate region of the guide line to obtain the central skeleton and edge line of the guide line; and to calculate the central skeleton and edge line of the guide line using a straight line fitting algorithm to obtain the slope and intercept of the guide line in the image coordinate system.

[0081] Furthermore, in one embodiment, the solution module is also used to calculate the average pixel value within the candidate region of the guide line and the average pixel value within the background region; calculate a dynamic segmentation threshold using the average pixel value within the candidate region of the guide line and the average pixel value within the background region; perform binarization processing on the candidate region of the guide line using the dynamic segmentation threshold to obtain a binarized region; and extract the central skeleton of the guide line within the binarized region.

[0082] Further, in one embodiment, the optimization module is used to maintain a sliding window of length N, calculate the effective fitted angle of each of the previous N frames based on the initial path deviation of each frame, and calculate the moving average angle based on the effective fitted angle of the previous N frames, where N is a positive integer greater than 1; calculate the fitted angle of the current frame based on the initial path deviation of the current frame, and calculate the absolute value of the deviation between the fitted angle of the current frame and the moving average angle of the previous N frames; if the absolute value of the deviation meets a preset condition, it is determined that the current ground features are clear and the extraction results are continuous, the fitted angle of the current frame is used as the original measurement value input to the Kalman filter, and the smoothed visual navigation parameters are output; if the absolute value of the deviation does not meet the preset condition, it is determined that the fitted value of the current frame is abnormal, the moving average angle is used as the original measurement value input to the Kalman filter, and the smoothed visual navigation parameters are output.

[0083] Furthermore, in one embodiment, the calculation module is used to determine the steering angle of the lead transport device based on visual navigation parameters; determine the instantaneous steering center based on the steering angle of the lead transport device; map the instantaneous steering center of the lead transport device to the entire cluster based on extended Ackerman geometric constraints; and calculate the steering angles of the remaining transport devices in the cluster based on the coordinate positions of the remaining transport devices relative to the lead transport device.

[0084] The functions of each module in the path tracking control device of the above-mentioned transportation equipment cluster correspond to the steps in the above-mentioned path tracking control method embodiment of the transportation equipment cluster, and their functions and implementation processes will not be described in detail here.

[0085] Thirdly, embodiments of this application provide a path tracking control device for a transportation equipment cluster. The path tracking control device for the transportation equipment cluster can be a personal computer (PC), a laptop computer, a server, or other device with data processing capabilities.

[0086] Reference Figure 8 , Figure 8 This is a schematic diagram of the hardware structure of the path tracking control device for a transportation equipment cluster involved in the embodiments of this application. In this embodiment, the path tracking control device for the transportation equipment cluster may include a processor, a memory, a communication interface, and a communication bus.

[0087] The communication bus can be of any type and is used to interconnect the processor, memory, and communication interface.

[0088] The communication interface includes input / output (I / O) interfaces, physical interfaces, and logical interfaces used for interconnecting devices within the path tracking control equipment of the transportation equipment cluster, as well as interfaces used for interconnecting the path tracking control equipment of the transportation equipment cluster with other devices (such as other computing devices or user equipment). Physical interfaces can be Ethernet interfaces, fiber optic interfaces, ATM interfaces, etc.; user equipment can be displays, keyboards, etc.

[0089] Memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0090] The processor can be a general-purpose processor, which can call the path tracking control program of the transportation equipment cluster stored in memory and execute the path tracking control method of the transportation equipment cluster provided in the embodiments of this application. For example, the general-purpose processor can be a central processing unit (CPU). The method executed when the path tracking control program of the transportation equipment cluster is called can be referred to in the various embodiments of the path tracking control method of the transportation equipment cluster of this application, and will not be repeated here.

[0091] Those skilled in the art will understand that Figure 8 The hardware structure shown does not constitute a limitation of this application and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0092] Fourthly, embodiments of this application also provide a readable storage medium.

[0093] The present application has a readable storage medium storing a path tracking control program for a transportation equipment cluster, wherein when the path tracking control program for the transportation equipment cluster is executed by a processor, it implements the steps of the path tracking control method for the transportation equipment cluster as described above.

[0094] The method implemented when the path tracking control program for the transportation equipment cluster is executed can be referred to in various embodiments of the path tracking control method for the transportation equipment cluster of this application, and will not be repeated here.

[0095] It should be noted that the sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0096] The terms "comprising" and "having," and any variations thereof, in the specification, claims, and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus. The terms "first," "second," and "third," etc., are used to distinguish different objects, etc., and do not indicate a sequence, nor do they limit "first," "second," and "third" to different types.

[0097] In the description of the embodiments of this application, terms such as "exemplary," "for example," or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplary," "for example," or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary," "for example," or "for instance" is intended to present the relevant concepts in a concrete manner.

[0098] In the description of the embodiments of this application, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in the text is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of this application, "multiple" means two or more.

[0099] In some processes described in the embodiments of this application, multiple operations or steps are included in a specific order. However, it should be understood that these operations or steps may not be executed in the order they appear in the embodiments of this application, or they may be executed in parallel. The sequence number of the operation is only used to distinguish different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed sequentially or in parallel, and these operations or steps may be combined.

[0100] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device to execute the methods described in the various embodiments of this application.

[0101] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for path tracking control of a transportation equipment cluster, characterized in that, The method for path tracking and control of the transportation equipment cluster includes: The initial path deviation is calculated based on the road images collected by the navigation transport equipment in the cluster. The initial path deviation is optimized to obtain visual navigation parameters; The steering angle of the lead transport equipment is determined based on visual navigation parameters, and the steering angle of the lead transport equipment is mapped to the steering angle of the other transport equipment in the cluster. Each transport device in the control cluster performs a steering action according to its corresponding steering angle.

2. The method for path tracking and control of a transportation equipment cluster as described in claim 1, characterized in that, The initial path deviation is calculated based on road images collected by the navigation transport equipment in the cluster, including: Extract candidate regions for guide lines from road images captured by navigation transport equipment; The slope and intercept of the guide line in the image coordinate system are calculated from the candidate regions of the guide line; Based on the camera's intrinsic parameters and the camera's installation geometry on the navigation transport equipment, the slope and intercept are converted into the initial path deviation of the actual driving path. The initial path deviation includes angular deviation and lateral displacement deviation.

3. The method for path tracking and control of a transportation equipment cluster as described in claim 2, characterized in that, The extraction of guide line candidate regions from road images collected by the navigation transport equipment includes: A trained visual segmentation model is used to perform semantic segmentation on road images collected by navigation and transportation equipment to obtain preliminary segmentation regions. The initial segmented region is used as the center to expand outwards, and the set pixel values ​​are used as candidate regions for guide lines.

4. The method for path tracking and control of a transportation equipment cluster as described in claim 2, characterized in that, The step of calculating the slope and intercept of the guide line in the image coordinate system from the candidate region of the guide line includes: The skeleton of the guide line candidate region is extracted and edge points are sampled to obtain the central skeleton and edge lines of the guide line. The guide line's central skeleton and edge lines are solved using a straight line fitting algorithm to obtain the guide line's slope and intercept in the image coordinate system.

5. The method for path tracking and control of a transportation equipment cluster as described in claim 4, characterized in that, The step of extracting the skeleton and sampling the edge points of the candidate region for the guide line to obtain the central skeleton and edge lines of the guide line includes: Calculate the average pixel value within the candidate region of the guide line and the average pixel value within the background region; The dynamic segmentation threshold is calculated using the average pixel values ​​within the candidate region of the guide line and the average pixel values ​​within the background region. The candidate regions for the guide lines are binarized using a dynamic segmentation threshold to obtain the binarized regions. Extract the central skeleton of the guide line within the binarized region.

6. The method for path tracking and control of a transportation equipment cluster as described in claim 1, characterized in that, The optimization of the initial path deviation to obtain visual navigation parameters includes: Maintain a sliding window of length N, calculate the effective fitted rotation angle for each of the first N frames based on the initial path deviation of each frame, and calculate the moving average rotation angle based on the effective fitted rotation angle of the first N frames, where N is a positive integer greater than 1. Calculate the fitted rotation angle of the current frame based on the initial path deviation of the current frame, and calculate the absolute value of the deviation between the fitted rotation angle of the current frame and the moving average rotation angle of the previous N frames. If the absolute value of the deviation meets the preset conditions, it is determined that the current ground features are clear and the extraction results are continuous. The current frame fitted angle is used as the original measurement value input to the Kalman filter, and the smoothed visual navigation parameters are output. If the absolute value of the deviation does not meet the preset conditions, the current frame fitting value is determined to be abnormal. The moving average rotation angle is used as the original measurement value input to the Kalman filter, and the smoothed visual navigation parameters are output.

7. The method for path tracking and control of a transportation equipment cluster as described in claim 1, characterized in that, The process of determining the steering angle of the lead transport equipment based on visual navigation parameters and mapping the steering angle of the lead transport equipment to the steering angles of the other transport equipment in the cluster includes: Determine the steering angle of the pilot transport equipment based on visual navigation parameters; The instantaneous turning center is determined based on the turning angle of the pilot transport equipment; Based on extended Ackerman geometric constraints, the instantaneous turning center of the lead transport equipment is mapped to the entire cluster, and the turning angles of the other transport equipment in the cluster are calculated according to the coordinate positions of the other transport equipment relative to the lead transport equipment.

8. A path tracking control device for a transportation equipment cluster, characterized in that, The path tracking control device for the transportation equipment cluster includes: The deviation calculation module is used to calculate the initial path deviation based on the road images collected by the navigation transport equipment in the cluster. The optimization module is used to optimize the initial path deviation and obtain visual navigation parameters; The calculation module is used to determine the turning angle of the lead transport equipment based on visual navigation parameters and map the turning angle of the lead transport equipment to the turning angle of the other transport equipment in the cluster. The control module is used to control each transport device in the cluster to perform steering actions according to the corresponding steering angle.

9. A path tracking control device for a transportation equipment cluster, characterized in that, The path tracking control device for the transportation equipment cluster includes a processor, a memory, and a path tracking control program for the transportation equipment cluster stored in the memory and executable by the processor, wherein when the path tracking control program for the transportation equipment cluster is executed by the processor, it implements the steps of the path tracking control method for the transportation equipment cluster as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a path tracking control program for a transport equipment cluster, wherein when the path tracking control program for the transport equipment cluster is executed by a processor, it implements the steps of the path tracking control method for a transport equipment cluster as described in any one of claims 1 to 7.

Images