Autonomous parking system, method and equipment for semi-trailer tractor, medium and vehicle

Abstract

The embodiment of the invention relates to the technical field of vehicles, and discloses a semi-trailer tractor autonomous parking system, method and equipment, a medium and a vehicle. According to the embodiment of the invention, at least the technical problems of trailer folding, path deviation, even collision and the like in the parking process caused by unreliable measurement results due to the fact that most of the existing autonomous parking technologies of the semitrailer mostly depend on single visual sensor or model estimation under illumination variation, shielding or complex backgrounds can be relieved; according to the method, the included angle between the main vehicle and the trailer can be accurately measured in real time, autonomous parking of the semi-trailer tractor is facilitated, the parking safety of the tractor is guaranteed, and the driving experience of a user is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicles, in particular to a semi-trailer tractor autonomous parking system, method, device, medium and vehicle. BACKGROUND

[0002] When the semi-trailer tractor (with trailer) is autonomously parked in a port, a logistics park or the like, its unique articulated structure leads to a complex kinematic model, and the traditional parking system is difficult to apply.

[0003] Currently, the core challenge of parking of the semi-trailer tractor is that the articulation angle (included angle) between the tractor and the trailer cannot be obtained in real time and accurately, which is a prerequisite for accurate path planning and stable control. The existing technology relies on a single visual sensor or model estimation, and in the case of light changes, occlusions or complex backgrounds, the measurement results are unreliable, which easily leads to risks such as trailer folding, path deviation and even collision during parking.

[0004] Therefore, there is an urgent need for a sensor solution and a corresponding control system that can directly and reliably perceive the real-time state of the main trailer, especially the included angle. SUMMARY

[0005] The purpose of the present application is to provide a semi-trailer tractor autonomous parking system, method, device, medium and vehicle, which can at least alleviate the technical problems that the existing semi-trailer tractor autonomous parking technology relies on a single visual sensor or model estimation, and in the case of light changes, occlusions or complex backgrounds, the measurement results are unreliable, which easily leads to risks such as trailer folding, path deviation and even collision during parking. The included angle between the main vehicle and the trailer can be accurately and real-time measured, which is beneficial to realize the autonomous parking of the semi-trailer tractor, ensures the safety of vehicle parking, and improves the user driving experience.

[0006] In order to solve the above technical problems, in a first aspect, the present application provides a semi-trailer tractor autonomous parking system, which at least comprises a blind-filling radar module, a surround-view camera module, a positioning module, a data processing module, a parking control module and a path tracking module;

[0007] The blind-filling radar module is at least arranged on the symmetrical two sides of the top surround of the tractor head, and is at least used for acquiring point clouds of a preset area of the front part and the articulated part of the trailer;

[0008] The surround-view camera module is at least used for acquiring multi-view image data of the vehicle;

[0009] The positioning module is at least used for acquiring vehicle pose information;

[0010] The data processing module is connected with the blind filling radar module and the surround view camera module respectively, and is used at least for obtaining and processing the preset area point cloud and the multi-view image data to unify to a bird's eye view BEV representation space, performing BEV space feature fusion and three-dimensional bounding box fitting operation, and then extracting preset point coordinates at the bottom of the bounding box; and after converting the preset point coordinates to a vehicle coordinate system, combining the hitch ball position information and obtaining a real-time articulation angle through geometric calculation;

[0011] The parking control module is connected with the surround view camera module, the positioning module and the data processing module respectively, and is used at least for obtaining the multi-view image data, the vehicle pose information and the real-time articulation angle; and identifying an idle parking space according to the multi-view image data, and then determining a target parking space in combination with the vehicle pose information; and taking at least the vehicle pose information, the real-time artication angle and the target parking space as path planning input, and obtaining a parking planning path in combination with a preset path planning algorithm;

[0012] The path tracking module is connected with the parking control module and the data processing module respectively, and is used at least for obtaining the parking planning path and the real-time articulation angle; and based on a vehicle error model, solving at least a vehicle steering angle and a vehicle speed control sequence in each control cycle, taking the parking planning path as a target and considering dynamic constraints of the articulation angle; and driving the vehicle to enter the target parking space along the parking planning path at least according to the vehicle steering angle and the vehicle speed control sequence.

[0013] Optionally, the data processing module is specifically used at least for extracting features of the preset area point cloud and the multi-view image data through a preset modal encoder respectively.

[0014] Wherein, after the multi-view image data is extracted through at least a Swin-T encoder, multi-scale features are fused by a feature pyramid network FPN to output multi-view perspective features; and the preset area point cloud is extracted as sparse 3D representation after being encoded by at least a VoxelNet.

[0015] Optionally, the data processing module is further configured to at least unify the multi-view perspective features and the sparse 3D representation into a BEV representation space; and, in the BEV representation space, unfold the sparse 3D representation along the Z-axis to form a planar BEV representation; and, in the BEV representation space, explicitly predict the discrete depth distribution of each pixel in the multi-view perspective features, and disperse each pixel in the multi-view perspective features into D discrete points along the camera's view frustum direction, and then perform feature alignment according to the corresponding depth probabilities to generate a camera feature point cloud of shape N×H×W×D through weighted reconstruction; and, in the BEV representation space, quantize the camera feature point cloud along the X-axis and Y-axis with a preset step size R, and then aggregate and flatten all features falling in each R×R grid along the Z-axis through BEV pooling operations to generate a BEV feature map;

[0016] Wherein, N represents the number of cameras included in the surround-view camera module, and H and W represent the size of the multi-view perspective feature.

[0017] Optionally, the data processing module is further configured to extract trailer features from the planar BEV representation and the BEV feature map to estimate the trailer's three-dimensional bounding box in the vehicle coordinate system in real time; and to determine the coordinates of the preset point or the equivalent feature point located at the bottom of the three-dimensional bounding box; and to project the preset point or the equivalent feature point into the vehicle coordinate system, at least in combination with the radar installation position and radar extrinsic parameters; and to obtain the real-time hinge angle through geometric calculation using the projected feature point coordinates and the towing ball position information.

[0018] The vehicle coordinate system is based on at least the center of the rear axle of the tractor or the hinge point.

[0019] Optionally, the parking control module is specifically used at least to take the vehicle pose information, the real-time articulation angle, and the target parking space as path planning inputs, and combine the A* algorithm to take the lowest cost path between the vehicle's starting position and the target parking space as the parking planning path.

[0020] Optionally, the parking planning path is obtained at least by finding new nodes by extending the nodes forward or backward under different steering angles;

[0021] The parking control module calculates a cost for each node and expands the node with the lowest cost.

[0022] The costs include at least the cost of the vehicle traveling to the destination and the heuristic cost;

[0023] The cost of the vehicle traveling to its destination includes at least the distance the vehicle travels forward, the distance the vehicle travels backward, the absolute angle of the steering wheel, and the change in the steering wheel angle.

[0024] The heuristic cost includes at least the absolute Euclidean distance between the current trailer position and the destination trailer position, the lateral distance between the current trailer position and the destination trailer position, the longitudinal distance between the current trailer position and the destination trailer position, the angle between the current trailer position and the destination trailer position, and the switching of the vehicle's direction of movement.

[0025] Based on the same concept, in a second aspect, the present invention also provides a method for autonomous parking of a semi-trailer tractor, which is executed by the autonomous parking system for a semi-trailer tractor described in any one of the first aspects;

[0026] The autonomous parking method for semi-trailer tractors includes at least the following:

[0027] The blind spot radar module acquires point clouds of a preset area of ​​the front and articulated part of the trailer; the surround view camera module acquires multi-view image data of the vehicle; and the positioning module acquires the vehicle's pose information.

[0028] The data processing module acquires and processes the point cloud of the preset region and the multi-view image data, unifying them into the bird's-eye view BEV representation space to perform BEV space feature fusion and three-dimensional bounding box fitting operations, thereby extracting the preset point coordinates at the bottom of the bounding box.

[0029] After the preset point coordinates are converted to the vehicle coordinate system by the data processing module, the real-time articulation angle is obtained by combining the traction ball position information and geometric calculation.

[0030] The parking control module acquires the multi-view image data, the vehicle pose information, and the real-time articulation angle, identifies vacant parking spaces based on the multi-view image data, and then determines the target parking space by combining the vehicle pose information.

[0031] The parking control module takes at least the vehicle pose information, the real-time articulation angle and the target parking space as path planning inputs, and combines them with a preset path planning algorithm to obtain the parking planning path.

[0032] The parking planning path and the real-time articulation angle are obtained through the path tracking module. Based on the vehicle error model, the parking planning path is used as the target in each control cycle, and the dynamic constraint of the articulation angle is considered to solve at least the vehicle steering angle and vehicle speed control sequence.

[0033] The path tracking module drives the vehicle along the parking planning path into the target parking space based at least on the vehicle steering angle and speed control sequence.

[0034] Based on the same concept, in a third aspect, the present invention also provides an electronic device including a memory and a processor, the memory storing a computer program executable on the processor, the processor executing the program to implement the steps of the semi-trailer tractor autonomous parking method described in any one of the second aspects.

[0035] Based on the same concept, in a fourth aspect, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the semi-trailer tractor autonomous parking method described in any one of the second aspects.

[0036] The technical solution provided in this invention firstly acquires a point cloud of a preset area at the front and articulated part of the trailer through a blind spot radar module, acquires multi-view image data of the vehicle through a surround-view camera module, and acquires vehicle pose information through a positioning module. Further, the preset area point cloud and multi-view image data are acquired and processed by a data processing module and unified into a bird's-eye view BEV representation space to perform BEV spatial feature fusion and 3D bounding box fitting operations, thereby extracting the preset point coordinates at the bottom of the bounding box. Further, the preset point coordinates are transformed to the vehicle coordinate system by the data processing module, and the real-time articulation angle is obtained through geometric calculations combined with the towing ball position information. Further, the parking control module acquires multi-view images. The system uses data, vehicle pose information, and real-time articulation angles, and identifies available parking spaces based on multi-view image data. It then combines this with the vehicle pose information to determine the target parking space. Further, the parking control module uses at least the vehicle pose information, real-time articulation angle, and target parking space as path planning inputs, combining them with a preset path planning algorithm to obtain a parking planning path. Further, the path tracking module acquires the parking planning path and real-time articulation angle, and based on the vehicle error model, uses the parking planning path as the target in each control cycle, considering dynamic constraints on the articulation angle, to at least solve for the vehicle steering angle and speed control sequence. Finally, the path tracking module drives the vehicle along the parking planning path into the target parking space based at least on the vehicle steering angle and speed control sequence.

[0037] Therefore, the embodiments of the present invention can at least alleviate the problem that existing semi-trailer tractor autonomous parking technologies rely heavily on a single visual sensor or model estimation. Under varying lighting conditions, occlusion, or complex backgrounds, the measurement results are unreliable, which can easily lead to technical problems such as trailer folding, path deviation, or even collisions during the parking process. The present invention can accurately and in real time measure the angle between the tractor and the trailer, which is conducive to realizing autonomous parking of semi-trailer tractors, ensuring vehicle parking safety, and improving the user's driving experience. Attached Figure Description

[0038] Figure 1This is a schematic diagram of the structure of an autonomous parking system for a semi-trailer tractor provided in an embodiment of the present invention;

[0039] Figure 2 This is a flowchart of a semi-trailer tractor autonomous parking method provided in an embodiment of the present invention;

[0040] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0042] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the application. The singular forms “a,” “said,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms, and “multiple” generally includes at least two unless the context clearly indicates otherwise.

[0043] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0044] It should be understood that although the terms first, second, third, etc., may be used in the embodiments of this application, these descriptions should not be limited to these terms. These terms are only used to distinguish the descriptions. For example, first may also be referred to as second without departing from the scope of the embodiments of this application, and similarly, second may also be referred to as first.

[0045] Depending on the context, the words “if” or “suppose” as used here can be interpreted as “when” or “in response to determination” or “in response to detection.” Similarly, depending on the context, the phrases “if determination” or “if detection (of the stated condition or event)” can be interpreted as “when determination” or “in response to determination” or “when detection (of the stated condition or event)” or “in response to detection (of the stated condition or event).”

[0046] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that an article or device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such an article or device. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the article or device that includes said element.

[0047] It should be noted that any symbols and / or numbers present in the specification that are not marked in the accompanying drawings are not reference numerals.

[0048] Figure 1 This is a schematic diagram of the structure of an autonomous parking system for a semi-trailer tractor provided in an embodiment of the present invention. This embodiment is applicable to at least any parking control scenario for a semi-trailer tractor, and the autonomous parking system for the semi-trailer tractor can be implemented using software and / or hardware. Figure 1 As shown, the semi-trailer tractor autonomous parking system includes at least a blind spot radar module 110, a surround view camera module 120, a positioning module 130, a data processing module 140, a parking control module 150, and a path tracking module 160.

[0049] The blind spot radar module 110 is installed on at least two symmetrical sides of the top of the tractor's cab, and is used to acquire point clouds of a preset area of ​​the front and articulated parts of the trailer.

[0050] The surround-view camera module 120 is used to acquire multi-view image data of the vehicle.

[0051] The positioning module 130 is used at least to obtain vehicle position and pose information.

[0052] The data processing module 140 is connected to the blind spot radar module 110 and the surround view camera module 120 respectively. It is used at least to acquire and process the point cloud and multi-view image data of the preset area and unify them into the bird's-eye view BEV representation space to perform BEV spatial feature fusion and three-dimensional bounding box fitting operations, thereby extracting the preset point coordinates at the bottom of the bounding box; and after converting the preset point coordinates to the vehicle coordinate system, it combines the traction ball position information and obtains the real-time hinge angle through geometric calculation.

[0053] The parking control module 150 is connected to the surround-view camera module 120, the positioning module 130, and the data processing module 140, respectively, and is used at least to acquire multi-view image data, vehicle pose information, and real-time articulation angle; and to identify vacant parking spaces based on the multi-view image data, and then determine the target parking space by combining the vehicle pose information; and to use at least the vehicle pose information, real-time articulation angle, and target parking space as path planning inputs, and obtain a parking planning path by combining a preset path planning algorithm.

[0054] The path tracking module 160 is connected to the parking control module 150 and the data processing module 140 respectively, and is used at least to acquire the parking planning path and the real-time articulation angle; and, based on the vehicle error model, in each control cycle, with the parking planning path as the target and considering the dynamic constraint of the articulation angle, to solve at least the vehicle steering angle and vehicle speed control sequence; and, at least according to the vehicle steering angle and vehicle speed control sequence, to drive the vehicle along the parking planning path into the target parking space.

[0055] The blind spot radar module 110 can be composed of multiple radar devices, such as multiple conventional lidars and at least two blind spot lidars (e.g., the Sagitar BP model can be selected), with at least two blind spot lidars set on the symmetrical sides of the top of the tractor's front end.

[0056] It is understood that the surround-view camera module 120 can also be composed of multiple cameras. For example, the surround-view camera module 120 may include at least a front-facing camera, left and right front-facing cameras, left and right rear-facing cameras, and a rear-facing camera. This module can provide rich visual information and can be used to assist the vehicle in performing environmental perception (such as parking line recognition, obstacle detection, etc.), and then fuse with the data identified by the aforementioned LiDAR. It is understood that the positioning module 130 can employ an inertial navigation / BDS / GPS integrated navigation system, which can at least provide high-precision vehicle pose information. In addition, the data processing module 140, parking control module 150, and path tracking module 160 can be integrated into an industrial control computer for unified implementation.

[0057] In one specific implementation, optionally, the data processing module 140 is at least specifically used to extract features from the point cloud of the preset region and the multi-view image data through a preset modal encoder.

[0058] Among them, the multi-view image data is extracted by the Swin-T encoder at least, and then multi-scale features are fused using the Feature Pyramid Network (FPN) to output multi-view perspective features; the point cloud of the preset region is extracted as a sparse 3D representation after being encoded by VoxelNet at least (the encoder corresponding to the conventional LiDAR can share weights with the encoder corresponding to the blind spot LiDAR to reduce computational overhead and improve inference speed).

[0059] In another specific implementation, optionally, the data processing module 140 is at least further configured to unify multi-view perspective features and sparse 3D representations into a BEV representation space; and, in the BEV representation space, unfold the sparse 3D representation along the Z-axis to form a planar BEV representation; and, in the BEV representation space, explicitly predict the discrete depth distribution of each pixel in the multi-view perspective features, and disperse each pixel in the multi-view perspective features into D discrete points along the camera frustum direction, and then perform feature alignment according to the corresponding depth probability to generate a camera feature point cloud of shape N×H×W×D through weighted reconstruction; and, in the BEV representation space, quantize the camera feature point cloud along the X-axis and Y-axis with a preset step size R (R can be, for example, 0.4m), and then aggregate and flatten all features falling in each R×R grid along the Z-axis through BEV pooling operation to generate a BEV feature map;

[0060] Where N represents the number of cameras included in the surround-view camera module 120, and H and W represent the size of the multi-view perspective features.

[0061] In another specific implementation, optionally, the data processing module 140 is further configured to extract trailer features from the planar BEV representation and BEV feature map to estimate the three-dimensional bounding box of the trailer in the vehicle coordinate system in real time; and to determine the coordinates of a preset point or equivalent feature point located at the bottom of the three-dimensional bounding box; and to project the preset point or equivalent feature point into the vehicle coordinate system, at least in combination with the radar installation position and radar extrinsic parameters; and to obtain the real-time articulation angle (which may be the yaw angle of the trailer relative to the vehicle's central axis) through geometric calculation using the projected feature point coordinates and the towing ball position information (usually derived from the vehicle's mechanical parameters).

[0062] The vehicle coordinate system shall have its origin at least at the center of the rear axle of the tractor or at the hinge point.

[0063] In another specific implementation, optionally, the parking control module 150 is at least specifically used to take the vehicle pose information, real-time articulation angle and target parking space as path planning inputs, and combine the A* algorithm to take the lowest cost path between the vehicle's starting position and the target parking space as the parking planning path.

[0064] In another specific implementation, optionally, the parking planning path is obtained at least by finding new nodes by extending the nodes forward or backward at different steering angles;

[0065] The parking control module 150 calculates a cost for each node and expands the node with the lowest cost.

[0066] The costs include at least the cost of the vehicle traveling to the destination and the heuristic costs.

[0067] The cost of a vehicle traveling to its destination includes at least the distance the vehicle travels forward, the distance the vehicle travels backward, the absolute angle of the steering wheel, and the change in steering wheel angle.

[0068] Heuristic costs include at least the absolute Euclidean distance between the current trailer position and the destination trailer position, the lateral distance between the current trailer position and the destination trailer position, the longitudinal distance between the current trailer position and the destination trailer position, the angle between the current trailer position and the destination trailer position, and the switching of the vehicle's direction of movement.

[0069] More specifically, considering the requirements for low latency and lightweight design in parking scenarios, this embodiment can also introduce the pre-computation and interval reduction mechanism proposed by BEVFusion to optimize the conversion efficiency from the camera view to the BEV. By pre-constructing the image-to-BEV mapping index and combining it with the feature sampling acceleration strategy during forward inference, the feature conversion latency can be reduced by more than 40 times, effectively meeting the real-time requirements in parking scenarios.

[0070] Furthermore, considering the potential for local spatial misalignment after fusing the converted camera and radar features in the BEV representation space, this embodiment employs a convolution-based BEV encoder for unified feature processing to further improve fusion accuracy. Multi-scale convolution operations can enhance the local perceptual ability of features, while mitigating spatial biases caused by modal differences and improving the system's ability to identify fine-grained obstacles.

[0071] Furthermore, based on the shared BEV feature map, this embodiment can also set up detection head modules for different tasks. For example, a center heatmap head of a specific category can be used to predict the center position of all targets (TransFusion), and multiple regression heads can be used to estimate the size of the targets. As another example, a 3D detection head and a parking space recognition detection head can be integrated to achieve accurate perception of obstacles and parking spaces in parking scenarios, respectively.

[0072] Furthermore, after finding the parking planning path, this path can be saved and sent to the vehicle trajectory optimization code. This embodiment optimizes the vehicle trajectory through path smoothing so that the controller can more easily follow the path; this objective can be achieved by optimizing a cost function. Trajectory planning can be defined as an optimization problem, which also includes constraints on the system state and control inputs (to prevent articulation folding and sudden changes in speed or steering). Additionally, collision avoidance constraints can be added. The optimized Collision Avoidance (OBCA) algorithm is used, and the solution is obtained using the IPOPT solver.

[0073] Furthermore, the optimized parking planning path can be sent back to the controller to control the semi-trailer tractor. The MPC will follow this path while ensuring that no articulation folding occurs. This can also be described as an optimization problem, whose cost function includes the difference between the state of the optimized path and the actual state, and may also include constraints on the state and control inputs.

[0074] Furthermore, the path tracking module 160 can be designed with a controller based on model predictive control (MPC). This controller can utilize a linearized vehicle error model, with the planned path as a reference and the real-time angle as an important feedback quantity, to continuously optimize and calculate steering, throttle, and braking control quantities, thereby achieving high-precision path tracking.

[0075] To accurately describe the system's motion characteristics, a discrete-time state-space model can be used to characterize the dynamic behavior of the tractor-trailer. The model can consider the following core elements:

[0076] 1. State variables: .

[0077] In the above formula, z represents the state variable, and x t and y t The Cartesian coordinates representing the center of the rear axle of the tractor unit. θ represents the heading angle of the tractor unit. t Indicates the articulation angle (knife angle) between the tractor and trailer, v t The speed of the tractor unit.

[0078] 2. Control variables: .

[0079] In the above formula, u represents the control variable. Indicates the steering angle of the front wheels of the tractor, a t This indicates the acceleration of the tractor (controlling speed changes).

[0080] 3. Discrete State Equations

[0081] ;

[0082] In the above formula, dt represents the sampling time, and L t L represents the wheelbase of the tractor unit. s This indicates the trailer wheelbase.

[0083] This model can fully capture the coupled motion relationship between the tractor and the trailer, especially the dynamic evolution of the articulation angle, providing a mathematical basis for folding knife risk prediction.

[0084] Furthermore, the core of the MPC controller is solving a constrained optimization problem, which can take the following form:

[0085] 1. Objective function design:

[0086] ;

[0087] In the above formula, N represents the prediction time domain length; z t+k|t z represents the system state predicted at time t+k from time t; ref,t+k The reference trajectory state at time t+k is represented by Q; the state error weight matrix (diagonal elements correspond to the importance of each state, such as position error having a higher weight than velocity error); and the control input weight matrix (suppressing excessive changes in steering angle and acceleration). f This represents the terminal state error weight matrix (which enhances the tracking accuracy of the final prediction point).

[0088] 2. Constraints:

[0089] (1) Dynamic constraints: (i.e., the discrete state equations mentioned above).

[0090] (2) State constraints

[0091] Position constraints: (Parking area boundary);

[0092] Speed ​​constraints: (In parking scenarios, v) max ≤5km / h);

[0093] Hinge angle constraint: (θ) max As a safety threshold, a 70° angle is typically used to avoid folding the knife.

[0094] (3) Control input constraints

[0095] Steering angle constraint: (Determined by the physical limits of the steering system, typically ±30°);

[0096] Acceleration constraints: (Considering comfort and tire grip, typically 2m / s) 2 );

[0097] Input rate of change constraint: (Suppressing shift mutations).

[0098] By constructing the optimization problem described above, the MPC controller can achieve high-precision trajectory tracking while satisfying all safety and physical constraints. It is understood that the above extreme values ​​or thresholds can be obtained through pre-experimentation, and will not be elaborated further.

[0099] The technical solution provided in this embodiment firstly acquires a point cloud of a preset area at the front and articulated part of the trailer through a blind spot radar module, acquires multi-view image data of the vehicle through a surround-view camera module, and acquires vehicle pose information through a positioning module. Further, the data processing module acquires and processes the preset area point cloud and multi-view image data, unifying them into a bird's-eye view BEV representation space to perform BEV spatial feature fusion and 3D bounding box fitting operations, thereby extracting the preset point coordinates at the bottom of the bounding box. Further, the data processing module transforms the preset point coordinates to the vehicle coordinate system, combines the towing ball position information, and obtains the real-time articulation angle through geometric calculation. Further, the parking control module acquires multi-view image data... Based on vehicle pose information and real-time articulation angle, and using multi-view image data to identify available parking spaces, the system then determines the target parking space by combining the vehicle pose information. Further, the parking control module uses at least the vehicle pose information, real-time articulation angle, and target parking space as path planning inputs, and combines them with a preset path planning algorithm to obtain a parking planning path. Further, the path tracking module acquires the parking planning path and real-time articulation angle, and based on the vehicle error model, uses the parking planning path as the target in each control cycle, considering dynamic constraints on the articulation angle, to at least solve for the vehicle steering angle and speed control sequence. Finally, the path tracking module drives the vehicle along the parking planning path into the target parking space based at least on the vehicle steering angle and speed control sequence.

[0100] Therefore, this embodiment can at least alleviate the problem that existing semi-trailer tractor autonomous parking technologies rely heavily on a single visual sensor or model estimation. Under varying lighting conditions, occlusion, or complex backgrounds, the measurement results are unreliable, which can easily lead to technical problems such as trailer folding, path deviation, or even collisions during parking. This embodiment can accurately and in real-time measure the angle between the tractor and the trailer, which is beneficial for realizing autonomous parking of semi-trailer tractors, ensuring vehicle parking safety, and improving the user's driving experience.

[0101] Figure 2 This is a flowchart of a semi-trailer tractor autonomous parking method provided by an embodiment of the present invention. This embodiment is applicable to at least any parking control scenario for semi-trailer tractors. The semi-trailer tractor autonomous parking method can be, but is not limited to, executed by the semi-trailer tractor autonomous parking system of this embodiment as the executing entity, which can be implemented in software and / or hardware. Figure 2 As shown, the autonomous parking method for semi-trailer tractors includes at least the following steps:

[0102] S1. Obtain point cloud data of the preset area of ​​the front and articulated part of the trailer through the blind spot radar module; obtain multi-view image data of the vehicle through the surround view camera module; obtain vehicle position and posture information through the positioning module.

[0103] S2. After the data processing module acquires and processes the point cloud and multi-view image data of the preset area, it is unified into the bird's-eye view BEV representation space to perform BEV space feature fusion and three-dimensional bounding box fitting operations, thereby extracting the preset point coordinates at the bottom of the bounding box.

[0104] S3. After converting the preset point coordinates to the vehicle coordinate system through the data processing module, the real-time articulation angle is obtained by combining the position information of the traction ball and through geometric calculation.

[0105] S4. The parking control module acquires multi-view image data, vehicle pose information and real-time articulation angle, and identifies vacant parking spaces based on the multi-view image data, and then determines the target parking space by combining the vehicle pose information.

[0106] S5. The parking control module takes at least the vehicle pose information, real-time articulation angle and target parking space as path planning inputs, and combines them with a preset path planning algorithm to obtain the parking planning path.

[0107] S6. Obtain the parking planning path and real-time articulation angle through the path tracking module. Based on the vehicle error model, take the parking planning path as the target in each control cycle and consider the dynamic constraint of the articulation angle to solve at least the vehicle steering angle and speed control sequence.

[0108] S7. Drive the vehicle along the parking planning path into the target parking space by the path tracking module based on the vehicle steering angle and speed control sequence.

[0109] Optionally, the data processing module is specifically used at least to extract features from the point cloud of the preset region and the multi-view image data through a preset modal encoder;

[0110] Among them, the multi-view image data is extracted by the Swin-T encoder at least, and then multi-scale features are fused using the Feature Pyramid Network (FPN) to output multi-view perspective features; the point cloud of the preset region is extracted as a sparse 3D representation after being encoded by VoxelNet at least.

[0111] Optionally, the data processing module is at least specifically used to unify multi-view perspective features and sparse 3D representations into a BEV representation space; and, in the BEV representation space, to unfold the sparse 3D representation along the Z-axis to form a planar BEV representation; and, in the BEV representation space, to explicitly predict the discrete depth distribution of each pixel in the multi-view perspective features, and to disperse each pixel in the multi-view perspective features into D discrete points along the camera's view frustum direction, and then to perform feature alignment according to the corresponding depth probability, so as to generate a camera feature point cloud of shape N×H×W×D through weighted reconstruction; and, in the BEV representation space, to quantize the camera feature point cloud along the X-axis and Y-axis with a preset step size R, and then to aggregate and flatten all features falling in each R×R grid along the Z-axis through BEV pooling operation to generate a BEV feature map;

[0112] Where N represents the number of cameras included in the surround view camera module, and H and W represent the size of the multi-view perspective features.

[0113] Optionally, the data processing module is at least specifically used to extract trailer features from the planar BEV representation and BEV feature map to estimate the three-dimensional bounding box of the trailer in the vehicle coordinate system in real time; and to determine the coordinates of a preset point or equivalent feature point located at the bottom of the three-dimensional bounding box; and to project the preset point or equivalent feature point into the vehicle coordinate system, at least in combination with the radar installation position and radar extrinsic parameters; and to obtain the real-time hinge angle through geometric calculation using the coordinates of the projected feature point and the position information of the towing ball.

[0114] The vehicle coordinate system shall have its origin at least at the center of the rear axle of the tractor or at the hinge point.

[0115] Optionally, the parking control module is specifically used to take the vehicle pose information, real-time articulation angle and target parking space as path planning inputs, and combine the A* algorithm to take the lowest cost path between the vehicle's starting position and the target parking space as the parking planning path.

[0116] Alternatively, the parking planning path is obtained at least by finding new nodes by extending the nodes forward or backward at different steering angles;

[0117] The parking control module calculates a cost for each node and expands the node with the lowest cost.

[0118] The costs include at least the cost of the vehicle traveling to the destination and the heuristic costs.

[0119] The cost of a vehicle traveling to its destination includes at least the distance the vehicle travels forward, the distance the vehicle travels backward, the absolute angle of the steering wheel, and the change in steering wheel angle.

[0120] Heuristic costs include at least the absolute Euclidean distance between the current trailer position and the destination trailer position, the lateral distance between the current trailer position and the destination trailer position, the longitudinal distance between the current trailer position and the destination trailer position, the angle between the current trailer position and the destination trailer position, and the switching of the vehicle's direction of movement.

[0121] The technical solution provided in this embodiment firstly acquires a point cloud of a preset area at the front and articulated part of the trailer through a blind spot radar module, acquires multi-view image data of the vehicle through a surround-view camera module, and acquires vehicle pose information through a positioning module. Further, the data processing module acquires and processes the preset area point cloud and multi-view image data, unifying them into a bird's-eye view BEV representation space to perform BEV spatial feature fusion and 3D bounding box fitting operations, thereby extracting the preset point coordinates at the bottom of the bounding box. Further, the data processing module transforms the preset point coordinates to the vehicle coordinate system, combines the towing ball position information, and obtains the real-time articulation angle through geometric calculation. Further, the parking control module acquires multi-view image data... Based on vehicle pose information and real-time articulation angle, and using multi-view image data to identify available parking spaces, the system then determines the target parking space by combining the vehicle pose information. Further, the parking control module uses at least the vehicle pose information, real-time articulation angle, and target parking space as path planning inputs, and combines them with a preset path planning algorithm to obtain a parking planning path. Further, the path tracking module acquires the parking planning path and real-time articulation angle, and based on the vehicle error model, uses the parking planning path as the target in each control cycle, considering dynamic constraints on the articulation angle, to at least solve for the vehicle steering angle and speed control sequence. Finally, the path tracking module drives the vehicle along the parking planning path into the target parking space based at least on the vehicle steering angle and speed control sequence.

[0122] Therefore, this embodiment can at least alleviate the problem that existing semi-trailer tractor autonomous parking technologies rely heavily on a single visual sensor or model estimation. Under varying lighting conditions, occlusion, or complex backgrounds, the measurement results are unreliable, which can easily lead to technical problems such as trailer folding, path deviation, or even collisions during parking. This embodiment can accurately and in real-time measure the angle between the tractor and the trailer, which is beneficial for realizing autonomous parking of semi-trailer tractors, ensuring vehicle parking safety, and improving the user's driving experience.

[0123] This embodiment provides an electronic device. Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. See also: Figure 3The electronic device 1000 includes a processor 1001 and a memory 1002. The memory 1002 stores computer-readable instructions. When the computer-readable instructions are executed by the processor 1001, the steps in any of the above-mentioned semi-trailer tractor autonomous parking methods are performed. Through the above technical solution, the processor 1001 and the memory 1002 are interconnected and communicate with each other through a communication bus and / or other forms of connection mechanism (not shown). The memory 1002 stores a computer program executable by the processor. When the electronic device 1000 is running, the processor 1001 executes the computer program to execute the semi-trailer tractor autonomous parking method in any optional implementation of the above embodiments, so as to achieve at least the following functions: acquiring a preset area point cloud of the front and articulated parts of the trailer through a blind spot radar module, acquiring multi-view image data of the vehicle through a surround view camera module, and acquiring vehicle pose information through a positioning module; acquiring and processing the preset area point cloud and multi-view image data through a data processing module and unifying them into a bird's-eye view BEV representation space to perform BEV spatial feature fusion and three-dimensional bounding box fitting operations, thereby extracting the preset point coordinates at the bottom of the bounding box. The system transforms the preset point coordinates into the vehicle coordinate system using a data processing module, and then calculates the real-time articulation angle using geometric calculations in conjunction with the traction ball position information. The parking control module acquires multi-view image data, vehicle pose information, and the real-time articulation angle, identifies available parking spaces based on the multi-view image data, and determines the target parking space based on the vehicle pose information. The parking control module uses at least the vehicle pose information, the real-time articulation angle, and the target parking space as path planning inputs, and obtains the parking planning path using a preset path planning algorithm. The path tracking module acquires the parking planning path and the real-time articulation angle, and based on the vehicle error model, uses the parking planning path as the target in each control cycle, considering the dynamic constraints of the articulation angle, to at least solve for the vehicle steering angle and speed control sequence. The path tracking module drives the vehicle along the parking planning path into the target parking space based at least on the vehicle steering angle and speed control sequence.

[0124] This embodiment provides a computer-readable storage medium storing a computer program. When executed by a processor, the program implements the autonomous parking method for a semi-trailer tractor provided in all embodiments of this application: acquiring a preset area point cloud of the front and articulated parts of the trailer through a blind spot radar module, acquiring multi-view image data of the vehicle through a surround-view camera module, and acquiring vehicle pose information through a positioning module; acquiring and processing the preset area point cloud and multi-view image data through a data processing module, unifying them into a bird's-eye view BEV representation space, performing BEV space feature fusion and three-dimensional bounding box fitting operations, and then extracting the preset point coordinates at the bottom of the bounding box; transforming the preset point coordinates to the vehicle coordinate system through the data processing module, combining the towing ball position information, and using geometric calculations... The system calculates the real-time articulation angle; acquires multi-view image data, vehicle pose information, and the real-time articulation angle through the parking control module, identifies available parking spaces based on the multi-view image data, and then determines the target parking space by combining the vehicle pose information; the parking control module uses at least the vehicle pose information, the real-time articulation angle, and the target parking space as path planning inputs, and obtains the parking planning path by combining it with a preset path planning algorithm; the path tracking module acquires the parking planning path and the real-time articulation angle, and based on the vehicle error model, uses the parking planning path as the target in each control cycle, and considers the dynamic constraints of the articulation angle, and at least solves the vehicle steering angle and speed control sequence; the path tracking module drives the vehicle to enter the target parking space along the parking planning path based on at least the vehicle steering angle and speed control sequence.

[0125] Any combination of one or more computer-readable media may be used. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device.

[0126] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including—but not limited to—electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of transmitting, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0127] The program code contained on a computer-readable medium may be transmitted using any suitable medium, including—but not limited to—wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0128] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, and C++—as well as conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0129] This embodiment provides a vehicle that integrates at least the semi-trailer tractor autonomous parking system described in any of the foregoing embodiments or implementation methods, which will not be repeated here.

[0130] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An autonomous parking system for a semi-trailer tractor, characterized in that, It includes at least a blind spot radar module, a surround view camera module, a positioning module, a data processing module, a parking control module, and a path tracking module; The blind spot radar module is installed on at least two symmetrical sides of the top of the tractor's cab, and is used to acquire point clouds of a preset area of ​​the front and articulated parts of the trailer. The surround-view camera module is used at least to acquire multi-view image data of the vehicle; The positioning module is used at least to acquire vehicle position and orientation information; The data processing module is connected to the blind spot radar module and the surround view camera module, respectively. It is used at least to acquire and process the point cloud of the preset area and the multi-view image data and unify them into the bird's-eye view BEV representation space, so as to perform BEV spatial feature fusion and three-dimensional bounding box fitting operations, thereby extracting the preset point coordinates at the bottom of the bounding box; and after converting the preset point coordinates to the vehicle coordinate system, it combines the traction ball position information and obtains the real-time hinge angle through geometric calculation. The parking control module is connected to the surround view camera module, the positioning module and the data processing module respectively, and is used at least to acquire the multi-view image data, the vehicle pose information and the real-time hinge angle; Furthermore, based on the multi-view image data, vacant parking spaces are identified, and then the target parking space is determined by combining the vehicle pose information; In addition, at least the vehicle pose information, the real-time articulation angle and the target parking space are used as path planning inputs, and a parking planning path is obtained by combining them with a preset path planning algorithm; The path tracking module is connected to the parking control module and the data processing module respectively, and is used at least to obtain the parking planning path and the real-time articulation angle; Furthermore, based on the vehicle error model, within each control cycle, taking the parking planning path as the target and considering the dynamic constraint of the articulation angle, at least the vehicle steering angle and speed control sequence are solved; and, at least according to the vehicle steering angle and speed control sequence, the vehicle is driven along the parking planning path into the target parking space.

2. The autonomous parking system for semi-trailer tractors according to claim 1, characterized in that, The data processing module is specifically used at least to extract features from the preset region point cloud and the multi-view image data using a preset modal encoder. The multi-view image data is extracted by at least a Swin-T encoder and then multi-scale features are fused using a Feature Pyramid Network (FPN) to output multi-view perspective features; the point cloud of the preset region is extracted as a sparse 3D representation after at least a VoxelNet encoding.

3. The semi-trailer tractor autonomous parking system according to claim 2, characterized in that, The data processing module is at least specifically used to unify the multi-view perspective features and the sparse 3D representation into a BEV representation space; and, in the BEV representation space, to unfold the sparse 3D representation along the Z-axis to form a planar BEV representation; and, in the BEV representation space, to explicitly predict the discrete depth distribution of each pixel in the multi-view perspective features, and to disperse each pixel in the multi-view perspective features into D discrete points along the camera's view frustum direction, and then to perform feature alignment according to the corresponding depth probabilities, so as to generate a camera feature point cloud with a shape of N×H×W×D through weighted reconstruction; Furthermore, in the BEV representation space, the camera feature point cloud is quantized along the X and Y axes with a preset step size R, and then all features falling in each R×R grid are aggregated and flattened along the Z axis through the BEV pooling operation to generate a BEV feature map. Wherein, N represents the number of cameras included in the surround-view camera module, and H and W represent the size of the multi-view perspective feature.

4. The autonomous parking system for semi-trailer tractors according to claim 3, characterized in that, The data processing module is at least specifically used to extract trailer features from the planar BEV representation and the BEV feature map to estimate the three-dimensional bounding box of the trailer in the vehicle coordinate system in real time; and to determine the coordinates of the preset point or the equivalent feature point located at the bottom of the three-dimensional bounding box; and to project the preset point or the equivalent feature point into the vehicle coordinate system, at least in combination with the radar installation position and radar extrinsic parameters. Furthermore, the real-time hinge angle is obtained through geometric calculation using the coordinates of the projected feature points and the position information of the traction ball; The vehicle coordinate system is based on at least the center of the rear axle of the tractor or the hinge point.

5. The autonomous parking system for a semi-trailer tractor according to claim 1, characterized in that, The parking control module is specifically used at least to take the vehicle pose information, the real-time articulation angle, and the target parking space as path planning inputs, and combine the A* algorithm to take the lowest cost path between the vehicle's starting position and the target parking space as the parking planning path.

6. The autonomous parking system for a semi-trailer tractor according to claim 5, characterized in that, The parking planning path is obtained at least by finding new nodes by extending the vehicle forward or backward at different steering angles; The parking control module calculates a cost for each node and expands the node with the lowest cost. The costs include at least the cost of the vehicle traveling to the destination and the heuristic cost; The cost of the vehicle traveling to its destination includes at least the distance the vehicle travels forward, the distance the vehicle travels backward, the absolute angle of the steering wheel, and the change in the steering wheel angle. The heuristic cost includes at least the absolute Euclidean distance between the current trailer position and the destination trailer position, the lateral distance between the current trailer position and the destination trailer position, the longitudinal distance between the current trailer position and the destination trailer position, the angle between the current trailer position and the destination trailer position, and the switching of the vehicle's direction of movement.

7. A method for autonomous parking of a semi-trailer tractor, characterized in that, Performed by the autonomous parking system for semi-trailer tractors as described in any one of claims 1-6; The autonomous parking method for semi-trailer tractors includes at least the following: The blind spot radar module acquires point clouds of a preset area of ​​the front and articulated part of the trailer; the surround view camera module acquires multi-view image data of the vehicle; and the positioning module acquires the vehicle's pose information. The data processing module acquires and processes the point cloud of the preset region and the multi-view image data, unifying them into the bird's-eye view BEV representation space to perform BEV space feature fusion and three-dimensional bounding box fitting operations, thereby extracting the preset point coordinates at the bottom of the bounding box. After the preset point coordinates are converted to the vehicle coordinate system by the data processing module, the real-time articulation angle is obtained by combining the traction ball position information and geometric calculation. The parking control module acquires the multi-view image data, the vehicle pose information, and the real-time articulation angle, identifies vacant parking spaces based on the multi-view image data, and then determines the target parking space by combining the vehicle pose information. The parking control module takes at least the vehicle pose information, the real-time articulation angle and the target parking space as path planning inputs, and combines them with a preset path planning algorithm to obtain the parking planning path. The parking planning path and the real-time articulation angle are obtained through the path tracking module. Based on the vehicle error model, the parking planning path is used as the target in each control cycle, and the dynamic constraint of the articulation angle is considered to solve at least the vehicle steering angle and vehicle speed control sequence. The path tracking module drives the vehicle along the parking planning path into the target parking space based at least on the vehicle steering angle and speed control sequence.

8. An electronic device comprising a memory and a processor, the memory storing a computer program executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the autonomous parking method for a semi-trailer tractor as described in claim 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps in the autonomous parking method for a semi-trailer tractor as described in claim 7.

10. A vehicle, characterized in that, It integrates at least the semi-trailer tractor autonomous parking system as described in any one of claims 1-6.

Images