Method, device, equipment and medium for determining articulation angle between tractor and trailer

Abstract

The application discloses a kind of determination methods, devices, equipment and medium of articulation angle between tractor and trailer, it is related to vehicle intelligent driving technical field, comprising: through the all-around view camera installed on tractor acquisition all-around view image data, and through the laser radar installed on tractor acquisition laser point cloud data;Image feature is obtained by feature extraction to all-around view image data, and point cloud feature is obtained by feature extraction to laser point cloud data;Image feature and point cloud feature are converted to bird's eye view perspective space respectively, obtain image bird's eye view feature and point cloud bird's eye view feature, and image bird's eye view feature is fused with point cloud bird's eye view feature, generates shared bird's eye view feature map;Based on shared bird's eye view feature map, determine the articulation angle between tractor and the trailer between being hung up.The application determines articulation angle by multimodal fusion, overcomes the defect that pure vision precision is low, pure radar point cloud is sparse and precision declines, realizes high-precision, low-cost articulation angle estimation.

Description

Technical Field

[0001] This invention relates to the field of intelligent driving technology for vehicles, and in particular to a method, apparatus, device, and medium for determining the articulation angle between a tractor and a trailer. Background Technology

[0002] In the autonomous driving system of a semi-trailer tractor, the articulation angle between the tractor and the trailer (i.e., the yaw angle of the trailer relative to the longitudinal axis of the tractor) is a key parameter affecting vehicle path planning, trajectory tracking control, and driving safety. If the articulation angle is not accurately estimated, it can easily lead to vehicle folding, collisions, or parking failures.

[0003] Currently, the methods for estimating the articulation angle of trailers mainly fall into the following two categories: Pure vision methods acquire images through cameras and estimate hinge angles using object detection and geometric / deep learning models. They are low-cost but greatly affected by lighting and occlusion, and monocular images lack depth information, resulting in limited accuracy.

[0004] Pure lidar methods estimate the articulation angle by fitting the trailer outline to a point cloud, which is unaffected by lighting conditions. However, the accuracy decreases when the trailer outline is not clear or the point cloud is sparse, and the cost is also high. Summary of the Invention

[0005] This invention provides a method, apparatus, equipment, and medium for determining the articulation angle between a tractor and a trailer, in order to solve the technical problems of existing pure vision methods being greatly affected by environmental factors such as lighting and occlusion, resulting in limited accuracy, and pure lidar methods experiencing accuracy degradation when point clouds are sparse or trailer outlines are not obvious.

[0006] According to one aspect of the present invention, a method for determining the articulation angle between a tractor and a trailer is provided, the method comprising: Surround view image data is collected by a surround view camera installed on the tractor, and laser point cloud data is collected by a lidar installed on the tractor. Feature extraction is performed on the panoramic image data to obtain image features, and feature extraction is performed on the laser point cloud data to obtain point cloud features; The image features and the point cloud features are respectively converted to the bird's-eye view space to obtain image bird's-eye view features and point cloud bird's-eye view features. The image bird's-eye view features and the point cloud bird's-eye view features are then fused to generate a shared bird's-eye view feature map. Based on the shared bird's-eye view feature map, the articulation angle between the tractor and the attached trailer is determined.

[0007] According to another aspect of the present invention, a device for determining the articulation angle between a tractor and a trailer is provided, the device comprising: The data acquisition module is used to acquire surround view image data through the surround view camera installed on the tractor and to acquire laser point cloud data through the lidar installed on the tractor. The feature extraction module is used to extract features from the panoramic image data to obtain image features, and to extract features from the laser point cloud data to obtain point cloud features. The shared bird's-eye view feature map generation module is used to convert the image features and the point cloud features to the bird's-eye view space respectively to obtain image bird's-eye view features and point cloud bird's-eye view features, and to fuse the image bird's-eye view features and the point cloud bird's-eye view features to generate a shared bird's-eye view feature map. The articulation angle determination module is used to determine the articulation angle between the tractor and the attached trailer based on the shared bird's-eye view feature map.

[0008] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor to enable the at least one processor to perform the method described in any one of the present invention.

[0009] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the method described in any one of the present invention.

[0010] The beneficial effects of this invention are as follows: Firstly, addressing the limitations of purely visual methods due to their susceptibility to environmental factors such as lighting, rain, fog, and occlusion, this invention introduces a lidar sensor. The lidar actively emits laser beams for detection, and its point cloud acquisition is independent of ambient light. Even in scenarios where purely visual methods are prone to failure, such as at night, in backlight, or in rain and fog, it can still stably provide the trailer's geometric structure information. By fusing the panoramic image with the lidar point cloud in a bird's-eye view space, this invention enhances its adaptability to environmental changes and effectively improves the accuracy of hinge angle estimation.

[0011] Secondly, addressing the limitations of pure vision methods, such as the difficulty of obtaining accurate depth information and the limited precision of monocular cameras, this invention integrates lidar point cloud data and surround view image data. LiDAR can directly provide accurate three-dimensional spatial information. Compared to pure vision methods that require depth estimation through monocular images, the depth information source of this invention is more direct and reliable, fundamentally eliminating the angle error caused by inaccurate depth estimation and effectively improving the estimation accuracy of the hinge angle.

[0012] Thirdly, addressing the issue of decreased accuracy in pure LiDAR methods when trailer outlines are indistinct or point clouds are sparse, this invention integrates image semantic features from a surround-view camera. When the LiDAR point cloud is sparse, the surround-view image still provides rich semantic information such as edges, textures, and shapes; when the trailer outline is indistinct, image features provide additional visual constraints, helping to compensate for insufficient geometric information. Through feature-level fusion in the bird's-eye view space, image semantic information and point cloud geometric information complement each other, enabling this invention to maintain high articulation angle estimation accuracy even when the performance of a single sensor deteriorates, effectively overcoming the inherent defects of pure LiDAR methods.

[0013] Fourthly, addressing the high cost of pure LiDAR solutions, this invention employs a multimodal fusion approach, combining a surround-view camera with LiDAR. Compared to pure LiDAR solutions that rely on high-beam LiDAR to obtain sufficiently dense point clouds for contour fitting, the multimodal fusion architecture of this invention allows for the use of LiDAR with lower performance requirements. The surround-view image provides supplementary information when the point cloud is sparse, thereby reducing the requirements for LiDAR beamwidth and performance, and effectively controlling overall hardware costs. Furthermore, the sensor configuration of this invention does not require modification to the mechanical connection structure between the tractor and trailer, making it easy to deploy on mass-produced commercial vehicles.

[0014] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 A flowchart illustrating a method for determining the articulation angle between a tractor and a trailer, provided in Embodiment 1 of the present invention; Figure 2 A flowchart illustrating another method for determining the articulation angle between a tractor and a trailer, provided in Embodiment 2 of the present invention; Figure 3 A flowchart illustrating another method for determining the articulation angle between a tractor and a trailer, provided in Embodiment 3 of the present invention; Figure 4 This is a schematic diagram of a device for determining the hinge angle between a tractor and a trailer, provided in Embodiment 4 of the present invention. Figure 5 This is a schematic diagram of the structure of an electronic device that implements the method for determining the hinge angle between the tractor and the trailer in this embodiment of the invention. Detailed Implementation

[0017] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0018] It should be noted that the terms "first," "second," "third," "fourth," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0019] Example 1 Figure 1 This is a flowchart illustrating a method for determining the articulation angle between a tractor and a trailer according to Embodiment 1 of the present invention. This embodiment is applicable to the real-time estimation of the articulation angle in scenarios where a semi-trailer tractor is reversing or parking. This method can be executed by a device for determining the articulation angle between the tractor and the trailer, which can be implemented in hardware and / or software. Figure 1 As shown, the method includes: S101. Surround view image data is collected by a surround view camera installed on the tractor, and laser point cloud data is collected by a lidar installed on the tractor.

[0020] In this invention, the tractor unit refers to the cab of a vehicle with driving capability used for towing trailers. In this embodiment, the tractor unit serves as the mounting platform for the surround-view camera and lidar, and its vehicle coordinate system serves as a reference for the unified alignment of all sensor data.

[0021] A surround-view camera refers to multiple wide-angle cameras installed at different locations on the tractor unit (such as forward, front side, rear side, and rearward) to simultaneously capture environmental images within a 360° range around the tractor unit. In this embodiment of the invention, the image data captured by the surround-view cameras serves as the input source for the image feature extraction module.

[0022] Surround view image data refers to raw perspective images covering a 360° area around the tractor unit, simultaneously acquired by multiple surround view cameras. The images retain realistic perspective projection relationships and contain rich visual semantic information such as texture, color, edges, and shape.

[0023] A lidar is a laser detection and ranging sensor installed on a tractor unit. It measures the distance to a target object by emitting laser pulses and receiving reflected signals. In this embodiment of the invention, the lidar is used to collect point cloud data of the area in front of the trailer, providing accurate three-dimensional spatial geometric information.

[0024] Laser point cloud data refers to the collection of three-dimensional spatial points acquired by lidar, with each point containing information such as three-dimensional coordinates and reflection intensity. Point cloud data accurately depicts the contours, edges, and spatial pose of the trailer surface, and is independent of ambient lighting conditions, allowing it to operate stably even at night or in backlit environments.

[0025] In one embodiment, the tractor unit is equipped with multiple surround-view cameras, specifically including: one forward-facing camera with a 150° field of view, mounted at the front of the tractor unit; two side-front cameras, each mounted below the left and right rearview mirrors; two side-rear cameras, each mounted on either side of the tractor unit's roof; and several rearward cameras mounted at the rear of the tractor unit. Each surround-view camera synchronously acquires image data via a time synchronization mechanism, with a sampling frequency of 10Hz, outputting surround-view image data covering a 360° area around the tractor unit. A rectangular checkerboard visual target is positioned at the center of the trailer's front end to enhance the surround-view cameras' detection and recognition capabilities of the trailer's front end.

[0026] Meanwhile, a forward-facing main LiDAR is installed at the front of the tractor unit to collect laser point cloud data of the environment in front of the vehicle; two rearward blind spot LiDARs are installed at the center line of the rear of the tractor unit to collect laser point cloud data of the area in front of the trailer. The LiDARs and surround-view cameras are synchronized in time via hardware synchronization signals to ensure that the data collected by each sensor are aligned in time. All sensor data are unified to the tractor unit's coordinate system through a pre-calibrated extrinsic parameter matrix.

[0027] The processor sends synchronous acquisition commands to the surround-view camera and LiDAR respectively, controlling the surround-view camera and LiDAR to start acquisition according to the set sampling frequency. After the acquisition is completed, the processor obtains the surround-view image data output by the surround-view camera and the laser point cloud data output by each LiDAR, associates each type of data with its respective timestamp and sensor identifier, and stores it for subsequent processing.

[0028] S102. Extract features from the panoramic image data to obtain image features, and extract features from the laser point cloud data to obtain point cloud features.

[0029] Image features refer to the high-dimensional semantic information extracted from panoramic image data. Deep learning encoders abstract visual information such as texture, color, edge, and shape in the original image into compact feature vectors or feature maps, preserving high-level semantic information such as the target's category, location, and contour, but they do not contain precise three-dimensional spatial coordinates.

[0030] Point cloud features refer to the three-dimensional spatial geometric information extracted from laser point cloud data. By using a voxel encoder, the disordered laser point cloud is converted into a structured high-dimensional feature representation, which preserves the target's accurate geometric information such as three-dimensional coordinates, contours, and dimensions, and is independent of ambient lighting conditions.

[0031] In one implementation, after the processor acquires the panoramic image data and the laser point cloud data, it performs feature extraction on each.

[0032] For panoramic image data, the processor inputs the data into the Swin Transformer encoder. The Swin Transformer encoder extracts features progressively from small image patches using a hierarchical window self-attention mechanism, gradually merging adjacent patches in deeper network layers to construct a hierarchical feature representation. The multi-level feature maps output by this encoder are then fed into a Feature Pyramid Network (FPN). The FPN fuses feature maps of different scales through top-down paths and lateral connections, ensuring that each feature map contains both detailed information from shallower networks (such as edges and textures) and semantic information from deeper networks (such as object category and overall shape). The final output is multi-scale perspective features, serving as the image features.

[0033] For laser point cloud data, the processor inputs the data into the VoxelNet encoder. The VoxelNet encoder first divides the unordered laser point cloud data into an equally spaced cubic voxel grid in three-dimensional space. For each voxel, a stacked voxel feature encoding layer encodes the point cloud features within the voxel, extracting the local geometric features of each voxel. Subsequently, a three-dimensional convolution further aggregates the features of adjacent voxels, transforming the original unordered point cloud into a structured sparse three-dimensional feature representation. Finally, the sparse three-dimensional features are output as the point cloud features.

[0034] After the above extraction is completed, the processor obtains image features and point cloud features, which are used for subsequent bird's-eye view spatial transformation and fusion processing.

[0035] S103. Convert the image features and point cloud features to the bird's-eye view space respectively to obtain the image bird's-eye view features and the point cloud bird's-eye view features, and fuse the image bird's-eye view features and the point cloud bird's-eye view features to generate a shared bird's-eye view feature map.

[0036] The bird's-eye view space refers to a spatial representation formed by uniformly transforming data collected from different sensors into a top-down, ground-parallel two-dimensional coordinate system. In this space, the position of an object is directly represented by two-dimensional coordinates (X, Y), and its size and orientation angle are immediately apparent, eliminating the distortion effect of "near objects appearing larger and farther objects appearing smaller" in perspective images. The bird's-eye view space is a common coordinate system for the unified alignment of multimodal sensor data, facilitating subsequent feature fusion and target detection.

[0037] Image bird's-eye view features refer to the feature representation obtained by transforming image features from a perspective view to a bird's-eye view space. Image bird's-eye view features retain semantic information such as texture, color, and edges in the original image, while also possessing unified spatial location attributes under the bird's-eye view, making it easy to align and merge with point cloud bird's-eye view features in the same space.

[0038] Point cloud bird's-eye view features refer to the feature representation obtained by compressing 3D point cloud features into a bird's-eye view space. Point cloud bird's-eye view features retain the precise geometric information such as 3D coordinates, contours, and dimensions of the original point cloud, while compressing 3D spatial features into 2D bird's-eye view features, eliminating the height dimension, making point cloud features and image features comparable and fusionable under a unified bird's-eye view space.

[0039] A shared bird's-eye view feature map refers to a unified feature map generated by fusing image bird's-eye view features and point cloud bird's-eye view features in the same bird's-eye view space. This feature map simultaneously contains semantic information of the image modality and geometric information of the point cloud modality.

[0040] In one implementation, after the processor obtains image features and point cloud features, it performs bird's-eye view spatial transformation on each.

[0041] For image features, the processor performs geometric projection using the intrinsic and extrinsic parameter matrices of the surround-view camera, transforming the image features from the perspective image coordinate system to the bird's-eye view space to obtain the image's bird's-eye view features. This transformation endows the image features originally existing in the perspective view with the spatial location attributes of the bird's-eye view, eliminating the scale inconsistency problem caused by perspective distortion.

[0042] For point cloud features, the processor performs pooling along the height direction, compressing the 3D point cloud features into a bird's-eye view space to obtain the point cloud bird's-eye view features. This transformation compresses the point cloud features from 3D space to a 2D plane, eliminating the height dimension, and enabling the point cloud features to align with the image bird's-eye view features in the bird's-eye view space.

[0043] After completing the above transformation, the processor stitches the image bird's-eye view features and the point cloud bird's-eye view features in the channel dimension to generate a fused feature containing multimodal information, and then further processes the fused feature through the encoder to generate a shared bird's-eye view feature map.

[0044] S104. Based on the shared bird's-eye view feature map, determine the articulation angle between the tractor and the attached trailer.

[0045] The articulation angle refers to the angle formed in the horizontal plane between the longitudinal axis of the tractor and the longitudinal axis of the trailer. In a semi-trailer tractor, the trailer is articulated to the tractor via a saddle, allowing the trailer to rotate relative to the tractor in the horizontal plane. The sign of the articulation angle indicates the direction of the trailer's deflection relative to the tractor (positive for leftward deflection and negative for rightward deflection), and its magnitude reflects the degree of deflection of the trailer relative to the tractor. It is a key parameter for path planning, trajectory tracking control, and driving safety.

[0046] In one implementation, after obtaining a shared bird's-eye view feature map, the processor determines the articulation angle between the tractor and the attached trailer based on the feature map.

[0047] Specifically, the processor determines the trailer's pose information in the bird's-eye view space based on the shared bird's-eye view feature map, and then calculates the hinge angle based on this pose information according to a preset hinge angle determination strategy. The hinge angle determination strategy can be based on feature regression or geometric projection, both of which use the shared bird's-eye view feature map as a basis for calculating the hinge angle.

[0048] In the feature regression-based approach, the processor extracts the feature vector corresponding to the trailer from the shared bird's-eye view feature map, maps the feature vector through a regression network, and directly outputs the articulation angle.

[0049] In the geometric projection-based approach, the processor determines the region of interest (ROI) corresponding to the trailer's front end in the surround-view image data based on the trailer's pose information in the shared bird's-eye view feature map. Within this ROI, it detects physical feature points at the trailer's front end, obtaining the pixel coordinates of these points in the image coordinate system. The processor then combines the intrinsic and extrinsic parameter matrices of the surround-view camera to project these pixel coordinates onto the vehicle coordinate system, obtaining the three-dimensional coordinates of the physical feature point. Finally, based on the geometric relationship between these three-dimensional coordinates and the fixed three-dimensional coordinates of the tractor's attachment point, it calculates the hinge angle.

[0050] The beneficial effects of the embodiments of the present invention are as follows: Firstly, addressing the limitations of purely visual methods due to their susceptibility to environmental factors such as lighting, rain, fog, and occlusion, this invention introduces a lidar sensor. The lidar actively emits laser beams for detection, and its point cloud acquisition is independent of ambient light. Even in scenarios where purely visual methods are prone to failure, such as at night, in backlight, or in rain and fog, it can still stably provide the trailer's geometric structure information. By fusing the panoramic image with the lidar point cloud in a bird's-eye view space, this invention enhances its adaptability to environmental changes and effectively improves the accuracy of hinge angle estimation.

[0051] Secondly, addressing the limitations of pure vision methods, such as the difficulty of obtaining accurate depth information and the limited precision of monocular cameras, this invention integrates lidar point cloud data and surround view image data. LiDAR can directly provide accurate three-dimensional spatial information. Compared to pure vision methods that require depth estimation through monocular images, the depth information source of this invention is more direct and reliable, fundamentally eliminating the angle error caused by inaccurate depth estimation and effectively improving the estimation accuracy of the hinge angle.

[0052] Thirdly, addressing the issue of decreased accuracy in pure LiDAR methods when trailer outlines are indistinct or point clouds are sparse, this invention integrates image semantic features from a surround-view camera. When the LiDAR point cloud is sparse, the surround-view image still provides rich semantic information such as edges, textures, and shapes; when the trailer outline is indistinct, image features provide additional visual constraints, helping to compensate for insufficient geometric information. Through feature-level fusion in the bird's-eye view space, image semantic information and point cloud geometric information complement each other, enabling this invention to maintain high articulation angle estimation accuracy even when the performance of a single sensor deteriorates, effectively overcoming the inherent defects of pure LiDAR methods.

[0053] Fourthly, addressing the high cost of pure LiDAR solutions, this invention employs a multimodal fusion approach, combining a surround-view camera with LiDAR. Compared to pure LiDAR solutions that rely on high-beam LiDAR to obtain sufficiently dense point clouds for contour fitting, the multimodal fusion architecture of this invention allows for the use of LiDAR with lower performance requirements. The surround-view image provides supplementary information when the point cloud is sparse, thereby reducing the requirements for LiDAR beamwidth and performance, and effectively controlling overall hardware costs. Furthermore, the sensor configuration of this invention does not require modification to the mechanical connection structure between the tractor and trailer, making it easy to deploy on mass-produced commercial vehicles.

[0054] Optional, also includes: S1. Based on the shared bird's-eye view feature map, the detection results are output through the 3D target detection head.

[0055] The 3D object detection head refers to a network module that performs 3D object detection based on a shared bird's-eye view feature map. It is a specific network layer in the deep learning model responsible for outputting the 3D object detection results. This detection head takes the shared bird's-eye view feature map as input, processes the feature map through convolutional layers or fully connected layers, and outputs detection results including the 3D bounding box of the object and its attribute information.

[0056] In this embodiment, the detection results include the three-dimensional bounding box of the trailer and the corresponding geometric center position, size, and orientation angle of the three-dimensional bounding box.

[0057] A 3D bounding box is the smallest rectangular box that encloses a target object in 3D space. Unlike 2D bounding boxes, which only provide the pixel position and extent of the target in the image plane, 3D bounding boxes provide the length, width, height, spatial position, and orientation of the target in the vehicle coordinate system, and can completely describe the area occupied and pose of the target in 3D space.

[0058] The geometric center position refers to the three-dimensional coordinates (X, Y, Z) of the geometric symmetry center of the three-dimensional bounding box in the vehicle coordinate system. This position represents the overall spatial position of the trailer.

[0059] Dimensions refer to the size of the three-dimensional bounding box in the three dimensions of length, width, and height, corresponding to the trailer's length, width, and height, respectively. Dimensional information is used to describe the physical size of the trailer in three-dimensional space.

[0060] The heading angle refers to the orientation angle of the trailer's longitudinal axis in the horizontal plane, reflecting the overall direction of the trailer in the horizontal plane. This heading angle is related to but not the same as the articulation angle—the heading angle is the absolute orientation of the trailer in the vehicle coordinate system, while the articulation angle is the relative deflection angle of the trailer's orientation relative to the longitudinal axis of the tractor.

[0061] In one implementation, after obtaining the shared bird's-eye view feature map, the processor inputs the shared bird's-eye view feature map into the 3D target detection head. The 3D target detection head uses the shared bird's-eye view feature map as input and employs a joint prediction method combining a center point heatmap and a regression head to output the overall 3D bounding box of the trailer, as well as the geometric center position, size, and orientation angle of the corresponding 3D bounding box.

[0062] Specifically, the center point heatmap branch in the 3D object detection head is used to predict the position of the trailer's geometric center on the shared bird's-eye view feature map, while the regression head branch regresses the trailer's size and orientation angle at that center position. Through the above processing, the processor obtains the trailer's 3D bounding box and its detection results, including the geometric center position, size, and orientation angle.

[0063] S2. Output the detection results to the collision detection module so that the collision detection module can calculate the sweep radius of the trailer and determine the collision risk between the trailer and the obstacle based on the detection results; and / or, output the detection results to the path planning module so that the path planning module can construct a kinematic model of the trailer based on the detection results.

[0064] The collision detection module is a functional module used to determine whether the trailer collides with surrounding obstacles during its movement.

[0065] The sweep radius refers to the maximum radius of the arc swept by the rear of the trailer relative to the tractor's engagement point during reversing or turning. The size of the sweep radius depends on the trailer's length, width, and current articulation angle, and is a key parameter for measuring the safe space required when the trailer is turning or reversing. A larger sweep radius means more space is needed when the trailer is turning or reversing, and a higher risk of collision with surrounding obstacles.

[0066] Collision risk refers to the safety assessment result obtained by comprehensively judging the trailer's three-dimensional bounding box, sweep radius, and obstacle position information. Collision risk is used to guide vehicle path planning and trajectory control to ensure driving safety.

[0067] The path planning module is a functional module used to generate a safe driving trajectory for the tractor and trailer combination.

[0068] A trailer kinematic model is a mathematical model used to describe the changes in position and posture of the tractor and trailer assembly during motion. Based on parameters such as trailer dimensions, attachment point position, and articulation angle, this model describes the changes in the trailer's trajectory at different articulation angles and vehicle speeds, and is the foundation for path planning and trajectory tracking control.

[0069] In one implementation, the processor outputs the detection results from the 3D target detection head to the collision detection module and / or path planning module for use by downstream modules.

[0070] The detection results include the trailer's three-dimensional bounding box, as well as the geometric center position, size, and orientation angle of the three-dimensional bounding box.

[0071] The processor outputs the detection results to the collision detection module. The collision detection module calculates the sweep radius of the trailer during its movement based on the trailer's three-dimensional bounding box, its geometric center position, size, and orientation angle. It also combines the position information of surrounding obstacles to comprehensively judge the collision risk between the trailer and the obstacles.

[0072] The processor can also output the detection results to the path planning module. The path planning module constructs a kinematic model of the trailer based on the trailer's three-dimensional bounding box, its geometric center position, size, and orientation angle, combined with the current hinge angle. This model is used to describe the pose change law of the tractor and trailer combination during the motion process, providing model support for path planning and trajectory tracking control.

[0073] The processor integrates the outputs of the collision detection module and the path planning module to generate a safe and feasible driving trajectory, and outputs it to the downstream vehicle control module for execution.

[0074] By using a shared bird's-eye view feature map, a 3D target detection head outputs detection results. These results include the trailer's 3D bounding box, its corresponding geometric center position, size, and orientation angle. The detection results are then output to a collision detection module, which calculates the trailer's sweep radius and assesses the collision risk between the trailer and obstacles. Alternatively, the detection results can be output to a path planning module, enabling it to construct a kinematic model of the trailer based on the detection results. The beneficial effects are: Firstly, the collision detection module calculates the sweep radius and assesses collision risk based on the trailer's 3D bounding box, accurately evaluating the safe distance between the trailer and surrounding obstacles during movement. Because the 3D bounding box precisely describes the trailer's size and orientation, the sweep radius calculation is more accurate, effectively preventing collisions caused by inaccurate estimates of trailer size or orientation, thus improving the safety of reversing and parking.

[0075] Secondly, the path planning module constructs a kinematic model of the trailer based on its three-dimensional bounding box, providing precise kinematic constraints for path planning and trajectory tracking control. Compared to relying solely on the hinge angle for trajectory prediction, the kinematic model incorporating trailer dimensions can more accurately describe the trailer's trajectory, making the planned driving trajectory safer and more feasible, and improving the accuracy and stability of control.

[0076] Optionally, the bird's-eye view features of the image are fused with the bird's-eye view features of the point cloud to generate a shared bird's-eye view feature map, including: The bird's-eye view features of the image and the bird's-eye view features of the point cloud are stitched together in the channel dimension to generate a multi-channel feature map; the multi-channel feature map is input into the bird's-eye view encoder, and after processing by the bird's-eye view encoder, a shared bird's-eye view feature map is output.

[0077] In this context, a multi-channel feature map refers to a feature map containing multiple information channels at each spatial location in a bird's-eye view space. Some channels store image bird's-eye view features, while others store point cloud bird's-eye view features. The multi-channel feature map concatenates these two modalities along the channel dimension, ensuring that each grid location in the bird's-eye view simultaneously possesses both image semantics and point cloud geometric information.

[0078] A bird's-eye view encoder is a neural network module used to further extract and fuse features from stitched multi-channel feature maps. This encoder performs convolutional processing on the multi-channel feature maps through convolutional layers, extracting cross-modal fusion features. This allows the semantic information of the image modality and the geometric information of the point cloud modality to fully interact and complement each other in the same space, ultimately outputting a shared bird's-eye view feature map.

[0079] In one implementation, after the processor obtains the bird's-eye view features of the image and the bird's-eye view features of the point cloud, it performs a fusion process on the two types of features.

[0080] First, the processor concatenates the image bird's-eye view features with the point cloud bird's-eye view features along the channel dimension. The image bird's-eye view features and the point cloud bird's-eye view features have the same spatial dimensions (i.e., the resolution of the bird's-eye view grid is consistent), but the number of channels differs. Through channel concatenation, the processor generates a multi-channel feature map that simultaneously contains semantic information from the image modality and geometric information from the point cloud modality at each spatial location.

[0081] The processor then inputs the multi-channel feature map into the bird's-eye view encoder. The bird's-eye view encoder extracts and fuses features from the multi-channel feature map through convolution operations, enabling image features and point cloud features to fully interact in a unified bird's-eye view space, generating a fused bird's-eye view feature representation.

[0082] Finally, the bird's-eye view encoder outputs a shared bird's-eye view feature map, which contains both semantic information from the surround-view camera and geometric information from the LiDAR.

[0083] This invention achieves deep feature-level fusion of data from two heterogeneous sensors, surround-view cameras and LiDAR, by concatenating image and point cloud bird's-eye view features along the channel dimension and then fusing them using a bird's-eye view encoder. The channel concatenation operation is simple and efficient, preserving the integrity of each modal feature while achieving spatial alignment. The bird's-eye view encoder further extracts cross-modal fusion features through convolution operations, enabling full interaction and complementarity between image semantic information and point cloud geometric information in a unified space. Compared to earlier fusion or decision-level fusion, the feature-level fusion employed in this invention achieves more comprehensive information interaction at the feature level, effectively overcoming the performance limitations of a single sensor in complex environments. The resulting shared bird's-eye view feature map provides a unified and rich basic feature representation for multiple downstream tasks such as trailer detection and articulation angle determination, avoiding redundant calculations and improving the overall efficiency and perception accuracy of the system.

[0084] Example 2 Figure 2 This is a flowchart illustrating another method for determining the articulation angle between a tractor and a trailer, provided in Embodiment 2 of the present invention. It further optimizes and expands the above technical solution and can be combined with the various optional embodiments described above. For example... Figure 2 As shown, the method includes: S201. Surround view image data is collected by a surround view camera installed on the tractor, and laser point cloud data is collected by a lidar installed on the tractor.

[0085] S202. Perform feature extraction on the panoramic image data to obtain image features, and perform feature extraction on the laser point cloud data to obtain point cloud features.

[0086] S203. Convert the image features and point cloud features to the bird's-eye view space respectively to obtain the image bird's-eye view features and the point cloud bird's-eye view features, and fuse the image bird's-eye view features and the point cloud bird's-eye view features to generate a shared bird's-eye view feature map.

[0087] S204. Locate the spatial position of the trailer in the shared bird's-eye view feature map, determine the feature region corresponding to the trailer, and extract the feature vector corresponding to the feature region from the shared bird's-eye view feature map.

[0088] The spatial location of the trailer refers to its coordinate position in the bird's-eye view space, usually represented by the two-dimensional coordinates (X, Y) of the trailer's geometric center or bottom center in the bird's-eye view space. This position reflects the overall distribution area of ​​the trailer in the bird's-eye view space.

[0089] The feature region corresponding to a trailer refers to the feature map region in the shared bird's-eye view feature map that corresponds to the spatial location of the trailer. The shared bird's-eye view feature map is a two-dimensional feature map, with each location corresponding to a feature vector. The feature region corresponding to a trailer is the feature map region occupied by the trailer, and the feature vector within this region contains the semantic and geometric information of the trailer.

[0090] The feature vector corresponding to the feature region refers to the feature vector extracted from the feature region corresponding to the trailer in the shared bird's-eye view feature map, which is used to characterize the trailer's attitude. This feature vector integrates the semantic information of the image modality and the geometric information of the point cloud modality, and is used for subsequent hinge angle determination processing.

[0091] In one implementation, after obtaining a shared bird's-eye view feature map, the processor locates the spatial position of the trailer within the feature map. Specifically, the shared bird's-eye view feature map is a two-dimensional grid structure, where each grid position corresponds to a feature vector containing semantic and geometric information of that position. The processor performs object detection processing on the shared bird's-eye view feature map to determine the position of the trailer's geometric center within the bird's-eye view grid, which is then used as the trailer's spatial position.

[0092] Based on the trailer's spatial location, the processor determines the corresponding feature region in the shared bird's-eye view feature map. This feature region covers the grid area occupied by the trailer in the bird's-eye view space, and its size is related to the trailer's dimensions. The processor extracts all feature vectors within this feature region from the shared bird's-eye view feature map, using them as the corresponding feature vectors for subsequent articulation angle determination processing.

[0093] S205. Input the feature vector into the angle regression head, perform nonlinear mapping on the feature vector through the angle regression head, output at least one angle regression value, and determine the hinge angle based on at least one angle regression value.

[0094] The angle regression head is a neural network module used to map feature vectors to angle values. It typically consists of fully connected layers, taking the feature vector corresponding to the trailer as input and outputting the angle regression value. The core function of this module is to map the semantic and geometric information in the high-dimensional feature vectors into continuous angle values ​​through nonlinear transformation, achieving an end-to-end regression mapping from features to angles.

[0095] Nonlinear mapping refers to the process of applying a nonlinear transformation to input features using the activation function of a neural network. Nonlinear mapping enables neural networks to fit complex nonlinear functional relationships, that is, complex mappings from high-dimensional feature vectors to angular values.

[0096] Angle regression value refers to the numerical result output by the angle regression head, used to characterize the size and direction of the hinge angle. The angle regression head can output a single angle regression value directly as the hinge angle, or it can output multiple angle regression values ​​(such as sine and cosine values, multiple candidate angles, etc.). The processor obtains the final hinge angle by post-processing multiple angle regression values.

[0097] In one implementation, after obtaining the feature vector corresponding to the trailer, the processor inputs the feature vector into the angle regression head. The angle regression head consists of fully connected layers, which perform non-linear mapping on the feature vector, mapping the high-dimensional feature vector into angle regression values ​​and outputting them.

[0098] The angle regression head can output at least one angle regression value. When a single angle regression value is output, the processor directly uses that value as the hinge angle. When multiple angle regression values ​​are output, the processor processes the values ​​according to a preset post-processing strategy (such as averaging, weighted fusion, or inverse trigonometric function calculation) to determine the final hinge angle.

[0099] This invention achieves end-to-end regression determination of hinge angles by performing nonlinear mapping on feature vectors and outputting angle regression values ​​through an angle regression head. Compared to traditional geometric calculation methods, the deep learning-based regression method can implicitly learn the complex nonlinear relationship between features and angles, fully utilize multimodal information in shared bird's-eye view feature maps, avoid manually designing geometric calculation rules, and has higher accuracy and adaptability. Furthermore, the regression head's calculation process is simple and efficient, meeting the real-time requirements of autonomous driving scenarios.

[0100] Example 3 Figure 3 This is a flowchart illustrating another method for determining the articulation angle between a tractor and a trailer, provided in Embodiment 3 of the present invention. It further optimizes and expands the above technical solution and can be combined with the various optional embodiments described above. For example... Figure 3 As shown, the method includes: S301. Based on the shared bird's-eye view feature map, determine the position and orientation of the trailer in the bird's-eye view space, and based on the position and orientation of the trailer in the bird's-eye view space, determine the corresponding region of interest in the surround view image data.

[0101] Here, "location" refers to the trailer's coordinate position in bird's-eye view space, typically represented by the two-dimensional coordinates of the trailer's geometric center or bottom center in bird's-eye view space. This location reflects the overall distribution area of ​​the trailer in bird's-eye view space and is the basis for subsequently determining the region of interest.

[0102] Orientation refers to the direction in which the longitudinal axis of the trailer points in the horizontal plane, reflecting the overall attitude of the trailer in the horizontal plane. The trailer's orientation is directly related to the articulation angle—the relative angle of the trailer's orientation to the longitudinal axis of the tractor. Once the trailer's orientation is determined, the articulation angle can be calculated by combining this with the tractor's own orientation.

[0103] The region of interest (ROI) refers to a local image area in the panoramic image data that may contain the front end of the trailer, determined by mapping the trailer's position and orientation in the bird's-eye view space back to the image plane.

[0104] In one implementation, the processor determines the trailer's position and orientation in the bird's-eye view space based on a shared bird's-eye view feature map. Specifically, the shared bird's-eye view feature map is a two-dimensional feature map, where each position corresponds to a feature vector containing semantic and geometric information about that position. The processor determines the trailer's position and orientation in the bird's-eye view space by performing object detection processing on the shared bird's-eye view feature map.

[0105] Based on the trailer's position and orientation in the bird's-eye view space, and combined with the intrinsic and extrinsic parameter matrix of the surround-view cameras, the processor maps the trailer's front-end position back from the bird's-eye view space to the surround-view image data, and determines the corresponding region of interest (ROI) within the surround-view image data. The ROI covers the area where the trailer's front-end might appear in the surround-view image.

[0106] S302. Detect the visual target set at the front end of the trailer within the region of interest, obtain the pixel coordinates of the geometric center of the visual target in the image coordinate system, and use the pixel coordinates as the pixel coordinates of the physical feature points at the front end of the trailer.

[0107] The visual target refers to an artificial marker with a specific pattern or shape, pre-set on the front surface of the trailer, used to assist the visual inspection system in identifying and locating the trailer. In this embodiment of the invention, the visual target is preferably a rectangular checkerboard pattern, whose regular corner distribution facilitates high-precision detection by image processing algorithms.

[0108] The geometric center refers to the geometric center point of a visual target pattern. For a rectangular checkerboard target, its geometric center is the intersection of the two diagonals of the rectangular area, or the average position of all checkerboard corner points. The geometric center has definite pixel coordinates in the image coordinate system and is a key reference point for subsequent coordinate projection and geometric calculations.

[0109] A physical feature point refers to an actual physical point on the front face of a trailer used to characterize the spatial position of the trailer. In this embodiment of the invention, the geometric center of a visual target is used as the image representation of the physical feature point—that is, the position of the physical feature point at the front face of the trailer in the image is represented by detecting the pixel coordinates of the target's geometric center in the image. The relative positional relationship between this physical feature point and the tractor's attachment point (center of the towing ball) is used to calculate the articulation angle.

[0110] In one implementation, after the processor determines the region of interest in the surround view image data, it detects a visual target positioned at the front end of the trailer within the region of interest.

[0111] The processor first preprocesses the panoramic image within the region of interest (e.g., grayscale conversion, binarization), and then extracts the pixel coordinates of all corner points of the visual target in the image using a checkerboard corner detection algorithm. Based on the extracted corner pixel coordinates, the processor calculates their average value to obtain the pixel coordinates of the geometric center of the visual target in the image coordinate system.

[0112] Since the visual target is fixedly mounted on the front surface of the trailer, its geometric center has a preset relative positional relationship with the physical feature points of the trailer front (obtained through pre-calibration). The processor uses the pixel coordinates of the detected target geometric center as the pixel coordinates of the physical feature points of the trailer front for subsequent coordinate projection and hinge angle calculation.

[0113] S303. By combining the intrinsic and extrinsic parameter matrices of the surround-view camera, the pixel coordinates are projected onto the vehicle coordinate system to obtain the three-dimensional coordinates of the physical feature points.

[0114] In one implementation, after obtaining the pixel coordinates of the physical feature point in the image coordinate system, the processor inputs these pixel coordinates to the coordinate transformation module. The coordinate transformation module pre-stores the intrinsic and extrinsic parameter matrices of the surround-view camera. The intrinsic parameter matrix describes the internal imaging parameters of the surround-view camera (such as focal length and optical center position), while the extrinsic parameter matrix describes the rotation and translation relationship of the surround-view camera relative to the vehicle coordinate system. The processor uses depth information combined with the intrinsic and extrinsic parameter matrices to transform the pixel coordinates from the image coordinate system to the vehicle coordinate system using a coordinate transformation formula, obtaining the three-dimensional coordinates of the physical feature point in the vehicle coordinate system. These three-dimensional coordinates reflect the precise position of the physical feature point at the front end of the trailer in real three-dimensional space and are used for subsequent geometric calculations of the hinge angle.

[0115] Optionally, by combining the intrinsic and extrinsic parameter matrices of the surround-view camera, the pixel coordinates are projected onto the vehicle coordinate system to obtain the three-dimensional coordinates of the physical feature points, including: The depth information of physical feature points is obtained using laser point cloud data; based on the depth information, intrinsic parameter matrix, and extrinsic parameter matrix, the pixel coordinates are transformed from the image coordinate system to the vehicle coordinate system to obtain the three-dimensional coordinates of the physical feature points.

[0116] In this invention, depth information refers to the distance of a target point relative to the surround-view camera. In an image coordinate system, a single pixel coordinate only determines that the point lies on a ray in space, but cannot uniquely determine its exact location. Only by supplementing with depth information can the pixel coordinate be uniquely converted into a coordinate point in three-dimensional space. In this embodiment, the depth information is provided by laser point cloud data. The lidar directly measures the distance by emitting laser pulses and receiving reflected signals, resulting in high ranging accuracy and independence from ambient light.

[0117] The vehicle coordinate system refers to a three-dimensional spatial coordinate system established with the tractor as the reference datum. The vehicle coordinate system is a common reference datum for the unified alignment of all sensor data. Through a pre-calibrated extrinsic parameter matrix, data collected by surround-view cameras and lidar can be unified under this coordinate system, achieving spatial alignment of multimodal data.

[0118] In one implementation, after the processor obtains the pixel coordinates of the physical feature points in the image coordinate system, it needs to convert the pixel coordinates into three-dimensional coordinates in the vehicle coordinate system.

[0119] Specifically, the processor spatially aligns the laser point cloud data with the surround-view image data. Since the LiDAR and surround-view camera have been pre-calibrated jointly, their extrinsic parameter matrices are known, allowing the processor to project the laser point cloud data onto the image plane containing the surround-view image data. Near the pixel coordinates of a physical feature point, the processor searches for the nearest neighbor laser point cloud and uses its depth value as the depth information of the physical feature point. If multiple laser point clouds exist within a preset search range, the processor can determine the depth value using interpolation or a weighted average method.

[0120] After obtaining depth information, the processor combines the intrinsic and extrinsic parameter matrices of the surround-view camera and transforms the pixel coordinates from the image coordinate system to the vehicle coordinate system through coordinate transformation to obtain the three-dimensional coordinates of the physical feature points in the vehicle coordinate system.

[0121] This invention achieves the following beneficial effects by acquiring depth information through laser point cloud data and transforming pixel coordinates to the vehicle coordinate system using the intrinsic and extrinsic parameter matrices of a surround-view camera: First, compared to pure vision methods that rely on monocular images for depth estimation, lidar directly measures distance, making the source of depth information more direct and reliable, thus eliminating coordinate errors caused by inaccurate depth estimation at the source; Second, lidar ranging is not affected by ambient lighting conditions, and can still stably provide accurate depth information in scenarios where pure vision methods are prone to failure, such as at night, in backlight, or in rain and fog; Third, by accurately converting pixel coordinates to three-dimensional coordinates in the vehicle coordinate system, precise coordinate input is provided for subsequent geometric calculations of the hinge angle, effectively improving the estimation accuracy of the hinge angle.

[0122] S304. Obtain the fixed three-dimensional coordinates of the tractor's coupling point in the vehicle coordinate system, and calculate the hinge angle based on the three-dimensional coordinates of the physical feature points and the fixed three-dimensional coordinates of the tractor's coupling point.

[0123] The tractor-trailer engagement point refers to the hinge point where the rear of the tractor unit connects to the trailer, specifically the center point of the towing ball (saddle). This point is the only physical connection between the tractor and trailer, and the trailer rotates relative to the tractor unit in the horizontal plane around this point. The position of the tractor-trailer engagement point in the vehicle coordinate system is fixed and does not change with vehicle movement or trailer attitude; it serves as the reference point for calculating the articulation angle.

[0124] Fixed 3D coordinates refer to the three-dimensional coordinate values ​​of the tractor's attachment point in the vehicle coordinate system. Since the tractor's attachment point is fixed at the rear of the tractor, its position relative to the tractor body remains constant; therefore, these coordinate values ​​remain fixed in the vehicle coordinate system. These fixed 3D coordinates can be pre-acquired through measurement or calibration and stored in the system for later retrieval.

[0125] In one implementation, the processor acquires the fixed three-dimensional coordinates of the tractor's coupling point in the vehicle coordinate system. These fixed three-dimensional coordinates are pre-obtained through measurement or calibration, stored in the system, and can be directly read and retrieved by the processor. These coordinates are fixed values ​​in the vehicle coordinate system and do not change with vehicle movement or trailer posture.

[0126] The processor calculates the horizontal vector between the three-dimensional coordinates of the physical feature points and the fixed three-dimensional coordinates of the tractor's attachment point. This horizontal vector reflects the offset direction and amount of the trailer's front end relative to the tractor's attachment point. Since the hinge angle is defined as the deflection angle of the trailer's longitudinal axis relative to the tractor's longitudinal axis, and the direction of the trailer's longitudinal axis can be characterized by the vector direction of the physical feature points of the trailer's front end relative to the tractor's attachment point, the processor calculates the hinge angle accordingly.

[0127] This invention determines the trailer's position and orientation based on a shared bird's-eye view feature map and maps it to a surround-view image to define the region of interest (ROI). This reduces the target detection range from the entire image to a local area, reducing computational load and improving detection efficiency and real-time performance. Simultaneously, the ROI localization is based on multimodal fusion perception results, remaining accurate and reliable even in complex environments such as varying lighting and occlusion, effectively reducing background interference and false detections, and improving the accuracy and stability of target detection. Furthermore, by projecting the pixel coordinates of the target's geometric center onto the vehicle coordinate system using the intrinsic and extrinsic parameter matrices of the surround-view camera, the three-dimensional coordinates of the physical feature points are obtained. Compared to pure vision methods that rely on monocular depth estimation, this invention provides a more direct and reliable source of depth information, effectively eliminating angle errors caused by inaccurate depth estimation and fundamentally improving the accuracy of articulation angle estimation. This provides a reliable articulation angle input for subsequent path planning and trajectory tracking control.

[0128] Optionally, the articulation angle is calculated based on the three-dimensional coordinates of the physical feature point and the fixed three-dimensional coordinates of the tractor attachment point, including: Obtain the first component of the three-dimensional coordinates of the physical feature point in the X direction and the second component in the Y direction in the horizontal plane of the vehicle coordinate system; obtain the third component of the fixed three-dimensional coordinates of the tractor attachment point in the X direction and the fourth component in the Y direction in the horizontal plane of the vehicle coordinate system; calculate the first difference between the first and third components and the second difference between the second and fourth components; and obtain the hinge angle by performing a ratio operation on the first and second differences using the arctangent function.

[0129] In the vehicle coordinate system, the horizontal plane refers to the horizontal plane formed by the X-axis and Y-axis, specifically the plane where the Z-axis coordinate is zero. In the vehicle coordinate system, the X-axis points in the direction of vehicle movement, the Y-axis points to the left of the vehicle, and the Z-axis points vertically upwards; the horizontal plane is parallel to the ground. The articulation angle is the deflection angle of the trailer relative to the tractor in the horizontal plane. Therefore, the calculation only needs to consider the X and Y coordinate components of the physical feature points and the attachment point in the horizontal plane; the Z-axis component is not included in the articulation angle calculation.

[0130] The first component refers to the three-dimensional coordinates of the physical feature point in the X-direction within the horizontal plane of the vehicle coordinate system. The second component refers to the three-dimensional coordinates of the physical feature point in the Y-direction within the horizontal plane of the vehicle coordinate system. The third component refers to the fixed three-dimensional coordinates of the tractor's coupling point in the X-direction within the horizontal plane of the vehicle coordinate system. The fourth component refers to the fixed three-dimensional coordinates of the tractor's coupling point in the Y-direction within the horizontal plane of the vehicle coordinate system.

[0131] In one implementation, after the processor obtains the three-dimensional coordinates of the physical feature point and the fixed three-dimensional coordinates of the tractor's attachment point, it extracts the X-direction and Y-direction components in the horizontal plane of the vehicle coordinate system from the two three-dimensional coordinates. The three-dimensional coordinates of the physical feature point correspond to the first component (X-direction) and the second component (Y-direction), and the fixed three-dimensional coordinates of the tractor's attachment point correspond to the third component (X-direction) and the fourth component (Y-direction).

[0132] The processor calculates the difference between the first and third components to obtain the offset of the physical feature point and the tractor attachment point in the horizontal X direction; it calculates the difference between the second and fourth components to obtain the offset of the two in the horizontal Y direction. The two offsets constitute a horizontal vector from the tractor attachment point to the physical feature point.

[0133] The processor calculates the ratio of the two offsets of the horizontal vector using the arctangent function to obtain the deflection angle of the horizontal vector relative to the longitudinal axis of the tractor, and uses this deflection angle as the hinge angle.

[0134] This invention extracts the X and Y coordinate components of physical feature points and tractor attachment points in the horizontal plane, calculates the difference, and then obtains the hinge angle through arctangent function ratio calculation. The geometric meaning is clear, the calculation process is simple and efficient, and it can meet the real-time requirements of hinge angle estimation in autonomous driving scenarios.

[0135] Example 4 Figure 4 This is a schematic diagram of a device for determining the articulation angle between a tractor and a trailer, provided in Embodiment 4 of the present invention. It is applicable to real-time estimation of the articulation angle in scenarios where a semi-trailer tractor is reversing or parking. Figure 4 As shown, the device includes: The data acquisition module 41 is used to acquire surround view image data through the surround view camera installed on the tractor and to acquire laser point cloud data through the lidar installed on the tractor. The feature extraction module 42 is used to extract features from the panoramic image data to obtain image features, and to extract features from the laser point cloud data to obtain point cloud features. The shared bird's-eye view feature map generation module 43 is used to convert the image features and the point cloud features to the bird's-eye view space respectively to obtain image bird's-eye view features and point cloud bird's-eye view features, and to fuse the image bird's-eye view features and the point cloud bird's-eye view features to generate a shared bird's-eye view feature map. The articulation angle determination module 44 is used to determine the articulation angle between the tractor and the attached trailer based on the shared bird's-eye view feature map.

[0136] Optionally, the hinge angle determining module 44 is specifically used for: The spatial location of the trailer is located in the shared bird's-eye view feature map, the feature region corresponding to the trailer is determined, and the feature vector corresponding to the feature region is extracted from the shared bird's-eye view feature map. The feature vector is input into the angle regression head, and the feature vector is non-linearly mapped through the angle regression head to output at least one angle regression value; The hinge angle is determined based on the at least one angle regression value.

[0137] Optionally, the hinge angle determining module 44 is further used for: Based on the shared bird's-eye view feature map, the position and orientation of the trailer in the bird's-eye view space are determined, and based on the position and orientation of the trailer in the bird's-eye view space, the corresponding region of interest is determined in the surround view image data; A visual target set at the front end of the trailer is detected in the region of interest, and the pixel coordinates of the geometric center of the visual target in the image coordinate system are obtained. The pixel coordinates are then used as the pixel coordinates of the physical feature points at the front end of the trailer. By combining the intrinsic and extrinsic parameter matrices of the surround-view camera, the pixel coordinates are projected onto the vehicle coordinate system to obtain the three-dimensional coordinates of the physical feature points; Obtain the fixed three-dimensional coordinates of the tractor's coupling point in the vehicle coordinate system, and calculate the hinge angle based on the three-dimensional coordinates of the physical feature point and the fixed three-dimensional coordinates of the tractor's coupling point.

[0138] Optionally, the hinge angle determining module 44 is further used for: The depth information of the physical feature points is obtained using the laser point cloud data; Based on the depth information, the intrinsic parameter matrix, and the extrinsic parameter matrix, the pixel coordinates are transformed from the image coordinate system to the vehicle coordinate system to obtain the three-dimensional coordinates of the physical feature point.

[0139] Optionally, the hinge angle determining module 44 is further used for: Obtain the first component of the three-dimensional coordinates of the physical feature point in the X direction and the second component in the Y direction in the horizontal plane of the vehicle coordinate system; Obtain the third component of the fixed three-dimensional coordinates of the tractor attachment point in the horizontal plane of the vehicle coordinate system in the horizontal direction and the fourth component in the horizontal direction. Calculate the first difference between the first component and the third component, and the second difference between the second component and the fourth component; The hinge angle is obtained by performing a ratio operation on the first difference and the second difference using the arctangent function.

[0140] Optionally, the device further includes a three-dimensional target detection module, specifically used for: Based on the shared bird's-eye view feature map, the detection results are output through a 3D target detection head; wherein, the detection results include the 3D bounding box of the trailer and the geometric center position, size and orientation angle of the corresponding 3D bounding box; The detection results are output to the collision detection module, so that the collision detection module can calculate the sweep radius of the trailer and determine the collision risk between the trailer and the obstacle based on the detection results; and / or, the detection results are output to the path planning module, so that the path planning module can construct a kinematic model of the trailer based on the detection results.

[0141] Optionally, the shared bird's-eye view feature map generation module 43 is specifically used for: The bird's-eye view features of the image and the bird's-eye view features of the point cloud are stitched together in the channel dimension to generate a multi-channel feature map; The multi-channel feature map is input into the bird's-eye view encoder, and the shared bird's-eye view feature map is output after processing by the bird's-eye view encoder.

[0142] The device for determining the articulation angle between the tractor and the trailer provided in this embodiment of the invention can execute the method for determining the articulation angle between the tractor and the trailer provided in this embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method.

[0143] According to embodiments of this disclosure, embodiments of the present invention also provide an electronic device, a readable storage medium, and a computer program product.

[0144] Figure 5 A schematic diagram of an electronic device 50 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0145] like Figure 5As shown, the electronic device 50 includes at least one processor 51 and a memory, such as a read-only memory (ROM) 52 and a random access memory (RAM) 53, communicatively connected to the at least one processor 51. The memory stores computer programs executable by the at least one processor. The processor 51 can perform various appropriate actions and processes based on the computer program stored in the ROM 52 or loaded from storage unit 58 into the RAM 53. The RAM 53 can also store various programs and data required for the operation of the electronic device 50. The processor 51, ROM 52, and RAM 53 are interconnected via a bus 54. An input / output (I / O) interface 55 is also connected to the bus 54.

[0146] Multiple components in electronic device 50 are connected to I / O interface 55, including: input unit 56, such as keyboard, mouse, etc.; output unit 57, such as various types of monitors, speakers, etc.; storage unit 58, such as disk, optical disk, etc.; and communication unit 59, such as network card, modem, wireless transceiver, etc. Communication unit 59 allows electronic device 50 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0147] Processor 51 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 51 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 51 performs the various methods and processes described above, such as the method for determining the articulation angle between the tractor and the trailer.

[0148] In some embodiments, the method for determining the articulation angle between the tractor and the trailer may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 58. In some embodiments, part or all of the computer program may be loaded into and / or installed on electronic device 50 via ROM 52 and / or communication unit 59. When the computer program is loaded into RAM 53 and executed by processor 51, one or more steps of the method for determining the articulation angle between the tractor and the trailer described above may be performed. Alternatively, in other embodiments, processor 51 may be configured to perform the method for determining the articulation angle between the tractor and the trailer by any other suitable means (e.g., by means of firmware).

[0149] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0150] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0151] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0152] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0153] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0154] A computing system can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product within the cloud computing service system to address the shortcomings of traditional physical hosts and virtual private servers, such as high management difficulty and weak business scalability.

[0155] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0156] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for determining the articulation angle between a tractor and a trailer, characterized in that, include: Surround view image data is collected by a surround view camera installed on the tractor, and laser point cloud data is collected by a lidar installed on the tractor. Feature extraction is performed on the panoramic image data to obtain image features, and feature extraction is performed on the laser point cloud data to obtain point cloud features; The image features and the point cloud features are respectively converted to the bird's-eye view space to obtain image bird's-eye view features and point cloud bird's-eye view features. The image bird's-eye view features and the point cloud bird's-eye view features are then fused to generate a shared bird's-eye view feature map. Based on the shared bird's-eye view feature map, the articulation angle between the tractor and the attached trailer is determined.

2. The method according to claim 1, characterized in that, Determining the articulation angle between the tractor and the attached trailer based on the shared bird's-eye view feature map includes: The spatial location of the trailer is located in the shared bird's-eye view feature map, the feature region corresponding to the trailer is determined, and the feature vector corresponding to the feature region is extracted from the shared bird's-eye view feature map. The feature vector is input into the angle regression head, and the feature vector is non-linearly mapped through the angle regression head to output at least one angle regression value; The hinge angle is determined based on the at least one angle regression value.

3. The method according to claim 1, characterized in that, Determining the articulation angle between the tractor and the attached trailer based on the shared bird's-eye view feature map includes: Based on the shared bird's-eye view feature map, the position and orientation of the trailer in the bird's-eye view space are determined, and based on the position and orientation of the trailer in the bird's-eye view space, the corresponding region of interest is determined in the surround view image data; A visual target set at the front end of the trailer is detected in the region of interest, and the pixel coordinates of the geometric center of the visual target in the image coordinate system are obtained. The pixel coordinates are then used as the pixel coordinates of the physical feature points at the front end of the trailer. By combining the intrinsic and extrinsic parameter matrices of the surround-view camera, the pixel coordinates are projected onto the vehicle coordinate system to obtain the three-dimensional coordinates of the physical feature points; Obtain the fixed three-dimensional coordinates of the tractor's coupling point in the vehicle coordinate system, and calculate the hinge angle based on the three-dimensional coordinates of the physical feature point and the fixed three-dimensional coordinates of the tractor's coupling point.

4. The method according to claim 3, characterized in that, The step of combining the intrinsic and extrinsic parameter matrices of the surround-view camera to project the pixel coordinates onto the vehicle coordinate system to obtain the three-dimensional coordinates of the physical feature points includes: The depth information of the physical feature points is obtained using the laser point cloud data; Based on the depth information, the intrinsic parameter matrix, and the extrinsic parameter matrix, the pixel coordinates are transformed from the image coordinate system to the vehicle coordinate system to obtain the three-dimensional coordinates of the physical feature point.

5. The method according to claim 3, characterized in that, The step of calculating the articulation angle based on the three-dimensional coordinates of the physical feature point and the fixed three-dimensional coordinates of the tractor attachment point includes: Obtain the first component of the three-dimensional coordinates of the physical feature point in the X direction and the second component in the Y direction in the horizontal plane of the vehicle coordinate system; Obtain the third component of the fixed three-dimensional coordinates of the tractor attachment point in the horizontal plane of the vehicle coordinate system in the horizontal direction and the fourth component in the horizontal direction. Calculate the first difference between the first component and the third component, and the second difference between the second component and the fourth component; The hinge angle is obtained by performing a ratio operation on the first difference and the second difference using the arctangent function.

6. The method according to claim 1, characterized in that, Also includes: Based on the shared bird's-eye view feature map, the detection results are output through a 3D target detection head; wherein, the detection results include the 3D bounding box of the trailer and the geometric center position, size and orientation angle of the corresponding 3D bounding box; The detection results are output to the collision detection module, so that the collision detection module can calculate the sweep radius of the trailer and determine the collision risk between the trailer and the obstacle based on the detection results; and / or, the detection results are output to the path planning module, so that the path planning module can construct a kinematic model of the trailer based on the detection results.

7. The method according to claim 1, characterized in that, The step of fusing the image bird's-eye view features with the point cloud bird's-eye view features to generate a shared bird's-eye view feature map includes: The bird's-eye view features of the image and the bird's-eye view features of the point cloud are stitched together in the channel dimension to generate a multi-channel feature map; The multi-channel feature map is input into the bird's-eye view encoder, and the shared bird's-eye view feature map is output after processing by the bird's-eye view encoder.

8. A device for determining the hinge angle between a tractor and a trailer, characterized in that, include: The data acquisition module is used to acquire surround view image data through the surround view camera installed on the tractor and to acquire laser point cloud data through the lidar installed on the tractor. The feature extraction module is used to extract features from the panoramic image data to obtain image features, and to extract features from the laser point cloud data to obtain point cloud features. The shared bird's-eye view feature map generation module is used to convert the image features and the point cloud features to the bird's-eye view space respectively to obtain image bird's-eye view features and point cloud bird's-eye view features, and to fuse the image bird's-eye view features and the point cloud bird's-eye view features to generate a shared bird's-eye view feature map. The articulation angle determination module is used to determine the articulation angle between the tractor and the attached trailer based on the shared bird's-eye view feature map.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a processor to perform the method of any one of claims 1-7.

Images