Hinge headstock hanging included angle visual estimation method and device

Abstract

The invention discloses a visual estimation method and device for an included angle of a hinge headstock, relates to the technical field of computer vision and intelligent perception and measurement of vehicles, and can solve the problems of high cost, poor environmental adaptability, insufficient robustness and lack of end-to-end direct prediction capability during measurement of the included angle of the headstock in the prior art. Comprising the following steps: synchronously acquiring image data and head-mounted included angle truth value data of a trailer, and performing time alignment processing on the image data and the head-mounted included angle truth value data to form a training data set; an AngleNet end-to-end head-hanging included angle estimation model is constructed, the estimation model comprises an input layer, a backbone network, a regression head and a loss function, the backbone network is a progressive deformable convolutional network GDNet, and the loss function adopts TMSE Loss; training the estimation model by using the training data set; and deploying the trained estimation model to a vehicle-mounted computing platform, processing an image acquired by the camera in real time, and outputting a head-mounted included angle estimation value. The method is used for hinge headstock hanging included angle visual estimation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision and intelligent perception and measurement of vehicles, and particularly relates to a hinged vehicle head-hitching angle visual estimation method and device. BACKGROUND

[0002] In the field of hinged commercial vehicles (such as tractor-trailer combinations), the angle between the vehicle head and the trailer (referred to as the "head-hitching angle") is a key parameter representing the running state, directly affecting the driving stability and handling performance, and is also the core input of intelligent functions such as 360° panoramic image and automatic driving.

[0003] At present, there are three types of head-hitching angle measurement schemes in the prior art: the method based on physical sensors relies on high-precision angle sensors or GPS / IMU combined navigation systems installed at the hinge point, which has the problems of high hardware cost, complex installation and vulnerability; the method based on laser radar extracts the trailer pose by processing point cloud data, which has high precision, but the system is complex, the cost is high, and the performance is significantly reduced in complex environments such as rain and snow; the method based on visual perception includes schemes relying on artificial markers, traditional image processing or deep learning, which have the problems of poor convenience, high maintenance cost, or are greatly disturbed by light and background, and lack of robustness, and most of the existing visual methods cannot realize end-to-end direct angle prediction, which has complex process and limited real-time performance. SUMMARY

[0004] In order to solve the problems of high cost, poor environmental adaptability, insufficient robustness and lack of end-to-end direct prediction ability in the prior art when measuring the head-hitching angle, the present application provides a hinged vehicle head-hitching angle visual estimation method and device. The technical solution is as follows:

[0005] According to a first aspect of the present application, a hinged vehicle head-hitching angle visual estimation method is provided, the method comprising the following steps:

[0006] Synchronously collecting image data and head-hitching angle true value data of the trailer, and performing time alignment processing on the image data and the head-hitching angle true value data to form a training data set, wherein the image data contains complete structural information of the trailer head;

[0007] An AngleNet end-to-end head-hitching angle estimation model is constructed, the estimation model comprising an input layer, a backbone network, a regression head and a loss function, wherein the backbone network is a progressive deformable convolution network GDNet, and the loss function adopts TMSE Loss;

[0008] The training data set is used to train the estimation model;

[0009] The trained estimation model is deployed to a vehicle-mounted computing platform to process images collected by a camera in real time and output a trailer hitch angle estimation value.

[0010] The application provides a trailer hitch angle visual estimation method, which comprises the following steps: synchronously collecting image data and trailer hitch angle true value data of a trailer, and performing time alignment processing on the image data and the trailer hitch angle true value data to form a training data set, wherein the image data contains complete structure information of a trailer head; constructing an AngleNet end-to-end trailer hitch angle estimation model, wherein the estimation model comprises an input layer, a backbone network, a regression head and a loss function, the backbone network is a progressive deformable convolution network GDNet, and the loss function adopts TMSE Loss; training the estimation model by using the training data set; and deploying the trained estimation model to a vehicle-mounted computing platform to process images collected by a camera in real time and output a trailer hitch angle estimation value.

[0011] As a further scheme of the application, the step of synchronously collecting image data and trailer hitch angle true value data of a trailer, and performing time alignment processing on the image data and the trailer hitch angle true value data to form a training data set comprises the following steps:

[0012] The image data is collected by using a camera at a first frequency;

[0013] The trailer hitch angle true value data is collected by using a combined navigation system at a second frequency, wherein the combined navigation system comprises a main positioning unit installed on a vehicle head and a secondary positioning unit installed on a trailer;

[0014] Based on timestamp information, the image data and the trailer hitch angle true value data with different sampling frequencies are aligned by using a linear interpolation method to form an image data and trailer hitch angle true value data pair.

[0015] The application only relies on a single camera and a combined navigation system to collect the data set, avoids expensive sensors and complex calibration processes, greatly reduces the hardware cost and application threshold, enables the system to quickly adapt to various vehicle models, and has strong generalizability and commercial value.

[0016] As a further scheme of the application, the linear interpolation method comprises the following steps:

[0017] Timestamps of each image data are obtained;

[0018] Trailer hitch angle true value data at the nearest time before and after the timestamp is found;

[0019] The head hanging angle true value data corresponding to the image data is calculated through a first formula, and the first formula comprises:

[0020] (1)

[0021] Wherein, t ang,i ≤t img ≤t ang,i+1 , theta i And theta i+1 Respectively, the head hanging angle true value data at t ang,i , t ang,i+1 Instant, theta(t img ) is the head hanging angle true value data corresponding to the image data at t img .

[0022] The method introduces a linear interpolation method in the data preprocessing stage, realizes accurate alignment of image data and combined navigation head hanging angle data under different sampling frequencies, ensures strict correspondence of image features and real angle labels in the time dimension, significantly expands the richness of the training data, and improves the precision and stability of model training.

[0023] As a further scheme of the present application: the progressive deformable convolutional network is constructed based on a ResNet18 basic network, comprising:

[0024] A convolutional layer is used to reduce image resolution and extract bottom edge and texture features;

[0025] A plurality of stage structures, wherein at least one stage introduces a deformable convolution DCNv2 structure for enhancing the adaptability of the model to the trailer deformation.

[0026] The method adopts a Grad-DeformNet main structure, combines the gradient guidance mechanism with the deformable convolution DCNv2, and combines the standard convolution, while keeping the network lightweight, improves the ability to capture complex image features, significantly enhances the generalization performance of the model under different vehicle types and different angles, and enhances the modeling ability of the model to non-rigid deformation (such as different perspective forms of trailers in images).

[0027] As a further scheme of the present application: the plurality of stage structures, specifically comprising:

[0028] The second convolution in the stage-2 stage adopts a DCNv2 structure for introducing first-layer geometric modeling capability;

[0029] All 3x3 convolutions in the stage-3 stage adopt a DCNv2 structure for improving spatial structure adaptability;

[0030] Stage-4 fully adopts the DCNv2 structure to achieve joint modeling of high-level semantics and geometric features.

[0031] The method of this invention adopts a progressive deformable convolutional fusion method of backbone network, which avoids excessive model computation and ensures that the advantages of deformable convolutional structure are fully utilized in deep layers. Because trailers have large deformations in images under different vehicle models and angles, ordinary convolution can only extract regular grid features and lacks the ability to capture deformation. However, the introduction of deformable convolution can adaptively focus on the edge and angle features of the trailer head area, enhancing the network's robustness to geometric changes.

[0032] As a further aspect of the present invention: training the estimation model using the training dataset specifically includes:

[0033] The parameters of the ResNet18 base network structure retained in the estimated model are initialized by loading the pre-trained weights from ImageNet.

[0034] The newly added DCNv2 structure and regression head in the estimation model are randomly initialized;

[0035] Set training parameters, including: learning rate, batch size, number of iterations, optimizer and regularization, and data augmentation;

[0036] Freeze the existing pre-trained weights of the ResNet18 base network structure and train only the newly added DCNv2 structure and regression head in the estimated model;

[0037] Unfreeze all structures and perform end-to-end training using the loss function.

[0038] This invention initializes the parameters of the ResNet18 basic network structure, the newly added DCNv2 structure, and the regression head, fully utilizing the low-level and mid-level feature representation capabilities of the ResNet18 basic network structure while ensuring the learnability of the newly added DCNv2 structure and regression head in the head-hanging angle estimation task. Furthermore, through a staged training approach, it achieves rapid convergence of the estimation model in the angle prediction task and optimal fitting between global features and the angle prediction results, thereby improving the model's training efficiency and prediction accuracy.

[0039] As a further aspect of the present invention: the loss function includes a mean squared error loss term and a smoothing loss constraint term, wherein the smoothing loss constraint term is used to constrain the continuity of the head-hanging angle estimates of adjacent frame images; the mean squared error loss term and the smoothing loss constraint term are calculated by a second formula and a third formula, respectively, wherein the second formula and the third formula include:

[0040] (2)

[0041] (3)

[0042] in, , The estimated head-hanging angles at times t and t+1 are θ. t Let L be the true value of the head-hanging angle at time t, where T is the image data sequence and one image corresponds to one time step t; MSE For the mean squared error loss term; L Smooth This is a smoothing loss constraint term;

[0043] The loss function is calculated using a fourth formula, which includes:

[0044] (4)

[0045] Where λ is the smoothing weight parameter, with a value range of [0.01, 1.0]; L total This represents the total loss.

[0046] The method of this invention introduces a smoothing loss constraint term on the basis of traditional MSE loss to constrain the continuity of the predicted angle between adjacent frames. This not only ensures the consistency between the predicted value and the actual angle, but also solves the problem of "flickering" that MSE is prone to in single-frame image prediction, which is prone to fluctuating. It reduces its overfitting to single-frame noise data, effectively reduces the jitter of the prediction curve, and makes the prediction results more stable and physically reasonable, making them more suitable for downstream vehicle control and decision-making.

[0047] As a further aspect of the present invention: adjusting the smoothing weight parameter according to the vehicle's operating status or the rate of change of the head-mounted angle, including:

[0048] When the rate of change of the head-mounted angle is detected to be lower than the preset threshold, the value of λ is increased to obtain a more stable prediction output.

[0049] When the rate of change of the head-mounted angle is detected to be higher than a preset threshold, the value of λ is reduced to enhance the response capability of the estimation model to rapid changes.

[0050] In the method of this invention, the smoothing weight parameter can be adaptively adjusted according to the vehicle's operating status or the rate of change of the head-mounted angle, thereby improving the model's adaptability and robustness under diverse working conditions.

[0051] As a further embodiment of the present invention: the input layer is used to receive three-channel RGB image data;

[0052] The regression head consists of two fully connected layers and is used to output a single, continuous head-hanging angle estimate.

[0053] According to a second aspect of the present invention, a visual estimation device for the hinged front end of a vehicle is provided, comprising:

[0054] The acquisition module is used to synchronously acquire image data and head-mounted angle ground truth data of the trailer, and perform time alignment processing on the image data and head-mounted angle ground truth data to form a training dataset, wherein the image data contains complete structural information of the trailer head;

[0055] The building module is used to build an end-to-end head-hanging angle estimation model for AngleNet. The estimation model includes an input layer, a backbone network, a regression head, and a loss function. The backbone network is a progressively deformable convolutional network GDNet, and the loss function is TMSE Loss.

[0056] A training module is used to train the estimated model using the training dataset;

[0057] The prediction module is used to deploy the trained estimation model to the vehicle computing platform, process the images captured by the camera in real time, and output the estimated value of the head-mounted angle.

[0058] The present invention provides a visual estimation device for the hinged trailer cab angle, comprising: an acquisition module, a construction module, a training module, and a prediction module. The acquisition module simultaneously acquires image data and ground truth data of the trailer cab angle from the trailer, and performs time alignment processing on the image data and ground truth data to form a training dataset. The image data contains complete structural information of the trailer cab. The construction module constructs an end-to-end AngleNet cab angle estimation model, which includes an input layer, a backbone network, a regression head, and a loss function. The backbone network is a progressively deformable convolutional network (GDNet), and the loss function is TMSE Loss. The training module trains the estimation model using the training dataset. The prediction module deploys the trained estimation model to an onboard computing platform, processes the images acquired by the camera in real time, and outputs the estimated cab angle value. The device of this invention can directly and robustly calculate high-precision cab angle values ​​from complex real-world scene images, while avoiding the complex processes of traditional methods that rely on multi-sensor fusion, feature point detection, or geometric modeling.

[0059] According to a third aspect of the present invention, a visual estimation device for the hinged front end angle is provided, the visual estimation device for the hinged front end angle includes a processor and a memory, the memory storing at least one computer instruction, the instruction being loaded and executed by the processor to perform the steps performed in the visual estimation method for the hinged front end angle described above.

[0060] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, the storage medium storing at least one computer instruction, the instruction being loaded and executed by a processor to perform the steps performed in the visual estimation method for the hinged front end angle as described in any of the preceding claims.

[0061] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0062] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0063] Figure 1 This is a flowchart of the visual estimation method for the hinged front hanging angle provided in an embodiment of the present invention;

[0064] Figure 2 This is a schematic diagram of camera installation in the visual estimation method for the hinged front hanging angle provided in an embodiment of the present invention;

[0065] Figure 3 This is a structural diagram of GDNet in the visual estimation method for the hinged front hanging angle provided in the embodiments of the present invention;

[0066] Figure 4 This is a structural diagram of the visual estimation device for the hinged front hanging angle provided in an embodiment of the present invention;

[0067] In the diagram: 1-tractor unit; 2-trailer; 3-camera; 4-main positioning unit of the integrated navigation system; 5-sub-positioning unit of the integrated navigation system. Detailed Implementation

[0068] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention.

[0069] This invention provides a visual estimation method for the hinged front suspension angle, such as... Figure 1 As shown, it includes the following steps:

[0070] Step 101: Synchronously collect image data and ground truth data of the trailer cab angle, and perform time alignment processing on the image data and ground truth data of the trailer cab angle to form a training dataset. The image data contains complete structural information of the trailer cab.

[0071] In practical use, the specific execution process of this invention involves five major modules: camera installation and adaptation, trailer image and trailer-to-trailer angle data acquisition and processing, model construction, model training, and model deployment and application. First, the camera needs to be adapted and installed according to the actual vehicle model. Different vehicle models differ in size and structure, such as vehicle height, width, and the distance from the tractor to the trailer connection point. Therefore, when actually installing the camera, adjustments need to be made for the specific vehicle model to ensure that effective trailer image information is acquired.

[0072] Generally, the camera should be installed behind the cab of the vehicle, facing the trailer, and the tilt angle should be adjusted so that the camera's field of view can always include the complete view of the trailer cab while the vehicle is in motion.

[0073] Specifically, the following factors need to be considered during actual installation:

[0074] (1) The horizontal and vertical positions of the camera installation: ensure that the vertical and horizontal views of the camera cover the overall height of the trailer and the width of the cab, and avoid trailer image cropping;

[0075] (2) Distance between camera and trailer: Adjust the front and rear positions to ensure that the camera can fully capture the content of the trailer;

[0076] (3) Pitch angle adjustment: By adjusting the pitch angle, the trailer head is fully displayed in the imaging image when the head-mounted angle is any value, providing sufficient information for subsequent algorithm processing.

[0077] like Figure 2 The diagram shows the installation position and orientation of the camera under the tractor unit. This installation method ensures consistent camera imaging across different vehicle models, providing a reliable data foundation for subsequent head-mounted angle measurements.

[0078] In one embodiment, the steps of simultaneously acquiring image data of the trailer and ground truth data of the cab-mounted angle, and performing time alignment processing on the image data and ground truth data of the cab-mounted angle to form a training dataset specifically include:

[0079] Use a camera to acquire image data at the highest frequency;

[0080] The combined navigation system is used to collect true data of the head-mounted angle at a second frequency. The combined navigation system includes a main positioning unit installed at the head of the vehicle and a secondary positioning unit installed on the trailer.

[0081] Based on timestamp information, a linear interpolation method is used to align image data with the true value data of the head-hanging angle at different sampling frequencies, forming image data and head-hanging angle true value data pairs.

[0082] In practical applications, the acquisition of image data and true-value data of the head-mounted angle mainly consists of two parts: a visual perception subsystem and a reference measurement subsystem. Specifically:

[0083] Visual perception subsystem: consists of the camera installed in step S101 above, which is responsible for acquiring video stream image data of the trailer area at a fixed frequency of 30Hz;

[0084] Reference Measurement Subsystem: This module mainly consists of an integrated navigation system (GNSS+IMU), as described above. Figure 2 As can be seen, the main positioning unit is fixedly installed on the top of the tractor frame to accurately measure the heading angle of the tractor, while the secondary positioning unit is fixedly installed at the front of the trailer to accurately measure the trailer's attitude. Finally, the true value data of the head-to-trailer angle is obtained through the relative relationship between the two. The heading angle is collected at a frequency of 100Hz, and the measurement error is about 0.1°.

[0085] The method of this invention can collect datasets using only a single camera and a combined navigation system, avoiding expensive sensors and complex calibration processes, significantly reducing hardware costs and application barriers, enabling the system to be quickly adapted to various vehicle models, and possessing strong scalability and commercial value.

[0086] In one embodiment, the linear interpolation method specifically includes:

[0087] Get the timestamp of each image data;

[0088] Find the true value of the head-hanging angle data for the closest time before and after the timestamp;

[0089] The true value of the head-hanging angle corresponding to the image data is calculated using a first formula, which includes:

[0090] (1)

[0091] Among them, t ang,i ≤t img ≤t ang,i+1 θ i and θ i+1 They correspond to t respectively ang,i t ang,i+1 True value data of the head-hanging angle at time t, θ(t) img ) for t img The true value of the head-hanging angle corresponding to the time-lapse image data.

[0092] In practical use, after obtaining the trailer image data and the true value data of the head-mounted angle, a linear interpolation preprocessing operation is performed. The image and angle data are aligned by the timestamp information. That is, for each image, the head-mounted angle data under the nearest timestamp before and after it is first found, and then the true value of the head-mounted angle corresponding to the image is obtained by linear interpolation. After completing the data preprocessing task, the overall data is represented as a data pair of "trailer image + head-mounted angle", that is, each trailer image has a corresponding head-mounted angle value.

[0093] The method of this invention introduces a linear interpolation method in the data preprocessing stage to achieve accurate alignment between image data and the angle data of the integrated navigation head under different sampling frequencies. This ensures a strict correspondence between image features and real angle labels in the time dimension, significantly expands the richness of training data, and improves the accuracy and stability of model training.

[0094] Step 102: Construct an end-to-end head-hanging angle estimation model for AngleNet. The estimation model includes an input layer, a backbone network, a regression head, and a loss function. The backbone network is a progressively deformable convolutional network GDNet, and the loss function is TMSE Loss.

[0095] In this embodiment, the AngleNet end-to-end angle estimation network is used. This network mainly consists of four parts: input, backbone network, regression head, and loss function. The backbone network is GDNet (Grad-DeformNet, Progressively Deformable Convolutional Network).

[0096] In one embodiment, the progressively deformable convolutional network is built upon the ResNet18 base network and includes:

[0097] A convolutional layer is used to reduce image resolution and extract low-level edge and texture features;

[0098] The model employs a multi-stage structure, with at least one stage incorporating a deformable convolutional DCNv2 structure to enhance its adaptability to trailer deformation.

[0099] In this embodiment, the backbone network is constructed with reference to the ResNet18 base network, specifically consisting of one convolutional layer, four stage structures, and a global average pooling layer. The specific structure is as follows: Figure 3 As shown. It should be noted that the global average pooling layer, while aggregating global spatial information, transforms the feature map to a fixed length, generating a compact feature representation suitable for subsequent regression head processing.

[0100] The method of this invention adopts the Grad-DeformNet backbone structure, combines the gradient guidance mechanism with deformable convolution DCNv2, and combines it with standard convolution. While maintaining the network's lightweight nature, it improves the ability to capture complex image features, significantly enhances the model's generalization performance under different vehicle types and angles, and enhances the model's ability to model non-rigid deformations (such as trailers in different perspective shapes in images).

[0101] In one embodiment, the multiple stage structures specifically include:

[0102] The second convolution in stage-2 uses the DCNv2 structure to introduce first-layer geometric modeling capabilities;

[0103] All 3×3 convolutions in the stage-3 phase use the DCNv2 structure to improve spatial structure adaptability;

[0104] Stage-4 fully adopts the DCNv2 structure to achieve joint modeling of high-level semantics and geometric features.

[0105] In this embodiment, the stage-1 stage consists of basic convolutional layers and does not include the DCNv2 module.

[0106] The method of this invention adopts a progressive deformable convolutional fusion method of backbone network, which avoids excessive model computation and ensures that the advantages of deformable convolutional structure are fully utilized in deep layers. Because trailers have large deformations in images under different vehicle models and angles, ordinary convolution can only extract regular grid features and lacks the ability to capture deformation. However, the introduction of deformable convolution can adaptively focus on the edge and angle features of the trailer head area, enhancing the network's robustness to geometric changes.

[0107] In one embodiment, the input layer is used to receive three-channel RGB image data;

[0108] The regression head, consisting of two fully connected layers, is used to output a single, continuous estimate of the head-hanging angle.

[0109] Specifically, the input includes the original trailer image and the actual cab angle at the corresponding timestamp. The trailer image is a 3*448*448 three-channel RGB image, specifically represented as follows: .

[0110] In one embodiment, the loss function includes a mean squared error loss term and a smoothing loss constraint term, wherein the smoothing loss constraint term is used to constrain the continuity of the head-hook angle estimates of adjacent frame images. The mean squared error loss term and the smoothing loss constraint term are calculated by a second formula and a third formula, respectively, wherein the second formula and the third formula include:

[0111] (2)

[0112] (3)

[0113] in, , The estimated head-hanging angles at times t and t+1 are θ. t Let L be the true value of the head-hanging angle at time t, where T is the image data sequence and one image corresponds to one time step t; MSE For the mean squared error loss term; L Smooth This is a smoothing loss constraint term;

[0114] The loss function is calculated using the fourth formula, which includes:

[0115] (4)

[0116] Where λ is the smoothing weight parameter, with a value range of [0.01, 1.0]; L total This represents the total loss.

[0117] In this embodiment, the smoothing weight parameter adopts a fixed value method, which is implemented by scene data under different working conditions to determine the optimal value range, thereby ensuring a balance between prediction accuracy and temporal continuity.

[0118] The method of this invention introduces a smoothing loss constraint term on the basis of traditional MSE loss to constrain the continuity of the predicted angle between adjacent frames. This not only ensures the consistency between the predicted value and the actual angle, but also solves the problem of "flickering" that MSE is prone to in single-frame image prediction, which is prone to fluctuating. It reduces its overfitting to single-frame noise data, effectively reduces the jitter of the prediction curve, and makes the prediction results more stable and physically reasonable, making them more suitable for downstream vehicle control and decision-making.

[0119] In one embodiment, adjusting the smoothing weight parameter based on the vehicle's operating status or the rate of change of the head-mounted clamp angle includes:

[0120] When the rate of change of the head-mounted angle is detected to be lower than the preset threshold, the value of λ is increased to obtain a more stable prediction output.

[0121] When the rate of change of the head-mounted angle is detected to be higher than a preset threshold, the value of λ is reduced to enhance the estimation model's ability to respond to rapid changes.

[0122] Specifically, the smoothing weight parameter can be adjusted according to the vehicle's operating status or the rate of change of the cab angle to balance the model's accuracy and stability. When the weight value is small, the model tends to fit the true labels more closely; when the weight value is large, the model focuses more on the continuity and stability of the output.

[0123] In the method of this invention, the smoothing weight parameter can be adaptively adjusted according to the vehicle's operating status or the rate of change of the head-mounted angle, thereby improving the model's adaptability and robustness under diverse working conditions.

[0124] Step 103: Train the estimated model using the training dataset;

[0125] In one embodiment, training the estimation model using a training dataset specifically includes:

[0126] By loading the pre-trained weights from ImageNet, the parameters of the ResNet18 base network structure retained in the estimated model are initialized.

[0127] The newly added DCNv2 structure and regression head in the estimation model are randomly initialized;

[0128] Set training parameters, including: learning rate, batch size, number of iterations, optimizer and regularization, and data augmentation;

[0129] Freeze the existing ResNet18 base network structure with pre-trained weights and train only the newly added DCNv2 structure and regression head in the estimation model;

[0130] Unfreeze all structures and perform end-to-end training using the loss function.

[0131] In practical applications, the initial learning rate can be set to 0.0001, and a learning rate decay strategy can be used to improve convergence stability. The batch size can be determined based on hardware computing power and GPU memory size, typically set to 16-32. The number of iterations can be set according to the dataset size, typically 50-100 rounds, to ensure the model fully learns the data features. The optimizer can be either Adam or SGD, combined with momentum, weight decay, and other techniques to improve training performance. Regularization can introduce Dropout to prevent overfitting, and combined with L2 weight decay to constrain the network, further improving the model's generalization ability and robustness in unseen scenarios. Enhancement techniques such as brightness adjustment and random horizontal flipping can also be introduced to augment the data and improve the model's generalization ability.

[0132] This invention initializes the parameters of the ResNet18 basic network structure, the newly added DCNv2 structure, and the regression head, fully utilizing the low-level and mid-level feature representation capabilities of the ResNet18 basic network structure while ensuring the learnability of the newly added DCNv2 structure and regression head in the head-hanging angle estimation task. Furthermore, through a staged training approach, it achieves rapid convergence of the estimation model in the angle prediction task and optimal fitting between global features and the angle prediction results, thereby improving the model's training efficiency and prediction accuracy.

[0133] Furthermore, after training to obtain the estimated model, the embodiments of the present invention also optimize and test the model using a validation dataset and a test dataset.

[0134] Step 104: Deploy the trained estimation model to the vehicle computing platform to process the images captured by the camera in real time and output the estimated value of the head-mounted angle.

[0135] In this embodiment, after obtaining the optimal accuracy training model, quantization processing is applied to the model to balance model execution time and accuracy performance. The model is then deployed on an onboard computing platform to achieve real-time inference on the device side. After processing by the model, the camera images can output the angle between the truck cab and the trailer in real time.

[0136] The visual estimation method for the hinged trailer cab angle provided in this invention first synchronously acquires image data and ground truth data of the trailer cab angle, and performs time alignment processing on the image data and ground truth data of the trailer cab angle to form a training dataset. The image data contains complete structural information of the trailer cab. Then, an end-to-end AngleNet cab angle estimation model is constructed. The estimation model includes an input layer, a backbone network, a regression head, and a loss function. The backbone network is a progressively deformable convolutional network GDNet, and the loss function is TMSE Loss. The estimation model is then trained using the training dataset. Finally, the trained estimation model is deployed to an onboard computing platform to process the images acquired by the camera in real time and output the estimated value of the cab cab angle. The method of this invention directly uses trailer images as input. The model can directly and robustly calculate high-precision head-mounted angle values ​​from complex real-world scene images without the need for additional markers or complex sensor arrangements. It balances real-time performance, low cost, and environmental adaptability, and can provide stable and accurate angle estimation under various working conditions. It provides reliable data for intelligent functions such as 360° panoramic imaging, reversing assistance, and autonomous driving of articulated commercial vehicles.

[0137] In summary, this invention relies solely on a single vehicle-mounted camera and a combined navigation system to complete the data collection and preparation work required for model training, eliminating the need for additional complex sensors or external auxiliary devices, thus significantly reducing hardware dependence and deployment costs. In practical applications, only one camera needs to be installed at a predetermined location on the vehicle, and the system can automatically complete the trailer angle perception and estimation, offering advantages such as easy installation, strong adaptability, and low application threshold.

[0138] Furthermore, thanks to the AngleNet recognition network proposed in this invention, supported by the Grad-DeformNet backbone structure and the TMSE loss function, the model can fully mine trailer image features and combine them with temporal constraints to achieve high-precision prediction of the trailer's angle. Under typical test conditions, the overall prediction accuracy can reach over 96%, effectively ensuring the stability, reliability, and safety of the vehicle during operation. Compared with existing methods that rely on multi-sensor fusion, the solution of this invention is not only simpler in terms of hardware deployment, but also has significant advantages in data processing efficiency, model inference real-time performance, and angle accuracy.

[0139] Based on the above Figure 1 The visual estimation method for the hinged front hanging angle described in the corresponding embodiment is described below as an embodiment of the device of the present invention, which can be used to execute the embodiment of the method of the present invention.

[0140] The hinged front suspension angle visual estimation device provided in this embodiment of the invention, such as... Figure 4 As shown, the device includes:

[0141] The acquisition module 201 is used to synchronously acquire image data and head-mounted angle ground truth data of the trailer, and perform time alignment processing on the image data and head-mounted angle ground truth data to form a training dataset. The image data contains complete structural information of the trailer head.

[0142] Module 202 is used to build an end-to-end head-hanging angle estimation model for AngleNet. The estimation model includes an input layer, a backbone network, a regression head, and a loss function. The backbone network is a progressively deformable convolutional network GDNet, and the loss function is TMSE Loss.

[0143] Training module 203 is used to train the estimated model using the training dataset;

[0144] The prediction module 204 is used to deploy the trained estimation model to the vehicle computing platform, process the images captured by the camera in real time, and output the estimated value of the head-mounted angle.

[0145] The visual estimation device for the hinged trailer head angle provided in this embodiment of the invention includes: an acquisition module 201, a construction module 202, a training module 203, and a prediction module 204. The acquisition module 201 simultaneously acquires image data and ground truth data of the trailer head angle of the trailer, and performs time alignment processing on the image data and ground truth data of the trailer head angle to form a training dataset. The image data contains complete structural information of the trailer head. The construction module 202 constructs an end-to-end AngleNet head angle estimation model. The estimation model includes an input layer, a backbone network, a regression head, and a loss function. The backbone network is a progressively deformable convolutional network GDNet, and the loss function is TMSE Loss. The training module 203 trains the estimation model using the training dataset. The prediction module 204 deploys the trained estimation model to an on-board computing platform, processes the images acquired by the camera in real time, and outputs the estimated value of the head angle. The device of this invention can directly and robustly calculate high-precision head-mounted angle values ​​from complex real-world scene images, while avoiding the complex process of traditional methods that rely on multi-sensor fusion, feature point detection, or geometric modeling.

[0146] Based on the above Figure 1 The corresponding embodiment describes a visual estimation method for the hinged front end hook angle. Another embodiment of the present invention also provides a visual estimation device for the hinged front end hook angle, which includes a processor and a memory. The memory stores at least one computer instruction, which is loaded and executed by the processor to achieve the above-described method. Figure 1 The corresponding embodiment describes a visual estimation method for the hinged front hanging angle.

[0147] Based on the above Figure 1 The corresponding embodiment of the invention describes a visual estimation method for the hinged front end hook angle. This embodiment also provides a computer-readable storage medium, such as a non-transitory computer-readable storage medium that is a read-only memory (ROM), random access memory (RAM), CD-ROM, magnetic tape, floppy disk, or optical data storage system. This storage medium stores at least one computer instruction for executing the above-described method. Figure 1 The visual estimation method for the hinged front hanging angle described in the corresponding embodiment will not be repeated here.

[0148] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the claims.

[0149] It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A method for estimating the angle of a hinged head-on clip, the method comprising: The method comprises the following steps: Synchronously collecting image data and head-hitch angle ground truth data of the trailer, and performing time alignment processing on the image data and the head-hitch angle ground truth data to form a training data set, wherein the image data contains complete structure information of the trailer head; An AngleNet end-to-end head-hitch angle estimation model is constructed, the estimation model comprising an input layer, a backbone network, a regression head and a loss function, wherein the backbone network is a progressive deformable convolution network GDNet, and the loss function adopts TMSELoss; The training data set is used to train the estimation model; The trained estimation model is deployed to a vehicle-mounted computing platform to process images collected by a camera in real time and output a head-hitch angle estimation value.

2. The method of visual estimation of the angle of the head-on hitch of a hinged vehicle according to claim 1, characterized by the fact that The step of synchronously collecting image data and head-hitch angle ground truth data of the trailer, and performing time alignment processing on the image data and the head-hitch angle ground truth data to form a training data set specifically comprises: The image data is collected at a first frequency using a camera; The head-hitch angle ground truth data is collected at a second frequency using a combined navigation system, wherein the combined navigation system comprises a main positioning unit installed on the vehicle head and a secondary positioning unit installed on the trailer; Based on timestamp information, the image data and the head-hitch angle ground truth data with different sampling frequencies are aligned using a linear interpolation method to form an image data and head-hitch angle ground truth data pair.

3. The method of visual estimation of the angle of the head-on hitch of a hinged vehicle according to claim 2, characterized by the fact that The linear interpolation method specifically comprises: Obtaining the timestamp of each image data; Finding the head-hitch angle ground truth data at the nearest time before and after the timestamp; Calculating the head-hitch angle ground truth data corresponding to the image data through a first formula, the first formula comprising: （1） Wherein, t ang,i ≤t img ≤t ang,i+1 , θ i and θ i+1 are the head hanging angle true value data at t ang,i , t ang,i+1 , respectively, and θ(t img ) is the head hanging angle true value data corresponding to the image data at t img .

4. The method of visual estimation of the angle of the head-on hitch of a hinged vehicle according to claim 1, characterized by the fact that The progressive deformable convolution network is constructed based on a ResNet18 base network and comprises: A convolution layer for reducing image resolution and extracting low-level edge and texture features; Multiple stage structures, at least one stage of which introduces a deformable convolution DCNv2 structure to enhance the adaptability of the model to trailer deformation.

5. The method of visual estimation of the angle of the head-on hitch of the articulated vehicle according to claim 4, characterized in that, The multiple stage structures specifically comprise: In the stage-2 stage, the second convolution adopts a DCNv2 structure to introduce first-layer geometric modeling capability; In the stage-3 stage, all 3x3 convolutions adopt a DCNv2 structure to improve spatial structure adaptability; In the stage-4 stage, a DCNv2 structure is completely adopted to realize joint modeling of high-level semantic and geometric features.

6. The method of visual estimation of the angle of the head-on hitch of the articulated vehicle according to claim 5, characterized in that, The training of the estimation model using the training data set specifically comprises: The ResNet18 base network structure remaining in the estimation model is parameterized and initialized by loading the pre-training weight of ImageNet; The newly added DCNv2 structure and regression head in the estimation model are randomly initialized; Training parameters are set, including learning rate, batch size, iteration number, optimizer and regularization, data enhancement; The ResNet18 base network structure with existing pre-training weight is frozen, and only the newly added DCNv2 structure and regression head in the estimation model are trained. All structures are thawed, and end-to-end training is performed in combination with the loss function.

7. The method of visual estimation of the angle of articulation of a hitch trailer according to claim 1, wherein, The loss function includes a mean square error loss term and a smooth loss constraint term, wherein the smooth loss constraint term is used to constrain the continuity of the head-hitch angle estimation value of adjacent frame images, and the mean square error loss term and the smooth loss constraint term are calculated by a second formula and a third formula respectively, the second formula and the third formula include: （2） （3） wherein, , are the head-pendant included angle estimation values at times t and t+1, respectively, θ t is the head-pendant included angle true value data at time t, T is a sequence of image data, and one image corresponds to one time step t; L MSE is a mean square error loss term; L Smooth is a smoothing loss constraint term; The loss function is calculated by a fourth formula, and the fourth formula includes: （4） where λ is a smoothing weight parameter with a value range of [0.01, 1.0], L total is the total loss.

8. The method of visual estimation of the angle of the head-on hitch of the articulated vehicle according to claim 7, characterized in that, The smooth weight parameter is adjusted according to the vehicle operating state or the head-hitch angle change rate, including: When it is detected that the change rate of the head-hitch angle is lower than a preset threshold, the value of λ is increased to obtain a more stable prediction output; When it is detected that the change rate of the head-hitch angle is higher than a preset threshold, the value of λ is reduced to enhance the response ability of the estimation model to rapid changes.

9. The method of visual estimation of the angle of articulation of a hitch trailer according to claim 1, wherein, The input layer is configured to receive three-channel RGB image data. The regression head is composed of two fully connected layers and is configured to output a single continuous head-hitch angle estimation value.

10. A device for estimating the angle of a hinged head-on clip, characterized by The input layer is configured to receive three-channel RGB image data. The regression head is composed of two fully connected layers and is configured to output a single continuous head-hitch angle estimation value. The acquisition module is configured to synchronously acquire image data and head-hitch angle true value data of a trailer, and perform time alignment processing on the image data and the head-hitch angle true value data to form a training data set, wherein the image data contains complete structure information of a head of the trailer. The construction module is configured to construct an AngleNet end-to-end head-hitch angle estimation model, wherein the estimation model includes an input layer, a backbone network, a regression head and a loss function, the backbone network is a progressive deformable convolution network GDNet, and the loss function adopts TMSE Loss. The training module is configured to train the estimation model using the training data set. The prediction module is configured to deploy the trained estimation model to a vehicle-mounted computing platform, process images collected by a camera in real time, and output a head-hitch angle estimation value.

Images