A method and device for estimating the angle of a hinged vehicle head suspension

Abstract

The application discloses a kind of hinge truck head hanging clip angle visual estimation method and device, it is related to computer vision and vehicle intelligent perception and measurement technical field, can solve the problems of high cost, poor environmental adaptability, insufficient robustness and lack of end-to-end direct prediction capability when existing technology head hanging clip angle measurement.Therein, the image data and head hanging clip angle true value data of trailer are synchronously collected, and the image data and head hanging clip angle true value data are time-aligned to form a training data set;An AngleNet end-to-end head hanging clip angle estimation model is constructed, the estimation model includes an input layer, a backbone network, a regression head and a loss function, wherein the backbone network is a progressive deformable convolution network GDNet, and the loss function uses TMSE Loss;The training data set is used to train the estimation model;The trained estimation model is deployed to a vehicle-mounted computing platform, real-time processing of the image collected by the camera is carried out, and a head hanging clip angle estimation value is output.The application is used for hinge truck head hanging clip angle visual estimation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision and intelligent perception and measurement of vehicles, and particularly relates to a hinged vehicle head-hitching angle visual estimation method and device. BACKGROUND

[0002] In the field of hinged commercial vehicles (such as tractor-trailer combinations), the angle between the vehicle head and the trailer (referred to as the "head-hitching angle") is a key parameter representing the running state, directly affecting the driving stability and handling performance, and is also the core input of intelligent functions such as 360° panoramic image and automatic driving.

[0003] At present, there are three types of head-hitching angle measurement schemes in the prior art: the method based on physical sensors relies on high-precision angle sensors or GPS / IMU combined navigation systems installed at the hinge point, which has the problems of high hardware cost, complex installation and vulnerability; the method based on laser radar extracts the trailer pose by processing point cloud data, which has high precision, but the system is complex, the cost is high, and the performance is significantly reduced in complex environments such as rain and snow; the method based on visual perception includes schemes relying on artificial markers, traditional image processing or deep learning, which have the problems of poor convenience, high maintenance cost, or are greatly disturbed by light and background, and lack of robustness, and most of the existing visual methods cannot realize end-to-end direct angle prediction, which has complex process and limited real-time performance. SUMMARY

[0004] In order to solve the problems of high cost, poor environmental adaptability, insufficient robustness and lack of end-to-end direct prediction ability in the prior art when measuring the head-hitching angle, the present application provides a hinged vehicle head-hitching angle visual estimation method and device. The technical solution is as follows:

[0005] According to a first aspect of the present application, a hinged vehicle head-hitching angle visual estimation method is provided, the method comprising the following steps:

[0006] Synchronously collecting image data and head-hitching angle true value data of the trailer, and performing time alignment processing on the image data and the head-hitching angle true value data to form a training data set, wherein the image data contains complete structural information of the trailer head;

[0007] An AngleNet end-to-end head-hitching angle estimation model is constructed, the estimation model comprising an input layer, a backbone network, a regression head and a loss function, wherein the backbone network is a progressive deformable convolution network GDNet, and the loss function adopts TMSE Loss;

[0008] The training data set is used to train the estimation model;

[0009] The trained estimation model is deployed to a vehicle-mounted computing platform to process images collected by a camera in real time and output a trailer hitch angle estimation value.

[0010] The application provides a trailer hitch angle visual estimation method, which comprises the following steps: synchronously collecting image data and trailer hitch angle true value data of a trailer, and performing time alignment processing on the image data and the trailer hitch angle true value data to form a training data set, wherein the image data contains complete structure information of a trailer head; constructing an AngleNet end-to-end trailer hitch angle estimation model, wherein the estimation model comprises an input layer, a backbone network, a regression head and a loss function, the backbone network is a progressive deformable convolution network GDNet, and the loss function adopts TMSE Loss; training the estimation model by using the training data set; and deploying the trained estimation model to a vehicle-mounted computing platform to process images collected by a camera in real time and output a trailer hitch angle estimation value.

[0011] As a further scheme of the application, the step of synchronously collecting image data and trailer hitch angle true value data of a trailer, and performing time alignment processing on the image data and the trailer hitch angle true value data to form a training data set comprises the following steps:

[0012] The image data is collected by using a camera at a first frequency;

[0013] The trailer hitch angle true value data is collected by using a combined navigation system at a second frequency, wherein the combined navigation system comprises a main positioning unit installed on a vehicle head and a secondary positioning unit installed on a trailer;

[0014] Based on timestamp information, the image data and the trailer hitch angle true value data with different sampling frequencies are aligned by using a linear interpolation method to form an image data and trailer hitch angle true value data pair.

[0015] The application only relies on a single camera and a combined navigation system to collect the data set, avoids expensive sensors and complex calibration processes, greatly reduces the hardware cost and application threshold, enables the system to quickly adapt to various vehicle models, and has strong generalizability and commercial value.

[0016] As a further scheme of the application, the linear interpolation method comprises the following steps:

[0017] Timestamps of each image data are obtained;

[0018] Trailer hitch angle true value data at the nearest time before and after the timestamp is found;

[0019] The head hanging angle true value data corresponding to the image data is calculated through a first formula, and the first formula comprises:

[0020] (1)

[0021] Wherein, t ang,i ≤t img ≤t ang,i+1 , theta i And theta i+1 Respectively, the head hanging angle true value data at t ang,i , t ang,i+1 Instant, theta(t img ) is the head hanging angle true value data corresponding to the image data at t img .

[0022] The method introduces a linear interpolation method in the data preprocessing stage, realizes accurate alignment of image data and combined navigation head hanging angle data under different sampling frequencies, ensures strict correspondence of image features and real angle labels in the time dimension, significantly expands the richness of the training data, and improves the precision and stability of model training.

[0023] As a further scheme of the present application: the progressive deformable convolutional network is constructed based on a ResNet18 basic network, comprising:

[0024] A convolutional layer is used to reduce image resolution and extract bottom edge and texture features;

[0025] A plurality of stage structures, wherein at least one stage introduces a deformable convolution DCNv2 structure for enhancing the adaptability of the model to the trailer deformation.

[0026] The method adopts a Grad-DeformNet main structure, combines the gradient guidance mechanism with the deformable convolution DCNv2, and combines the standard convolution, while keeping the network lightweight, improves the ability to capture complex image features, significantly enhances the generalization performance of the model under different vehicle types and different angles, and enhances the modeling ability of the model to non-rigid deformation (such as different perspective forms of trailers in images).

[0027] As a further scheme of the present application: the plurality of stage structures, specifically comprising:

[0028] The second convolution in the stage-2 stage adopts a DCNv2 structure for introducing first-layer geometric modeling capability;

[0029] All 3x3 convolutions in the stage-3 stage adopt a DCNv2 structure for improving spatial structure adaptability;

[0030] Stage-4 stage fully adopts DCNv2 structure, and is used for realizing joint modeling of high layer semantics and geometric features.

[0031] The progressive deformable convolution fusion mode of the trunk network is adopted in the method, so that the model calculation amount is avoided to be too large, and the advantage of the deformable convolution structure is ensured to be fully utilized in the deep layer.

[0032] As a further scheme of the present application, the training of the estimation model by using the training data set specifically comprises:

[0033] The ResNet18 basic network structure reserved in the estimation model is parameterized and initialized by loading the pre-training weight of ImageNet;

[0034] The DCNv2 structure and the regression head newly added in the estimation model are randomly initialized;

[0035] The training parameters are set, including a learning rate, a batch size, an iteration number, an optimizer and regularization, and data enhancement;

[0036] The ResNet18 basic network structure with the pre-training weight is frozen, and only the DCNv2 structure and the regression head newly added in the estimation model are trained;

[0037] All the structures are unfrozen, and end-to-end training is performed in combination with the loss function.

[0038] The method parameterizes and initializes the ResNet18 basic network structure, the newly added DCNv2 structure and the regression head, fully utilizes the bottom and middle feature expression ability of the ResNet18 basic network structure, and ensures the learnability of the newly added DCNv2 structure and the regression head in the head-hanging angle estimation task. In addition, through the stage-by-stage training mode, the estimation model realizes the rapid convergence in the angle prediction task, the optimal fitting of the global feature and the angle prediction result, and the training efficiency and the prediction precision of the model are improved.

[0039] As a further scheme of the present application, the loss function comprises a mean square error loss term and a smooth loss constraint term, wherein the smooth loss constraint term is used to constrain the continuity of the head-hanging angle estimation value of adjacent frame images, and the mean square error loss term and the smooth loss constraint term are calculated through a second formula and a third formula respectively, and the second formula and the third formula comprise:

[0040] (2)

[0041] (3)

[0042] wherein, , are the head-cant angle estimation value at time t and t+1 respectively, θ t is the true value of the head-cant angle at time t, T is the image data sequence, and one image corresponds to a time step t; L MSE is the mean square error loss term; L Smooth is the smooth loss constraint term;

[0043] The loss function is calculated by a fourth formula, and the fourth formula comprises:

[0044] (4)

[0045] wherein, λ is a smooth weight parameter, and the value range is [0.01, 1.0]; L total is the total loss.

[0046] The method introduces a smooth loss constraint term on the basis of the traditional MSE loss, and the continuity of the angle prediction value of the adjacent frames is constrained, which not only ensures the consistency of the prediction value and the true angle, but also solves the “flicker” problem that the MSE is prone to appear large and small in single frame image prediction, reduces the overfitting of the single frame noise data, effectively reduces the jitter phenomenon of the prediction curve, makes the prediction result more stable and physically reasonable, and is more suitable for the downstream vehicle control and decision-making link.

[0047] As a further scheme of the present application: the smooth weight parameter is adjusted according to the vehicle running state or the head-cant angle change rate, comprising:

[0048] When the change rate of the head-cant angle is detected to be lower than a preset threshold, the value of λ is increased to obtain a more stable prediction output;

[0049] When the change rate of the head-cant angle is detected to be higher than a preset threshold, the value of λ is reduced to enhance the response ability of the estimation model to rapid changes.

[0050] The smooth weight parameter in the method can be adaptively adjusted according to the vehicle running state or the head-cant angle change rate, and the adaptability and robustness of the model under diversified working conditions are improved.

[0051] As a further scheme of the present application: the input layer is used for receiving three-channel RGB image data;

[0052] The regression head is composed of two full connection layers and is used for outputting a single continuous head-cant angle estimation value.

[0053] According to a second aspect of the present application, a device for estimating a hitching angle of a trailer head is provided, comprising:

[0054] An acquisition module is configured to synchronously acquire image data and true value data of the hitching angle of the trailer head, and perform time alignment processing on the image data and the true value data to form a training data set, wherein the image data contains complete structural information of the trailer head.

[0055] A construction module is configured to construct an end-to-end estimation model of the hitching angle of the trailer head, AngleNet, which comprises an input layer, a backbone network, a regression head and a loss function, wherein the backbone network is a gradually deformable convolution network GDNet, and the loss function adopts TMSE Loss.

[0056] A training module is configured to train the estimation model using the training data set.

[0057] A prediction module is configured to deploy the trained estimation model to a vehicle-mounted computing platform, process images collected by a camera in real time and output an estimated value of the hitching angle of the trailer head.

[0058] The device for estimating the hitching angle of the trailer head comprises an acquisition module, a construction module, a training module and a prediction module. The acquisition module synchronously acquires image data and true value data of the hitching angle of the trailer head, and performs time alignment processing on the image data and the true value data to form a training data set, wherein the image data contains complete structural information of the trailer head. The construction module constructs an end-to-end estimation model of the hitching angle of the trailer head, AngleNet, which comprises an input layer, a backbone network, a regression head and a loss function, wherein the backbone network is a gradually deformable convolution network GDNet, and the loss function adopts TMSE Loss. The training module trains the estimation model using the training data set. The prediction module deploys the trained estimation model to a vehicle-mounted computing platform, processes images collected by a camera in real time and outputs an estimated value of the hitching angle of the trailer head. The device can directly and robustly solve a high-precision hitching angle value from a complex real scene image, and simultaneously avoids the complex process of relying on multi-sensor fusion, feature point detection or geometric modeling in traditional methods.

[0059] According to a third aspect of the present application, a device for estimating a hitching angle of a trailer head is provided, comprising a processor and a memory, wherein the memory stores at least one computer instruction, and the instruction is loaded and executed by the processor to implement the steps performed in the hitching angle estimation method of any one of the above aspects.

[0060] According to a fourth aspect of the present application, a computer readable storage medium is provided, in which at least one computer instruction is stored, the instruction being loaded and executed by a processor to implement the steps performed in the articulated vehicle head suspension angle visual estimation method according to any one of the above aspects.

[0061] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present application. BRIEF DESCRIPTION OF DRAWINGS

[0062] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and serve to explain the principles of the present application, together with the description.

[0063] Figure 1 is a flowchart of the articulated vehicle head suspension angle visual estimation method provided by the embodiments of the present application;

[0064] Figure 2 is a schematic diagram of camera installation in the articulated vehicle head suspension angle visual estimation method provided by the embodiments of the present application;

[0065] Figure 3 is a structure diagram of GDNet in the articulated vehicle head suspension angle visual estimation method provided by the embodiments of the present application;

[0066] Figure 4 is a structure diagram of the articulated vehicle head suspension angle visual estimation device provided by the embodiments of the present application;

[0067] In the figure: 1 - tractor head; 2 - trailer; 3 - camera; 4 - main positioning unit of the integrated navigation; 5 - auxiliary positioning unit of the integrated navigation. DETAILED DESCRIPTION

[0068] The exemplary embodiments will be described in detail herein with reference to the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application.

[0069] The embodiments of the present application provide an articulated vehicle head suspension angle visual estimation method, as shown in Figure 1 The method comprises the following steps:

[0070] Step 101, synchronously collecting image data and head suspension angle true value data of the trailer, and performing time alignment processing on the image data and the head suspension angle true value data to form a training data set, wherein the image data contains complete structure information of the trailer head;

[0071] In actual use, the specific implementation process of the present application involves five major modules of camera installation and adaptation, trailer image and head-hitch angle data collection and processing, model construction, model training, and model deployment and application. First, the camera needs to be adapted and installed in combination with the actual vehicle model. Different vehicle models differ in vehicle size and structure, such as vehicle height, width, and the distance from the vehicle head to the trailer connection part. Therefore, when installing the camera in practice, adjustments need to be made for the specific vehicle model to ensure that effective trailer image information is collected.

[0072] In general, the camera should be installed behind the vehicle head, facing the trailer direction, and the pitch angle of the installation needs to be adjusted so that the camera field of view always contains the complete appearance of the trailer head during vehicle driving.

[0073] Specifically, in actual installation, the following factors need to be considered:

[0074] (1) Horizontal and vertical position of camera installation: Ensure that the vertical and horizontal angles of view of the camera cover the overall height of the trailer and the width of the vehicle head, avoiding trailer image cropping;

[0075] (2) Distance from camera to trailer: Adjust the front and rear positions to ensure that the camera can capture the trailer content completely;

[0076] (3) Pitch angle adjustment: Adjust the pitch angle so that the trailer head is completely presented in the imaging picture when the head-hitch angle is any value, providing sufficient information for subsequent algorithm processing.

[0077] As shown in Figure 2 , the installation position and direction of the camera under the tractor vehicle model are shown. Through this installation method, different vehicle models can ensure consistent camera imaging effect, providing a reliable data basis for subsequent head-hitch angle measurement.

[0078] In one embodiment, the step of synchronously collecting image data and head-hitch angle true value data of the trailer and performing time alignment processing on the image data and head-hitch angle true value data to form a training data set comprises:

[0079] Collecting image data using a camera at a first frequency;

[0080] Collecting head-hitch angle true value data using a combined navigation system at a second frequency, wherein the combined navigation system includes a main positioning unit installed on the vehicle head and a secondary positioning unit installed on the trailer;

[0081] Based on the timestamp information, the image data and the head-hitch angle true value data of different sampling frequencies are aligned using a linear interpolation method to form an image data and head-hitch angle true value data pair.

[0082] In actual use, the collection of image data and the head-hitch angle true value data is mainly composed of a visual perception subsystem and a reference measurement subsystem.

[0083] The visual perception subsystem is composed of the camera installed in the step S101, which is responsible for acquiring video stream image data of the trailer area at a fixed frequency of 30HZ;

[0084] The reference measurement subsystem is mainly composed of a combined navigation system (GNSS+IMU), which is installed on the front of the vehicle and the front of the trailer. Figure 2 It can be seen that the main positioning unit is fixedly installed on the top of the vehicle frame and is used for accurately measuring the attitude heading angle of the vehicle head, and the auxiliary positioning unit is fixedly installed on the front of the trailer and is used for accurately measuring the attitude of the trailer, and finally the head-hitch angle true value data is obtained through the relative relationship between the two. The collection frequency of the heading angle is 100HZ, and the measurement error is about 0.1°.

[0085] The method of the application only relies on a single camera and a combined navigation system to collect data sets, avoids expensive sensors and complex calibration processes, greatly reduces hardware costs and application thresholds, enables the system to quickly adapt to various vehicle models, and has strong generalizability and commercial value.

[0086] In one embodiment, the linear interpolation method specifically includes:

[0087] acquiring the timestamp of each image data;

[0088] finding the head-hitch angle true value data at the nearest time before and after the timestamp;

[0089] calculating the head-hitch angle true value data corresponding to the image data through a first formula, the first formula including:

[0090] (1)

[0091] wherein, t ang,i ≤t img ≤t ang,i+1 , θ i and θ i+1 are the head-hitch angle true value data at t ang,i , t ang,i+1 , and θ(t img ) is the head-hitch angle true value data corresponding to the image data at t img .

[0092] In practical use, after obtaining the trailer image data and the true value data of the head-mounted angle, a linear interpolation preprocessing operation is performed. The image and angle data are aligned by the timestamp information. That is, for each image, the head-mounted angle data under the nearest timestamp before and after it is first found, and then the true value of the head-mounted angle corresponding to the image is obtained by linear interpolation. After completing the data preprocessing task, the overall data is represented as a data pair of "trailer image + head-mounted angle", that is, each trailer image has a corresponding head-mounted angle value.

[0093] The method of this invention introduces a linear interpolation method in the data preprocessing stage to achieve accurate alignment between image data and the angle data of the integrated navigation head under different sampling frequencies. This ensures a strict correspondence between image features and real angle labels in the time dimension, significantly expands the richness of training data, and improves the accuracy and stability of model training.

[0094] Step 102: Construct an end-to-end head-hanging angle estimation model for AngleNet. The estimation model includes an input layer, a backbone network, a regression head, and a loss function. The backbone network is a progressively deformable convolutional network GDNet, and the loss function is TMSE Loss.

[0095] In this embodiment, the AngleNet end-to-end angle estimation network is used. This network mainly consists of four parts: input, backbone network, regression head, and loss function. The backbone network is GDNet (Grad-DeformNet, Progressively Deformable Convolutional Network).

[0096] In one embodiment, the progressively deformable convolutional network is built upon the ResNet18 base network and includes:

[0097] A convolutional layer is used to reduce image resolution and extract low-level edge and texture features;

[0098] The model employs a multi-stage structure, with at least one stage incorporating a deformable convolutional DCNv2 structure to enhance its adaptability to trailer deformation.

[0099] In this embodiment, the backbone network is constructed with reference to the ResNet18 base network, specifically consisting of one convolutional layer, four stage structures, and a global average pooling layer. The specific structure is as follows: Figure 3 As shown. It should be noted that the global average pooling layer, while aggregating global spatial information, transforms the feature map to a fixed length, generating a compact feature representation suitable for subsequent regression head processing.

[0100] The method adopts a Grad-DeformNet backbone structure, combines a gradient guiding mechanism with a deformable convolution DCNv2, and combines a standard convolution, so that the capturing capability of complex image features is improved while the network is lightweight, the generalization performance of the model under different vehicle types and different angles is significantly enhanced, and the modeling capability of the model for non-rigid deformation (such as different perspective forms of a trailer in an image) is enhanced.

[0101] In one embodiment, the plurality of stage structures specifically include:

[0102] The second convolution in the stage-2 stage adopts a DCNv2 structure, for introducing first-layer geometric modeling capability;

[0103] All 3x3 convolutions in the stage-3 stage adopt a DCNv2 structure, for improving spatial structure adaptability;

[0104] The stage-4 stage completely adopts a DCNv2 structure, for realizing joint modeling of high-level semantic and geometric features.

[0105] In the embodiment, the stage-1 stage is composed of basic convolution layers, and does not contain a DCNv2 module.

[0106] In the method, a progressive deformable convolution fusion mode of the backbone network is adopted, so that the model calculation amount is avoided from being too large, and the advantages of the deformable convolution structure are fully utilized in the deep layer, because the deformation of the trailer in the image is large under different vehicle types and different angles, the ordinary convolution can only extract regular grid features, and lacks the capture of deformation, and the introduction of the deformable convolution can adaptively focus on the edge and angle features of the trailer head area, and the robustness of the network to geometric changes is enhanced.

[0107] In one embodiment, an input layer is configured to receive three-channel RGB image data;

[0108] A regression head is composed of two full connection layers, and is configured to output a single continuous head-hitch angle estimation value.

[0109] Specifically, the input part includes an original trailer image and a real head-hitch angle under a corresponding timestamp , wherein the trailer image is a three-channel RGB image with a size of 3*448*448, and is specifically represented as .

[0110] In one embodiment, the loss function includes a mean square error loss term and a smooth loss constraint term, wherein the smooth loss constraint term is used to constrain the continuity of the head-hitch angle estimation value of adjacent frame images, and the mean square error loss term and the smooth loss constraint term are calculated through a second formula and a third formula respectively, and the second formula and the third formula include:

[0111] (2)

[0112] (3)

[0113] wherein, , are the head-pose angle estimation values at time t and t+1, respectively, and t is the head-pose angle true value data at time t, T is a sequence of image data, and one image corresponds to one time step t; L MSE is a mean square error loss term; L Smooth is a smoothing loss constraint term.

[0114] The loss function is calculated by a fourth formula, and the fourth formula includes:

[0115] (4)

[0116] wherein, λ is a smoothing weight parameter, and the value range is [0.01, 1.0]; L total is a total loss.

[0117] In this embodiment, the smoothing weight parameter adopts a fixed value mode, and the optimal value range is determined by implementing scene data under different working conditions, so as to ensure the balance between the prediction accuracy and the time sequence continuity.

[0118] On the basis of the traditional MSE loss, the method introduces a smoothing loss constraint term to constrain the continuity of the angle prediction value under adjacent frames of images, not only ensures the consistency of the prediction value and the true angle, but also solves the "flicker" problem that the MSE is prone to appear large and small on single frame image prediction, reduces the overfitting of the single frame noise data, effectively reduces the jitter phenomenon of the prediction curve, makes the prediction result more stable and physically reasonable, and is more suitable for the downstream vehicle control and decision-making link.

[0119] In one embodiment, the smoothing weight parameter is adjusted according to the vehicle operating state or the head-pose angle change rate, including:

[0120] When the change rate of the head-pose angle is detected to be lower than a preset threshold, the value of λ is increased to obtain a more stable prediction output;

[0121] When the change rate of the head-pose angle is detected to be higher than a preset threshold, the value of λ is reduced to enhance the response ability of the estimation model to rapid changes.

[0122] Specifically, the smoothing weight parameter can also be adjusted according to the vehicle operating state or the head-hanging angle change rate, for adjusting the balance between the precision and the stability of the model. When the weight value is small, the model is more inclined to fit the real label; when the weight value is large, the model pays more attention to the continuity and stability of the output.

[0123] In the method, the smoothing weight parameter can be adaptively adjusted according to the vehicle operating state or the head-hanging angle change rate, thereby improving the adaptability and robustness of the model under diversified working conditions.

[0124] Step 103, training the estimation model using the training data set;

[0125] In one embodiment, the estimation model is trained using the training data set, specifically including:

[0126] The ResNet18 basic network structure remaining in the estimation model is parameterized and initialized by loading the pre-training weight of ImageNet;

[0127] The newly added DCNv2 structure and regression head in the estimation model are randomly initialized;

[0128] The training parameters are set, including: learning rate, batch size, iteration number, optimizer and regularization, data enhancement;

[0129] The existing pre-training weight of the ResNet18 basic network structure is frozen, and only the newly added DCNv2 structure and regression head in the estimation model are trained;

[0130] All structures are unfrozen, and end-to-end training is performed in combination with the loss function.

[0131] In actual use, the initial learning rate can be set to 0.0001, and a learning rate decay strategy can be used to improve the convergence stability; the batch size can be combined with the hardware computing power and the size of the video memory, and is generally set to 16-32; the iteration number can be set according to the size of the data set, and is generally set to 50-100 rounds to ensure that the model fully learns the data features; the optimizer can select the Adam or SGD optimizer, and combine the momentum, weight decay and other techniques to improve the training effect. Regularization can introduce Dropout to prevent model overfitting, and can combine L2 weight decay to constrain the network, further improving the generalization ability and robustness of the model in unseen scenarios; brightness adjustment, random horizontal flip and other enhancement means can also be introduced to enhance the data, to improve the generalization ability of the model.

[0132] The method of the present application initializes the parameters of the newly added DCNv2 structure and the regression head through the ResNet18 basic network structure, fully utilizes the bottom and middle feature expression capabilities of the ResNet18 basic network structure, and also ensures the learnability of the newly added DCNv2 structure and the regression head in the head hanging angle estimation task. In addition, through the phased training mode, the estimation model realizes rapid convergence in the angle prediction task, and the optimal fitting of global features and angle prediction results, thereby improving the training efficiency of the model and the prediction accuracy of the model.

[0133] In addition, after obtaining the estimation model through training, the model is optimized and tested through the verification data set and the test data set.

[0134] Step 104, deploying the trained estimation model to the vehicle-mounted computing platform, processing the image collected by the camera in real time and outputting the head hanging angle estimation value.

[0135] In this embodiment, after obtaining the optimal precision training model, the model is applied to quantitative processing to balance the time consumption and precision performance of the model, and then the model is deployed on the vehicle-mounted computing platform to realize end-side real-time inference. After the camera image is processed by the model, the angle value between the vehicle head and the trailer can be output in real time.

[0136] The hinge vehicle head hanging angle visual estimation method provided by the embodiment of the present application first synchronously collects image data and head hanging angle true value data of the trailer, and performs time alignment processing on the image data and the head hanging angle true value data to form a training data set, wherein the image data contains complete structure information of the trailer head. Then, an AngleNet end-to-end head hanging angle estimation model is constructed, the estimation model comprising an input layer, a backbone network, a regression head and a loss function, wherein the backbone network is a progressive deformable convolution network GDNet, and the loss function adopts TMSE Loss. The training data set is used to train the estimation model. Finally, the trained estimation model is deployed to the vehicle-mounted computing platform to process the image collected by the camera in real time and output the head hanging angle estimation value. The method of the present application directly uses the trailer image as the input, and the model can directly and robustly solve the high-precision head hanging angle value from the complex real scene image, without the need for additional markers or complex sensor arrangement, and can provide stable and accurate angle estimation under various working conditions, and can provide reliable data for intelligent functions such as 360° panoramic image, reversing assistance and automatic driving of the articulated commercial vehicle.

[0137] In summary, the present application only relies on a single vehicle-mounted camera and a set of combined navigation system to complete the data collection and preparation required for model training, without the need for additional installation of complex sensors or external auxiliary devices, greatly reducing the hardware dependence and deployment cost. In the actual application stage, only one camera needs to be installed at the predetermined position of the vehicle, and the system can automatically complete the trailer angle perception and estimation, with the advantages of easy installation, strong adaptability, low application threshold, etc.

[0138] In addition, thanks to the AngleNet recognition network proposed in the present application, under the assistance of Grad-DeformNet backbone structure and TMSE loss function, the model can fully mine the trailer image features and combine the time sequence constraint to realize high-precision prediction of the head-hung angle. In typical test conditions, the overall prediction accuracy can reach more than 96%, effectively ensuring the stability, reliability and safety of the vehicle during operation. Compared with the existing method relying on multi-sensor fusion, the present application scheme is not only more concise in hardware deployment, but also has significant advantages in data processing efficiency, model inference real-time performance and angle accuracy.

[0139] Based on the above Figure 1 The hinge trailer head-hung angle visual estimation method described in the corresponding embodiment is as follows: the device embodiment of the present application can be used to execute the method embodiment of the present application.

[0140] The hinge trailer head-hung angle visual estimation device provided by the embodiment of the present application is as shown in the figure, which comprises: Figure 4 As shown in the figure, the device comprises:

[0141] The acquisition module 201 is used for synchronously acquiring image data and head-hung angle true value data of the trailer, and performing time alignment processing on the image data and the head-hung angle true value data to form a training data set, wherein the image data contains complete structure information of the trailer head;

[0142] The construction module 202 is used for constructing an AngleNet end-to-end head-hung angle estimation model, and the estimation model comprises an input layer, a backbone network, a regression head and a loss function, wherein the backbone network is a progressive deformable convolution network GDNet, and the loss function adopts TMSE Loss;

[0143] The training module 203 is used for training the estimation model by using the training data set;

[0144] The prediction module 204 is used for deploying the trained estimation model to a vehicle-mounted computing platform, processing the image collected by the camera in real time and outputting the head-hung angle estimation value.

[0145] The hinge trailer head hanging angle visual estimation device provided by the embodiment of the application comprises a collection module 201, a construction module 202, a training module 203 and a prediction module 204. The collection module 201 synchronously collects image data and head hanging angle true value data of a trailer, and performs time alignment processing on the image data and the head hanging angle true value data to form a training data set, wherein the image data contains complete structure information of a trailer head. The construction module 202 constructs an AngleNet end-to-end head hanging angle estimation model, wherein the estimation model comprises an input layer, a backbone network, a regression head and a loss function, the backbone network is a progressive deformable convolution network GDNet, and the loss function adopts TMSE Loss. The training module 203 trains the estimation model by using the training data set. The prediction module 204 deploys the trained estimation model to a vehicle-mounted computing platform, processes images collected by a camera in real time, and outputs a head hanging angle estimation value. The device can directly and robustly solve a high-precision head hanging angle value from a complex real scene image, and simultaneously avoids a complex process of relying on multi-sensor fusion, feature point detection or geometric modeling in a traditional method.

[0146] Based on the above Figure 1 The hinge trailer head hanging angle visual estimation method described in the corresponding embodiment, another embodiment of the application further provides a hinge trailer head hanging angle visual estimation device, which comprises a processor and a memory, and at least one computer instruction is stored in the memory, the instruction is loaded and executed by the processor to realize the above Figure 1 The hinge trailer head hanging angle visual estimation method described in the corresponding embodiment.

[0147] Based on the above Figure 1 The hinge trailer head hanging angle visual estimation method described in the corresponding embodiment, the embodiment of the application further provides a computer readable storage medium, for example, a non-transitory computer readable storage medium can be a read only memory (English: Read Only Memory, ROM), a random access memory (English: Random Access Memory, RAM), a CD-ROM, a magnetic tape, a floppy disk and an optical data storage system and the like. The storage medium stores at least one computer instruction for executing the above Figure 1 The hinge trailer head hanging angle visual estimation method described in the corresponding embodiment, which will not be described here.

[0148] Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. It is intended that the specification and examples be considered as exemplary only, with the true scope and spirit of the application being indicated by the following claims.

[0149] It is to be understood that the application is not limited to the precise construction hereinafter described and as shown in the attached drawings, and that various changes in form and detail can be made therein without departing from the scope thereof. The scope of the application should only be limited by the appended claims.

Claims

1. A visual estimation method for the hinged front end angle of a vehicle, characterized in that, The method includes the following steps: Simultaneously acquire image data and ground truth data of the trailer cab angle, and perform time alignment processing on the image data and ground truth data of the trailer cab angle to form a training dataset, wherein the image data contains complete structural information of the trailer cab; An end-to-end head-hanging angle estimation model for AngleNet is constructed. The estimation model includes an input layer, a backbone network, a regression head, and a loss function. The backbone network is a progressively deformable convolutional network (GDNet) built on the ResNet18 base network. The progressively deformable convolutional network includes one convolutional layer, four stage structures, and a global average pooling layer. In the four stage structures: the second convolution in stage-2 uses a DCNv2 structure to introduce first-layer geometric modeling capabilities; all 3×3 convolutions in stage-3 use a DCNv2 structure to improve spatial structure adaptability; and stage-4 uses a complete DCNv2 structure to achieve joint modeling of high-level semantic and geometric features. The loss function is TMSE Loss, which includes a mean squared error loss term and a smoothing loss constraint term. The mean squared error loss term and the smoothing loss constraint term are calculated using a first formula and a second formula, respectively. The first and second formulas include: （1） （2） in, , The estimated head-hanging angles at times t and t+1 are θ. t Let L be the true value of the head-hanging angle at time t, where T is the image data sequence and one image corresponds to one time step t; MSE For the mean squared error loss term; L Smooth This is a smoothing loss constraint term; The loss function is calculated using a third formula, which includes: （3） Where λ is the smoothing weight parameter with a value range of [0.01, 1.0], L total Total loss; The estimated model is trained using the training dataset; The trained estimation model is deployed to the vehicle computing platform to process the images captured by the camera in real time and output the estimated value of the head-mounted angle.

2. The visual estimation method for the hinged front end hanging angle according to claim 1, characterized in that, The steps of synchronously acquiring image data and ground truth data of the trailer's cab-over angle, and performing time alignment processing on the image data and ground truth data of the cab-over angle to form a training dataset specifically include: The image data is acquired using a camera at a first frequency; The true value data of the head-mounted angle is collected at a second frequency using a combined navigation system, wherein the combined navigation system includes a main positioning unit installed at the head of the vehicle and a secondary positioning unit installed on the trailer; Based on timestamp information, a linear interpolation method is used to align image data with the true value data of the head-hanging angle at different sampling frequencies, forming image data and head-hanging angle true value data pairs.

3. The visual estimation method for the hinged front end hanging angle according to claim 2, characterized in that, The linear interpolation method specifically includes: Obtain the timestamp of each of the image data; Find the true value of the head-hanging angle at the nearest moment before and after the timestamp; The true value of the head-hook angle corresponding to the image data is calculated using a fourth formula, which includes: （4） Among them, t ang,i ≤t img ≤t ang,i+1 θ i and θ i+1 They correspond to t respectively ang,i t ang,i+1 True value data of the head-hanging angle at time t, θ(t) img ) for t img The true value of the head-hanging angle corresponding to the time-lapse image data.

4. The visual estimation method for the hinged front end hanging angle according to claim 1, characterized in that, The step of training the estimation model using the training dataset specifically includes: The parameters of the ResNet18 base network structure retained in the estimated model are initialized by loading the pre-trained weights from ImageNet. The newly added DCNv2 structure and regression head in the estimation model are randomly initialized; Set training parameters, including: learning rate, batch size, number of iterations, optimizer and regularization, and data augmentation; Freeze the existing pre-trained weights of the ResNet18 base network structure and train only the newly added DCNv2 structure and regression head in the estimated model; Unfreeze all structures and perform end-to-end training using the loss function.

5. The method for visually estimating the hinged front end angle according to claim 1, characterized in that, The smoothing weight parameters are adjusted according to the vehicle's operating status or the rate of change of the head-mounted clamp angle, including: When the rate of change of the head-mounted angle is detected to be lower than the preset threshold, the value of λ is increased to obtain a more stable prediction output. When the rate of change of the head-mounted angle is detected to be higher than a preset threshold, the value of λ is reduced to enhance the response capability of the estimation model to rapid changes.

6. The method for visually estimating the hinged front end angle according to claim 1, characterized in that, The input layer is used to receive three-channel RGB image data; The regression head consists of two fully connected layers and is used to output a single, continuous head-hanging angle estimate.

7. A visual estimation device for the hinged front end hanging angle, characterized in that, include: The acquisition module is used to synchronously acquire image data and head-mounted angle ground truth data of the trailer, and perform time alignment processing on the image data and head-mounted angle ground truth data to form a training dataset, wherein the image data contains complete structural information of the trailer head; The module is used to construct an end-to-end head-hook angle estimation model for AngleNet. The estimation model includes an input layer, a backbone network, a regression head, and a loss function. The backbone network is a progressively deformable convolutional network (GDNet) built on the ResNet18 base network. The progressively deformable convolutional network includes one convolutional layer, four stage structures, and a global average pooling layer. In the four stage structures: the second convolution in stage-2 uses a DCNv2 structure to introduce first-layer geometric modeling capabilities; all 3×3 convolutions in stage-3 use a DCNv2 structure to improve spatial structure adaptability; and stage-4 uses a complete DCNv2 structure to achieve joint modeling of high-level semantics and geometric features. The loss function uses TMSE Loss, which includes a mean squared error loss term and a smoothing loss constraint term. The smoothing loss constraint term constrains the continuity of head-hook angle estimates between adjacent frames. The mean squared error loss term and the smoothing loss constraint term are calculated using a first formula and a second formula, respectively. The first formula and the second formula include: （1） （2） in, , The estimated head-hanging angles at times t and t+1 are θ. t Let L be the true value of the head-hanging angle at time t, where T is the image data sequence and one image corresponds to one time step t; MSE For the mean squared error loss term; L Smooth This is a smoothing loss constraint term; The loss function is calculated using a third formula, which includes: （3） Where λ is the smoothing weight parameter with a value range of [0.01, 1.0], L total Total loss; A training module is used to train the estimated model using the training dataset; The prediction module is used to deploy the trained estimation model to the vehicle computing platform, process the images captured by the camera in real time, and output the estimated value of the head-mounted angle.

Images