Real-time monitoring method, device and equipment for included angle of commercial vehicle head hanger and medium

Abstract

The invention discloses a real-time monitoring method, device and equipment for a commercial vehicle head hanging included angle and a medium, and belongs to the technical field of computer vision and commercial vehicle safety auxiliary driving. The technical problems of strong dependence on manual identification or specific geometric characteristics, weak environmental interference resistance and insufficient model generalization in the existing commercial vehicle head hanging included angle monitoring can be solved. Comprising the steps that firstly, an image containing a trailer front edge area is collected through a fisheye camera installed at the tail of a tractor, the image is preprocessed to extract an area of interest, and data are enhanced; then the preprocessed image is input into a trained head-mounted included angle prediction network MFARNet, end-to-end included angle regression is carried out by fusing trailer target frame features and key point features, sine and cosine prediction values of a head-mounted included angle are output, and the prediction network MFARNet is constructed based on YOLOv8-Pose improvement; and finally obtaining a head hanging included angle prediction value according to the sine and cosine prediction values. The method is used for real-time monitoring of the included angle of the commercial headstock.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and commercial vehicle safety assistance driving technology, specifically relating to a method, device, equipment and medium for real-time monitoring of the front-mounted angle of a commercial vehicle. Background Technology

[0002] As modern logistics systems rapidly evolve towards higher efficiency and intelligence, articulated vehicles, as the core carriers of long-haul transportation, are experiencing a continuous increase in scale and complexity. Unlike ordinary single-unit vehicles, tractor-trailers are articulated via a saddle, making the articulation point extremely sensitive under conditions such as low-speed turns, U-turns, reversing, and intersections. Even slight operational errors can induce sudden angle changes, trailer swaying, a sharp increase in driver workload, and a significant rise in the risk of collisions and rollovers. This is particularly pronounced in high-frequency operation scenarios in cities and industrial parks, becoming a core bottleneck restricting efficiency and safety. Against this backdrop, how to achieve real-time, accurate, and reliable monitoring of the vehicle's cab-trailer angle has become an important research direction and industry demand for improving driver assistance levels and reducing accident rates.

[0003] Currently, in the field of commercial vehicle trailer angle monitoring, real-time measurement of the trailer angle is mainly achieved through visual means. Existing solutions mainly fall into three categories: First, trailer angle measurement based on manual features (markers / auxiliary lines) relies on preset features for geometric calculation, but requires additional installation and is easily affected by environmental interference, resulting in poor robustness and high maintenance costs; Second, methods based on geometric modeling (inherent features / optical flow, etc.) utilize the inherent contours of the trailer or visual features such as optical flow, but its accuracy is severely affected by lighting, occlusion, or dirt, and the stability of feature extraction is insufficient; Third, methods based on deep learning indirectly deduce the angle through segmentation, depth estimation, etc., but are prone to result deviations due to errors in the preceding tasks, or rely on weakly representative features such as tires, resulting in limited model generalization ability. Summary of the Invention

[0004] To address the technical problems of existing commercial vehicle cab-over angle monitoring methods, such as strong reliance on manual markings or specific geometric features, weak resistance to environmental interference, and insufficient model generalization, this invention provides a real-time monitoring method and device for commercial vehicle cab-over angle. The technical solution is as follows:

[0005] According to a first aspect of the present invention, a method for real-time monitoring of the front-end clamping angle of a commercial vehicle is provided, the method comprising the following steps:

[0006] Images containing the leading edge area of ​​the trailer are acquired by a fisheye camera mounted at the rear of the tractor unit, and the images are preprocessed to extract the region of interest and enhance the data.

[0007] The preprocessed image is input into the trained head-mounted angle prediction network MFARNet. By fusing the trailer target box features and key point features, end-to-end angle regression is performed, and the sine and cosine predicted values ​​of the head-mounted angle are output. The prediction network MFARNet is built based on YOLOv8-Pose.

[0008] The predicted value of the head-hanging angle is obtained based on the predicted values ​​of sine and cosine.

[0009] The real-time monitoring method for the trailer-mounted angle of a commercial vehicle provided by this invention first acquires an image containing the leading edge region of the trailer using a fisheye camera installed at the rear of the tractor unit. The image is then preprocessed to extract the region of interest and enhance the data. Next, the preprocessed image is input into a trained trailer-mounted angle prediction network, MFARNet. End-to-end angle regression is performed by fusing trailer bounding box features and keypoint features, outputting the sine and cosine predicted values ​​of the trailer-mounted angle. The prediction network MFARNet is an improved version of YOLOv8-Pose. Finally, the predicted trailer-mounted angle value is obtained based on the sine and cosine predicted values. This invention effectively overcomes the dependence of traditional geometric measurement methods on external markers and strict calibration through multimodal regression and end-to-end design, achieving higher accuracy angle estimation under purely visual conditions. Simultaneously, the multimodal feature fusion mechanism enhances robustness to complex lighting, occlusion, and different vehicle models. Furthermore, the end-to-end overall design significantly reduces system latency and complexity, facilitating lightweight deployment on embedded platforms.

[0010] As a further aspect of the present invention: the trained head-hanging angle prediction network MFARNet is obtained through the following training method:

[0011] The ground truth value of the head-mounted angle is obtained by using a combined navigation system installed on the tractor and trailer, and images from the vehicle-mounted camera are collected simultaneously to construct a training dataset. The annotation information of the dataset includes the target box at the leading edge of the trailer, key points, and the ground truth value of the head-mounted angle.

[0012] Construct an MFARNet network based on YOLOv8-Pose improvement and train it end-to-end using the training dataset;

[0013] The total loss function used in training is:

[0014] L Total =λ1L box +λ2L cls +λ3L pose +λ4L SCL

[0015] Among them, L Total L box Lcls L pose L SCL λ1, λ2, λ3, and λ4 represent the total loss, bounding box loss, classification loss, keypoint loss, and angle regression loss, respectively; λ1, λ2, λ3, and λ4 represent the gain coefficients corresponding to the bounding box loss, classification loss, keypoint loss, and angle regression loss, respectively.

[0016] The method of this invention obtains high-precision ground truth data through a combined navigation system for network training, and adopts an end-to-end training method that includes multi-task loss terms, which helps to improve the overall accuracy and convergence stability of the model prediction.

[0017] As a further aspect of the present invention: the angle regression loss L SCL Calculated using the following formula:

[0018] L SCL =L cos +λ5L smooth

[0019] L cos =1-(P (sin,cos) *T (sin,cos) )

[0020] Among them, L cos For the sine and cosine loss of the included angle, L smooth To smooth the L1 loss, λ5 is the gain coefficient for smoothing the L1 loss, P (sin,cos) T represents the predicted sine and cosine values ​​of the head-hanging angle. (sin,cos) Let be the sine and cosine values ​​of the true value of the angle between the heads.

[0021] The method of this invention effectively avoids the boundary discontinuity problem commonly found in angle regression by employing a composite angle regression loss function, significantly improving the smoothness and stability of the angle prediction. At the same time, this loss function takes into account both the directionality and numerical accuracy of the angle, enabling the model to maintain continuous and interpretable angle output in actual vehicle turning scenarios. In addition, this method also improves the training convergence speed and generalization performance, helping to reduce estimation errors under small sample conditions.

[0022] As a further aspect of the present invention: the preprocessed image is input into the trained head-mounted angle prediction network MFARNet, and end-to-end angle regression is performed by fusing trailer target box features and key point features to output the sine and cosine predicted values ​​of the head-mounted angle, specifically including:

[0023] Multi-scale feature extraction is performed on the preprocessed image to obtain semantic and detail features at different levels;

[0024] The extracted multi-scale features are fused to generate fused features;

[0025] Based on the fusion features, predict the bounding box features and key point features of the trailer's leading edge;

[0026] The predicted target box features are fused with the key point features, and the sine and cosine predicted values ​​of the head-hanging angle are calculated by regression.

[0027] The method of this invention first extracts and fuses multi-scale features, and then jointly predicts the target box and key points at the leading edge of the trailer. This method can comprehensively utilize the semantic and detailed information of the image, enhance the model's ability to represent the trailer structure, and provide a more robust feature foundation for subsequent angle regression.

[0028] As a further aspect of the present invention: the fusion of the predicted target box features and the key point features, and the calculation of the sine and cosine predicted values ​​of the head-hanging angle through regression, specifically includes:

[0029] The target bounding box features and the keypoint features are reconstructed and concatenated to obtain fused features. These fused features are then input into a multilayer perceptron for regression calculation to obtain the predicted sine and cosine values ​​of the head-hanging angle; or...

[0030] The target bounding box features and the key point features are enhanced and fused through attention mechanism and feature cross-fusion to obtain the sine and cosine predicted values ​​of the head-hanging angle.

[0031] The method of this invention provides a flexible implementation of feature fusion. It can ensure real-time performance with low computational cost by combining reconstruction splicing with multilayer perceptrons, or it can enhance the model's ability to model complex features by using attention mechanisms or feature cross-fusion. Thus, it provides an optional solution that is suitable for different scenarios between accuracy and efficiency.

[0032] As a further aspect of the present invention: the preprocessing of the image to extract the region of interest and enhance the data specifically includes:

[0033] Based on the installation parameters of the fisheye camera, a region of interest containing the trailer connection part and the leading edge structure is extracted from the image;

[0034] Data augmentation is performed on the image within the region of interest. The data augmentation operations include at least one of brightness adjustment, contrast adjustment, color perturbation, translation, scaling, and horizontal flipping.

[0035] The enhanced image is then normalized and standardized.

[0036] The method of this invention reduces irrelevant background interference by extracting regions of interest and performing diverse data augmentation on the original image, thereby enhancing the model's adaptability to installation errors, pitch angle changes, and different lighting and viewing conditions. At the same time, this method can also effectively reduce estimation bias caused by environmental changes such as day-night transitions and rain or snow, thus improving the overall robustness in real-world complex driving environments.

[0037] As a further aspect of the present invention, the method further includes:

[0038] The predicted head-mounted angle is compared with a preset safety threshold.

[0039] When the predicted value of the head-mounted angle exceeds the threshold range, an audible and visual alarm is triggered.

[0040] The predicted head-mounted angle is displayed in real time when it is within the safe threshold range.

[0041] The method of this invention compares the predicted angle with a safety threshold and links it with an audible and visual alarm and a display interface, which can provide drivers with intuitive status feedback and hazard warnings, thus helping to improve the safety of driving operations.

[0042] According to a second aspect of the present invention, a real-time monitoring device for the front-end clamping angle of a commercial vehicle is provided, comprising:

[0043] The acquisition module is used to acquire images containing the leading edge area of ​​the trailer using a fisheye camera installed at the rear of the tractor, and to preprocess the images to extract the region of interest and enhance the data;

[0044] The prediction module is used to input the preprocessed image into the trained head-mounted angle prediction network MFARNet, and perform end-to-end angle regression by fusing trailer target box features and key point features, and output the sine and cosine predicted values ​​of the head-mounted angle. The prediction network MFARNet is built based on YOLOv8-Pose.

[0045] The module is used to obtain the predicted value of the head-hanging angle based on the predicted values ​​of the sine and cosine.

[0046] The present invention provides a real-time monitoring device for the cab-mounted trailer angle of a commercial vehicle, comprising: a data acquisition module, a prediction module, and an acquisition module; the data acquisition module acquires images containing the leading edge region of the trailer using a fisheye camera installed at the rear of the tractor, and preprocesses the images to extract the region of interest and enhance the data; the prediction module inputs the preprocessed images into a trained cab-mounted trailer angle prediction network MFARNet, and performs end-to-end angle regression by fusing trailer target box features and key point features, outputting the sine and cosine predicted values ​​of the cab-mounted trailer angle, wherein the prediction network MFARNet is constructed based on an improvement of YOLOv8-Pose; the acquisition module obtains the predicted value of the cab-mounted trailer angle based on the sine and cosine predicted values. The device of this invention does not rely on additional markers or sensors. It can achieve real-time, non-contact, high-precision estimation of the head-to-trailer angle between the tractor and trailer using only a single fisheye camera and an end-to-end multi-task deep learning model. It has the advantages of simple deployment and low cost. The overall system structure is simple and can adapt to different vehicle models and installation locations. The end-to-end design effectively reduces the accumulation of external geometric calculation errors in traditional solutions. At the same time, the device can achieve millisecond-level real-time detection of the angle, which can meet the usage needs of various scenarios such as commercial vehicle reversing assistance and fleet formation.

[0047] According to a third aspect of the present invention, a real-time monitoring device for the front-end angle of a commercial vehicle is provided. The real-time monitoring device for the front-end angle of a commercial vehicle includes a processor and a memory. The memory stores at least one computer instruction, which is loaded and executed by the processor to perform the steps in the real-time monitoring method for the front-end angle of a commercial vehicle as described above.

[0048] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, the storage medium storing at least one computer instruction, the instruction being loaded and executed by a processor to perform the steps performed in the real-time monitoring method for the cab-off angle of a commercial vehicle as described in any of the preceding claims.

[0049] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0050] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0051] Figure 1 This is a flowchart of a real-time monitoring method for the cab-mounted angle of a commercial vehicle provided in an embodiment of the present invention;

[0052] Figure 2 This is a schematic diagram of the tractor, trailer, camera, and integrated navigation device provided in the embodiments of the present invention.

[0053] Figure 3 This is a structural diagram of the MFARNet network provided in an embodiment of the present invention.

[0054] Figure 4 This is a flowchart of the multi-level alarm mechanism provided in the embodiments of the present invention;

[0055] Figure 5 This is a structural diagram of the real-time monitoring device for the front-end clamping angle of a commercial vehicle provided in an embodiment of the present invention.

[0056] Among them, 1-tractor, 2-first positioning unit of the integrated navigation system, 3-fisheye camera, 4-second positioning unit of the integrated navigation system, and 5-trailer. Detailed Implementation

[0057] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention.

[0058] This invention provides a method for real-time monitoring of the cab-mounted angle of a commercial vehicle, such as... Figure 1 As shown, it includes the following steps:

[0059] Step 101: Acquire an image containing the leading edge area of ​​the trailer using a fisheye camera installed at the rear of the tractor unit, and preprocess the image to extract the region of interest and enhance the data.

[0060] Specifically, the fisheye camera, with its ultra-wide-angle view, can be installed in the middle of the rear of the tractor unit to effectively cover the entire cab connection area, avoiding image loss during sharp turns or uneven lighting, thus improving the completeness of observation and the continuity of angle estimation. After image acquisition, this method employs preprocessing operations including ROI extraction, data augmentation, and normalization, and directly utilizes network learning of distorted image features, avoiding coordinate errors and computational overhead associated with traditional geometric correction, significantly improving system real-time performance. Simultaneously, the wide-angle view expands the visible range, enhancing detection stability and coverage, and reducing system calibration difficulty and overall algorithm complexity, making the solution more lightweight and easier for mass production deployment.

[0061] In one embodiment, image preprocessing to extract regions of interest and enhance data specifically includes:

[0062] Based on the installation parameters of the fisheye camera, the region of interest containing the trailer connection part and the leading edge structure is extracted from the image;

[0063] Data augmentation is performed on the image within the region of interest. The data augmentation operations include at least one of brightness adjustment, contrast adjustment, color perturbation, translation, scaling, and horizontal flipping.

[0064] The enhanced image is then normalized and standardized.

[0065] In practical use, the first step is to determine the Region of Interest (ROI) containing the trailer connection points and front-end structure based on the acquired images of the trailer's leading edge and the camera installation parameters and imaging relationships. ROI extraction effectively removes redundant background information, reduces the network's input dimensions, and allows for a more focused attention on key areas in the image, improving the accuracy and stability of angle prediction. Next, image data enhancement techniques are applied, including random brightness and contrast adjustments, color perturbations, translation and scaling, and random horizontal flipping, to improve the network's robustness to diverse trailer postures, lighting variations, and camera installation differences. This module ensures the model can learn more diverse visual feature distributions during training, maintaining high-precision angle prediction performance in real-world environments. Finally, after all enhancement operations, image normalization and standardization are performed. This standardization operation normalizes the mean and standard deviation of each channel, ensuring consistent distribution across different channels, thereby accelerating model convergence and improving training stability.

[0066] The method of this invention reduces irrelevant background interference by extracting regions of interest and performing diverse data augmentation on the original image, thereby enhancing the model's adaptability to installation errors, pitch angle changes, and different lighting and viewing conditions. At the same time, this method can also effectively reduce estimation bias caused by environmental changes such as day-night transitions and rain or snow, thus improving the overall robustness in real-world complex driving environments.

[0067] Step 102: Input the preprocessed image into the trained head-mounted angle prediction network MFARNet. By fusing the trailer target box features and key point features, perform end-to-end angle regression and output the sine and cosine predicted values ​​of the head-mounted angle.

[0068] In practical applications, the prediction network MFARNet is built upon an improvement upon YOLOv8-Pose. For example... Figure 3 As shown, the MFARNet network mainly consists of five parts: the feature extraction backbone, the neck, the head, the angle regression, and the loss function.

[0069] In one embodiment, the trained head-hanging angle prediction network MFARNet is obtained through the following training method:

[0070] The ground truth value of the nose-mounted angle is obtained by using a combined navigation system installed on the tractor and trailer, and images from the vehicle-mounted camera are collected simultaneously to construct a training dataset. The annotation information of the dataset includes the target box at the leading edge of the trailer, key points, and the ground truth value of the nose-mounted angle.

[0071] Construct an MFARNet network based on YOLOv8-Pose improvement and train it end-to-end using the training dataset;

[0072] The total loss function used in training is:

[0073] L Total =λ1L box +λ2L cls +λ3L pose +λ4L SCL

[0074] Among them, L Total L box L cls L pose L SCL λ1, λ2, λ3, and λ4 represent the total loss, bounding box loss, classification loss, keypoint loss, and angle regression loss, respectively; λ1, λ2, λ3, and λ4 represent the gain coefficients corresponding to the bounding box loss, classification loss, keypoint loss, and angle regression loss, respectively.

[0075] In specific implementation, such as Figure 2 As shown, a camera is first installed at the center of the rear of the tractor unit, and the first and second positioning units of the integrated navigation system are installed at the front of the tractor and trailer to acquire ground truth data of the head-mounted angle. After the equipment installation and adjustment are completed, image sequences of the tractor and trailer are collected under different head-mounted angle conditions to construct a dataset for model training. Subsequently, the model is optimized and tested using the validation dataset and the test dataset.

[0076] Furthermore, this invention abandons the traditional single image processing flow and innovatively designs a head-hanging angle regression estimation model based on multi-feature angle fusion (Multi-FeatureAngle RegressionNetwork, MFARNet), achieving end-to-end accurate prediction from perception to understanding.

[0077] The method of this invention obtains high-precision ground truth data through a combined navigation system for network training, and adopts an end-to-end training method that includes multi-task loss terms, which helps to improve the overall accuracy and convergence stability of the model prediction.

[0078] In one embodiment, the angle regression loss L SCL Calculated using the following formula:

[0079] L SCL =L cos +λ5L smooth

[0080] L cos =1-(P (sin,cos) *T (sin,cos) )

[0081] Among them, L cos For the sine and cosine loss of the included angle, L smooth To smooth the L1 loss, λ5 is the gain coefficient for smoothing the L1 loss, P (sin,cos) T represents the predicted sine and cosine values ​​of the head-hanging angle. (sin,cos) Let be the sine and cosine values ​​of the true value of the angle between the heads.

[0082] In this embodiment, based on the losses of the bounding box, classification, and key points, a composite loss term SCLoss is fused for angle prediction. It combines directional consistency constraints and angle numerical smoothing constraints, which can simultaneously ensure the geometric consistency of the predicted direction and the accuracy of the angle estimation.

[0083] The method of this invention effectively avoids the boundary discontinuity problem commonly found in angle regression by employing a composite angle regression loss function, significantly improving the smoothness and stability of the angle prediction. At the same time, this loss function takes into account both the directionality and numerical accuracy of the angle, enabling the model to maintain continuous and interpretable angle output in actual vehicle turning scenarios. In addition, this method also improves the training convergence speed and generalization performance, helping to reduce estimation errors under small sample conditions.

[0084] In one embodiment, the preprocessed image is input into a trained MFARNet head-mount angle prediction network. End-to-end angle regression is performed by fusing trailer bounding box features and keypoint features to output the sine and cosine predicted values ​​of the head-mount angle. Specifically, this includes:

[0085] Multi-scale feature extraction is performed on the preprocessed image to obtain semantic and detail features at different levels;

[0086] The extracted multi-scale features are fused to generate fused features;

[0087] Based on the fusion features, predict the bounding box features and key point features of the trailer's leading edge;

[0088] The predicted bounding box features are fused with the key point features, and the predicted values ​​of the sine and cosine of the head-hanging angle are calculated by regression.

[0089] In this embodiment of the invention, the Backbone part of the MFARNet network structure mainly consists of Conv, C2f, and SPPF structures, which perform multi-scale feature extraction on the input RGB image to obtain semantic and detail information at different levels. Specifically, Conv is composed of regular convolution, BN, and SILU; the C2f structure is used to extract the depth features of the input; SPPF is used to solve the problem of detecting targets at different scales; and C in the network represents the concatenation operation. The Neck part is responsible for fusing multi-scale features to improve the robustness of keypoint and trailer target box localization. The Head module is responsible for predicting trailer keypoints and trailer target boxes, providing input features for the subsequent angle regression module. The angle regression module mainly consists of feature reconstruction processing and a multilayer perceptron structure.

[0090] The method of this invention first extracts and fuses multi-scale features, and then jointly predicts the target box and key points at the leading edge of the trailer. This method can comprehensively utilize the semantic and detailed information of the image, enhance the model's ability to represent the trailer structure, and provide a more robust feature foundation for subsequent angle regression.

[0091] In one embodiment, the predicted bounding box features are fused with keypoint features, and the sine and cosine predicted values ​​of the head-hanging angle are calculated by regression, specifically including:

[0092] The target bounding box features and keypoint features are reconstructed and concatenated to obtain fused features. These fused features are then input into a multilayer perceptron for regression calculation to obtain the predicted sine and cosine values ​​of the head-hanging angle; or...

[0093] The target bounding box features and key point features are enhanced and fused through attention mechanism and feature cross-fusion to obtain the sine and cosine predicted values ​​of the head-hanging angle.

[0094] In this embodiment of the invention, the angle regression module employs a multimodal fusion structure based on target bounding box features and keypoint features for angle regression. The two types of features are fused through reconstruction and concatenation operations. Specifically, the detection features and keypoint features output by the Head are first reconstructed and concatenated to obtain fused features; then, the fused features are fed into a multilayer perceptron, ultimately outputting the predicted angle between the tractor and trailer. Furthermore, enhanced fusion processing can be performed using structures such as an attention mechanism and a cross-fusion module to further improve the model's ability to represent complex poses.

[0095] The method of this invention provides a flexible implementation of feature fusion. It can ensure real-time performance with low computational cost by combining reconstruction splicing with multilayer perceptrons, or it can enhance the model's ability to model complex features by using attention mechanisms or feature cross-fusion. Thus, it provides an optional solution that is suitable for different scenarios between accuracy and efficiency.

[0096] Step 103: Obtain the predicted value of the head-hanging angle based on the predicted values ​​of sine and cosine.

[0097] Specifically, after obtaining the model inference results, the predicted head-hanging angle is calculated based on the angle regression components output by the model. The specific formula is as follows:

[0098] x = arctan2(sinx, cosx)

[0099] Where x is the predicted value of the head-hanging angle, and sinx and cosx are the predicted values ​​of the sine and cosine of the head-hanging angle, respectively.

[0100] In one embodiment, the above method further includes:

[0101] Compare the predicted head-hanging angle with the preset safety threshold;

[0102] When the predicted value of the head-mounted angle exceeds the threshold range, an audible and visual alarm is triggered.

[0103] When the predicted value of the head-hanging angle is within the safe threshold range, it is displayed in real time.

[0104] It should be noted that the predicted head-mounted angle is used for subsequent vehicle safety status assessment and alarm control. For example... Figure 4 As shown, for vehicles with a tractor-trailer structure, this invention designs a multi-level alarm mechanism:

[0105] After calculating the predicted head-hung angle, a multi-level early warning mechanism is used for judgment and display. The specific operation is as follows:

[0106] When the included angle k1≤x≤k2, it is considered a safe zone, and the included angle is only displayed in real time on the vehicle display terminal.

[0107] When x < k1, the task vehicle enters the danger zone. At this time, the central control interface flashes a red warning light and announces "The left turn angle is too large, please pay attention to the operation" until the vehicle leaves the danger zone.

[0108] When x > k2, the task vehicle enters the danger zone. At this time, the central control interface flashes a red warning light and announces "The right turn angle is too large, please pay attention to the operation" until the vehicle leaves the danger zone.

[0109] In the above process, x represents the currently calculated predicted angle of the trailer, in degrees; k1 and k2 are both alarm thresholds. Based on functional requirements and actual testing, k1 can be set to -45° and k2 can be set to 45° in this invention. Furthermore, the alarm thresholds can be dynamically adjusted by combining vehicle geometric parameters, trailer length, and other information to achieve an adaptive multi-scenario alarm strategy for different vehicles.

[0110] The method of this invention compares the predicted angle with a safety threshold and links it with an audible and visual alarm and a display interface, which can provide drivers with intuitive status feedback and hazard warnings, thus helping to improve the safety of driving operations.

[0111] The real-time monitoring method for the trailer-mounted angle of a commercial vehicle provided in this invention first acquires an image containing the leading edge region of the trailer using a fisheye camera installed at the rear of the tractor unit. The image is then preprocessed to extract the region of interest and enhance the data. Next, the preprocessed image is input into a trained trailer-mounted angle prediction network, MFARNet. End-to-end angle regression is performed by fusing trailer bounding box features and keypoint features, outputting the sine and cosine predicted values ​​of the trailer-mounted angle. The prediction network MFARNet is an improved version of YOLOv8-Pose. Finally, the predicted value of the trailer-mounted angle is obtained based on the sine and cosine predicted values. This invention effectively overcomes the dependence of traditional geometric measurement methods on external markers and strict calibration through multimodal regression and end-to-end design, achieving higher accuracy angle estimation under purely visual conditions. Simultaneously, the multimodal feature fusion mechanism enhances robustness to complex lighting, occlusion, and different vehicle models. Furthermore, the end-to-end overall design significantly reduces system latency and complexity, facilitating lightweight deployment on embedded platforms.

[0112] In summary, thanks to the single fisheye camera and integrated navigation system employed in this invention, the system is able to acquire high-quality real training data. The proposed MFARNet network fully utilizes the multi-source fusion features of the bounding box and keypoints, significantly improving the estimation accuracy of the head-hung angle. Furthermore, with the assistance of SCLoss optimization, the model's estimation performance is further enhanced.

[0113] Compared to other vision solutions, this method has a simple system structure, is easy to deploy, and is more suitable for industrial applications. The overall angle estimation accuracy is more than 10% higher than that of conventional vision solutions.

[0114] Based on the above Figure 1 The real-time monitoring method for the cab-mounted angle of commercial vehicles described in the corresponding embodiments is described below as an embodiment of the device of the present invention, which can be used to execute the method embodiment of the present invention.

[0115] The real-time monitoring device for the front-end clamping angle of commercial vehicles provided in this embodiment of the invention, such as... Figure 3As shown, the device includes: a data acquisition module 201, a prediction module 202, and an acquisition module 203;

[0116] The acquisition module 201 is used to acquire images containing the leading edge area of ​​the trailer through a fisheye camera installed at the rear of the tractor, and to preprocess the images to extract the region of interest and enhance the data.

[0117] The prediction module 202 is used to input the preprocessed image into the trained head-mounted angle prediction network MFARNet, and perform end-to-end angle regression by fusing trailer target box features and key point features, and output the sine and cosine predicted values ​​of the head-mounted angle. The prediction network MFARNet is built based on YOLOv8-Pose improvement.

[0118] The module 203 is used to obtain the predicted value of the head-hanging angle based on the predicted values ​​of sine and cosine.

[0119] The real-time monitoring device for the cab-mounted trailer angle provided in this embodiment of the invention includes: a data acquisition module 201, a prediction module 202, and an acquisition module 203; the data acquisition module 201 acquires images containing the leading edge region of the trailer using a fisheye camera installed at the rear of the tractor, and preprocesses the images to extract the region of interest and enhance the data; the prediction module 202 inputs the preprocessed images into a trained cab-mounted trailer angle prediction network MFARNet, and performs end-to-end angle regression by fusing trailer target box features and key point features, outputting the sine and cosine predicted values ​​of the cab-mounted trailer angle, wherein the prediction network MFARNet is constructed based on an improvement of YOLOv8-Pose; the acquisition module 203 obtains the predicted value of the cab-mounted trailer angle based on the sine and cosine predicted values. The device of this invention does not rely on additional markers or sensors. It can achieve real-time, non-contact, high-precision estimation of the head-to-trailer angle between the tractor and trailer using only a single fisheye camera and an end-to-end multi-task deep learning model. It has the advantages of simple deployment and low cost. The overall system structure is simple and can adapt to different vehicle models and installation locations. The end-to-end design effectively reduces the accumulation of external geometric calculation errors in traditional solutions. At the same time, the device can achieve millisecond-level real-time detection of the angle, which can meet the usage needs of various scenarios such as commercial vehicle reversing assistance and fleet formation.

[0120] Based on the above Figure 1 In addition to the real-time monitoring method for the cab-off angle of a commercial vehicle described in the corresponding embodiment, another embodiment of the present invention provides a real-time monitoring device for the cab-off angle of a commercial vehicle. This device includes a processor and a memory. The memory stores at least one computer instruction, which is loaded and executed by the processor to achieve the above-described method. Figure 1 The corresponding embodiment describes a method for real-time monitoring of the clamping angle of a commercial vehicle's front-end.

[0121] Based on the above Figure 1 The real-time monitoring method for the cab-mounted angle of commercial vehicles described in the corresponding embodiments of the present invention also provides a computer-readable storage medium. For example, a non-transitory computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a CD-ROM, magnetic tape, a floppy disk, or an optical data storage system, etc. This storage medium stores at least one computer instruction for executing the above-described method. Figure 1 The real-time monitoring method for the cab-mounted angle of commercial vehicles described in the corresponding embodiments will not be repeated here.

[0122] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the claims.

[0123] It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A method of real-time monitoring of the head-on cornering angle of a commercial vehicle, characterized in that, The method includes the following steps: Images containing the leading edge area of ​​the trailer are acquired by a fisheye camera mounted at the rear of the tractor unit, and the images are preprocessed to extract the region of interest and enhance the data. The preprocessed image is input into the trained head-mounted angle prediction network MFARNet. By fusing the trailer target box features and key point features, end-to-end angle regression is performed, and the sine and cosine predicted values ​​of the head-mounted angle are output. The prediction network MFARNet is constructed based on YOLOv8-Pose. The predicted value of the head-hanging angle is obtained based on the predicted values ​​of sine and cosine.

2. The method of real-time monitoring of the head-on corner angle of a commercial vehicle as claimed in claim 1, wherein, The trained head-hanging angle prediction network MFARNet was obtained through the following training method: The ground truth value of the nose-mounted angle is obtained by using a combined navigation system installed on the tractor and trailer, and images from the vehicle-mounted camera are collected simultaneously to construct a training dataset. The annotation information of the dataset includes the target box at the leading edge of the trailer, key points, and the ground truth value of the nose-mounted angle. Construct an MFARNet network based on YOLOv8-Pose improvement and train it end-to-end using the training dataset; The total loss function used in training is: L Total = λ1L box + λ2L cls + λ3L pose + λ4L SCL Among them, L Total L box L cls L pose L SCL λ1, λ2, λ3, and λ4 represent the total loss, bounding box loss, classification loss, keypoint loss, and angle regression loss, respectively; λ1, λ2, λ3, and λ4 represent the gain coefficients corresponding to the bounding box loss, classification loss, keypoint loss, and angle regression loss, respectively.

3. The method of real-time monitoring of the head-on corner angle of a commercial vehicle as claimed in claim 2, wherein, The angle regression loss L SCL Calculated using the following formula: L SCL =L cos +λ5L smooth L cos =1-(P (sin,cos) *T (sin,cos) ) Among them, L cos For the sine and cosine loss of the included angle, L smooth To smooth the L1 loss, λ5 is the gain coefficient for smoothing the L1 loss, P (sin,cos) T represents the predicted sine and cosine values ​​of the head-hanging angle. (sin,cos) Let be the sine and cosine values ​​of the true value of the angle between the heads.

4. The method of real-time monitoring of the head-on corner angle of a commercial vehicle of claim 1, wherein, The process involves inputting the preprocessed image into the trained MFARNet head-mount angle prediction network, fusing trailer bounding box features and keypoint features to perform end-to-end angle regression, and outputting the sine and cosine predicted values ​​of the head-mount angle. Specifically, this includes: Multi-scale feature extraction is performed on the preprocessed image to obtain semantic and detail features at different levels; The extracted multi-scale features are fused to generate fused features; Based on the fusion features, predict the bounding box features and key point features of the trailer's leading edge; The predicted target box features are fused with the key point features, and the sine and cosine predicted values ​​of the head-hanging angle are calculated by regression.

5. The method of real-time monitoring of the head-on corner angle of a commercial vehicle as claimed in claim 4, wherein, The step of fusing the predicted target box features with the key point features, and calculating the sine and cosine predicted values ​​of the head-hanging angle through regression, specifically includes: The target bounding box features and the keypoint features are reconstructed and concatenated to obtain fused features. These fused features are then input into a multilayer perceptron for regression calculation to obtain the predicted sine and cosine values ​​of the head-hanging angle; or... The target bounding box features and the key point features are enhanced and fused through attention mechanism and feature cross-fusion to obtain the sine and cosine predicted values ​​of the head-hanging angle.

6. The real-time monitoring method for the cab-mounted angle of a commercial vehicle according to claim 1, characterized in that, The preprocessing of the image to extract the region of interest and enhance the data specifically includes: Based on the installation parameters of the fisheye camera, a region of interest containing the trailer connection part and the leading edge structure is extracted from the image; Data augmentation is performed on the image within the region of interest. The data augmentation operations include at least one of brightness adjustment, contrast adjustment, color perturbation, translation, scaling, and horizontal flipping. The enhanced image is then normalized and standardized.

7. The method of real-time monitoring of the head-on corner angle of a commercial vehicle of claim 1, wherein, The method further includes: The predicted head-mounted angle is compared with a preset safety threshold. When the predicted value of the head-mounted angle exceeds the threshold range, an audible and visual alarm is triggered. The predicted head-mounted angle is displayed in real time when it is within the safe threshold range.

8. A real-time monitoring device for the head-on corner angle of a commercial vehicle, characterized in that include: The acquisition module is used to acquire images containing the leading edge area of ​​the trailer using a fisheye camera installed at the rear of the tractor, and to preprocess the images to extract the region of interest and enhance the data; The prediction module is used to input the preprocessed image into the trained head-mounted angle prediction network MFARNet, and perform end-to-end angle regression by fusing trailer target box features and key point features, and output the sine and cosine predicted values ​​of the head-mounted angle. The prediction network MFARNet is built based on YOLOv8-Pose. The module is used to obtain the predicted value of the head-hanging angle based on the predicted values ​​of the sine and cosine.

9. A real-time monitoring device for the head-on cornering angle of a commercial vehicle, characterized in that The real-time monitoring device for the cab-mounted angle of the commercial vehicle includes a processor and a memory. The memory stores at least one computer instruction, which is loaded and executed by the processor to perform the steps in the real-time monitoring method for the cab-mounted angle of the commercial vehicle according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The storage medium stores at least one computer instruction, which is loaded and executed by a processor to perform the steps in the real-time monitoring method for the cab-off angle of a commercial vehicle as described in any one of claims 1 to 7.

Images