Logistics vehicle lane deviation detection method and device, equipment and storage medium

Abstract

The invention relates to the field of logistics transportation, and discloses a logistics vehicle lane deviation detection method, device and equipment and a storage medium. The method comprises the following steps: acquiring historical logistics vehicle driving images, and marking the historical logistics vehicle driving images to construct a training data set; constructing an initial lane departure recognition model based on a LaneNet network; training the initial lane departure recognition model through the training data set, and adjusting parameters of the initial lane departure recognition model to obtain a trained lane departure recognition model; collecting a to-be-detected image sample in real time, inputting the to-be-detected image sample into the trained lane departure identification model, and outputting a departure prediction value of the logistics vehicle and the lane line; and determining whether to start an early warning mechanism based on the deviation prediction value of the logistics vehicle and the lane line. According to the logistics vehicle lane deviation detection method provided by the invention, an automatic detection and early warning mechanism technology is adopted, the accident rate is reduced, the logistics vehicle operation efficiency is improved, and the safety management and control of the logistics transportation process are realized.

Description

Technical Field

[0001] This invention relates to the field of logistics transportation technology, and in particular to a method, apparatus, equipment and storage medium for detecting lane departure of logistics vehicles. Background Technology

[0002] Lane departure is a common safety hazard in logistics transportation. Existing logistics fleet management systems mainly focus on vehicle speed, position, and basic driving status, lacking real-time monitoring and early warning functions for lane departure. This increases the risk of traffic accidents due to lane departure during transportation, affecting the safety and efficiency of logistics transportation.

[0003] Therefore, existing technologies still need to be improved and developed. Summary of the Invention

[0004] The main objective of this invention is to solve the problem that existing logistics fleet management systems lack real-time monitoring of lane departure.

[0005] The first aspect of this invention provides a method for detecting lane departure of logistics vehicles, comprising: acquiring historical images of logistics vehicles driving, annotating the historical images of logistics vehicles driving to construct a training dataset; constructing an initial lane departure recognition model based on a LaneNet network; training the initial lane departure recognition model using the training dataset, adjusting the parameters of the initial lane departure recognition model to obtain a trained lane departure recognition model; acquiring test image samples in real time, inputting the test image samples into the trained lane departure recognition model, and outputting a predicted value of the lane departure between the logistics vehicle and the lane line; and determining whether to activate a warning mechanism based on the predicted value of the lane departure between the logistics vehicle and the lane line.

[0006] Optionally, in a first implementation of the first aspect of the present invention, the historical logistics vehicle driving images are classified into normal driving state images and lane deviation state images; the logistics vehicle position information and lane line position information in the normal driving state images are labeled, and the logistics vehicle position information, lane line position information, and vehicle deviation direction and degree information in the lane deviation state images are labeled; the labeled and classified normal driving state images and lane deviation state images are organized to form a training dataset containing different road types, lighting conditions, and weather conditions.

[0007] Optionally, in a second implementation of the first aspect of the present invention, a LaneNet network framework is constructed, the LaneNet network framework including multiple network layers, semantic segmentation branches and instance embedding branches; a multi-scale feature fusion mechanism is introduced into the LaneNet network framework to construct an initial lane departure recognition model.

[0008] Optionally, in a third implementation of the first aspect of the present invention, multi-scale feature images are extracted through the multi-layer network; a multi-scale feature fusion module is embedded between the multi-layer network layers, the multi-scale feature fusion module is used to fuse and enhance the multi-scale feature images extracted by the multi-layer network layers to generate a fused feature map containing multi-scale information; the multi-scale feature fusion module assigns adaptive weights to the multi-scale feature images extracted by the multi-layer network layers based on the fused feature map to construct an initial lane departure recognition model.

[0009] Optionally, in the fourth implementation of the first aspect of the present invention, the training dataset is subjected to data augmentation processing to generate a data-augmented training dataset; the data-augmented training dataset is divided into multiple batches according to batch size and sequentially input into the initial lane departure recognition model; the initial lane departure recognition model performs forward propagation calculation on each batch of training datasets to output the predicted offset values ​​between the logistics vehicle and the lane line; based on the predicted offset values ​​between the logistics vehicle and the lane line and the actual offset values ​​between the logistics vehicle and the lane line marked in the training dataset, a loss function of the initial lane departure recognition model is constructed; the gradient value of the loss with respect to the parameters of the initial lane departure recognition model is calculated by taking the derivative of the loss function through the backpropagation algorithm; and the parameters of the initial lane departure recognition model are adjusted according to the gradient value using a preset optimizer to obtain a trained lane departure recognition model.

[0010] Optionally, in the fifth implementation of the first aspect of the present invention, a real-time image of the logistics vehicle is acquired using an onboard camera as a sample image to be tested; the sample image to be tested is preprocessed to obtain a preprocessed image to be tested; the preprocessed image to be tested is input into the trained lane departure recognition model, and multi-scale feature images are extracted through the multi-layer network of the trained lane departure recognition model; the multi-scale feature images are fused and enhanced by the multi-scale feature fusion module of the trained lane departure recognition model to generate a fused feature map containing multi-scale information; the fused feature map is semantically segmented through the semantic segmentation branch of the trained lane departure recognition model to generate a binary segmentation map containing lane line pixels; the lane line pixels in the binary segmentation map are feature-represented through the instance embedding branch of the trained lane departure recognition model to generate a feature vector; and the predicted value of the offset between the logistics vehicle and the lane line is calculated based on the binary segmentation map and the feature vector.

[0011] Optionally, in a sixth implementation of the first aspect of the present invention, a safe deviation threshold for the logistics vehicle is set according to the logistics vehicle driving safety standard; the predicted deviation value between the logistics vehicle and the lane line is compared with the safe deviation threshold; if the predicted deviation value is less than the safe deviation threshold, the warning mechanism is not activated; if the predicted deviation value is greater than or equal to the safe deviation threshold, the warning mechanism is activated.

[0012] A second aspect of the present invention provides a lane departure detection device for logistics vehicles. The lane departure detection device includes: an acquisition module for acquiring historical images of logistics vehicles driving and labeling the historical images to construct a training dataset; a training module for constructing an initial lane departure recognition model based on a LaneNet network, training the initial lane departure recognition model using the training dataset, and adjusting the parameters of the initial lane departure recognition model to obtain a trained lane departure recognition model; a calculation module for real-time acquisition of image samples to be tested, inputting the image samples to be tested into the trained lane departure recognition model, and outputting a predicted value of the lane departure between the logistics vehicle and the lane line; and an early warning module for determining whether to activate an early warning mechanism based on the predicted value of the lane departure between the logistics vehicle and the lane line.

[0013] Optionally, in a first implementation of the second aspect of the present invention, the acquisition module includes: a classification unit, used to classify the historical logistics vehicle driving images into normal driving state images and lane departure state images; a labeling unit, used to label the logistics vehicle position information and lane line position information in the normal driving state images, and label the logistics vehicle position information, lane line position information, and vehicle deviation direction and degree information in the lane departure state images; and a sorting unit, used to sort the labeled and classified normal driving state images and lane departure state images to form a training dataset containing different road types, lighting conditions, and weather conditions.

[0014] Optionally, in a second implementation of the second aspect of the present invention, the training module includes: a building unit for building a LaneNet network framework, the LaneNet network framework including multi-layer network layers, semantic segmentation branches and instance embedding branches; and a construction unit for introducing a multi-scale feature fusion mechanism into the LaneNet network framework to construct an initial lane departure recognition model.

[0015] Optionally, in a third implementation of the second aspect of the present invention, the construction unit is specifically used to extract multi-scale feature images through the multi-layer network; embed a multi-scale feature fusion module between the multi-layer network, the multi-scale feature fusion module being used to fuse and enhance the multi-scale feature images extracted by the multi-layer network to generate a fused feature map containing multi-scale information; the multi-scale feature fusion module assigns adaptive weights to the multi-scale feature images extracted by the multi-layer network based on the fused feature map to construct an initial lane departure recognition model.

[0016] Optionally, in a fourth implementation of the second aspect of the present invention, the training module further includes: a data processing unit, configured to perform data augmentation processing on the training dataset to generate a data-augmented training dataset; divide the data-augmented training dataset into multiple batches according to batch size, and input them sequentially into the initial lane departure recognition model; an offset prediction unit, configured to perform forward propagation calculation on each batch of training dataset using the initial lane departure recognition model, and output the offset prediction value between the logistics vehicle and the lane line; a gradient calculation unit, configured to construct the loss function of the initial lane departure recognition model based on the offset prediction value between the logistics vehicle and the lane line and the actual offset value between the logistics vehicle and the lane line marked in the training dataset; calculate the gradient value of the loss with respect to the parameters of the initial lane departure recognition model by differentiating the loss function using a backpropagation algorithm; and a training unit, configured to adjust the parameters of the initial lane departure recognition model according to the gradient value using a preset optimizer to obtain a trained lane departure recognition model.

[0017] Optionally, in a fifth implementation of the second aspect of the present invention, the calculation module includes: an image processing unit, configured to acquire a real-time image of the logistics vehicle's movement via an onboard camera as a sample image to be tested; preprocess the sample image to be tested to obtain a preprocessed image to be tested; a feature fusion unit, configured to input the preprocessed image to be tested into the trained lane departure recognition model, extract multi-scale feature images through the multi-layer network of the trained lane departure recognition model; fuse and enhance the multi-scale feature images through the multi-scale feature fusion module of the trained lane departure recognition model to generate a fused feature map containing multi-scale information; a first generation unit, configured to perform semantic segmentation on the fused feature map through the semantic segmentation branch of the trained lane departure recognition model to generate a binary segmentation map containing lane line pixels; a second generation unit, configured to perform feature representation on the lane line pixels in the binary segmentation map through the instance embedding branch of the trained lane departure recognition model to generate a feature vector; and a calculation unit, configured to calculate the predicted offset value between the logistics vehicle and the lane line based on the binary segmentation map and the feature vector.

[0018] Optionally, in a sixth implementation of the second aspect of the present invention, the warning module includes: a setting unit, configured to set a safe deviation threshold for the logistics vehicle according to the logistics vehicle driving safety standard; a comparison unit, configured to compare the predicted deviation value between the logistics vehicle and the lane line with the safe deviation threshold; if the predicted deviation value is less than the safe deviation threshold, the warning mechanism is not activated; if the predicted deviation value is greater than or equal to the safe deviation threshold, the warning mechanism is activated.

[0019] A third aspect of the present invention provides a logistics vehicle lane departure detection device, comprising: a memory and at least one processor, wherein the memory stores computer-readable instructions, and the memory and the at least one processor are interconnected via a circuit; the at least one processor invokes the computer-readable instructions in the memory to cause the logistics vehicle lane departure detection device to perform the various steps of the logistics vehicle lane departure detection method described above.

[0020] A fourth aspect of the present invention provides a computer-readable storage medium storing computer-readable instructions that, when executed on a computer, cause the computer to perform the various steps of the logistics vehicle lane departure detection method described above.

[0021] Beneficial Effects: In the technical solution of this invention, historical logistics vehicle driving images are acquired and annotated to construct a training dataset containing different road types, lighting conditions, and weather conditions. An initial lane departure recognition model is built based on the LaneNet network, and the model is trained using the training dataset, adjusting parameters to obtain a trained model. Real-time acquisition of test image samples is input into the trained model, outputting the predicted value of the logistics vehicle's lane departure. Finally, the predicted value of the lane departure is compared with a safety threshold to determine whether to activate the warning mechanism. This invention constructs a LaneNet network model and introduces multi-scale feature fusion technology to obtain a trained lane departure recognition model. Through the trained lane departure recognition model, automated detection and real-time warning of logistics vehicle lane departure are achieved, reducing reliance on manual monitoring, lowering labor costs and monitoring workload. The real-time detection and warning mechanism can promptly detect and handle lane departure situations, effectively reducing the incidence of traffic accidents and improving the safety and management efficiency of logistics transportation. Attached Figure Description

[0022] Figure 1 This is a first flowchart of a lane departure detection method for logistics vehicles provided in an embodiment of the present invention; Figure 2 This is a second flowchart of the lane departure detection method for logistics vehicles provided in an embodiment of the present invention; Figure 3 This is a third flowchart of the logistics vehicle lane departure detection method provided in the embodiments of the present invention; Figure 4 This is a fourth flowchart of the logistics vehicle lane departure detection method provided in this embodiment of the invention; Figure 5 This is a fifth flowchart of the logistics vehicle lane departure detection method provided in the embodiments of the present invention; Figure 6 This is a sixth flowchart of the logistics vehicle lane departure detection method provided in this embodiment of the invention; Figure 7 This is the seventh flowchart of the logistics vehicle lane departure detection method provided in the embodiments of the present invention; Figure 8 A schematic diagram of a logistics vehicle lane departure detection device provided in an embodiment of the present invention; Figure 9 This is another structural schematic diagram of the logistics vehicle lane departure detection device provided in an embodiment of the present invention; Figure 10 This is a schematic diagram of the structure of the logistics vehicle lane departure detection device provided in an embodiment of the present invention. Detailed Implementation

[0023] This invention provides a method, apparatus, device, and storage medium for detecting lane departure of logistics vehicles. The method includes: acquiring and annotating historical images of logistics vehicles to construct a training dataset containing different road types, lighting conditions, and weather conditions; building an initial lane departure recognition model based on a LaneNet network, training the model using the training dataset, and adjusting parameters to obtain a trained model; real-time acquisition of test image samples and inputting them into the trained model, outputting predicted values ​​of the lane departure between the logistics vehicle and the lane line; finally, comparing the predicted values ​​with a safety threshold to determine whether to activate a warning mechanism. This invention constructs a LaneNet network model and introduces multi-scale feature fusion technology to obtain a trained lane departure recognition model. This model enables automated detection and real-time warning of lane departure of logistics vehicles, reducing reliance on manual monitoring, lowering labor costs, and reducing monitoring workload. The real-time detection and warning mechanism can promptly detect and handle lane departure situations, effectively reducing the incidence of traffic accidents and improving the safety and management efficiency of logistics transportation.

[0024] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" or "having" and any variations thereof are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0025] For ease of understanding, the specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1 The first embodiment of the logistics vehicle lane departure detection method in this invention includes: S100. Obtain historical logistics vehicle driving images and annotate the historical logistics vehicle driving images to construct a training dataset; In this embodiment, images of the logistics vehicle in motion are captured in real time by an onboard camera installed on the vehicle. The onboard camera captures images of the logistics vehicle at a fixed frequency, such as every 10 seconds, covering road types such as highways, urban expressways, and rural roads, as well as lighting and weather conditions such as sunny days, cloudy days, rainy days, and nighttime. Annotation tools are used to classify the images, for example, into 15,000 images of normal driving conditions and 5,000 images of lane departure conditions. For normal driving images, the vehicle center pixel coordinates, lane line key point coordinates, and termination point are annotated. For lane departure images, in addition to annotating the vehicle center pixel coordinates, lane line key point coordinates, and termination point, the direction of departure (e.g., rightward deviation) and the degree (e.g., lateral deviation of 60 cm) are also annotated. Finally, a training dataset containing different scenarios is constructed based on these annotated images.

[0026] S200, Construct an initial lane departure recognition model based on the LaneNet network; In this embodiment, a LaneNet network framework is constructed, comprising multiple network layers, a semantic segmentation branch, and an instance embedding branch. The shallow network extracts high-resolution feature maps to capture lane line edge details; the deep network extracts low-resolution feature maps to obtain the global direction of the lane lines; the semantic segmentation branch outputs binary segmented images to distinguish lane line pixels from non-lane line pixels; and the instance embedding branch generates feature vectors to represent lane line pixels using feature vectorization. Simultaneously, a multi-scale feature fusion module is embedded between network layers to fuse feature maps of different scales through feature concatenation, constructing an initial lane departure recognition model.

[0027] S300. Train the initial lane departure recognition model using the training dataset, adjust the parameters of the initial lane departure recognition model, and obtain the trained lane departure recognition model. In this embodiment, the training dataset is first augmented by random cropping, horizontal flipping, and color adjustments such as brightness, contrast, and saturation changes to increase sample diversity and improve the model's robustness to complex lighting and viewing angle changes. The augmented training dataset is then divided into multiple batches according to a preset batch size and sequentially input into the initial lane departure recognition model. After forward propagation of each batch of training datasets by the initial lane departure recognition model, the predicted offset values ​​between the logistics vehicle and the lane lines are output, for example, a lateral offset of 75 cm and a rightward deviation. The difference between the predicted value and the labeled actual offset value is calculated based on the Focal Loss function, for example, an actual offset of 80 cm, to quantify the prediction bias of the initial lane departure recognition model. To minimize the Focal Loss function, the gradient of the loss with respect to the parameters of the initial lane departure recognition model is calculated using the backpropagation algorithm. The optimizer updates the parameters of the initial lane departure recognition model based on the direction and magnitude of the gradient, causing the Focal Loss function to gradually converge. Specifically, the optimizer can employ the commonly used Stochastic Gradient Descent (SGD) technique to reduce prediction error by iteratively adjusting the weights and biases of the initial lane departure warning model. The learning rate strategy supports either a fixed learning rate or dynamic adjustment to balance training speed and convergence performance. The training process of the initial lane departure warning model continues iteratively until the Focal Loss function converges, ultimately yielding a parameter-optimized trained lane departure warning model.

[0028] S400: Real-time acquisition of image samples to be tested, inputting the image samples to be tested into the trained lane departure recognition model, and outputting the predicted value of the deviation between the logistics vehicle and the lane line; In this embodiment, the image samples to be tested are acquired in real time by the vehicle-mounted camera. They are first preprocessed by scaling the image to the model's preset size and normalizing the pixel values. Then, the scaled and adjusted image samples are input into the trained lane departure recognition model. Multi-scale feature images are extracted through the multi-layer network of the trained lane departure recognition model. The multi-scale feature fusion module then fuses and enhances the fused feature map to generate a fused feature map. The semantic segmentation branch then performs semantic segmentation on the fused feature map to generate a binary segmentation map. The instance embedding branch performs feature representation on the lane line pixels in the segmentation map to generate feature vectors. Finally, the predicted value of the offset between the logistics vehicle and the lane line is calculated based on the binary segmentation map and the feature vectors.

[0029] S500: Based on the predicted offset value between the logistics vehicle and the lane line, determine whether to activate the early warning mechanism.

[0030] In this embodiment, a preset safety deviation threshold is set according to the safety standards for logistics vehicle driving. The predicted deviation value between the logistics vehicle and the lane line is compared with the safety deviation threshold. If the predicted deviation value is less than the safety deviation threshold, it is determined to be a normal driving state and the warning mechanism is not activated. If the predicted deviation value is greater than or equal to the safety deviation threshold, it is determined to be a dangerous deviation state and the warning mechanism is activated immediately. The corresponding warning signal is issued through the vehicle warning system. At the same time, the deviation time, vehicle location information and other relevant data are recorded to the logistics fleet management platform for subsequent analysis and processing.

[0031] As an example, the safety offset threshold is set to 50 cm laterally according to the safety standards for logistics vehicles. When the predicted offset is 35 cm < the threshold, normal driving is determined; when the predicted offset is 60 cm ≥ the threshold, an early warning is immediately activated: the vehicle system emits an 85-decibel audible alarm, the dashboard red light flashes, and the offset time (e.g., 10:30:22 AM on June 18, 2025) and the vehicle's GPS coordinates (e.g., 31 degrees North latitude, 121 degrees East longitude) are recorded and sent to the management platform.

[0032] This embodiment provides a lane departure detection method for logistics vehicles, aiming to address the lack of real-time monitoring of lane departure in existing logistics fleet management systems. Through multi-scale feature fusion technology, the trained lane departure recognition model integrates shallow detail features with deep semantic information, significantly enhancing lane line recognition capabilities and resolving the false detection or missed detection problems caused by environmental interference in traditional methods. Optimizing detection real-time performance and accuracy, the trained lane departure recognition model, combined with data augmentation technology and the FocalLoss loss function, effectively alleviates the imbalance between lane line and non-lane line pixels, improving the sensitivity and detection accuracy of the trained lane departure recognition model. Simultaneously, real-time image processing and early warning mechanisms can trigger alarms within millisecond response times, reducing the risk of traffic accidents caused by driver fatigue or distraction. Reducing reliance on manual labor and operating costs: The automated lane departure monitoring system replaces manual monitoring, reducing the consumption of human resources in fleet management, avoiding blind spots in manual monitoring, and providing data support for driver behavior assessment and safety management strategy optimization through real-time recording and analysis of lane departure events. This invention provides a low-cost, high-reliability lane departure detection solution for logistics vehicles. Through real-time detection and early warning mechanisms, it enables safe control of the logistics transportation process, reduces accident rates, and improves fleet operating efficiency.

[0033] Please see Figure 2 The second embodiment of the logistics vehicle lane departure detection method in this invention includes: S110. Classify the historical logistics vehicle driving images into normal driving state images and lane departure state images. S120. Label the location information of the logistics vehicle and the lane line in the normal driving state image, and label the location information of the logistics vehicle, the lane line, and the direction and degree of vehicle deviation in the lane deviation state image. S130. Organize the labeled and classified normal driving state images and lane departure state images to form a training dataset containing different road types, lighting conditions, and weather conditions.

[0034] In this embodiment, images of the logistics vehicle are captured at regular time intervals using an onboard camera, covering various road types, lighting conditions, and weather conditions. An annotation tool is used to classify the images into two categories: normal driving and lane departure. For normal driving images, the vehicle's position information in the image, such as center and corner coordinates, and lane line position information, such as start point, end point, and key point coordinates, is annotated. For lane departure images, in addition to annotating the vehicle's position information and lane line position information, the direction and degree of vehicle departure are also annotated. Finally, the annotated images are compiled into a training dataset containing different scenarios, ensuring that the dataset covers various road environments and weather conditions.

[0035] As an example, 20,000 images were collected and classified into 15,000 normal driving images and 5,000 offset images. The vehicle center coordinates and lane line key point coordinates were labeled. The offset images were additionally labeled with information such as "60 cm to the right". The dataset was organized according to scenarios such as sunny days, rainy days, and highways.

[0036] Please see Figure 3 The third embodiment of the logistics vehicle lane departure detection method in this invention includes: S210. Construct the LaneNet network framework, which includes multiple network layers, a semantic segmentation branch, and an instance embedding branch. S220. Introduce a multi-scale feature fusion mechanism into the LaneNet network framework to construct an initial lane departure recognition model.

[0037] In this embodiment, the basic network architecture is first constructed: the LaneNet network framework serves as the backbone network, and its multi-layer network adopts the ENet lightweight structure. Through the stacking of convolutional and pooling layers, features are extracted from the input image. The shallow network layers extract high-resolution feature maps to capture details such as lane line edges and dashed lines, while the deep network layers extract low-resolution feature maps to obtain the global direction and semantic information of the lane lines. The semantic segmentation branch outputs a binary semantic segmentation image based on the shallow features, and performs binary classification of lane line pixels and background. The instance embedding branch generates feature vectors for lane line pixels, distinguishes different lane line instances through a clustering algorithm, and finally obtains the instance segmentation result of the lane lines.

[0038] Then, a multi-scale feature fusion mechanism is introduced: a multi-scale feature fusion module is embedded between multiple network layers. This module extracts high-resolution small-scale feature maps from shallow networks and low-resolution large-scale feature maps from deep networks, fusing them through feature addition or concatenation. Specifically, if feature addition is used, the number of channels is first unified through convolution, and then the feature map size is adjusted through upsampling or downsampling, adding the corresponding elements. If feature concatenation is used, feature maps of different scales are directly concatenated along the channel dimension to form a fused feature map containing multi-scale information. Based on the distribution of details and semantic information in the fused feature map, the multi-scale feature fusion module assigns adaptive weights to feature maps of different scales, such as 60% weight for shallow features and 40% weight for deep features, to enhance the model's ability to recognize lane lines in complex lighting and occlusion scenarios.

[0039] As an example: the feature maps extracted by the shallow network layers in the multi-layer network capture the edge details of the blurred lane lines in heavy rain, and the feature maps extracted by the deep network layers in the multi-layer network obtain the overall curvature of the lane lines. The fused feature maps are generated by splicing them together through the multi-scale feature fusion module, so that the initial lane departure recognition model can still accurately identify the partially occluded lane lines in night scenes.

[0040] Please see Figure 4 The fourth embodiment of the logistics vehicle lane departure detection method in this invention includes: S221. Extract multi-scale feature images through the multi-layer network; S222. A multi-scale feature fusion module is embedded between the multi-layer network layers. The multi-scale feature fusion module is used to fuse and enhance the multi-scale feature images extracted by the multi-layer network layers to generate a fused feature map containing multi-scale information. S223. Based on the fused feature map, the multi-scale feature fusion module assigns adaptive weights to the multi-scale feature images extracted by the multi-layer network layer to construct an initial lane departure recognition model.

[0041] In this embodiment, a multi-scale feature fusion module is first embedded: an independent multi-scale feature fusion module is added between the shallow and deep layers of the multi-layer network. The multi-scale feature fusion module integrates with the features of each network layer. Figure 1 In a one-to-one correspondence, convolution and upsampling / downsampling operations are used to unify the feature map size, preparing for subsequent fusion. For example, a multi-scale feature fusion module is embedded between shallow and deep network layers to ensure that feature maps of different scales can be processed compatiblely.

[0042] Then, the feature distribution is statistically analyzed and the fusion method is adjusted: the multi-scale feature fusion module calculates the resolution and semantic information intensity of each level of feature map, and selects the fusion method based on the resolution and semantic information intensity. If the shallow feature map has high resolution but weak semantics, and the deep feature map has strong semantics but low resolution, then feature concatenation is used for the shallow features to preserve details, and weighted phasing is used for the deep features to enhance semantics.

[0043] Next, adaptive fusion weight thresholds are set: For low-level feature maps, which have small receptive fields and rich details, a higher fusion weight threshold, such as 0.7, is set to allow shallow features to dominate the fusion process while preserving lane line details. For high-level feature maps, which have large receptive fields and complete semantics, a lower fusion weight threshold, such as 0.3, is set to allow deeper features to supplement global direction information. For example, in a rainstorm scenario, low-level features account for 70% of the weight to identify blurred lane line edges, while high-level features account for 30% to infer the overall lane direction.

[0044] Finally, the feature extraction weight allocation is corrected: the multi-scale feature fusion module dynamically adjusts the feature extraction weights of multiple network layers based on the details and semantic contributions of the adaptively fused feature map. For example, when the fused feature map shows that a certain layer's features contribute significantly to the recognition of occluded lane lines, the weight of the convolutional kernel at that layer is increased by 20%, and the overall network parameters are optimized through backpropagation, ultimately constructing an initial lane departure recognition model that can adapt to different scenarios.

[0045] Please see Figure 5 The fifth embodiment of the logistics vehicle lane departure detection method in this invention includes: S310. Perform data augmentation processing on the training dataset to generate a data-augmented training dataset; S320. Divide the augmented training dataset into multiple batches according to the batch size, and input them sequentially into the initial lane departure recognition model; S330. Perform forward propagation calculation on the training dataset of each batch using the initial lane departure recognition model, and output the predicted value of the deviation between the logistics vehicle and the lane line. S340. Based on the predicted offset value between the logistics vehicle and the lane line and the actual offset value between the logistics vehicle and the lane line labeled in the training dataset, construct the loss function of the initial lane offset recognition model; S350. The loss function is differentiated using the backpropagation algorithm to calculate the gradient value of the loss with respect to the parameters of the initial lane departure recognition model; S360. Using a preset optimizer, adjust the parameters of the initial lane departure recognition model according to the gradient value to obtain a trained lane departure recognition model.

[0046] In this embodiment, data augmentation is first performed: image rotation, horizontal flipping, scaling, adding noise, and adjusting brightness and contrast are used to expand sample diversity, thereby obtaining an enhanced training dataset. For example, the random rotation angle is controlled within ±15 degrees to simulate vehicle turning scenarios; the image is horizontally flipped with a 50% probability to enhance the model's adaptability to different driving directions; the brightness adjustment range is set to ±20% to cope with changes in lighting conditions such as sunny and cloudy days; and Gaussian noise is added to simulate rainy days or camera shake scenarios.

[0047] The enhanced training dataset is then divided into multiple batches according to batch size: for example, setting the batch size to 64, the 40,000 enhanced images are divided into 625 batches, which are then sequentially input into the initial lane departure recognition model. After each batch of data is forward-propagated by the model, the predicted deviation value between the logistics vehicle and the lane line is output, such as "75 cm to the right".

[0048] Next, based on the actual offset between the predicted value and the labeled value, such as "80 cm to the right", the FocalLoss loss function is constructed: by introducing a modulation factor, the weights of positive and negative samples are dynamically adjusted, the weights of easily classified negative samples are reduced, and the weights of difficult-to-classify positive samples are increased, thus solving the problem of sample imbalance.

[0049] The gradient of the FocalLoss loss function with respect to the model parameters is then calculated using the backpropagation algorithm: the gradient is derived from the output layer to each subsequent network layer, calculating the gradient values ​​of parameters such as convolutional kernel weights and bias terms. The optimizer updates the initial lane departure warning model parameters based on the gradient direction and magnitude, causing the FocalLoss loss function to gradually converge. Specifically, the optimizer can employ the commonly used Stochastic Gradient Descent (SGD) to iteratively adjust the weights and biases of the initial lane departure warning model, reducing prediction error. The learning rate strategy supports fixed or dynamically adjusted rates to balance training speed and convergence performance. The training process of the initial lane departure warning model continues iteratively until the FocalLoss loss function converges, ultimately yielding a trained lane departure warning model with optimized parameters.

[0050] Please see Figure 6 The sixth embodiment of the logistics vehicle lane departure detection method in this invention includes: S410. Real-time image capture of logistics vehicle movement is obtained through vehicle-mounted cameras and used as image samples for testing. S420. Preprocess the image sample to be tested to obtain the preprocessed image to be tested; S430. Input the preprocessed image to be tested into the trained lane departure recognition model, and extract multi-scale feature images through the multi-layer network of the trained lane departure recognition model. S440. After fusing and enhancing the multi-scale feature image through the multi-scale feature fusion module of the trained lane departure recognition model, a fused feature map containing multi-scale information is generated. S450. Using the semantic segmentation branch of the trained lane departure recognition model, perform semantic segmentation on the fused feature map to generate a binary segmentation map containing lane line pixels. S460. Through the instance embedding branch of the trained lane departure recognition model, feature representation is performed on the lane line pixels in the binary segmentation map to generate a feature vector; S470. Based on the binary segmentation map and feature vector, calculate the predicted offset value between the logistics vehicle and the lane line.

[0051] In this embodiment, after obtaining the trained lane departure recognition model through the above embodiments, it is possible to detect whether a logistics vehicle is deviating from its lane in real time. First, image acquisition and preprocessing are performed: an onboard camera is installed on the vehicle's windshield to acquire images of the road ahead in real time; the images are preprocessed, including adjusting the image to the model's fit size using a scaling algorithm, and then normalizing the pixel values ​​to the [0,1] range to eliminate the influence of lighting differences. For example, the original 1280×720 pixel image is scaled to the input size of the trained lane departure recognition model to facilitate subsequent feature extraction.

[0052] Then, feature extraction and fusion are performed: In the multi-layer network of the trained lane departure recognition model, the shallow network extracts high-resolution feature maps (capturing lane line edges and dashed line details), and the deep network extracts low-resolution feature maps (obtaining the global direction of lane lines); the multi-scale feature fusion module fuses feature maps of different scales by splicing or adding them together. For example, the shallow 512×256 pixel feature map and the deep 128×64 pixel feature map are spliced ​​together after being made the same size to generate a fused feature map containing multi-scale information.

[0053] Next, a two-branch processing and offset calculation are performed: the semantic segmentation branch performs convolutional classification on the fused feature map, outputting a binary segmentation map with lane line pixel values ​​of 1 and background values ​​of 0; the instance embedding branch generates feature vectors for lane line pixels in the segmentation map, and uses the MeanShift algorithm for clustering, setting distance and vector thresholds to filter dense pixels and distinguish different lane line instances; based on the relative position of the vehicle center pixel coordinates and the lane line, the lateral offset distance and direction are calculated. For example, if the vehicle center coordinates are (320, 240) and the left lane line keypoint coordinates are (100, 300) - (200, 400), a predicted offset of 45 cm to the left is calculated.

[0054] Please see Figure 7The seventh embodiment of the logistics vehicle lane departure detection method in this invention includes: S510. Set the safety deviation threshold for logistics vehicles according to the safety standards for logistics vehicle operation. S520. Compare the predicted deviation value of the logistics vehicle from the lane line with the safe deviation threshold. S530. If the predicted offset value is less than the safe offset threshold, the early warning mechanism is not activated; if the predicted offset value is greater than or equal to the safe offset threshold, the early warning mechanism is activated.

[0055] In this embodiment, a safety deviation threshold is set according to the safety standards for logistics vehicle driving. The real-time deviation prediction value is compared with the threshold. If the deviation prediction value is less than the threshold, it is determined to be normal driving and no warning is activated. If the deviation prediction value is greater than or equal to the threshold, it is determined to be a dangerous deviation and the warning mechanism is activated immediately. The warning signal is issued through the vehicle system, including sound and light warnings, and relevant data such as deviation time and vehicle position are recorded to the management platform.

[0056] As an example, the safety offset threshold is set to 50 cm laterally according to the safety standards for logistics vehicles. When the predicted offset is 35 cm < the threshold, normal driving is determined; when the predicted offset is 60 cm ≥ the threshold, an early warning is immediately activated: the vehicle system emits an 85-decibel audible alarm, the dashboard red light flashes, and the offset time (e.g., 10:30:22 AM on June 18, 2025) and the vehicle's GPS coordinates (e.g., 31 degrees North latitude, 121 degrees East longitude) are recorded and sent to the management platform.

[0057] The above describes the lane departure detection method for logistics vehicles in the embodiments of the present invention. The following describes the lane departure detection device for logistics vehicles in the embodiments of the present invention. Please refer to [link / reference]. Figure 8 One embodiment of the logistics vehicle lane departure detection device in this invention includes: The acquisition module 10 is used to acquire historical logistics vehicle driving images and to annotate the historical logistics vehicle driving images to construct a training dataset. Training module 20 is used to build an initial lane departure recognition model based on the LaneNet network, train the initial lane departure recognition model using the training dataset, adjust the parameters of the initial lane departure recognition model, and obtain a trained lane departure recognition model. The calculation module 30 is used to collect the image samples to be tested in real time, input the image samples to be tested into the trained lane departure recognition model, and output the predicted value of the deviation between the logistics vehicle and the lane line. The early warning module 40 is used to determine whether to activate the early warning mechanism based on the predicted value of the offset between the logistics vehicle and the lane line.

[0058] In this embodiment, by constructing a deep learning network model and introducing multi-scale feature fusion technology, lane departure during the logistics vehicle's driving process is detected in real time, which reduces labor costs and monitoring workload, lowers the accident rate, improves fleet operation efficiency, and achieves safe management and control of the logistics transportation process.

[0059] Please see Figure 9 In this embodiment, the acquisition module 10 includes: Classification unit 11 is used to classify the historical logistics vehicle driving images into normal driving state images and lane departure state images. The annotation unit 12 is used to annotate the logistics vehicle position information and lane line position information in the normal driving state image, and to annotate the logistics vehicle position information, lane line position information, and vehicle deviation direction and degree information in the lane deviation state image. The sorting unit 13 is used to sort the labeled and classified normal driving state images and lane departure state images to form a training dataset containing different road types, lighting conditions and weather conditions.

[0060] Please see Figure 9 In this embodiment, the training module 20 includes: The building unit 21 is used to build the LaneNet network framework, which includes multiple network layers, semantic segmentation branches, and instance embedding branches. The building unit 22 is used to introduce a multi-scale feature fusion mechanism into the LaneNet network framework to build an initial lane departure recognition model.

[0061] Please see Figure 9 In this embodiment, the construction unit 22 is specifically used to extract multi-scale feature images through the multi-layer network layers; embed a multi-scale feature fusion module between the multi-layer network layers, the multi-scale feature fusion module is used to fuse and enhance the multi-scale feature images extracted by the multi-layer network layers to generate a fused feature map containing multi-scale information; the multi-scale feature fusion module assigns adaptive weights to the multi-scale feature images extracted by the multi-layer network layers based on the fused feature map to construct an initial lane departure recognition model.

[0062] Please see Figure 9 In this embodiment, the training module 20 includes: The data processing unit 23 is used to perform data augmentation processing on the training dataset to generate a data-augmented training dataset; divide the data-augmented training dataset into multiple batches according to the batch size, and input them sequentially into the initial lane departure recognition model; The offset prediction unit 24 is used to perform forward propagation calculation on the training dataset of each batch through the initial lane offset recognition model, and output the offset prediction value between the logistics vehicle and the lane line. The gradient calculation unit 25 is used to construct the loss function of the initial lane deviation recognition model based on the predicted deviation value between the logistics vehicle and the lane line and the actual deviation value between the logistics vehicle and the lane line labeled in the training dataset; and to calculate the gradient value of the loss with respect to the parameters of the initial lane deviation recognition model by taking the derivative of the loss function through the backpropagation algorithm. Training unit 26 is used to adjust the parameters of the initial lane departure recognition model according to the gradient value using a preset optimizer to obtain a trained lane departure recognition model.

[0063] Please see Figure 9 In this embodiment, the computing module 30 includes: Image processing unit 31 is used to acquire real-time images of logistics vehicles using an on-board camera as image samples to be tested; and to preprocess the image samples to be tested to obtain preprocessed images to be tested. The feature fusion unit 32 is used to input the preprocessed test image into the trained lane departure recognition model, extract multi-scale feature images through the multi-layer network of the trained lane departure recognition model, and then fuse and enhance the multi-scale feature images through the multi-scale feature fusion module of the trained lane departure recognition model to generate a fused feature map containing multi-scale information. The first generation unit 33 is used to perform semantic segmentation on the fused feature map through the semantic segmentation branch of the trained lane departure recognition model to generate a binary segmentation map containing lane line pixels. The second generation unit 34 is used to perform feature representation on the lane line pixels in the binary segmentation map through the instance embedding branch of the trained lane departure recognition model, and generate a feature vector. The calculation unit 35 is used to calculate the predicted offset value between the logistics vehicle and the lane line based on the binary segmentation map and the feature vector.

[0064] Please see Figure 9 In this embodiment, the early warning module 40 includes: Setting unit 41 is used to set the safety deviation threshold of logistics vehicle according to the logistics vehicle driving safety standard. The comparison unit 42 is used to compare the predicted value of the deviation between the logistics vehicle and the lane line with the safe deviation threshold; if the predicted deviation value is less than the safe deviation threshold, the warning mechanism is not activated; if the predicted deviation value is greater than or equal to the safe deviation threshold, the warning mechanism is activated.

[0065] This invention provides a lane departure detection method for logistics vehicles. By constructing a deep learning network model and introducing multi-scale feature fusion technology, the method can detect lane departures of logistics vehicles in real time. It uses an automated lane departure monitoring system to replace manual monitoring, reducing the consumption of human resources in fleet management, realizing safe control of the logistics transportation process, reducing the accident rate and improving the operational efficiency of the fleet.

[0066] above Figure 8 and Figure 9 The logistics vehicle lane departure detection device in this embodiment of the invention is described in detail from the perspective of modular functional entities. The logistics vehicle lane departure detection device in this embodiment of the invention is described in detail from the perspective of hardware processing.

[0067] Figure 10 This is a schematic diagram of the structure of a lane departure detection device for logistics vehicles provided in an embodiment of the present invention. The lane departure detection device 1000 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 1100 (e.g., one or more processors) and a memory 1200, and one or more storage media 1300 (e.g., one or more mass storage devices) for storing application programs 1310 or data 1320. The memory 1200 and storage media 1300 can be temporary or persistent storage. The program stored in the storage media 1300 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the lane departure detection device 1000. Furthermore, the processor 1100 may be configured to communicate with the storage media 1300 and execute the series of instruction operations in the storage media 1300 on the lane departure detection device 1000.

[0068] The logistics vehicle lane departure detection device 1000 may also include one or more power supplies 1400, one or more wired or wireless network interfaces 1500, one or more input / output interfaces 1600, and / or one or more operating systems 1330 such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will understand that... Figure 10 The illustrated device structure does not constitute a limitation on the logistics vehicle lane departure detection device 1000, and may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.

[0069] The present invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the steps of the logistics vehicle lane departure detection method.

[0070] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the system, device, or unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0071] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0072] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting lane departure of logistics vehicles, characterized in that, The logistics vehicle lane departure detection method includes: Historical logistics vehicle driving images are acquired and annotated to construct a training dataset; An initial lane departure recognition model was constructed based on the LaneNet network; The initial lane departure recognition model is trained using the training dataset, and the parameters of the initial lane departure recognition model are adjusted to obtain a trained lane departure recognition model. Real-time acquisition of image samples to be tested, inputting the image samples to be tested into the trained lane departure recognition model, and outputting the predicted value of the deviation between the logistics vehicle and the lane line; The decision to activate the early warning mechanism is based on the predicted deviation value between the logistics vehicle and the lane line.

2. The lane departure detection method for logistics vehicles according to claim 1, characterized in that, The step of annotating the historical logistics vehicle driving images to construct a training dataset includes: The historical logistics vehicle driving images are classified into normal driving state images and lane departure state images. The location information of the logistics vehicle and the lane line in the normal driving state image are labeled, and the location information of the logistics vehicle, the lane line, and the direction and degree of vehicle deviation in the lane deviation state image are labeled. The labeled and categorized images of normal driving conditions and lane departure conditions are organized to form a training dataset that includes different road types, lighting conditions, and weather conditions.

3. The lane departure detection method for logistics vehicles according to claim 1, characterized in that, The steps for constructing the initial lane departure recognition model based on the LaneNet network include: A LaneNet network framework is constructed, which includes multiple network layers, a semantic segmentation branch, and an instance embedding branch. A multi-scale feature fusion mechanism is introduced into the LaneNet network framework to construct an initial lane departure recognition model.

4. The lane departure detection method for logistics vehicles according to claim 3, characterized in that, The step of introducing a multi-scale feature fusion mechanism into the LaneNet network framework to construct an initial lane departure recognition model includes: Multi-scale feature images are extracted through the multi-layer network; A multi-scale feature fusion module is embedded between the multi-layer network. The multi-scale feature fusion module is used to fuse and enhance the multi-scale feature images extracted by the multi-layer network to generate a fused feature map containing multi-scale information. The multi-scale feature fusion module assigns adaptive weights to the multi-scale feature images extracted by the multi-layer network based on the fused feature map, and constructs an initial lane departure recognition model.

5. The lane departure detection method for logistics vehicles according to claim 1, characterized in that, The step of training the initial lane departure recognition model using the training dataset and adjusting the parameters of the initial lane departure recognition model to obtain the trained lane departure recognition model includes: The training dataset is augmented to generate an augmented training dataset. The augmented training dataset is divided into multiple batches according to the batch size, and then sequentially input into the initial lane departure recognition model; The initial lane departure recognition model is used to perform forward propagation calculations on the training dataset of each batch, and outputs the predicted value of the deviation between the logistics vehicle and the lane line. Based on the predicted offset values ​​between the logistics vehicle and the lane line and the actual offset values ​​between the logistics vehicle and the lane line labeled in the training dataset, the loss function of the initial lane offset recognition model is constructed. The gradient value of the loss with respect to the parameters of the initial lane departure recognition model is calculated by differentiating the loss function using the backpropagation algorithm. The parameters of the initial lane departure recognition model are adjusted according to the gradient value using a preset optimizer to obtain a trained lane departure recognition model.

6. The lane departure detection method for logistics vehicles according to claim 1, characterized in that, The steps of real-time acquisition of image samples to be tested, inputting the image samples to be tested into the trained lane departure recognition model, and outputting the predicted value of the deviation between the logistics vehicle and the lane line include: The vehicle-mounted camera captures real-time images of the logistics vehicle's movement, which are then used as image samples for testing. The image sample to be tested is preprocessed to obtain the preprocessed image to be tested; The preprocessed image to be tested is input into the trained lane departure recognition model, and multi-scale feature images are extracted through the multi-layer network of the trained lane departure recognition model. After the multi-scale feature image is fused and enhanced by the multi-scale feature fusion module of the trained lane departure recognition model, a fused feature map containing multi-scale information is generated. The fused feature map is semantically segmented using the semantic segmentation branch of the trained lane departure recognition model to generate a binary segmentation map containing lane line pixels. The lane departure recognition model is used to embed the lane line pixels in the binary segmentation image to generate feature vectors. Based on the binary segmentation map and feature vector, the predicted offset value between the logistics vehicle and the lane line is calculated.

7. The lane departure detection method for logistics vehicles according to claim 1, characterized in that, The step of determining whether to activate the early warning mechanism based on the predicted deviation value between the logistics vehicle and the lane line includes: Set the safety deviation threshold for logistics vehicles according to the safety standards for logistics vehicle operation; The predicted deviation of the logistics vehicle from the lane line is compared with the safe deviation threshold. If the predicted offset value is less than the safe offset threshold, the early warning mechanism is not activated; if the predicted offset value is greater than or equal to the safe offset threshold, the early warning mechanism is activated.

8. A lane departure detection device for logistics vehicles, characterized in that, The logistics vehicle lane departure detection includes: The acquisition module is used to acquire historical logistics vehicle driving images and annotate the historical logistics vehicle driving images to construct a training dataset; The training module is used to build an initial lane departure recognition model based on the LaneNet network, train the initial lane departure recognition model using the training dataset, adjust the parameters of the initial lane departure recognition model, and obtain a trained lane departure recognition model. The calculation module is used to collect test image samples in real time, input the test image samples into the trained lane departure recognition model, and output the predicted value of the deviation between the logistics vehicle and the lane line. The early warning module is used to determine whether to activate the early warning mechanism based on the predicted deviation value between the logistics vehicle and the lane line.

9. A lane departure detection device for logistics vehicles, characterized in that, It includes a memory and at least one processor, wherein the memory stores computer-readable instructions; The at least one processor invokes the computer-readable instructions in the memory to perform the steps of the logistics vehicle lane departure detection method as described in any one of claims 1-7.

10. A computer-readable storage medium storing computer-readable instructions thereon, characterized in that, When the computer-readable instructions are executed by a processor, they implement the steps of the logistics vehicle lane departure detection method as described in any one of claims 1-7.

Images