Training and Detection Methods for Panoramic Lane Line Models in Autonomous Driving

Abstract

This invention discloses a method for training and detecting lane lines in an autonomous driving panoramic view model, comprising the following steps: 1) Data acquisition, cleaning, and annotation: obtaining multiple video segments from multiple perspectives using a camera, and annotating lane lines in a segmented polyline manner; 2) Model regression branch training: for the front-view and rear-view images, keeping their original state, directly feeding them into the model for training; for the four images (left front, right left rear, right front, right rear), rotating them counterclockwise or clockwise respectively, then merging them horizontally into a single image, and feeding the rotated and merged image into the model for training; 3) Model classification branch training; 4) Model deployment and inference. In this way, this invention can greatly overcome the problem of failing to detect lane lines at small side angles due to the prior direction of the Line Anchor, enabling it to detect lane lines well both front-to-back and sideways.

Description

Technical Field

[0001] This invention belongs to the field of autonomous driving technology, specifically relating to a method for training and detecting autonomous driving panoramic lane line models. Background Technology

[0002] Autonomous driving technology is the core technology of current intelligent vehicles, consisting of three main modules: perception fusion module, decision-making and planning module, and control module. Lane detection, as a crucial sub-module within the perception fusion module, directly impacts the control accuracy and speed of the entire autonomous driving system through its detection accuracy and real-time performance.

[0003] Current academic research on lane detection algorithms mainly focuses on the forward-facing camera scenario, and there are four main schools of thought: (1) based on traditional computer vision algorithms (CN201310450258.7 Lane detection method, system, lane departure warning method and system); (2) based on deep learning semantic segmentation, which is further divided into conventional semantic segmentation (CN202011612255.5 A lane detection method and device based on deep neural network in low light environment) and lane-specific semantic segmentation (CN202011550418.1 A lane detection and segmentation method based on attention space convolutional neural network); (3) based on deep learning and anchor line detection (Tabelini, L., et al. (2020). "Keep your Eyes on the Lane: Attention-guided Lane"). (4) End-to-end lane detection based on deep learning (CN201910628238.1 An end-to-end lane detection method and system).

[0004] However, for autonomous driving systems to achieve large-scale mass production, a single forward-facing camera is far from sufficient. The Tesla Model S series, already in mass production, and the BAIC Jihu Alfa S equipped with Huawei's autonomous driving AI system, all feature cameras positioned at different viewpoints on the vehicle. A diagram illustrating a multi-view perception camera configuration is shown below. Figure 1 As shown (the hardware configuration used in this patent).

[0005] from Figure 1As can be seen, for multi-view perception, the perception range becomes 360 degrees. In addition to detecting lane lines in the forward view, it is also necessary to detect lane lines in the side view. Since the sample spaces of the side and the front are not the same, directly applying the trained forward lane line model to the side view places particularly high demands on the generalization ability of the network itself. In fact, for some models, the direction of the side lane lines relative to the horizontal plane changes significantly, making them unusable for side view detection.

[0006] Through communication and benchmarking, it was learned that almost all autonomous driving panoramic lane detection algorithms in the industry are based on School 2, namely semantic segmentation. This approach trains a unified semantic segmentation network across multiple perspectives to ensure consistent performance in side-view and front-view scenarios. While segmentation-based methods offer greater interpretability and versatility, the models themselves are inefficient in embedded systems. Furthermore, the generated segmentation masks require complex post-processing logic, including morphological processing and clustering, which still cannot guarantee consistent results. In fact, the academic community has proven that segmentation-based methods are inherently flawed for open, easily occluded, and directionally variable continuous geometric curves, such as lane lines.

[0007] While line detection based on approach 3 has significantly lower operational efficiency and requires less post-processing, it imposes certain limitations on the detection scenario due to the prior setting of the anchor line direction. Specifically, current lane line models based on line detection in the literature are only used for forward look-ahead and have a strong prior assumption about the anchor line direction, namely, diagonally forward (see...). Figure 2 The angle θ between the lane line and the horizontal plane is generally limited to a certain angular region, namely θ. min <θ<θ max From a side view, the lane lines may appear to be lateral (see...). Figure 3 ), at this time its angle θ min >θ or θ < θ max Therefore, in this case, detection-based models have difficulty detecting lane lines. Summary of the Invention

[0008] To address the aforementioned shortcomings of existing technologies, the purpose of this invention is to provide a method for training and detecting autonomous driving panoramic lane line models, thereby solving the problem that lane line detection models are difficult to adapt to side-view lane line detection.

[0009] The technical solution of this invention is implemented as follows:

[0010] 1. A method for training and detecting lane line models for autonomous driving, characterized by the following steps:

[0011] 1) Data collection, cleaning and labeling: Multiple data collection cameras are installed on the vehicle body to obtain multiple video data from multiple perspectives; a certain number of images are extracted according to a certain frame extraction frequency, lane lines in different scenarios are selected, and lane lines are labeled in a segmented polyline manner to mark the lane lines.

[0012] 2) Model Regression Branch Training: The lane line model includes a backbone network, a lane line regression branch, and a lane line classification branch. The regression branch is trained first, while the classification branch is frozen, using an initial learning rate of n1. During training, the front-view and rear-view images are fed directly into the model in their original state. For the left-front and right-front images, the left-front image is rotated counter-clockwise (denoted as font-left-rotated), and the right-front image is rotated clockwise (denoted as font-right-rotated). Then, they are merged horizontally. During merging, the font-left image of the rotated left-front image is... The left-rotated image is on the left, and the rotated right front image is on the right. For the two rear images, the left rear image is rotated clockwise (denoted as rear-left-rotated), and the right rear image is rotated counterclockwise (denoted as rear-right-rotated). Then, they are merged horizontally. When merging, the rotated right rear image is on the left and the rotated left rear image is on the right. After rotation and merging, the image is fed into the model for training.

[0013] 3) Model classification branch training: The initial learning rate n2 is used, which is more than four times that of n1. During training, for the front-view and rear-view images, their original states are directly fed into the model for training. For the left-front and right-front images, the left-front image is first rotated counterclockwise (denoted as font-left-rotated), and the right-front image is rotated clockwise (denoted as font-right-rotated). Then, they are merged horizontally. After merging, the left-front image with font-left-rotated is on the left, and the right-front image with font-right-rotated is on the right. For the rear-back and right-back images, the left-back image is first rotated clockwise (denoted as rear-left-rotated), and the right-back image is rotated counterclockwise (denoted as rear-right-rotated). Then, they are merged horizontally. After merging, the right-back image with rear-right-rotated is on the left, and the left-back image with rear-left-rotated is on the right. After rotation and merging, they are fed into the model for training.

[0014] 4) Model deployment and inference: Using PyTorch's JIT module, the model is exported as an ONNX module and fixed-point quantization is performed. Finally, ONNX runtime is used to convert the model into a TRT engine, enabling the model to run efficiently on embedded devices.

[0015] During model inference, the process needs to correspond to that during model training. The front and back images should remain unchanged. The left image should be rotated counterclockwise or clockwise and then merged. The right image should be rotated clockwise or counterclockwise and then merged. Then, the model inference results should be post-processed.

[0016] Thus, the present invention can greatly overcome the problem that lane lines at small angles on the side cannot be detected due to the prior direction of the Line Anchor, and can detect lane lines well both front and rear and on the side.

[0017] Furthermore: after the images are horizontally merged and then resized to the same scale, the angle between a lane line and the horizontal plane will increase. Taking the left front image as an example:

[0018]

[0019]

[0020] Where H is the height of the image; u, v are the pixel coordinates of the lane line endpoints; and Δ is the position from the intersection of the lane line and the lower edge of the image to the lower left corner. Thus, by processing the image using the method of this invention, the angle between the lane line and the horizontal plane is significantly increased, greatly enhancing the generalization performance of the lane line model under multiple viewpoints.

[0021] Further: The input images on the left and right sides are rotated counterclockwise and clockwise respectively, with the rotation centers at the bottom left and bottom right corners, and the rotation angles between 2 and 4 degrees; for a point on a lane line on the left and right sides, its coordinates before and after the transformation are:

[0022]

[0023]

[0024] After sorting, we get:

[0025]

[0026]

[0027] Furthermore, when training the classification branch of the model, the regression branch is simultaneously fine-tuned. For regression branch training, the initial learning rate n1 is 0.0002, and for classification branch training, the initial learning rate n2 is 0.001. This allows for simultaneous fine-tuning of the regression branch while training the classification branch, minimizing the impact of training the classification branch on the regression branch.

[0028] Furthermore: the value of n2 is 4 times or more than that of n1.

[0029] Furthermore, the reasoning process includes the following steps:

[0030] (1) For a certain moment, first collect six view images simultaneously captured by six cameras. For the front view image and the back view image, directly resize to the input size required by the network. For the two images of the left front and right front after rotation, merge left and right and resize. For the two images of the right rear and left rear after rotation, merge right and left and resize to keep the sample space consistent with that during training. Then stack these four images and feed them into the model.

[0031] (2) The TRT model performs inference on the images and detects lane lines in each image;

[0032] (3) In post-processing, linear NMS is used to suppress the maximum value of overlapping lane lines; for the front and rear images, the lane line positions can be directly regressed after suppression; for the two merged composite images from the side, after suppression, the left and right sides are separated by the image midline, and then the lane lines detected on both sides are extracted to obtain the continuous pixel coordinates (u) of each lane line. detect ,v detect Since the image is merged from left to right, the actual coordinates of the lane line on the left are:

[0033] u after_left =2*u detect_left (7)

[0034] v after_left =v detect_left (8)

[0035] The actual coordinates of the lane line on the right are:

[0036]

[0037] v after_right =v detect_right (10)

[0038] Since the side view image is rotated before being fed into the model for inference, the detected lane line point coordinates are the coordinates of the actual lane lines after rotation. To obtain the actual lane line coordinates, the lane line points need to be rotated in the opposite direction, that is, according to formulas (5) and (6), based on... and Calculate and

[0039] Through reasoning, the actual lane line coordinates can be obtained, and because the images are merged together, the batch size of the model is reduced from six to four, further reducing the computing power consumed during model reasoning.

[0040] Furthermore, six cameras are installed on the vehicle body, one each in the front, rear, left front, left rear, right front, and right rear directions. This provides a comprehensive view of the vehicle, offering essential data for subsequent image processing.

[0041] Furthermore, the perspectives of two adjacent cameras partially overlap, thus covering 360 degrees around the vehicle. This 360-degree coverage provides a comprehensive view of the vehicle, offering essential data for subsequent image processing.

[0042] Furthermore: During training, the images on the left and right sides are randomly rotated and merged, and then mixed with the front-view and back-view images as a batch for training; while training, the images are flipped according to a certain probability.

[0043] Furthermore, the rotation angle is between 2 and 4 degrees.

[0044] In summary, the present invention has the following beneficial effects:

[0045] 1. This invention can greatly overcome the problem that lane lines at small angles on the side cannot be detected due to the prior direction of the Line Anchor, and enables it to detect lane lines well both front and rear and on the side.

[0046] 2. Because the images are merged together, the batch size of the model is reduced from six to four, further reducing the computing power consumed during model inference.

[0047] 3. The lane line model architecture, model training process, and model inference process proposed in this invention can simultaneously output the position and category of lane lines, and greatly enhance the generalization performance of lane line models under multi-view conditions. At the same time, it can achieve real-time performance of 20FPS+ on embedded systems, and can serve as the core algorithm for three subsystems in autonomous driving systems: Lane Departure Warning (LDW), Lane Keeping System (LKS), and Lane Change Assistance (LCA).

[0048] 4. The algorithm of this invention has been deployed on the applicant's internal test vehicle. After a long period of actual testing, it can stably perform real-time lane detection and vehicle control at a speed of 20 FPS. Attached Figure Description

[0049] Figure 1 This is a schematic diagram of a multi-view (panoramic view) camera configuration;

[0050] Figure 2 This is a diagram of the Anchor Line;

[0051] Figure 3 It is a side view with a small angle between the lane line and the horizontal line;

[0052] Figure 4 This is an example diagram labeled with the present invention;

[0053] Figure 5 This is a schematic diagram of the model architecture of the present invention;

[0054] Figures 6-1 and 6-2 are schematic diagrams of the training method of the present invention;

[0055] Figures 7-1 and 7-2 are curves showing the changes in model metrics during the training of classification branches in this invention.

[0056] Figure 8 This is a flowchart of the model reasoning process of this invention;

[0057] Figure 9 This is a diagram demonstrating the reasoning effect of the model of this invention. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0059] It should be noted that similar reference numerals and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the figures, or the orientation or positional relationship commonly used when the product is in use. They are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third," etc., are only used to distinguish descriptions and should not be construed as indicating or implying relative importance. In addition, the terms "horizontal," "vertical," etc., do not indicate that the component is required to be absolutely horizontal or suspended, but can be slightly tilted. For example, "horizontal" simply means that its direction is more horizontal than "vertical," and does not mean that the structure must be completely horizontal, but can be slightly tilted. In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0060] See Figure 1 — Figure 9 The present invention provides a method for training and detecting a panoramic lane line model for autonomous driving, the specific implementation steps of which are as follows:

[0061] 1. Data collection, cleaning, and labeling: according to... Figure 1As shown, multiple cameras and radars are installed on the vehicle body to collect data from multiple perspectives, obtaining multiple video segments from various angles. Then, a certain number of images are extracted according to a specific frame-sampling frequency. Following the principle of data diversity, lane lines for different scenarios are filtered, and then the lane lines are labeled using a segmented polyline method, indicating the lane line type. (See [reference]). Figure 4 .

[0062] Six cameras are installed on the vehicle body, one each in the front, rear, left front, left rear, right front, and right rear directions, ensuring that the views of adjacent cameras partially overlap, thus covering 360 degrees around the vehicle. The LiDAR is mounted on the top of the vehicle body.

[0063] 2. Model regression branch training: See lane line model. Figure 5 This lane line model is a common anchor-line lane line model in the industry, including a backbone network, lane line regression branch, and lane line classification branch. Initially, only the regression branch is trained, while the classification branch is frozen. This is because training both the classification and regression branches simultaneously with completely random parameter initialization can easily lead to model non-convergence. The data preprocessing module of this patent is shown in Figure 6. During training, the front-view and rear-view images are directly fed into the model in their original state. However, for side images, due to the significant difference in their sample space and front-rear dimensions, especially the small angle between the lane lines and the horizontal plane, directly feeding them into the model would, in practice, greatly interfere with the training effect of the front-view model. Therefore, this patent proposes rotating and merging the left and right images. For the two front-left and right images, the left-front image is first rotated counterclockwise (denoted as font-left-rotated), and the right-front image is rotated clockwise (denoted as font-right-rotated), and then merged horizontally. The rotated left-front image (font-left-rotated) is on the left, and the rotated right-front image (font-right-rotated) is on the right. For the rear left and right rear images, the left rear image is first rotated clockwise (denoted as rear-left-rotated), and the right rear image is rotated counterclockwise (denoted as rear-right-rotated). Then, they are merged horizontally. When merged, the rotated right rear image (rear-right-rotated) is on the left, and the rotated left rear image (rear-left-rotated) is on the right. After rotation and merging, the images are fed into the model for training.

[0064] After rotation and merging, the images are fed into the model for training. The rotation angle in this invention is typically around 2-4 degrees. This results in images whose sample space can effectively match the preceding and following images, ensuring consistency of the training data. Specifically: because the images are horizontally merged and then resized to the same scale, the angle between a lane line and the horizontal plane will increase (see Figure 6-1). Taking the left front image as an example:

[0065]

[0066]

[0067] Where H is the height of the image; u,v are the pixel coordinates of the lane line endpoints; and Δ is the position from the intersection of the lane line and the bottom edge of the image to the lower left corner.

[0068] Therefore, the angle between the transformed lane lines and the horizontal plane increases, which is more conducive to lane line detection. Furthermore, before merging, this patent rotates the input images on both sides counterclockwise or clockwise, further increasing the angle between the lane lines and the horizontal plane. The rotation centers are the lower left and lower right corners, respectively, as shown in Figure 6-2. For a point on a lane line on the left and right sides, its coordinates before and after transformation are:

[0069]

[0070]

[0071] After sorting, we get:

[0072]

[0073]

[0074] During training, images from the left and right sides are randomly rotated and merged, and then mixed with the front-view and back-view images as a batch for training. Simultaneously, images are flipped with a certain probability to further increase the diversity of the sample space, thereby improving the model's generalization ability.

[0075] 3. Model Classification Branch Training: After training the lane regression branch of the model, the classification branch is trained, while the regression branch is fine-tuned. To minimize the impact of training the classification branch on the regression branch, this patent uses different learning rates for different branches: a very low initial learning rate n1 (e.g., 0.0002) is used for the regression branch + backbone network, while a higher initial learning rate n2 (e.g., 0.001) is used for the classification branch. The value of n2 is a multiple of n1, such as four times or more. During training, the training method proposed in this patent is used in the same way as in the second step (Figures 6-1 and 6-2). For the front-view and rear-view images, their original state is kept and directly fed into the model for training. For the two front images, left-front and right-front, the left-front image is first rotated counterclockwise (denoted as font-left-rotated), and the right-front image is rotated clockwise (denoted as font-right-rotated). Then, they are merged horizontally. When merged, the left-front image after rotation is font-left-rotated on the left and the right-front image after rotation is font-right-rotated on the right. For the two rear images, left-rear and right-rear, the left-rear image is first rotated clockwise (denoted as rear-left-rotated), and the right-rear image is rotated counterclockwise (denoted as rear-right-rotated). Then, they are merged horizontally. When merged, the right-rear image after rotation is rear-right-rotated on the left and the left-rear image after rotation is rear-left-rotated on the right. After rotation and merging, they are fed into the model for training. The rotation angle of this invention is usually around 2-4 degrees. The resulting images have a sample space that can effectively match those before and after, ensuring the consistency of the training data. Figure 7 shows the metric changes during training. It can be seen that while maintaining a relatively constant regression accuracy, the model's classification accuracy continuously increases. Practice has proven that this two-step training method, compared to a one-step method, more easily achieves high accuracy in both lane line position regression and class detection simultaneously.

[0076] 4. Model Deployment and Inference: Utilizing PyTorch's JIT module, the model is exported as an ONNX module and subjected to fixed-point quantization. Finally, the ONNX runtime is used to convert the model into a TRT engine, enabling efficient model execution on embedded devices. The deployed model is only a sub-module of the overall inference process. This patent proposes a complete inference process adapted to the model training and camera mounting methods, as detailed below. Figure 8 As shown:

[0077] (1) At a certain moment, six view images simultaneously captured by six cameras are collected. For the front-view and rear-view images, they are directly resized to the input size required by the network. For the side-front and side-rear images, they are rotated and merged clockwise or counterclockwise as shown in Figure 6 to keep the sample space consistent with that during training. Specifically, for the two rotated images on the front side, the left front and right front are merged and resized; for the two rotated images on the rear side, the right rear and left rear are merged and resized. Then, these four images are stacked and fed into the model.

[0078] (2) The TRT model infers the images and detects the lane lines on each image.

[0079] (3) In post-processing, linear NMS is used to suppress the maximum value of overlapping lane lines. For the front and rear images, the lane line positions can be directly regressed after suppression. For the two merged composite images from the side, after suppression, the left and right sides are separated by the image midline, and the detected lane lines on both sides are extracted to obtain the continuous pixel coordinates (u) of each lane line. detect ,v detect Since the image is merged from left to right, the actual coordinates of the lane line on the left are:

[0080] u after_left =2*u detect_left (7)

[0081] v after_left =v detect_left (8)

[0082] The actual coordinates of the lane line on the right are:

[0083]

[0084] v after_right =v detect_right (10)

[0085] Since the side view image is rotated before being fed into the model for inference, the coordinates of the detected lane line points are the coordinates of the actual lane lines after rotation. To obtain the actual lane line coordinates, the lane line points need to be rotated in the opposite direction, that is, according to formulas (5) and (6), based on... and Calculate and The lane marking process is now complete.

[0086] This invention significantly overcomes the problem of failing to detect lane lines at small angles on the sides due to the prior orientation of the Line Anchor, enabling it to detect lane lines well both front-to-back and on the sides. Because the images are merged, the batch size of the model is reduced from six to four, further reducing the computational power consumed during model inference.

[0087] Finally, it should be noted that the above examples of the present invention are merely illustrative and not intended to limit the implementation of the invention. Although the applicant has described the present invention in detail with reference to preferred embodiments, those skilled in the art can make other variations and modifications based on the above description. It is impossible to exhaustively list all possible implementations here. All obvious variations or modifications derived from the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for training and detecting lane line models for autonomous driving, characterized in that... Includes the following steps: 1) Data collection, cleaning and labeling: Multiple data collection cameras are installed on the vehicle body to obtain multiple video data from multiple perspectives; a certain number of images are extracted according to a certain frame extraction frequency, lane lines in different scenarios are selected, and lane lines are labeled in a segmented polyline manner to mark the lane lines. 2) Model regression branch training: The lane line model includes the model backbone network, lane line regression branch, and lane line classification branch. The regression branch is trained first, while the classification branch is frozen. The initial learning rate is n1. During training, the front-view and rear-view images are fed directly into the model in their original state. For the two front images (left and right front), the left front image is first rotated counterclockwise (font-left-rotated), and the right front image is rotated clockwise (font-right-rotated). Then, they are merged horizontally. After merging, the rotated left front image (font-left-rotated) is on the left, and the rotated right front image (font-right-rotated) is on the right. For the two rear images (left and right rear), the left rear image is first rotated clockwise (rear-left-rotated), and the right rear image is rotated counterclockwise (rear-right-rotated). Then, they are merged horizontally. After merging, the rotated right rear image (rear-right-rotated) is on the left, and the rotated left rear image (rear-left-rotated) is on the right. After rotation and merging, the images are fed into the model for training. The rotation angle is between 2 and 4 degrees. 3) Model classification branch training: The initial learning rate used is n2, and the value of n2 is a multiple of n1; During training, the front-view and rear-view images are fed directly into the model in their original state. For the two front images (left and right), the left front image is first rotated counterclockwise (font-left-rotated), and the right front image is rotated clockwise (font-right-rotated). Then, they are merged horizontally. After merging, the left front image (font-left-rotated) is on the left and the right front image (font-right-rotated) is on the right. For the two rear images (left and right), the left rear image is first rotated clockwise (rear-left-rotated), and the right rear image is rotated counterclockwise (rear-right-rotated). Then, they are merged horizontally. After merging, the right rear image (rear-right-rotated) is on the left and the left rear image (rear-left-rotated) is on the right. After rotation and merging, they are fed into the model for training. 4) Model deployment and inference: Using PyTorch's JIT module, the model is exported as an ONNX module and fixed-point quantization is performed. Finally, ONNX runtime is used to convert the model into a TRT engine, enabling the model to run efficiently on embedded devices. During model inference, it needs to correspond to the model training. For the front and back images, directly resize them to the input size required by the network. For the two rotated images on the front side, merge the left and right front images and resize them. For the two rotated images on the back side, merge the right and left back images and resize them. Then, post-process the model inference results. The rotation angle is between 2 and 4 degrees.

2. The method for training and detecting autonomous driving panoramic lane line models according to claim 1, characterized in that: After merging the images horizontally and then resizing them to the same scale, the angle between a lane line and the horizontal plane will increase. The change in the angle of the left front image is as follows: Where H is the height of the image; u,v are the pixel coordinates of the lane line endpoints; and Δ is the position from the intersection of the lane line and the bottom edge of the image to the lower left corner.

3. The method for training and detecting autonomous driving panoramic lane line models according to claim 1 or 2, characterized in that: The input images on the left and right sides are rotated counterclockwise and clockwise respectively, with the rotation centers at the bottom left and bottom right corners. For a point on a lane line on the left and right sides, its coordinates before and after the transformation are: After sorting, we get:

4. The method for training and detecting autonomous driving panoramic lane line models according to claim 1, characterized in that: The value of n2 is 4 times or more than that of n1.

5. The method for training and detecting autonomous driving panoramic lane line models according to any one of claims 1, 2, and 4, characterized in that: The reasoning process includes the following steps: (1) For a certain moment, first collect six view images simultaneously captured by six cameras. For the front view image and the back view image, directly resize to the input size required by the network. For the two images of the left front and right front after rotation, merge left and right and resize. For the two images of the right rear and left rear after rotation, merge right and left and resize to keep the sample space consistent with that during training. Then stack these four images and feed them into the model. (2) The TRT model performs inference on the images and detects lane lines in each image; (3) In post-processing, linear NMS is used to suppress the maximum value of overlapping lane lines; for the front and rear images, the lane line positions can be directly regressed after suppression; for the two merged composite images from the side, after suppression, the left and right sides are separated by the image midline, and then the lane lines detected on both sides are extracted to obtain the continuous pixel coordinates (u) of each lane line. detect ,v detect Since the image is merged from left to right, the actual coordinates of the lane line on the left are: in after_left =2*u detect_left (7) v after_left =v detect_left (8) The actual coordinates of the lane line on the right are: v after_right =v detect_right (10) Since the side view image is rotated before being fed into the model for inference, the detected lane line point coordinates are the coordinates of the actual lane lines after rotation. To obtain the actual lane line coordinates, the lane line points need to be rotated in the opposite direction, that is, according to formulas (5) and (6), based on... and Calculate and 6. The method for training and detecting autonomous driving panoramic lane line models according to claim 5, characterized in that: Six cameras are installed in total, one each in the front, rear, left front, left rear, right front, and right rear directions of the vehicle.

7. The method for training and detecting autonomous driving panoramic lane line models according to claim 6, characterized in that: The perspectives of two adjacent cameras partially overlap, thus covering 360 degrees around the vehicle.

8. The method for training and detecting autonomous driving panoramic lane line models according to any one of claims 1, 2, and 4, characterized in that: During training, the images on the left and right sides are randomly rotated and merged, and then mixed with the front-view and back-view images as a batch for training. At the same time, the images are flipped according to a certain probability.

9. The method for training and detecting autonomous driving panoramic lane line models according to any one of claims 1 or 4, characterized in that: When training the classification branch of the model, the regression branch is also fine-tuned. For the regression branch training, the initial learning rate n1 is 0.0002, and for the classification branch, the initial learning rate n2 is 0.001.

Images