Lane detection model training method, lane detection method, medium and controller

Abstract

This invention discloses a training method for a lane detection model, as well as a lane detection method, medium, and controller. The training method includes: acquiring a training dataset, which includes lane training images and their corresponding lane label data; inputting the lane training images from the training dataset into the lane detection model for feature extraction to obtain a feature map; projecting multiple predefined line anchors onto the feature map and extracting features from the regions where each predefined line anchor is located to obtain each line anchor feature; using an internal attention mechanism within the line anchors and an attention mechanism between different line anchors to perform feature interaction and fusion on the features of each line anchor to obtain predicted lane data; determining a model loss term based on the lane label data and the predicted lane data; and adjusting the model parameters of the lane detection model based on the model loss term to obtain a trained lane detection model. The lane detection model trained by this method effectively handles lane detection problems in complex road conditions.

Description

Technical Field

[0001] This invention relates to the field of intelligent transportation technology, and in particular to a training method for a lane detection model, as well as a lane detection method, medium, controller, and vehicle. Background Technology

[0002] In intelligent transportation systems, lane detection is one of the key technologies for realizing functions such as autonomous driving and advanced driver assistance systems. In complex road conditions, such as vehicle obstruction, dense lane lines, and road surface damage, traditional lane detection methods often suffer from insufficient feature extraction, poor feature interaction and fusion effects, and insufficient stability of lane position regression. These problems often make it difficult to accurately detect lane lines, leading to decreased detection accuracy, affecting system reliability, and failing to meet the high accuracy and robustness requirements of lane detection in complex road conditions in practical applications. Summary of the Invention

[0003] This invention aims to at least partially solve one of the technical problems in related technologies. To this end, one objective of this invention is to propose a training method for a lane detection model. The trained lane detection model can effectively handle lane detection problems in complex road conditions, improving the performance of lane detection under complex road conditions, especially in terms of detection accuracy and stability under difficult conditions such as occlusion and high density.

[0004] The second objective of this invention is to provide a lane detection method.

[0005] A third objective of this invention is to provide a computer-readable storage medium.

[0006] The fourth objective of this invention is to provide a controller.

[0007] The fifth objective of this invention is to provide a vehicle.

[0008] To achieve the above objectives, a first aspect of the present invention proposes a training method for a lane detection model. The training method includes: acquiring a training dataset, which includes lane training images and their corresponding lane label data; inputting the lane training images in the training dataset into a lane detection model for feature extraction processing to obtain a feature map; projecting multiple predefined anchors onto the feature map and extracting features from the regions where each predefined anchor is located to obtain anchor features; using an internal anchor attention mechanism and an inter-anchor attention mechanism to perform feature interaction and fusion on the anchor features to obtain predicted lane data; determining a model loss term based on the lane label data and the predicted lane data; and adjusting the model parameters of the lane detection model based on the model loss term to obtain a trained lane detection model.

[0009] The training method of the lane detection model according to the embodiment of the present invention extracts multiple line anchor features by re-extracting the corresponding feature map of the lane image based on predefined line anchors, and uses the internal attention mechanism of line anchors and the attention mechanism between different line anchors to perform feature interaction and fusion of each line anchor feature. This can effectively handle the lane detection problem in complex road conditions and improve the performance of lane detection in complex road conditions, especially the detection accuracy and stability under difficult conditions such as occlusion and density.

[0010] In addition, the training method for the lane detection model proposed in the above embodiments of the present invention may also have the following additional technical features: According to one embodiment of the present invention, the lane detection model includes an image feature extraction module, a line anchor projection and feature extraction module, an in-lane attention module, an inter-lane attention module, and a fusion module. The input terminal of the image feature extraction module is used to input the lane training image. The output terminal of the image feature extraction module is connected to the input terminal of the line anchor projection and feature extraction module. The output terminal of the line anchor projection and feature extraction module is connected to the input terminals of the in-lane attention module and the inter-lane attention module, respectively. The output terminals of the in-lane attention module and the inter-lane attention module are connected to the output terminal of the fusion module. The output terminal of the fusion module is used to output the predicted lane data. The lane training images in the training dataset are input into the lane detection model for feature extraction processing to obtain a feature map, and multiple predefined line anchors are projected onto the feature map. The process involves extracting features from the regions where each predefined anchor is located to obtain anchor features, and then using intra-anchor attention mechanisms and inter-anchor attention mechanisms to interact and fuse these anchor features. This includes: using the image feature extraction module to extract features from the lane training image to obtain the feature map; using the anchor projection and feature extraction module to project multiple predefined anchors onto the feature map and extract features from the regions where each predefined anchor is located to obtain anchor features; using the lane-internal attention module to process the anchor features using the intra-anchor attention mechanism to obtain anchor attention features; using the inter-lane attention module to process the anchor features using the inter-anchor attention mechanism to obtain inter-anchor features; and using the fusion module to fuse the inter-anchor features and the anchor attention features to obtain the predicted lane data.

[0011] According to an embodiment of the present invention, the step of using the anchor projection and feature extraction module to project a plurality of predefined anchors onto the feature map and extract features of the region where each predefined anchor is located includes: projecting each predefined anchor onto the feature map according to the ratio between the image where each predefined anchor is located and the feature map; and extracting features from the projected feature map according to the region where each predefined anchor is located to obtain the features of each anchor.

[0012] According to one embodiment of the present invention, the step of using the lane-in-lane attention module to process each line anchor feature using the line anchor internal attention mechanism to obtain each line anchor attention feature includes: calculating the attention weight matrix within the line anchor based on each line anchor feature to obtain a first attention weight matrix corresponding to each line anchor feature; and obtaining each line anchor attention feature based on each line anchor feature and its corresponding first attention weight matrix.

[0013] According to an embodiment of the present invention, the process of using the inter-lane attention module to process the inter-lane anchor features using an inter-lane attention mechanism to obtain inter-lane anchor features includes: performing feature concatenation on the inter-lane anchor features to obtain concatenated features; calculating an attention weight matrix between different inter-lane anchors based on the concatenated features to obtain a second attention weight matrix; and obtaining the inter-lane anchor features based on the second attention weight matrix and the inter-lane anchor features.

[0014] According to one embodiment of the present invention, the lane label data includes the actual lane type, the actual lane length, and the actual lane key point coordinate set; the predicted lane data includes the predicted lane type, the predicted lane length, and the predicted lane key point coordinate set; and the step of determining the model loss term based on the lane label data and the predicted lane data includes: determining a classification loss term based on the actual lane type and the predicted lane type; determining a regression impairment term based on the actual lane length and the predicted lane length; determining a key point loss term based on the actual lane key point coordinate set and the predicted lane key point coordinate set; and determining a model loss term based on the classification loss term, the regression impairment term, and the key point loss term.

[0015] To achieve the above objectives, a second aspect of the present invention provides a lane detection method, the detection method comprising: inputting a lane image to be tested into a pre-trained lane detection model to obtain actual lane lines, wherein the pre-trained lane detection model is obtained using the training method of the lane detection model proposed in the first aspect of the present invention.

[0016] To achieve the above objectives, a third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it implements the training method for the lane detection model proposed in the first aspect of the present invention, or the lane detection method proposed in the second aspect of the present invention.

[0017] To achieve the above objectives, a fourth aspect of the present invention provides a controller, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, it implements the lane detection model training method proposed in the first aspect of the present invention, or the lane detection method proposed in the second aspect of the present invention.

[0018] To achieve the above objectives, a fifth aspect of the present invention provides a vehicle including a controller as provided in the fourth aspect of the present invention.

[0019] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0020] Figure 1 This is a flowchart of a training method for a lane detection model according to an embodiment of the present invention; Figure 2 This is a schematic diagram of a lane detection model according to an embodiment of the present invention; Figure 3 This is a flowchart of the feature processing of a lane detection model according to an embodiment of the present invention; Figure 4 This is a flowchart illustrating the process of obtaining the anchor feature according to an embodiment of the present invention; Figure 5 This is a flowchart illustrating the process of obtaining line anchor attention features according to an embodiment of the present invention; Figure 6 This is a flowchart illustrating the process of obtaining the features between the anchors according to an embodiment of the present invention; Figure 7 This is a flowchart illustrating the determination of the model loss term according to an embodiment of the present invention; Figure 8 This is a flowchart of a lane detection method according to an embodiment of the present invention; Figure 9 This is a structural block diagram of the controller according to an embodiment of the present invention; Figure 10 This is a schematic diagram of a vehicle according to an embodiment of the present invention. Detailed Implementation

[0021] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0022] The training method for the lane detection model, the lane detection method, the medium, the controller, and the vehicle of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0023] Figure 1 This is a flowchart of a training method for a lane detection model according to an embodiment of the present invention. Figure 1 The training method for the lane detection model may include: S101, Obtain the training dataset, which includes lane training images and their corresponding lane label data.

[0024] Specifically, a large number of images containing lane lines are collected and labeled. The labeling content may include lane type, lane length, and lane coordinate set, etc., to obtain a training dataset.

[0025] S102, input the lane training images in the training dataset into the lane detection model for feature extraction to obtain a feature map, project multiple predefined line anchors onto the feature map and extract the features of the regions where each predefined line anchor is located to obtain the features of each line anchor, and use the internal attention mechanism of the line anchor and the attention mechanism between different line anchors to perform feature interaction and fusion of the features of each line anchor to obtain the predicted lane data.

[0026] Specifically, to improve the accuracy and stability of the trained lane detection model in lane line detection, the lane detection model in this embodiment of the invention extracts features from the input lane training image to obtain a feature map when predicting lane data in the input lane training image. Predefined line anchors are projected onto this feature map, and feature extraction processing is performed on the projected feature map based on the region where each predefined line anchor is located, resulting in multiple line anchor features. An internal attention mechanism within the line anchors and an inter-line anchor attention mechanism are used to perform feature interaction and fusion of the line anchor features, and the lane detection model outputs predicted lane data.

[0027] S103, determine the model loss term based on lane label data and predicted lane data.

[0028] Specifically, the model loss term is determined based on lane label data and predicted lane data using a pre-built loss function.

[0029] S104. Based on the model loss term, adjust the model parameters of the lane detection model to obtain a trained lane detection model.

[0030] Specifically, based on the model loss term calculated during this training, the lane detection model parameters are optimized using the backpropagation algorithm, adjusting the model parameters. The lane detection model is then continuously trained using the above steps until the model loss term corresponding to the loss function converges to its minimum, at which point the trained lane detection model is obtained.

[0031] The training method of the lane detection model in this embodiment of the invention extracts multiple line anchor features from the feature map corresponding to the lane image based on predefined line anchors, and uses the internal attention mechanism of line anchors and the attention mechanism between different line anchors to perform feature interaction and fusion of each line anchor feature. This can effectively handle the lane detection problem in complex road conditions and improve the performance of lane detection in complex road conditions, especially the detection accuracy and stability under difficult conditions such as occlusion and density.

[0032] In one embodiment of the present invention, such as Figure 2 and Figure 3 As shown, the lane detection model may include an image feature extraction module, a line anchor projection and feature extraction module, an intra-lane attention module, an inter-lane attention module, and a fusion module. The input of the image feature extraction module is used to input the lane training image. The output of the image feature extraction module is connected to the input of the line anchor projection and feature extraction module. The output of the line anchor projection and feature extraction module is connected to the inputs of the intra-lane attention module and the inter-lane attention module, respectively. The outputs of the intra-lane attention module and the inter-lane attention module are connected to the output of the fusion module. The output of the fusion module is used to output predicted lane data. The lane training images from the training dataset are input into the lane detection model for feature extraction to obtain a feature map. Multiple predefined line anchors are projected onto the feature map, and features of the regions where each predefined line anchor is located are extracted to obtain the features of each line anchor. Intra-line anchor attention mechanisms and inter-line anchor attention mechanisms are used to perform feature interaction and fusion of the features of each line anchor. This may include: S201, using the image feature extraction module, performs feature extraction processing on the lane training image to obtain a feature map.

[0033] In practice, the image feature extraction module utilizes a backbone network to extract features from the input lane training image, obtaining a feature map of the lane training image. The backbone network can be a classic convolutional neural network, such as ResNet (Residual Network) or VGG (Visual Geometry Group), which, through multi-layer convolution and pooling operations, progressively extracts low-level geometric features, mid-level shape features, and high-level semantic features from the lane training image, providing rich image information for subsequent lane anchor feature processing.

[0034] In one specific embodiment, the backbone network can be ResNet-50, which has good feature extraction capabilities and a deep structure, enabling it to effectively extract multi-level features from images. The lane training images input to the lane detection model are first processed by the convolutional layers and pooling layers of ResNet-50, ultimately yielding a feature map of size H×W×C, where H is the height of the feature map, W is the width of the feature map, and C is the number of feature channels.

[0035] S202, using the anchor projection and feature extraction module, multiple predefined anchors are projected onto the feature map and the features of the region where each predefined anchor is located are extracted to obtain the features of each anchor.

[0036] In this embodiment of the invention, the predefined line anchor is a predefined line segment model with a specific length and direction, used to characterize the potential lane line structure in an image. In this embodiment of the invention, multiple predefined line anchors have different lengths and / or directions.

[0037] Specifically, multiple predefined line anchors are projected onto the feature map extracted by the image feature extraction module. Each predefined line anchor corresponds to a region on the feature map. By sampling and integrating the features of this region, multiple line anchor features that can characterize the location of the predefined line anchors and the features of potential lane lines are obtained.

[0038] In this embodiment of the invention, the predefined line anchor is a pre-designed structured line segment model, representing the possible location and direction of lane lines. The location of the predefined line anchor is its coordinates or region in the original image (the image defining the line anchor), used for localization. The line anchor features extracted in this embodiment of the invention not only contain location information but also the visual image features of "potential lane lines" in the feature map, serving as the basis for determining which line segment model in the image is most likely a lane line.

[0039] It should be noted that potential lane line features do not refer to the features of actual lane lines, but rather to the features extracted from the feature map based on the position of the line anchors that indicate the possible presence of lane lines. These features are visual information (such as color, texture, edges, and orientation) of the areas where lane lines may appear, inferred by feature extraction based on prior knowledge of predefined line anchors.

[0040] S203 utilizes the lane-in-lane attention module to process the features of each line anchor using the line anchor internal attention mechanism, thereby obtaining the attention features of each line anchor.

[0041] Specifically, the lane-in-lane attention module uses the line anchor internal attention mechanism to process the features of each line anchor.

[0042] In practice, a lane-line keypoint attention mechanism (a type of line anchor internal attention mechanism) can be adopted. On the same lane line, key points at different locations contain rich local and global information. By constructing an intra-lane attention model and calculating the attention weights among the features of these key points, information interaction and fusion can be promoted. For example, when a portion of the lane line lacks features due to occlusion, surrounding key points can provide crucial context for the missing part based on their spatial and semantic relationships, effectively improving the integrity and anti-interference capability of lane line features.

[0043] S204. Using the inter-lane attention module, the inter-lane anchor features are processed by the inter-lane anchor attention mechanism to obtain the inter-lane anchor features.

[0044] Specifically, the lane-to-lane attention module uses different inter-lane anchor attention mechanisms to process the features of each lane anchor.

[0045] S205 utilizes a fusion module to fuse the features between line anchors and the attention features of each line anchor to obtain predicted lane data.

[0046] Specifically, the fusion module fuses the line anchor attention features output by the lane-in-lane attention module and the line anchor features output by the inter-lane attention module to obtain predicted lane data.

[0047] In one embodiment of the present invention, such as Figure 4 As shown, the anchor projection and feature extraction module projects multiple predefined anchors onto a feature map and extracts features from the regions where each predefined anchor is located. This can include: S301, Based on the proportional relationship between the image where each predefined anchor is located and the feature map, project each predefined anchor onto the feature map; S302, based on the region where each predefined anchor is located, feature extraction is performed on the projected feature map to obtain the features of each anchor.

[0048] In this embodiment of the invention, the starting point coordinates and ending point coordinates of the line anchor are defined in the image coordinate system to form line segment models in different directions, thereby obtaining the predefined line anchor.

[0049] Specifically, the image containing the predefined line anchor can be determined based on its start and end coordinates. Then, based on the proportional relationship between the image containing each predefined line anchor and the feature map, each line segment model is projected onto the feature map. For the region covered by the predefined line anchor on the feature map, an interpolation method, such as bilinear interpolation, is used to obtain the feature values ​​within that region. These values ​​are then integrated to obtain the feature vector corresponding to each predefined line anchor, i.e., the line anchor feature.

[0050] In one embodiment of the present invention, such as Figure 5 As shown, the lane-internal attention module is used to process the features of each line anchor using the line anchor internal attention mechanism to obtain the attention features of each line anchor, which may include: S401, calculate the attention weight matrix within each anchor based on the features of each anchor, and obtain the first attention weight matrix corresponding to each anchor feature; S402, based on the features of each anchor line and its corresponding first attention weight matrix, obtain the attention features of each anchor line.

[0051] Specifically, the anchor feature of each predefined anchor is denoted as... Where L is the number of sampling points of the predefined line anchor on the feature map. This represents the number of feature channels. The attention weight matrix (first attention weight matrix) within the predefined anchor is calculated based on the features of each anchor using a self-attention mechanism (internal attention mechanism within the anchor). :

[0052] in, This is the first attention weight matrix. , For querying the matrix, Features of line anchors This represents the number of feature channels.

[0053] Based on the anchor features and their corresponding first attention weight matrix, the anchor attention features are calculated. for: F.

[0054] In one embodiment of the present invention, such as Figure 6 As shown, the inter-lane attention module is used to process the features of each lane anchor using an inter-lane attention mechanism to obtain inter-lane anchor features, which may include: S501, perform feature splicing on the features of each anchor line to obtain spliced ​​features; S502, calculate the attention weight matrix between different line anchors based on the splicing features to obtain the second attention weight matrix; S503, based on the second attention weight matrix and the features of each anchor, obtain the features between anchors.

[0055] Specifically, assuming there is There are 1 predefined anchor, and the anchor feature of each predefined anchor is: ,Will The anchor features of each predefined anchor are concatenated into a feature matrix (concatenated features). .

[0056] Based on splicing characteristics Calculate the attention weight matrix (second attention weight matrix) between different predefined line anchors. :

[0057] in, This is the second attention weight matrix. For splicing features, K is the key matrix and V is the value matrix.

[0058] Different anchor features are evaluated using the second attention weight matrix B. Weighted fusion is performed to obtain the fused features, namely the line-anchor features.

[0059] In one embodiment of the present invention, such as Figure 7 As shown, lane label data may include the actual lane type, actual lane length, and actual lane key point coordinate set; predicted lane data may include the predicted lane type, predicted lane length, and predicted lane key point coordinate set. The model loss term is determined based on the lane label data and predicted lane data, and may include: S601, determine the classification loss term based on the actual lane type and the predicted lane type; S602, determine the regression damage term based on the actual lane length and the predicted lane length; S603, determine the key point loss term based on the actual lane key point coordinate set and the predicted lane key point coordinate set; S604, determine the model loss term based on the classification loss term, regression damage term, and key point loss term.

[0060] The total loss function of the lane detection model in this embodiment of the invention includes a classification loss function, a regression impairment function, and a keypoint loss function. Specifically, the classification loss function is used to calculate a classification loss term based on the actual lane type and the predicted lane type. The regression impairment function is used to calculate a regression impairment term based on the actual lane length and the predicted lane length. The keypoint loss function is used to calculate a keypoint loss term based on the coordinate sets of the actual lane keypoints and the predicted lane keypoints. The model loss term can be obtained from the calculated classification loss term, regression impairment term, and keypoint loss term. The lane detection model parameters are optimized and adjusted using a backpropagation algorithm.

[0061] This invention optimizes the lane detection model by adding an adjacent keypoint loss function (a geometric constraint loss function for the keypoint sequence of the lane line) to the overall loss function of the lane detection model. It should be noted that a lane line can be considered as a series of key points, and there are certain positional and geometric relationships between adjacent key points. By defining the adjacent keypoint loss function, the difference between the predicted and actual positions of adjacent key points is constrained, and the stability of the overall lane position regression process is improved by leveraging the correlation between adjacent key points.

[0062] In practice, the adjacent keypoint loss can be based on Euclidean distance or other suitable distance metrics to calculate the distance error between the predicted adjacent keypoint pairs and the actual adjacent keypoint pairs, and this error (keypoint loss term) can be included in the model loss term of the lane detection model. This guides the lane detection model to pay more attention to the overall structure of the lane line and the relationship between adjacent keypoints during the training process, thereby accelerating the convergence of the lane detection model and improving the lane detection performance under difficult conditions.

[0063] For example, the adjacent keypoint loss definition represents the lane line as a sequence of keypoints. Adjacent keypoint pairs are: Loss at adjacent key points Defined as:

[0064] The training method for the lane detection model in this embodiment of the invention improves the performance of the trained lane detection model in complex road conditions, especially the detection accuracy and stability under difficult conditions such as occlusion and density, by improving feature extraction, feature interaction fusion and model optimization strategies.

[0065] In complex scenarios, traditional feature interaction methods often fail to fully integrate effective information, resulting in insufficient lane line feature representation. This invention employs a dual attention mechanism, both within and between lane anchors, to enhance feature representation at both local and global levels. Compared to traditional single interaction methods, this approach better addresses scenarios with occlusion and high density, improving lane line feature integrity and robustness, and ultimately increasing detection accuracy.

[0066] Traditional model optimization lacks constraints on the overall lane line structure, leading to unstable position regression and slow model convergence in challenging scenarios. This invention employs a model optimization strategy based on adjacent keypoint loss, effectively avoiding the accumulation of prediction errors from isolated keypoints, improving the overall stability of lane position regression, and accelerating model convergence. It significantly enhances lane detection accuracy and result consistency, particularly in occluded and dense scenarios.

[0067] Traditional methods struggle to effectively represent the varied morphologies of lane lines under complex road conditions, resulting in insufficient feature extraction. Anchor-based lane detection structures capture multiple lane line morphologies using prior structures, extracting features more accurately than existing methods. This enhances the representation of lane line structures in complex scenarios and provides a rich information foundation for subsequent detection.

[0068] This invention provides a lane detection method.

[0069] Figure 8 This is a flowchart of a lane detection method according to an embodiment of the present invention. Figure 8 As shown, lane detection methods may include: S701, the image of the lane to be tested is input into the pre-trained lane detection model to obtain the actual lane lines. The trained lane detection model is obtained using the training method of the lane detection model described above.

[0070] The lane detection method in this embodiment of the invention uses the above-mentioned training method to obtain a lane detection model for lane detection, which can effectively handle lane detection problems in complex road conditions and improve the performance of lane detection in complex road conditions, especially the detection accuracy and stability under difficult conditions such as occlusion and density.

[0071] This invention provides a computer-readable storage medium.

[0072] In one embodiment, a computer program is stored on a computer-readable storage medium, which, when executed by a processor, implements the training method for the lane detection model as described above.

[0073] In one embodiment, a computer program is stored on a computer-readable storage medium, and when executed by a processor, the computer program implements the lane detection method described above.

[0074] This invention provides a controller.

[0075] In one embodiment, the controller may include a memory and a processor. The memory stores a computer program, which, when executed by the processor, implements the training method for the lane detection model as described above.

[0076] In one embodiment, the controller may include a memory and a processor, the memory storing a computer program that, when executed by the processor, implements the lane detection method described above.

[0077] Figure 9 This is a structural block diagram of the controller according to an embodiment of the present invention.

[0078] like Figure 9 As shown, the controller 500 includes a processor 501 and a memory 503. The processor 501 and the memory 503 are connected, for example, via a bus 502. Optionally, the controller 500 may also include a transceiver 504. It should be noted that in practical applications, the transceiver 504 is not limited to one, and the structure of the controller 500 does not constitute a limitation on the embodiments of the present invention.

[0079] Processor 501 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this invention. Processor 501 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0080] Bus 502 may include a pathway for transmitting information between the aforementioned components. Bus 502 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 502 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 9 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0081] The memory 503 is used to store a computer program corresponding to the training method of the lane detection model or the lane detection method of the above embodiments of the present invention. The computer program is controlled and executed by the processor 501. The processor 501 is used to execute the computer program stored in the memory 503 to implement the content shown in the foregoing method embodiments.

[0082] The controller 500 includes, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (such as in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 9 The controller 500 shown is merely an example and should not be construed as limiting the functionality and scope of use of embodiments of the present invention.

[0083] The computer storage medium and controller in this embodiment of the invention, based on the lane detection model trained by the above-mentioned training method, can effectively handle lane detection problems in complex road conditions and improve the performance of lane detection in complex road conditions, especially the detection accuracy and stability under difficult conditions such as occlusion and density.

[0084] This invention provides a vehicle.

[0085] Figure 10 This is a schematic diagram of a vehicle according to an embodiment of the present invention. Figure 10 As shown, vehicle 1000 may include controller 500 as described above.

[0086] The vehicle in this embodiment of the invention, based on the controller 500, can effectively handle lane detection problems in complex road conditions, improve the performance of lane detection in complex road conditions, especially the detection accuracy and stability under difficult conditions such as obstruction and density.

[0087] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0088] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0089] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0090] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this invention and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0091] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0092] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0093] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "over," and "on top" of the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0094] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A training method for a lane detection model, characterized in that, The training method includes: Obtain a training dataset, which includes lane training images and their corresponding lane label data; The lane training images in the training dataset are input into the lane detection model for feature extraction to obtain a feature map. Multiple predefined line anchors are projected onto the feature map and the features of the regions where each predefined line anchor is located are extracted to obtain the features of each line anchor. The line anchor internal attention mechanism and the different line anchors inter-attention mechanism are used to perform feature interaction and fusion on the features of each line anchor to obtain the predicted lane data. The model loss term is determined based on the lane label data and the predicted lane data; Based on the model loss term, the model parameters of the lane detection model are adjusted to obtain a trained lane detection model.

2. The training method for the lane detection model according to claim 1, characterized in that, The lane detection model includes an image feature extraction module, a line anchor projection and feature extraction module, an intra-lane attention module, an inter-lane attention module, and a fusion module. The input of the image feature extraction module is used to input the lane training image. The output of the image feature extraction module is connected to the input of the line anchor projection and feature extraction module. The output of the line anchor projection and feature extraction module is connected to the inputs of the intra-lane attention module and the inter-lane attention module, respectively. The outputs of the intra-lane attention module and the inter-lane attention module are connected to the output of the fusion module. The output of the fusion module is used to output the predicted lane data. The process involves inputting the lane training image from the training dataset into the lane detection model for feature extraction to obtain a feature map. Multiple predefined line anchors are projected onto the feature map, and features of the regions where each predefined line anchor is located are extracted to obtain the features of each line anchor. The model employs intra-line anchor attention mechanisms and inter-line anchor attention mechanisms to perform feature interaction and fusion on the features of each line anchor, including: The image feature extraction module is used to perform feature extraction processing on the lane training image to obtain the feature map; Using the anchor projection and feature extraction module, multiple predefined anchors are projected onto the feature map and features of the region where each predefined anchor is located are extracted to obtain the features of each anchor. Using the lane-in-attention module, each line anchor feature is processed by the line anchor internal attention mechanism to obtain each line anchor attention feature; The inter-lane attention module is used to process the inter-lane anchor features using the inter-lane anchor attention mechanism to obtain the inter-lane anchor features; The fusion module is used to fuse the features between the line anchors and the attention features of each line anchor to obtain the predicted lane data.

3. The training method for the lane detection model according to claim 2, characterized in that, The step of using the anchor projection and feature extraction module to project multiple predefined anchors onto the feature map and extract features from the region where each predefined anchor is located includes: Based on the proportional relationship between the image containing each predefined anchor and the feature map, each predefined anchor is projected onto the feature map; Based on the region where each predefined anchor is located, feature extraction is performed on the projected feature map to obtain the features of each anchor.

4. The training method for the lane detection model according to claim 2, characterized in that, The process of using the lane-in-lane attention module to process each line anchor feature using the line anchor internal attention mechanism to obtain each line anchor attention feature includes: Calculate the attention weight matrix within the anchor based on each anchor feature to obtain the first attention weight matrix corresponding to each anchor feature. Each line anchor attention feature is obtained based on its respective line anchor feature and its corresponding first attention weight matrix.

5. The training method for the lane detection model according to claim 2, characterized in that, The method utilizes the inter-lane attention module to process each of the line anchor features using an inter-lane attention mechanism to obtain inter-lane anchor features, including: The features of each anchor line are spliced ​​together to obtain the spliced ​​features; Based on the splicing features, the attention weight matrix between different line anchors is calculated to obtain the second attention weight matrix; The features between line anchors are obtained based on the second attention weight matrix and each of the line anchor features.

6. The training method for the lane detection model according to claim 1, characterized in that, The lane label data includes the actual lane type, actual lane length, and actual lane key point coordinate set; the predicted lane data includes the predicted lane type, predicted lane length, and predicted lane key point coordinate set; and the step of determining the model loss term based on the lane label data and the predicted lane data includes: The classification loss term is determined based on the actual lane type and the predicted lane type; The regression damage term is determined based on the actual lane length and the predicted lane length; The key point loss term is determined based on the actual lane key point coordinate set and the predicted lane key point coordinate set. The model loss term is determined based on the classification loss term, the regression damage term, and the keypoint loss term.

7. A lane detection method, characterized in that, The detection method includes: The lane image to be tested is input into a pre-trained lane detection model to obtain the actual lane lines. The pre-trained lane detection model is obtained using the training method for lane detection models as described in any one of claims 1-6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the training method for the lane detection model as described in any one of claims 1-6, or the lane detection method as described in claim 7.

9. A controller, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the computer program is executed by the processor, it implements the training method for the lane detection model as described in any one of claims 1-6, or the lane detection method as described in claim 7.

10. A vehicle, characterized in that, Includes the controller as described in claim 9.

Images