Intelligent driving assistance control system and method based on deep learning

Abstract

The invention provides an intelligent driving assistance control system and method based on deep learning, and the method comprises the steps: collecting a road image in front of a vehicle, and determining a contour feature map of a lane line of the road image under different convolution scales; extracting lane line attention of different attention areas in each contour feature map, and further determining lane line vanishing points in the road image according to all the lane line attention and the lane line cluster in the road image; performing feature anchoring on the lane line contour in the image of the road in front of the vehicle through preset different lane line-shaped anchors to obtain anchoring losses of the lane line contour under the different lane line-shaped anchors, and determining boundary guiding features of the lane line contour under the constraint of the lane line vanishing points according to all the anchoring losses; and performing guide connection on the lane line feature points of the road in front of the vehicle based on the boundary guide features to generate a lane line in front of the vehicle. According to the scheme of the invention, the method can achieve the continuous reconstruction of the lane line under the condition of no obvious visual clues.

Description

Technical Field

[0001] This application relates to the field of deep learning technology, and more specifically, to a deep learning-based intelligent driving assistance control system and method. Background Technology

[0002] Deep learning is a machine learning method based on artificial neural networks. By constructing multi-layered nonlinear processing units, it can extract features and learn representations from data. Compared with traditional machine learning, deep learning can automatically mine high-level abstract features from large-scale data, reducing the dependence on manual feature design. It is widely used in computer vision, natural language processing, speech recognition and other fields. In lane detection, deep learning effectively improves the perception of lane features in complex environments through end-to-end training, and has become one of the core technologies in current intelligent driving systems.

[0003] With the rapid development of intelligent driving assistance systems and autonomous driving technology, lane line generation, as a fundamental module of vehicle environmental perception, has become one of the key prerequisites for realizing automatic vehicle control, path planning, and behavioral decision-making. Traditional lane line generation methods mostly rely on obvious visual cues in images, such as the continuous texture, color contrast, and edge features of lane lines. However, in real road environments, lane lines are subject to various anomalies such as wear, occlusion, breakage, lighting interference, and rain / snow coverage, causing lane lines to appear discontinuous, blurred, or disappearing in images. Faced with these challenges, current deep learning-driven lane line generation methods, while performing excellently in standard environments, still commonly suffer from problems such as lane line recognition interruption, positioning offset, and continuity breakage in scenarios with severe visual information loss. In such cases, traditional detection methods that rely on local visual feature inference struggle to accurately recover the complete topological structure of lane lines, severely affecting the perception stability and control decision safety of the system. Therefore, how to continuously reconstruct lane lines under conditions without obvious visual cues has become a technical challenge facing the industry. Summary of the Invention

[0004] This application provides a deep learning-based intelligent driving assistance control system and method that can reconstruct lane lines continuously without obvious visual cues.

[0005] In a first aspect, this application provides a lane line generation method for an intelligent driving assistance control system to generate lane lines for the road ahead of the vehicle. The method includes the following steps: Real-time acquisition of road images in front of the target vehicle, and then determination of lane line contour feature maps at different convolution scales in the road images; Extract lane line attention from different attention regions within each contour feature map, and then determine the lane line vanishing point in the road image based on all lane line attention and lane line clusters in the road image. By using different preset lane line anchors, the lane line contour in the road image in front of the target vehicle is feature anchored, and the anchoring loss of the lane line contour under different lane line anchors is obtained. Based on all the anchoring losses, the boundary guidance features of the lane line contour under the lane line vanishing point constraint are determined. Based on the boundary guidance features, the lane line feature points of the road ahead of the target vehicle are guided and connected to generate the lane line ahead of the target vehicle.

[0006] In some embodiments, determining the lane line contour feature maps of the road image at different convolutional scales specifically includes: The road image is preprocessed to obtain a preprocessed road image; By extracting features from the preprocessed road image using different convolutional branch networks, the contour feature maps of lane lines in the road image at different convolutional scales are obtained. Each convolutional branch network uses a convolutional kernel with a different receptive field size.

[0007] In some embodiments, extracting lane line attention for different attention regions within each contour feature map specifically includes: Based on the self-attention mechanism, an attention weight matrix is ​​generated for different attention regions within each contour feature map; Select one contour feature map as the selected contour feature map; By using the attention weight matrix of all attention regions within the selected contour feature map, the lane line features of different attention regions within the selected contour feature map are reconstructed using weighted reconstruction, thus obtaining the lane line attention of each attention region within the selected contour feature map. Continue to determine the lane line attention for each attention region within the remaining contour feature map.

[0008] In some embodiments, determining the lane vanishing point in the road image based on all lane line attention and lane line clusters within the road image specifically includes: Identify lane clusters within the road image; Based on all lane line attention, a directional consistency analysis is performed on the lane line contours within the road image to obtain the directional residuals of the lane line contours at different contour feature points. The vanishing points of lane lines in the road image are determined by the directional residuals of the lane line clusters and lane line contours at different contour feature points.

[0009] In some embodiments, the lane line contours in the road image ahead of the target vehicle are feature-anchored using preset different lane line anchors, and the anchoring loss of the lane line contours under different lane line anchors specifically includes: By using each preset lane line anchor, the shape anchoring of the lane line contour in the road image in front of the target vehicle is performed, and the shape anchoring error of the lane line contour under different lane line anchors is obtained. By anchoring the lane line contour in the road image in front of the target vehicle using each preset lane line anchor, the position anchoring error of the lane line contour under different lane line anchors is obtained. The anchorage loss of lane profiles under different lane shape anchorages is determined based on the shape anchorage error and position anchorage error of all lane profiles.

[0010] In some embodiments, determining the boundary guidance features of the lane line profile under lane line vanishing point constraints based on all anchoring losses specifically includes: Determine the optimal lane alignment anchorage of the road ahead of the target vehicle based on all anchoring losses; Based on the optimal lane line anchor, the lane line profile is subjected to boundary constraint analysis through the lane line vanishing point to obtain the boundary guidance features of the lane line profile under the constraint of the lane line vanishing point.

[0011] In some embodiments, generating lane lines in front of the target vehicle by guiding the connection of lane line feature points on the road ahead based on the boundary guidance features specifically includes: The boundary guidance features are used to smoothly connect the lane line feature points of the road ahead of the target vehicle to obtain the regressed lane lines in the road ahead image of the target vehicle. The lane lines in front of the target vehicle are generated based on the returned lane lines.

[0012] Secondly, this application provides a deep learning-based intelligent driving assistance control system, which includes a lane line generation unit, the lane line generation unit comprising: The acquisition module is used to acquire road images in front of the target vehicle in real time, and then determine the contour feature maps of lane lines in the road images at different convolution scales. The processing module is used to extract lane line attention in different attention regions within each contour feature map, and then determine the lane line vanishing point in the road image based on all lane line attention and lane line clusters in the road image. The processing module is also used to perform feature anchoring on the lane line contour in the road image in front of the target vehicle through preset different lane line anchors, to obtain the anchoring loss of the lane line contour under different lane line anchors, and to determine the boundary guidance feature of the lane line contour under the lane line vanishing point constraint based on all the anchoring losses. The execution module is used to guide and connect the lane line feature points of the road in front of the target vehicle based on the boundary guidance features, and generate the lane line in front of the target vehicle.

[0013] Thirdly, this application provides a computer device including a memory and a processor, the memory storing code, and the processor being configured to acquire the code and execute the lane line generation method described above.

[0014] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the lane line generation method described above.

[0015] The technical solutions provided by the embodiments disclosed in this application have the following beneficial effects: The intelligent driving assistance control system and method based on deep learning provided in this application involves: acquiring road images in front of the target vehicle in real time, and then determining the contour feature maps of lane lines in the road images at different convolutional scales; extracting lane line attention in different attention regions within each contour feature map, and then determining the lane line vanishing point in the road image based on all lane line attention and lane line clusters in the road image; anchoring the lane line contours in the road images in front of the target vehicle using preset lane line anchors to obtain the anchoring loss of the lane line contours under different lane line anchors, and determining the boundary guidance features of the lane line contours under the constraint of lane line vanishing points based on all anchoring losses; and guiding the connection of lane line feature points in front of the target vehicle based on the boundary guidance features to generate the lane lines in front of the target vehicle.

[0016] Therefore, this application can guide the connection of lane line feature points on the road ahead of the target vehicle based on the boundary guidance features to generate lane lines ahead of the target vehicle. First, the contour feature maps of lane lines in the road image at different convolutional scales are determined. These contour feature maps can represent different levels of spatial features in the road image, enabling the identification of potential lane line structures through high-level semantic information even when lane lines exhibit wear, occlusion, breaks, or blurring. Second, an attention mechanism can focus on the most valuable regions for lane line judgment in the road image, strengthening the lane line structure in the feature space and improving the ability to infer and complete missing lane lines. Next, lane line vanishing points in the road image are obtained through all lane line attention and lane line clusters within the road image. These vanishing points can serve as core constraint points for the lane line geometry, helping the intelligent driving assistance control system infer the trend of lane lines and recover broken or occluded parts in the road image, improving the adaptability of the intelligent driving assistance control system to complex scenarios such as changes in lighting, occlusion, and road anomalies. Furthermore, the... By determining the anchoring loss of lane line contours under different lane alignment anchors, the intelligent driving assistance control system can evaluate the adaptability of each lane alignment anchor. The calculation of anchoring loss helps the intelligent driving assistance control system infer the shape and position of the lane line through global information, enabling the restoration of the continuity of the lane line in broken or occluded areas and ensuring the complete reconstruction of the lane line. Then, the boundary guidance features of the lane line contour under the constraint of the lane line vanishing point are determined. By combining the boundary guidance features with the lane line vanishing point constraint, it is ensured that the lane line always conforms to the actual geometric structure of the lane line during the inference process in discontinuous or occluded areas, avoiding the lane line breakage problem caused by insufficient local information in traditional methods. Finally, by guiding the connection of lane line feature points, the intelligent driving assistance control system can infer the reasonable position of the lane line in the missing information area, thereby restoring the complete lane line topology and avoiding over-reliance on local information, so that the continuity of the lane line can still be maintained in complex or extreme scenarios. In summary, the solution of this application can realize the continuous reconstruction of lane lines under the condition of no obvious visual cues. Attached Figure Description

[0017] Figure 1 This is an exemplary flowchart of a lane line generation method according to some embodiments of this application; Figure 2 This is a schematic flowchart illustrating the process of determining the vanishing point of a lane line according to some embodiments of this application; Figure 3 This is a schematic flowchart illustrating the process of determining anchoring loss according to some embodiments of this application; Figure 4 This is a schematic diagram of the structure of a lane line generation unit according to some embodiments of this application; Figure 5This is a schematic diagram of the structure of a computer device implementing a lane line generation method according to some embodiments of this application. Detailed Implementation

[0018] To better understand the technical solution of this application, the technical solution of this application will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0019] refer to Figure 1 The figure is an exemplary flowchart of a lane line generation method according to some embodiments of this application. The lane line generation method 100 mainly includes the following steps: In step 101, road images in front of the target vehicle are acquired in real time, and then the contour feature maps of lane lines in the road images at different convolution scales are determined.

[0020] In practice, the vehicle-mounted camera captures real-time images of the road in front of the target vehicle.

[0021] In some embodiments, determining the contour feature maps of lane lines in the road image at different convolutional scales can be achieved using the following steps: The road image is preprocessed to obtain a preprocessed road image; By extracting features from the preprocessed road image using different convolutional branch networks, the contour feature maps of lane lines in the road image at different convolutional scales are obtained. Each convolutional branch network uses a convolutional kernel with a different receptive field size.

[0022] In specific implementation, the road image is preprocessed to obtain the preprocessed road image. The preprocessing can be carried out in the following way: the road image is preprocessed, including image size adjustment, noise reduction (such as using Gaussian filtering), color space conversion (such as converting RGB to grayscale format), etc., so as to improve the standardization and feature extraction effect when the road image is input into the neural network.

[0023] It should be noted that the convolutional branch network described in this application refers to the existence of multiple sub-networks in the backbone neural network structure. Each sub-network uses convolutional kernels of different sizes for feature extraction to achieve multi-scale information fusion. Each convolutional branch network corresponds to a convolutional scale.

[0024] In specific implementation, the preprocessed road image is used to extract features through different convolutional branch networks to obtain the contour feature maps of lane lines at different convolutional scales. This can be achieved in the following way: First, the preprocessed road image is input into the backbone neural network (e.g., Xception (ExtremeInception) in DeepLabV3). Feature maps are extracted through continuously stacked convolutional layers and stride convolutions, and the obtained feature maps are used as intermediate feature maps. Then, a dilated spatial pyramid pooling module (e.g., dilated spatial pyramid pooling technology in DeepLabV3) is constructed on the intermediate feature maps. Then, multiple convolutional branch networks are set up, and each convolutional branch network uses dilated convolutional kernels with different dilation rates to perform convolution operations. Specifically, 3×3 dilated convolutions with dilation rates of 6, 12, and 18, a standard 1×1 convolution without dilation, and a global average pooling branch are set up. The feature maps obtained after the convolution operations of different convolutional branch networks are used as the contour feature maps of the road image at the corresponding convolutional scales.

[0025] It should be noted that different dilation rates in this application correspond to different receptive field sizes, which can capture multi-scale features from fine-grained to coarse-grained in road images without increasing the computational load. In addition, it should be noted that the contour feature map in this application represents the spatial distribution feature map characterizing the contour morphology of lane lines.

[0026] In step 102, lane line attention is extracted from different attention regions within each contour feature map, and then the lane line vanishing point in the road image is determined based on all lane line attention and the lane line clusters in the road image.

[0027] In some embodiments, extracting lane line attention for different attention regions within each contour feature map can be achieved using the following steps: Based on the self-attention mechanism, an attention weight matrix is ​​generated for different attention regions within each contour feature map; Select one contour feature map as the selected contour feature map; By using the attention weight matrix of all attention regions within the selected contour feature map, the lane line features of different attention regions within the selected contour feature map are reconstructed using weighted reconstruction, thus obtaining the lane line attention of each attention region within the selected contour feature map. Continue to determine the lane line attention for each attention region within the remaining contour feature map.

[0028] In specific implementation, based on the self-attention mechanism, the attention weight matrix for different attention regions within each contour feature map can be generated in the following way: For each contour feature map, the entire contour feature map is divided into grid regions of fixed size (e.g., 4×4 or 8×8, not limited here) at equal intervals along the horizontal and vertical directions, and each of the divided grid regions is used as an attention region. Each attention region consists of multiple consecutive feature units (e.g., feature units can be 8×8 or 16×16 pixel blocks). Then, for each attention region, based on the self-attention mechanism (e.g., local self-attention in Swing Transformer), the query vector, key vector, and value vector of each feature unit are extracted from the attention region. Further, the dot product between each query vector and all key vectors is calculated, and all the obtained dot products are scaled (e.g., the dot product is divided by the square root of the key vector dimension). Then, the scaled values ​​are normalized (e.g., Softmax normalization). (maximum normalization), then construct a matrix from all the normalized values, and use the resulting matrix as the attention weight matrix of the attention region, thus obtaining the attention weight matrix of different attention regions within each contour feature map.

[0029] It should be noted that the attention weight matrix described in this application represents a numerical matrix showing the interdependencies of each feature unit within the attention region. Each element in the attention weight matrix represents the degree of attention one feature unit pays to another feature unit.

[0030] In specific implementation, the lane line attention of different attention regions within the selected contour feature map is reconstructed by weighting the lane line features of all attention regions within the selected contour feature map using the attention weight matrix of all attention regions. This can be achieved in the following way: First, the vector set composed of each value vector of each attention region within the selected contour feature map is used as the lane line feature, thus obtaining the lane line features of different attention regions within the selected contour feature map. Then, for each attention region within the selected contour feature map, the lane line features of the attention region are weighted and summed with the attention weight matrix of the attention region. That is, each row of attention weight in the attention weight matrix is ​​used as a set of weighting coefficients. Then, all value vectors in the lane line features of the attention region are linearly combined. Furthermore, all the values ​​obtained from the weighted sum are arranged in descending order to form a vector, and the resulting vector is used as the lane line attention of the attention region, thus obtaining the lane line attention of each attention region within the selected contour feature map.

[0031] It should be noted that the lane line attention representation described in this application is a weighted vector characterizing the intensity of lane line-related feature responses.

[0032] In some embodiments, reference Figure 2 As shown in the figure, this is a schematic flowchart of determining lane vanishing points in some embodiments of this application. In this embodiment, determining the lane vanishing points in the road image based on all lane line attention and lane line clusters in the road image can be achieved by the following steps: First, in step 1021, lane line clusters within the road image are determined; Secondly, in step 1022, the directional consistency analysis of the lane line contours in the road image is performed based on all lane line attention to obtain the directional residuals of the lane line contours at different contour feature points. Then, in step 1023, the vanishing points of the lane lines in the road image are determined by the directional residuals of the lane line clusters and lane line contours at different contour feature points.

[0033] When implemented, determining the lane clusters within the road image can be achieved in the following manner: First, edge detection (e.g., the Canny edge detection algorithm) is performed on the road image to extract the edge image. Then, Hough transform is used to perform line detection on the edge image to obtain edge pixels that conform to the straight line portion. Next, the B-spline algorithm is used to perform curve detection on the curve portion of the edge image to obtain edge pixels that conform to the curve portion. Finally, an existing clustering algorithm (e.g., hierarchical clustering algorithm) is used to cluster all the obtained edge pixels to obtain multiple edge pixel clusters. All obtained edge pixel clusters are used as lane clusters within the road image.

[0034] It should be noted that the lane line cluster mentioned in this application refers to a data cluster composed of edge pixels that reflect the structural features of lane lines within the road image in front of the target vehicle.

[0035] In specific implementation, the directional consistency analysis of the lane line contours within the road image is performed based on all lane line attention values. The directional residuals of the lane line contours at different contour feature points can be obtained in the following way: First, acquire the edge image of the road image. Multiply each lane line attention value by the pixel value of all edge pixels in the edge image, and use the multiplied values ​​as lane line attention values. Each lane line attention value corresponds to an edge pixel. Then, calculate the average of all multiplied values ​​and use the average as a contour feature point threshold. Select lane line attention values ​​greater than or equal to the contour feature point threshold from all lane line attention values. The edge pixels corresponding to the lane line attention values ​​of the feature point threshold are used as contour feature points. Then, a fixed-size local contour feature point window (e.g., a 5×5 or 7×7 pixel area) is constructed with each contour feature point as the center. The gradient direction of each pixel is calculated within the local contour feature point window (e.g., by using the Sobel operator to perform differential processing on all gradient directions, then summing the values ​​obtained after differential processing, and using the summed value as the direction residual of the lane line contour at the contour feature point, thus obtaining the direction residual of the lane line contour at different contour feature points).

[0036] It should be noted that the lane line contour described in this application refers to the distribution area composed of all contour feature points. In addition, it should be noted that the contour feature points described in this application refer to the pixels that describe the lane line contour, and the directional residual represents the degree of deviation of the gradient direction of the contour feature points in the local area from the main direction of the lane line contour.

[0037] In specific implementation, determining the vanishing points of lane lines in the road image by using the directional residuals of lane line clusters and lane line contours at different contour feature points can be achieved in the following way: Select one lane line cluster from the lane line clusters in the road image as the selected lane line cluster. Use an existing linear fitting algorithm (such as the least squares support vector machine algorithm) to perform linear fitting on all edge pixels in the selected lane line cluster, and use the fitted curve as the directional fitting curve of the selected lane line cluster. At the same time, use the sum of the directional residuals at each contour feature point belonging to the selected lane line cluster as the weight of the directional fitting curve of the selected lane line cluster. Continue to determine the directional fitting curves of the remaining lane line clusters and the weights of the directional fitting curves. Then, calculate the derivative of the directional fitting curve of each lane line cluster, and use the edge pixels at the position of minimum derivative as the vanishing points of lane lines in the road image.

[0038] It should be noted that, in this application, the lane vanishing point refers to the edge pixels in the road image where the road extension direction tends to converge under the action of perspective projection, and the lane vanishing point reflects the vanishing position at the far end of the road.

[0039] In step 103, the lane line contour in the road image in front of the target vehicle is feature-anchored by using preset lane line anchors of different lane line shapes to obtain the anchoring loss of the lane line contour under different lane line anchors. Based on all the anchoring losses, the boundary guidance features of the lane line contour under the lane line vanishing point constraint are determined.

[0040] In some embodiments, reference Figure 3 As shown in the figure, this is a flowchart illustrating the process of determining anchoring loss in some embodiments of this application. In this embodiment, the lane line contours in the road image ahead of the target vehicle are anchored using preset lane line anchors of different lane line shapes. The anchoring loss of the lane line contours under different lane line anchors can be achieved by the following steps: By using each preset lane line anchor, the shape anchoring of the lane line contour in the road image in front of the target vehicle is performed, and the shape anchoring error of the lane line contour under different lane line anchors is obtained. By anchoring the lane line contour in the road image in front of the target vehicle using each preset lane line anchor, the position anchoring error of the lane line contour under different lane line anchors is obtained. The anchorage loss of lane profiles under different lane shape anchorages is determined based on the shape anchorage error and position anchorage error of all lane profiles.

[0041] In some embodiments, the lane line contours in the road image ahead of the target vehicle are feature-anchored using preset lane line anchors. The anchoring loss of the lane line contours under different lane line anchors can be achieved by the following steps: By using each preset lane line anchor, the shape anchoring of the lane line contour in the road image in front of the target vehicle is performed, and the shape anchoring error of the lane line contour under different lane line anchors is obtained. By anchoring the lane line contour in the road image in front of the target vehicle using each preset lane line anchor, the position anchoring error of the lane line contour under different lane line anchors is obtained. The anchorage loss of lane profiles under different lane shape anchorages is determined based on the shape anchorage error and position anchorage error of all lane profiles.

[0042] It should be noted that the lane line anchor described in this application represents a preset reference template used to describe the geometric structural features of typical lane lines. The lane line anchor plays a role in geometric shape alignment, position matching, and error measurement during image analysis and target matching. Specifically, the lane line anchor forms a standard curve template by structurally modeling common lane line shapes in reality (such as straight lines, gently curved lines, moderately curved lines, and sharply curved lines), which includes a standard curve and standard control points on the standard curve. This template is used to perform structural fitting and shape matching on the lane line contours extracted from the actual image.

[0043] In specific implementation, the shape anchoring of the lane line contour in the road image ahead of the target vehicle is performed by pre-set lane line anchors. The shape anchoring error of the lane line contour under different lane line anchors can be achieved in the following way: First, obtain each contour feature point of the lane line contour in the road image ahead of the target vehicle, and use the B-spline fitting method to construct the fitting curve of each contour feature point. The fitting curve is composed of multiple fitting points. For each pre-set lane line anchor, align and match the fitting curve with the standard curve of the lane line anchor. Then, calculate the mean square error between each fitting point on the fitting curve and the corresponding standard control point on the standard curve of the lane line anchor, and use the obtained mean square error as the shape anchoring error of the lane line contour under the lane line anchor.

[0044] It should be noted that the shape anchoring error described in this application refers to the degree of difference between the geometry of the lane line profile and the preset lane line anchor.

[0045] In practice, the lane line contour in the road image in front of the target vehicle is anchored by each preset lane line anchor. The position anchoring error of the lane line contour under different lane line anchors can be achieved in the following way: For each lane line anchor, the fitting curve of each contour feature point is obtained. After aligning and matching the fitting curve with the standard curve of the lane line anchor, the Euclidean distance between each contour feature point and the nearest standard control point on the standard curve is calculated. Then, the sum of all Euclidean distances is taken as the position anchoring error of the lane line contour under the lane line anchor.

[0046] It should be noted that the shape anchoring error mentioned in this application refers to the degree of deviation in the positional distribution between the contour feature points of the lane line profile and the preset standard control points of the lane line shape anchor.

[0047] In practice, the anchoring loss of lane profiles under different lane profile anchors can be determined by the following method based on the shape anchoring error and position anchoring error of all lane profiles: for each lane profile anchor, the shape anchoring error of the lane profile under the lane profile anchor is added to the position anchoring error of the lane profile under the lane profile anchor, and the sum is taken as the anchoring loss of the lane profile under the lane profile anchor.

[0048] It should be noted that the anchoring loss described in this application refers to the overall deviation index generated when the lane line profile and the lane line anchor are anchored under conditions without obvious visual cues.

[0049] In some embodiments, determining the boundary guidance features of the lane line profile under lane line vanishing point constraints based on all anchoring losses can be achieved using the following steps: Determine the optimal lane alignment anchorage of the road ahead of the target vehicle based on all anchoring losses; Based on the optimal lane line anchor, the lane line profile is subjected to boundary constraint analysis through the lane line vanishing point to obtain the boundary guidance features of the lane line profile under the constraint of the lane line vanishing point.

[0050] In practice, determining the optimal lane alignment anchor for the road ahead of the target vehicle based on all anchoring losses can be achieved in the following way: select the minimum anchoring loss from all anchoring losses and take the lane alignment anchor corresponding to the minimum anchoring loss as the optimal lane alignment anchor.

[0051] In specific implementation, based on the optimal lane alignment anchor, boundary constraint analysis of the lane profile is performed through the lane vanishing point. The boundary guidance features of the lane profile under the constraint of the lane vanishing point can be obtained in the following way: First, after aligning and matching the fitted curves of the optimal lane alignment anchor and each profile feature point, the profile feature points closest to the lane alignment anchor are selected sequentially from the head of the lane alignment anchor. All selected profile feature points are used as lane line feature points within the longitudinal distance interval of the road ahead of the target vehicle. Then, for each lane line feature point, a geometric convergence center is constructed from the lane vanishing point. The direction vector from the lane line feature point to the lane line vanishing point is used as the constraint vector. Then, the direction vector between the lane line feature points and the edge pixels on the outermost edge of the lane line contour is constructed, and the obtained direction vector is used as the boundary vector. Further, the cosine value of the angle between the constraint vector and the boundary vector is calculated. Then, the obtained cosine value of the angle is divided by the Euclidean distance between the lane line feature point and the lane line vanishing point, and the value obtained by division is used as the boundary guidance value of the lane line feature point. Finally, the sum of the boundary guidance values ​​of all lane line feature points is used as the boundary guidance feature of the lane line contour under the constraint of the lane line vanishing point.

[0052] It should be noted that the boundary guidance features described in this application refer to the geometric features that guide the direction of the lane line edge when assisting in the reconstruction of the lane line contour under obvious visual cues. The boundary guidance features reflect the ability of the lane line edge direction to guide the extension of the lane line shape.

[0053] Additionally, it should be noted that the lane line feature points described in this application represent key edge pixels that reflect the lane line geometry of the road ahead of the target vehicle.

[0054] In step 104, the lane line feature points of the road ahead of the target vehicle are guided and connected based on the boundary guidance features to generate the lane line ahead of the target vehicle.

[0055] In some embodiments, generating lane lines in front of the target vehicle by guiding the connection of lane line feature points on the road ahead of the target vehicle based on the boundary guidance features can be achieved through the following steps: The boundary guidance features are used to smoothly connect the lane line feature points of the road ahead of the target vehicle to obtain the regressed lane lines in the road ahead image of the target vehicle. The lane lines in front of the target vehicle are generated based on the returned lane lines.

[0056] In specific implementation, the process of smoothly connecting lane line feature points of the road ahead of the target vehicle using the boundary guidance feature to obtain the regressed lane line in the image of the road ahead of the target vehicle can be achieved in the following way: obtain the boundary guidance value of each lane line feature point, calculate the mean of all boundary guidance values, and use the obtained mean as the boundary guidance threshold. Select the lane line feature points corresponding to the boundary guidance values ​​that are greater than or equal to the boundary guidance threshold, and use all the selected lane line feature points as fitting candidate points. Then, divide all the boundary guidance values ​​of the fitting candidate points by the boundary guidance feature, and normalize all the values ​​obtained by division. Use the normalized values ​​as the fitting weights of the fitting candidate points in the B-spline fitting algorithm. Then, use the B-spline fitting algorithm to perform curve fitting on all the fitting candidate points, and use the fitted curve as the regressed lane line in the image of the road ahead of the target vehicle.

[0057] It should be noted that the return lane line mentioned in this application refers to the lane line trajectory within the road image in front of the target vehicle.

[0058] In specific implementation, the lane line in front of the target vehicle can be generated based on the returned lane line in the following way: the returned lane line is transformed by inverse perspective transformation using the intrinsic parameters (including focal length, principal point, etc.) and extrinsic parameters (including the rotation and translation matrix of the camera relative to the vehicle coordinate system) of the vehicle camera, the returned lane line is transformed from the image coordinate system to the road coordinate system of the real world, a continuous lane line point sequence is generated, and the continuous lane line point sequence is used as the lane line in front of the target vehicle.

[0059] Furthermore, in another aspect of this application, in some embodiments, this application provides a deep learning-based intelligent driving assistance control system, which includes a lane line generation unit, with reference to... Figure 4 The figure is a schematic diagram of the structure of a lane line generation unit according to some embodiments of this application. The lane line generation unit 400 includes: a data acquisition module 401, a processing module 402, and an execution module 403, which are described below: The acquisition module 401 in this application is mainly used to acquire road images in front of the target vehicle in real time, and then determine the contour feature maps of lane lines in the road images at different convolution scales. Processing module 402, in this application, is used to extract lane line attention in different attention regions within each contour feature map, and then determine the lane line vanishing point in the road image based on all lane line attention and lane line clusters in the road image. It should be noted that the processing module 402 described in this application is also used to perform feature anchoring on the lane line contour in the road image in front of the target vehicle through preset different lane line anchors, to obtain the anchoring loss of the lane line contour under different lane line anchors, and to determine the boundary guidance feature of the lane line contour under the lane line vanishing point constraint based on all the anchoring losses. The execution module 403 in this application is mainly used to guide and connect the lane line feature points of the road in front of the target vehicle based on the boundary guidance features, so as to generate the lane line in front of the target vehicle.

[0060] In addition, this application also provides a computer device, the computer device including a memory and a processor, the memory storing code, and the processor being configured to acquire the code and execute the lane line generation method described above.

[0061] In some embodiments, reference Figure 5 The figure is a schematic diagram of the structure of a computer device implementing a lane line generation method according to some embodiments of this application. The lane line generation method in the above embodiments can be achieved through... Figure 5The computer device shown is used to implement this, and the computer device 500 includes at least one processor 501, a communication bus 502, a memory 503, and at least one communication interface 504.

[0062] The processor 501 may be a general-purpose central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more devices used to control the execution of the lane line generation method in this application.

[0063] The communication bus 502 can be used to transmit information between the aforementioned components.

[0064] Memory 503 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CDROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital versatile optical discs, Blu-ray discs, etc.), magnetic disks or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. Memory 503 may exist independently and be connected to processor 501 via communication bus 502. Memory 503 may also be integrated with processor 501.

[0065] The memory 503 stores program code for executing the scheme of this application, and its execution is controlled by the processor 501. The processor 501 executes the program code stored in the memory 503. The program code may include one or more software modules. The method described in the above method embodiments can be implemented by the processor 501 and one or more software modules in the program code in the memory 503.

[0066] Communication interface 504 uses any transceiver-like device to communicate with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc.

[0067] In a specific implementation, as one example, a computer device may include multiple processors, each of which may be a single-core (single CPU) processor or a multi-core (multi CPU) processor. Here, a processor may refer to one or more devices, circuits, and / or processing cores used to process data (e.g., computer program instructions).

[0068] The aforementioned computer device can be a general-purpose computer device or a special-purpose computer device. In specific implementations, the computer device can be a desktop computer, a portable computer, a network server, a handheld digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, or an embedded device. This application does not limit the type of computer device.

[0069] In addition, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the lane line generation method described above.

[0070] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0071] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A lane line generation method for an intelligent driving assistance control system to generate a lane line of a road ahead of a vehicle, characterized by, The method comprises the following steps: Real-time acquisition of a road image in front of a target vehicle, and then determination of lane line profile feature maps of the road image under different convolution scales; Extraction of lane line attentions of different attention regions in each profile feature map, and then determination of lane line vanishing points in the road image according to all lane line attentions and lane line clusters in the road image; Feature anchoring of lane line profiles in the road image in front of the target vehicle through preset different lane line shape anchors, to obtain anchoring losses of the lane line profiles under different lane line shape anchors, and determination of boundary guidance features of the lane line profiles under the constraint of the lane line vanishing points according to all the anchoring losses; Guided connection of lane line feature points of the road in front of the target vehicle based on the boundary guidance features, to generate lane lines in front of the target vehicle.

2. The method of claim 1, wherein, The determination of the lane line profile feature maps of the road image under different convolution scales specifically comprises: Preprocessing of the road image to obtain a preprocessed road image; Feature extraction of the preprocessed road image through different convolution branch networks to obtain lane line profile feature maps of the road image under different convolution scales, wherein each convolution branch network adopts a convolution kernel with a different receptive field size.

3. The method of claim 1, wherein, The extraction of the lane line attentions of different attention regions in each profile feature map specifically comprises: Generation of attention weight matrices of different attention regions in each profile feature map based on a self-attention mechanism; Selection of a profile feature map as a selected profile feature map; Weighted reconstruction of lane line features of different attention regions in the selected profile feature map through the attention weight matrices of all attention regions in the selected profile feature map, to obtain lane line attentions of each attention region in the selected profile feature map; Continued determination of lane line attentions of each attention region in the remaining profile feature maps.

4. The method of claim 1, wherein, The determination of the lane line vanishing points in the road image according to all the lane line attentions and the lane line clusters in the road image specifically comprises: Determination of lane line clusters in the road image; Direction consistency analysis of lane line profiles in the road image according to all the lane line attentions, to obtain direction residuals of the lane line profiles at different profile feature points; Determination of the lane line vanishing points in the road image through the lane line clusters in the road image and the direction residuals of the lane line profiles at different profile feature points.

5. The method of claim 1, wherein, The feature anchoring of the lane line profiles in the road image in front of the target vehicle through preset different lane line shape anchors to obtain anchoring losses of the lane line profiles under different lane line shape anchors specifically comprises: Shape anchoring of the lane line profiles in the road image in front of the target vehicle through each preset lane line shape anchor, to obtain shape anchoring errors of the lane line profiles under different lane line shape anchors; Position anchoring of the lane line profiles in the road image in front of the target vehicle through each preset lane line shape anchor, to obtain position anchoring errors of the lane line profiles under different lane line shape anchors; Determination of the anchoring losses of the lane line profiles under different lane line shape anchors according to the shape anchoring errors and the position anchoring errors of all the lane line profiles.

6. The method of claim 1, wherein, The boundary guiding feature of the lane line contour under the constraint of the lane line vanishing point is determined according to all the anchoring losses, and specifically includes: An optimal lane line shape anchor of a road ahead of the target vehicle is determined according to all the anchoring losses; Based on the optimal lane line shape anchor, a boundary constraint analysis is performed on the lane line contour by the lane line vanishing point, to obtain a boundary guiding feature of the lane line contour under the constraint of the lane line vanishing point.

7. The method of claim 1, wherein, The lane line feature points of the road ahead of the target vehicle are guided and connected based on the boundary guiding feature, to generate a lane line ahead of the target vehicle, and specifically includes: The lane line feature points of the road ahead of the target vehicle are smoothly connected through the boundary guiding feature, to obtain a regression lane line in the road image ahead of the target vehicle; The lane line ahead of the target vehicle is generated according to the regression lane line.

8. A deep learning-based intelligent driving assistance control system including a lane line generation unit, characterized by, The lane line generation unit includes: A collection module is configured to collect a road image ahead of a target vehicle in real time, and further determine a contour feature map of a lane line of the road image under different convolution scales; A processing module is configured to extract lane line attentions of different attention regions in each contour feature map, and further determine a lane line vanishing point in the road image according to all the lane line attentions and a lane line cluster in the road image; The processing module is further configured to perform feature anchoring on a lane line contour in the road image ahead of the target vehicle by using preset different lane line shape anchors, to obtain anchoring losses of the lane line contour under different lane line shape anchors, and determine a boundary guiding feature of the lane line contour under the constraint of the lane line vanishing point according to all the anchoring losses; An execution module is configured to guide and connect lane line feature points of the road ahead of the target vehicle based on the boundary guiding feature, to generate a lane line ahead of the target vehicle. 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-8 when the computer program is executed by the processor. The processor executes the computer program to implement the lane line generation method in any one of claims 1 to 7.

10. A computer readable storage medium storing a computer program, characterized in that, The computer program is executed by the processor to implement the lane line generation method in any one of claims 1 to 7.

Images