A Lane Line Direction Detection Method, Device, Computer Device, and Storage Medium

Abstract

The present invention discloses a lane line direction detection method, device, computer device and storage medium, which relates to the technical field of computer vision processing. The method is as follows: after obtaining a lane image, first input the lane image into a Bezier curve parameter prediction model to output the Bezier curve parameters of the lane edge line, then based on the output parameters, separate to obtain a new lane image containing only lane information, and then input the new lane image into a lane line detection model to output the lane line position information, and finally, according to the lane line position information and the Bezier curve parameters of the lane edge line, determine the Bezier curve parameters of the lane line and the lane curve. Thus, by utilizing the advantages of good continuity, obvious demarcation and not being easily blocked by vehicles of the left and right lane edges, the purpose of predicting the lane line direction according to the left and right lane edge directions and spacing can be achieved, and further the accuracy of the lane line direction detection result can be improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision processing, and particularly relates to a lane line trend detection method, device, computer device, and storage medium. Background Art

[0002] Lane line detection is usually defined as detecting the position and trend of lane lines in a vehicle traveling video or image. Lane line detection is an important part of assisted driving and autonomous driving. Lane departure warning and lane keeping can timely correct the driver's inadvertent operations, reduce traffic accidents caused by wrong operations and fatigue driving, thus effectively ensuring driving safety and reducing the accident rate of driving.

[0003] Due to the intermittent road surface printing, narrow and thin lines, and easy occlusion by the vehicle in front of the lane lines, the existing direct lane line detection methods based on deep learning technology have the problem of low accuracy of lane line trend detection results. Therefore, how to improve the accuracy of lane line trend detection is an urgent research topic for those skilled in the art. Summary of the Invention

[0004] The purpose of the present invention is to provide a lane line trend detection method, device, computer device, and computer-readable storage medium to solve the problem of low accuracy of lane line trend detection results in the existing lane line detection methods.

[0005] To achieve the above purpose, the present invention adopts the following technical solutions:

[0006] In the first aspect, a lane line trend detection method is provided, including:

[0007] Obtain a lane image;

[0008] Input the lane image into a pre-trained Bezier curve parameter prediction model, and output the Bezier curve parameters of the lane edge line. Among them, the lane edge line includes a left lane edge line and a right lane edge line, and the Bezier curve parameters include the coordinate positions of multiple Bezier curve key points in the two-dimensional coordinate system of the lane image. The multiple Bezier curve key points include a curve start point, a curve end point, and at least one curve control point;

[0009] Determine the left lane edge curve in the two-dimensional coordinate system according to the Bezier curve parameters of the left lane edge line, and determine the right lane edge curve in the two-dimensional coordinate system according to the Bezier curve parameters of the right lane edge line;

[0010] Perform lane information separation processing on the lane image according to the left lane edge curve and the right lane edge curve to obtain a new lane image containing only lane information, where the lane information refers to the image information between the left lane edge curve and the right lane edge curve in the lane image;

[0011] Input the new lane image into a pre-trained lane line detection model to output lane line position information, where the lane line position information includes the column positions of the lane lines on at least one row anchor, and the row anchor refers to a row of multiple segmentation units horizontally arranged in the lane image, and the column position refers to a certain segmentation unit among the multiple segmentation units;

[0012] Determine the Bézier curve parameters of the lane line according to the Bézier curve parameters of the lane edge line, the left lane edge curve, the right lane edge curve, and the column position of the lane line on a certain row anchor;

[0013] Determine the lane curve on the two-dimensional coordinate system according to the Bézier curve parameters of the lane line to obtain the trend of the lane line.

[0014] Based on the above invention content, a new solution for predicting the lane line trend based on the left and right lane edge trends is provided. That is, after obtaining the lane image, first input the lane image into a pre-trained Bézier curve parameter prediction model to output the Bézier curve parameters of the lane edge line, then perform lane information separation processing on the lane image based on the output parameters to obtain a new lane image containing only lane information, then input the new lane image into a pre-trained lane line detection model to output lane line position information, and finally determine the Bézier curve parameters and lane curve of the lane line according to the lane line position information and the Bézier curve parameters of the lane edge line. Thus, the advantages of good continuity, obvious demarcation, and not being easily blocked by vehicles of the left and right lane edge lines can be utilized to achieve the purpose of predicting the lane line trend based on the left and right lane edge trends, and further improve the accuracy of the lane line trend detection result, which is convenient for practical application and promotion.

[0015] In a possible design, the Bézier curve parameter prediction model adopts a pipeline model including a backbone network unit, a feature flipping and fusion unit, an average pooling unit, a convolution processing unit, a regression branch unit, and a classification branch unit, where the backbone network unit, the feature flipping and fusion unit, the average pooling unit, and the convolution processing unit are connected in sequence, and the regression branch unit and the classification branch unit are respectively connected to the convolution processing unit;

[0016] The backbone network unit is used to extract the first lane edge feature from the lane image;

[0017] The feature flipping and fusion unit is used to aggregate the left - right reversed lane edge features in the first lane edge feature to obtain the second lane edge feature;

[0018] The mean pooling unit is used to convert the high - dimensional second lane edge feature into a low - dimensional third lane edge feature;

[0019] The convolutional processing unit is used to perform conversion processing on the third lane edge feature using two one - dimensional convolutions with a dimension of 1×3 to obtain the fourth lane edge feature;

[0020] The regression branch unit is used to predict the Bezier curve parameters of at least two lane edge lines according to the fourth lane edge feature, where the Bezier curve parameters include the coordinate positions of multiple Bezier curve key points on the two - dimensional coordinate system of the lane image, and the multiple Bezier curve key points include a curve start point, a curve end point, and at least one curve control point;

[0021] The classification branch unit is used to predict the existence probability of the at least two lane edge lines according to the fourth lane edge feature, so as to summarize with the Bezier curve parameters of the at least two lane edge lines to obtain the Bezier curve parameters of the lane edge lines, where the lane edge lines include a left - hand lane edge line and a right - hand lane edge line.

[0022] In a possible design, the backbone network unit adopts a residual network ResNet and a RESA network.

[0023] In a possible design, the lane line detection model adopts a lane line detection network based on global image features and row anchors for position selection, where the row anchor refers to a row of multiple segmentation units horizontally arranged in the lane image.

[0024] In a possible design, determining the Bezier curve parameters of the lane line according to the Bezier curve parameters of the lane edge line, the left - hand lane edge curve, the right - hand lane edge curve, and the column position of the lane line on a certain row anchor includes:

[0025] According to the column position of the lane line on a certain row anchor and the column positions of the left - hand lane edge curve and the right - hand lane edge curve on the certain row anchor, determine the ratio ρ according to the following formula:

[0026]

[0027] In the formula, d la,lsrepresents the distance from the column position of the lane line on the certain row anchor to the column position of the left lane edge curve on the certain row anchor, d rs,ls represents the distance from the column position of the right lane edge curve on the certain row anchor to the column position of the left lane edge curve on the certain row anchor;

[0028] According to the Bezier curve parameters of the lane edge line, the column position of the lane line on the certain row anchor, and the ratio ρ, the Bezier curve parameters of the lane line are calculated according to the following formula:

[0029]

[0030] In the formula, P la,start represents the coordinate position of the curve starting point of the lane line on the two-dimensional coordinate system, P la,row represents the coordinate position of the column position of the lane line on the certain row anchor on the two-dimensional coordinate system, k represents a non-zero natural number, P la,k represents the coordinate position of the k-th curve control point of the lane line along the starting point to the ending point direction on the two-dimensional coordinate system, P ls,k represents the coordinate position of the k-th curve control point of the left lane edge line along the starting point to the ending point direction on the two-dimensional coordinate system, P rs,k represents the coordinate position of the k-th curve control point of the right lane edge line along the starting point to the ending point direction on the two-dimensional coordinate system, P la,end represents the coordinate position of the curve ending point of the lane line on the two-dimensional coordinate system, P ls,rs represents the intersection position of the left lane edge curve and the right lane edge curve on the two-dimensional coordinate system.

[0031] In a possible design, the certain row anchor has the largest road width among the at least one row anchor, where the road width refers to the distance from the column position of the right lane edge curve on the corresponding row anchor to the column position of the left lane edge curve on the corresponding row anchor.

[0032] In a possible design, the multiple Bezier curve key points include a curve starting point, a curve ending point, and two curve control points.

[0033] In a second aspect, a lane line direction detection device is provided, including a lane image acquisition module, a curve parameter prediction module, an edge curve determination module, an information separation and processing module, a lane line detection module, a curve parameter determination module, and a lane curve determination module;

[0034] The lane image acquisition module is used to acquire a lane image;

[0035] The curve parameter prediction module is communicatively connected to the lane image acquisition module, and is configured to input the lane image into a pre-trained Bezier curve parameter prediction model, and output the Bezier curve parameters of the lane edge line. Wherein, the lane edge line includes a left lane edge line and a right lane edge line, and the Bezier curve parameters include the coordinate positions of multiple Bezier curve key points on the two-dimensional coordinate system of the lane image. The multiple Bezier curve key points include a curve start point, a curve end point and at least one curve control point;

[0036] The edge curve determination module is communicatively connected to the curve parameter prediction module, and is configured to determine the left lane edge curve on the two-dimensional coordinate system according to the Bezier curve parameters of the left lane edge line, and determine the right lane edge curve on the two-dimensional coordinate system according to the Bezier curve parameters of the right lane edge line;

[0037] The information separation and processing module is communicatively connected to the lane image acquisition module and the edge curve determination module respectively, and is configured to perform lane information separation and processing on the lane image according to the left lane edge curve and the right lane edge curve, so as to obtain a new lane image containing only lane information. Wherein, the lane information refers to the image information between the left lane edge curve and the right lane edge curve in the lane image;

[0038] The lane line detection module is communicatively connected to the information separation and processing module, and is configured to input the new lane image into a pre-trained lane line detection model, and output lane line position information. Wherein, the lane line position information includes the column positions of the lane line on at least one row anchor. The row anchor refers to a row of multiple segmentation units horizontally arranged in the lane image, and the column position refers to a certain segmentation unit among the multiple segmentation units;

[0039] The curve parameter determination module is communicatively connected to the edge curve determination module and the lane line detection module respectively, and is configured to determine the Bezier curve parameters of the lane line according to the Bezier curve parameters of the lane edge line, the left lane edge curve, the right lane edge curve and the column position of the lane line on a certain row anchor;

[0040] The lane curve determination module is communicatively connected to the curve parameter determination module, and is configured to determine the lane curve on the two-dimensional coordinate system according to the Bezier curve parameters of the lane line, so as to obtain the trend of the lane line.

[0041] In a third aspect, the present invention provides a computer device, including a memory, a processor, and a transceiver that are communicatively connected in sequence. Among them, the memory is used to store computer programs, the transceiver is used to send and receive messages, and the processor is used to read the computer programs and execute the lane line direction detection method as described in the first aspect or any possible design in the first aspect.

[0042] In a fourth aspect, the present invention provides a computer-readable storage medium, on which instructions are stored. When the instructions are run on a computer, they execute the lane line direction detection method as described in the first aspect or any possible design in the first aspect.

[0043] In a fifth aspect, the present invention provides a computer program product containing instructions. When the instructions are run on a computer, they cause the computer to execute the lane line direction detection method as described in the first aspect or any possible design in the first aspect.

[0044] Beneficial effects of the above solutions:

[0045] (1) The present invention creatively provides a new solution for predicting the lane line direction based on the directions of the left and right lane edges at intervals. That is, after obtaining the lane image, the lane image is first input into a pre-trained Bessel curve parameter prediction model, and the Bessel curve parameters of the lane edge line are output. Then, based on the output parameters, the lane image is subjected to lane information separation processing to obtain a new lane image containing only lane information. Then, the new lane image is input into a pre-trained lane line detection model, and the lane line position information is output. Finally, based on the lane line position information and the Bessel curve parameters of the lane edge line, the Bessel curve parameters and lane curve of the lane line are determined. Thus, the advantages of good continuity, obvious demarcation, and not being easily blocked by vehicles of the left and right lane edge lines can be utilized to achieve the purpose of predicting the lane line direction based on the directions of the left and right lane edges at intervals, thereby improving the accuracy of the lane line direction detection result and facilitating practical application and promotion. Description of the Drawings

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0047] Figure 1 It is a schematic flowchart of the lane line direction detection method provided by the embodiment of the present application.

[0048] Figure 2Schematic diagram of the Bessel curve parameter prediction model provided by the embodiment of the present application.

[0049] Figure 3 Example diagram of pipeline processing of the Bessel curve parameter prediction model provided by the embodiment of the present application.

[0050] Figure 4 Schematic diagram of the structure of the feature flip fusion unit provided by the embodiment of the present application.

[0051] Figure 5 Example diagram of the left and right lane edge curves in a two-dimensional coordinate system provided by the embodiment of the present application, where Figure 5 (a) in shows the situation on a straight road, Figure 5 (b) in shows the situation on a curved road.

[0052] Figure 6 Schematic diagram of the structure of the lane line detection model provided by the embodiment of the present application.

[0053] Figure 7 Example diagram of the relationship between the row anchor and the lane line column position provided by the embodiment of the present application.

[0054] Figure 8 Schematic diagram of the structure of the lane line direction detection device provided by the embodiment of the present application.

[0055] Figure 9 Schematic diagram of the structure of the computer device provided by the embodiment of the present application. Detailed implementation manners

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the present invention in combination with the drawings and the descriptions of the embodiments or the prior art. Obviously, the following descriptions of the drawings' structures are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings. It should be noted here that the descriptions of these embodiment manners are used to help understand the present invention, but do not constitute a limitation to the present invention.

[0057] It should be understood that although terms such as first and second may be used herein to describe various objects, these objects should not be limited by these terms. These terms are only used to distinguish one object from another. For example, the first object can be called the second object, and similarly, the second object can be called the first object, without departing from the scope of the exemplary embodiments of the present invention.

[0058] It should be understood that for the term "and / or" that may appear in this text, it is merely a relational description of associated objects, indicating that three relationships can exist. For example, A and / or B can represent three situations: A exists alone, B exists alone, or both A and B exist simultaneously. Another example is that A, B, and / or C can represent any one of A, B, and C or any combination of them. For the term " / and" that may appear in this text, it is a description of another relational object relationship, indicating that two relationships can exist. For example, A / and B can represent two situations: A exists alone or both A and B exist simultaneously. Additionally, for the character " / " that may appear in this text, it generally indicates that the associated objects before and after are in an "or" relationship.

[0059] Embodiment:

[0060] As Figure 1 shown, the lane line direction detection method provided in the first aspect of this embodiment can, but is not limited to, be executed by a computer device with certain computing resources, such as an on-board computer, a personal computer (Personal Computer, PC, referring to a multi-purpose computer suitable for personal use in terms of size, price, and performance; desktop computers, laptops, small laptops, tablet computers, and ultrabooks all belong to personal computers), a smart phone, a personal digital assistant (Personal Digital Assistant, PDA), or a wearable device and other electronic devices. As Figure 1 shown, the lane line direction detection method can, but is not limited to, include the following steps S1 to S7.

[0061] S1. Obtain a lane image.

[0062] In step S1, the lane image can, but is not limited to, be obtained by an on-vehicle camera in real time during vehicle travel and transmitted to a local computer device through a wired or wireless transmission method.

[0063] S2. Input the lane image into a pre-trained Bezier curve parameter prediction model, and output the Bezier curve parameters of the lane edge line. Among them, the lane edge line includes, but is not limited to, a left lane edge line and a right lane edge line. The Bezier curve parameters include, but are not limited to, the coordinate positions of multiple Bezier curve key points in the two-dimensional coordinate system of the lane image. The multiple Bezier curve key points include, but are not limited to, a curve start point, a curve end point, and at least one curve control point.

[0064] In step S2, the Bezier curve parameters are used to fit the direction of the lane edge line. That is, the Bezier curve is a very important parametric curve in computer graphics, and its general formula is as follows:

[0065]

[0066] In the formula, t represents a variable, B(t) represents a B-spline curve, P0 represents the starting point of the curve, and P n represents the ending point of the curve, n is a non-zero natural number greater than 1 and represents the order of the B-spline curve, and i is a natural number. When i is a non-zero natural number not greater than n, P i represents the i-th curve control point along the direction from the starting point of the curve to the ending point of the curve. Thus, when n is equal to 2, the multiple B-spline curve key points include the starting point of the curve, the ending point of the curve, and one curve control point, such that the subsequent left lane edge curve, right lane edge curve, and lane curve are respectively second-order B-spline curves; and when n is equal to 3, the multiple B-spline curve key points include the starting point of the curve, the ending point of the curve, and two curve control points, such that the subsequent left lane edge curve, right lane edge curve, and lane curve are respectively third-order B-spline curves (which can more accurately fit the trend of the lane edge line and lane line compared to the second-order B-spline curve). Considering that although the higher the order, the higher the accuracy of the trend fitting, but the processing time required for prediction based on the B-spline curve parameter prediction model will also be more. In order to quickly obtain the B-spline curve parameters of the lane edge line, preferably, the multiple B-spline curve key points include the starting point of the curve, the ending point of the curve, and two curve control points (that is, the subsequent left lane edge curve, right lane edge curve, and lane curve are respectively third-order B-spline curves). In addition, before inputting the lane image into the B-spline curve parameter prediction model, preprocessing such as image grayscale processing and image filtering processing can be performed on the lane image first. Among them, the image grayscale processing (which is an existing conventional preprocessing method) is used to reduce the data volume and improve the real-time performance of subsequent detection, and the image filtering processing (which is an existing conventional preprocessing method) is used to reduce the interference of uncertain noise points in the road image.

[0067] In the step S2, specifically, the B-spline curve parameter prediction model adopts a pipeline model including but not limited to a backbone network unit, a feature flipping and fusion unit, an average pooling unit, a convolution processing unit, a regression branch unit, and a classification branch unit. Among them, the backbone network unit, the feature flipping and fusion unit, the average pooling unit, and the convolution processing unit are connected in sequence, and the regression branch unit and the classification branch unit are respectively connected to the convolution processing unit.

[0068] As Figures 2 - 3As shown, the backbone network unit is used to extract the first lane edge feature from the lane image, which can but is not limited to using a residual network ResNet and a RESA network. The ResNet is a new network structure proposed in response to the problem of network degradation (i.e., as the number of convolutional neural network layers increases, the prediction error increases) found after theoretically deepening the number of convolutional neural network layers. That is, by using a shortcut connection, only the residual term is learned because residual learning is easier than learning the original features. If the learned residual value F(x) is 0, it is equivalent to making an identity mapping. Specifically, the ResNet can but is not limited to using the ResNet_34 network structure. The RESA (Recurrent Feature-Shift Aggregator) network is also an existing network structure and can make four choices (i.e., up, down, left, and right), repeating each direction K times, and the step size will gradually increase each time, making the RESA network have characteristics such as high computational efficiency, efficient information aggregation, and strong compatibility, and is very suitable for extracting lane edge features.

[0069] The feature flipping and fusion unit is used to aggregate the left and right reverse lane edge features in the first lane edge feature to obtain the second lane edge feature. As Figure 4 shown, the working principle of the feature flipping and fusion unit is as follows: Considering that the existence of the left lane edge line is very likely to indicate its corresponding right lane edge line, and the structure of the straight left lane edge line can help describe the straight right lane edge line, this characteristic can be utilized to fuse the feature map with its horizontally flipped version. Specifically, each feature map is transformed using two separate convolutional and normalization layers, that is, the feature maps before and after flipping, and then the two transformed feature maps are added together and the output based on the ReLU activation function is made to obtain the second lane edge feature. In addition, in Figure 4 , a 3x3 deformable convolution can also be used to flip the feature map and learn the bias on the original feature map for feature alignment to solve the problem of slight misalignment of the images captured by the camera.

[0070] The mean pooling unit is used to convert the high-dimensional second lane edge feature into a low-dimensional third lane edge feature. For example, converting the second lane edge feature with a dimension of into a third lane edge feature with a dimension of The third lane edge feature, where H represents the number of vertical pixels in the lane image and W represents the number of horizontal pixels in the lane image; the convolution processing unit is used to perform a transformation process on the third lane edge feature using two one-dimensional convolutions with a dimension of 1×3 to obtain a fourth lane edge feature; the regression branch unit is used to predict the Bezier curve parameters of at least two lane edge lines according to the fourth lane edge feature, where the Bezier curve parameters include, but are not limited to, the coordinate positions of multiple Bezier curve key points in the two-dimensional coordinate system of the lane image, and the multiple Bezier curve key points include, but are not limited to, a curve start point, a curve end point, and at least one curve control point. For example, when the multiple Bezier curve key points include a curve start point, a curve end point, and two curve control points, the data dimension obtained by the foregoing prediction can be That is, to predict the Bezier curve parameters of the lane edge lines at, and predict the coordinate positions (including the ordinate and abscissa) of the curve start point, the curve end point, and the two curve control points in the two-dimensional coordinate system using 8 digits respectively; the classification branch unit is used to predict the existence probability of the at least two lane edge lines according to the fourth lane edge feature (when the multiple Bezier curve key points include a curve start point, a curve end point, and two curve control points, the data dimension obtained by the prediction can be ), so as to summarize with the Bezier curve parameters of the at least two lane edge lines to obtain the Bezier curve parameters of the lane edge lines, where the lane edge lines include a left lane edge line and a right lane edge line. In the foregoing summarization process, the Bezier curve parameters of the two lane edge lines with the largest and the second largest existence probabilities can be used as the Bezier curve parameters of the left lane edge line and the right lane edge line respectively. In addition, the training method of the Bezier curve parameter prediction model is the existing conventional method.

[0071] S3. Determine the left lane edge curve in the two-dimensional coordinate system according to the Bezier curve parameters of the left lane edge line, and determine the right lane edge curve in the two-dimensional coordinate system according to the Bezier curve parameters of the right lane edge line.

[0072] In the step S3, the left lane edge curve and the right lane edge curve can be specifically determined based on the general formula of the Bezier curve (at this time, t can take the value of 1), as Figure 5 shown.

[0073] S4. Based on the left lane edge curve and the right lane edge curve, perform lane information separation processing on the lane image to obtain a new lane image that only contains lane information, where the lane information refers to the image information between the left lane edge curve and the right lane edge curve in the lane image.

[0074] In step S4, the lane information separation processing is a conventional matte extraction process, that is, other image information except the lane information is extracted from the lane image to obtain the new lane image, as Figure 6 shown in the input image. In addition, through the above-mentioned lane information separation processing, the data volume can be further reduced, and the real-time performance and accuracy of subsequent detection can be improved.

[0075] S5. Input the new lane image into a pre-trained lane line detection model, and output the lane line position information, where the lane line position information includes the column positions of the lane lines on at least one row anchor, and the row anchor refers to a row of multiple segmentation units horizontally arranged in the lane image, and the column position refers to a certain segmentation unit among the multiple segmentation units.

[0076] In step S5, specifically, the lane line detection model uses a lane line detection network that selects positions based on global image features and row anchors, where the row anchor refers to a row of multiple segmentation units horizontally arranged in the lane image. As Figure 6 shown, the working principle of the lane line detection network is: use global image features to select the correct lane line column positions (i.e., the segmentation units containing lane line information) on each predefined row anchor, as Figure 7 shown. Specifically, the lane line detection network can be implemented by, but not limited to, the lane line detection network with ultra-fast structure awareness proposed by the team of Zhejiang University (i.e., Ultra Fast Structure-aware Deep Lane Detection), and the calculation of its prediction formula and loss function will not be elaborated here.

[0077] S6. Determine the Bezier curve parameters of the lane line according to the Bezier curve parameters of the lane edge line, the left lane edge curve, the right lane edge curve, and the column position of the lane line on a certain row anchor.

[0078] In step S6, specifically, steps S61 to S62 are included but are not limited to: S61. Based on the column position of the lane line on a certain row anchor and the column positions of the left lane edge curve and the right lane edge curve on the certain row anchor (i.e., the segmentation units intersecting the left lane edge curve and the right lane edge curve, respectively), determine the ratio ρ according to the following formula:

[0079]

[0080] Where, d la,ls represents the distance from the column position of the lane line on the row anchor to the column position of the left lane edge curve on the row anchor (i.e., the number of segmentation units separated), d rs,ls represents the distance (i.e., the number of segmentation units) between the column position of the right lane edge curve on the row anchor and the column position of the left lane edge curve on the row anchor. S62. Calculate the Bezier curve parameters of the lane line based on the Bezier curve parameters of the lane edge line, the column position of the lane line on the row anchor, and the ratio ρ according to the following formula:

[0081]

[0082] Where, P la,start represents the coordinate position of the starting point of the lane line curve in the two-dimensional coordinate system, P la,row represents the coordinate position of the column position of the lane line on the row anchor in the two-dimensional coordinate system, k represents a non-zero natural number, P la,k represents the coordinate position of the kth curve control point of the lane line along the direction from the starting point to the end point in the two-dimensional coordinate system, P ls,k represents the coordinate position of the kth curve control point along the left lane edge line from the starting point to the end point in the two-dimensional coordinate system, P rs,k The coordinate position of the kth curve control point along the right lane edge line from the starting point to the end point in the two-dimensional coordinate system, P la,end represents the coordinate position of the curve end point of the lane line in the two-dimensional coordinate system, P ls,rs represents the intersection position of the left lane edge curve and the right lane edge curve in the two-dimensional coordinate system.

[0083] Based on the foregoing steps S61 to S62, the advantages of good continuity, distinct demarcation, and not being easily blocked by vehicles of the left and right lane edge lines can be specifically utilized. According to the trends of the left and right lane edges, the trend of the lane line can be predicted by spacing, thereby improving the accuracy of the lane line trend detection result. In addition, in order to improve the detection accuracy of the lane curve starting point, preferably, the certain row anchor has the largest road width among the at least one row anchor, where the road width refers to the distance from the column position of the right lane edge curve on the corresponding row anchor to the column position of the left lane edge curve on the corresponding row anchor (i.e., the number of separated segmentation units).

[0084] S7. Determine the lane curve on the two-dimensional coordinate system according to the B-spline curve parameters of the lane line, and obtain the trend of the lane line.

[0085] In the step S7, similarly, the lane curve can be specifically determined based on the general formula of the B-spline curve (at this time, t can take the value of 1), and the trend of the lane line can be obtained.

[0086] Based on the lane line trend detection method described in the foregoing steps S1 to S7, a new solution for predicting the lane line trend by spacing based on the trends of the left and right lane edges is provided. That is, after obtaining the lane image, the lane image is first input into a pre-trained B-spline curve parameter prediction model, and the B-spline curve parameters of the lane edge line are output. Then, based on the output parameters, lane information separation processing is performed on the lane image to obtain a new lane image containing only lane information. Then, the new lane image is input into a pre-trained lane line detection model, and the lane line position information is output. Finally, according to the lane line position information and the B-spline curve parameters of the lane edge line, the B-spline curve parameters and lane curve of the lane line are determined. Thus, the advantages of good continuity, distinct demarcation, and not being easily blocked by vehicles of the left and right lane edge lines can be utilized to achieve the purpose of predicting the lane line trend by spacing based on the trends of the left and right lane edges, thereby improving the accuracy of the lane line trend detection result and facilitating practical application and promotion.

[0087] As Figure 8 shown, in the second aspect of this embodiment, a virtual device for implementing the lane line trend detection method described in the first aspect is provided, including a lane image acquisition module, a curve parameter prediction module, an edge curve determination module, an information separation processing module, a lane line detection module, a curve parameter determination module, and a lane curve determination module;

[0088] The lane image acquisition module is used to acquire a lane image;

[0089] The curve parameter prediction module is communicatively connected to the lane image acquisition module, and is configured to input the lane image into a pre-trained Bessel curve parameter prediction model, and output the Bessel curve parameters of the lane edge line. Among them, the lane edge line includes a left lane edge line and a right lane edge line, and the Bessel curve parameters include the coordinate positions of multiple Bessel curve key points on the two-dimensional coordinate system of the lane image. The multiple Bessel curve key points include a curve start point, a curve end point, and at least one curve control point;

[0090] The edge curve determination module is communicatively connected to the curve parameter prediction module, and is configured to determine the left lane edge curve on the two-dimensional coordinate system according to the Bessel curve parameters of the left lane edge line, and determine the right lane edge curve on the two-dimensional coordinate system according to the Bessel curve parameters of the right lane edge line;

[0091] The information separation processing module is communicatively connected to the lane image acquisition module and the edge curve determination module respectively, and is configured to perform lane information separation processing on the lane image according to the left lane edge curve and the right lane edge curve, and obtain a new lane image that only contains lane information. Among them, the lane information refers to the image information located between the left lane edge curve and the right lane edge curve in the lane image;

[0092] The lane line detection module is communicatively connected to the information separation processing module, and is configured to input the new lane image into a pre-trained lane line detection model, and output the lane line position information. Among them, the lane line position information includes the column positions of the lane line on at least one row anchor. The row anchor refers to a row of multiple segmentation units horizontally arranged in the lane image, and the column position refers to a certain segmentation unit among the multiple segmentation units;

[0093] The curve parameter determination module is communicatively connected to the edge curve determination module and the lane line detection module respectively, and is configured to determine the Bessel curve parameters of the lane line according to the Bessel curve parameters of the lane edge line, the left lane edge curve, the right lane edge curve, and the column position of the lane line on a certain row anchor;

[0094] The lane curve determination module is communicatively connected to the curve parameter determination module, and is configured to determine the lane curve on the two-dimensional coordinate system according to the Bessel curve parameters of the lane line, and obtain the trend of the lane line.

[0095] For the working process, working details and technical effects of the foregoing device provided in the second aspect of this embodiment, reference may be made to the lane line trend detection method described in the first aspect, and details are not described herein again.

[0096] As Figure 9 shown, in the third aspect of this embodiment, a computer device for executing the lane line direction detection method described in the first aspect is provided, including a memory, a processor, and a transceiver that are communicatively connected in sequence. Among them, the memory is used to store computer programs, the transceiver is used to send and receive messages, and the processor is used to read the computer programs and execute the lane line direction detection method described in the first aspect. Specifically, for example, the memory may include, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a flash memory, a first input first output (FIFO), and / or a first input last output (FILO), etc.; the processor may be, but is not limited to, a microprocessor of the STM32F105 series. In addition, the computer device may also include, but is not limited to, a power module, a display screen, and other necessary components.

[0097] For the working process, working details, and technical effects of the foregoing computer device provided in the third aspect of this embodiment, reference may be made to the lane line direction detection method described in the first aspect, which will not be elaborated herein.

[0098] In the fourth aspect of this embodiment, a computer-readable storage medium storing instructions for the lane line direction detection method described in the first aspect is provided, that is, instructions are stored on the computer-readable storage medium, and when the instructions run on a computer, the lane line direction detection method described in the first aspect is executed. Among them, the computer-readable storage medium refers to a carrier for storing data, and may include, but is not limited to, a floppy disk, an optical disc, a hard disk, a flash memory, a USB flash drive, and / or a memory stick, etc. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0099] For the working process, working details, and technical effects of the foregoing computer-readable storage medium provided in the fourth aspect of this embodiment, reference may be made to the lane line direction detection method described in the first aspect, which will not be elaborated herein.

[0100] In the fifth aspect of this embodiment, a computer program product containing instructions is provided, and when the instructions run on a computer, the computer is made to execute the lane line direction detection method described in the first aspect. Among them, the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0101] Finally, it should be noted that the above are only preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A lane line direction detection method, characterized in that, Including: Obtain a lane image; Input the lane image into a pre-trained Bessel curve parameter prediction model, and output the Bessel curve parameters of the lane edge line. Among them, the lane edge line includes a left lane edge line and a right lane edge line, and the Bessel curve parameters include the coordinate positions of multiple Bessel curve key points on the two-dimensional coordinate system of the lane image. The multiple Bessel curve key points include a curve start point, a curve end point, and at least one curve control point; Determine the left lane edge curve on the two-dimensional coordinate system according to the Bessel curve parameters of the left lane edge line, and determine the right lane edge curve on the two-dimensional coordinate system according to the Bessel curve parameters of the right lane edge line; Perform lane information separation processing on the lane image according to the left lane edge curve and the right lane edge curve to obtain a new lane image containing only lane information. Among them, the lane information refers to the image information between the left lane edge curve and the right lane edge curve in the lane image; Input the new lane image into a pre-trained lane line detection model, and output the lane line position information. Among them, the lane line position information includes the column positions of the lane line on at least one row anchor. The row anchor refers to a row of multiple segmentation units horizontally arranged in the lane image, and the column position refers to a certain segmentation unit among the multiple segmentation units; Determine the Bessel curve parameters of the lane line according to the Bessel curve parameters of the lane edge line, the left lane edge curve, the right lane edge curve, and the column position of the lane line on a certain row anchor; Determine the lane curve on the two-dimensional coordinate system according to the Bessel curve parameters of the lane line to obtain the trend of the lane line; 2. The lane line direction detection method according to claim 1, wherein, The Bessel curve parameter prediction model adopts a pipeline model including a backbone network unit, a feature flipping and fusion unit, an average pooling unit, a convolution processing unit, a regression branch unit, and a classification branch unit. Among them, the backbone network unit, the feature flipping and fusion unit, the average pooling unit, and the convolution processing unit are connected in sequence, and the regression branch unit and the classification branch unit are respectively connected to the convolution processing unit; The backbone network unit is used to extract the first lane edge feature from the lane image; The feature flipping and fusion unit is used to aggregate the left and right reverse lane edge features in the first lane edge feature to obtain the second lane edge feature; The average pooling unit is used to convert the high-dimensional second lane edge feature into a low-dimensional third lane edge feature; The convolution processing unit is used to perform conversion processing on the third lane edge feature using two one-dimensional convolutions with a dimension of 1×3 to obtain the fourth lane edge feature; The regression branch unit is used to predict the Bezier curve parameters of at least two lane edge lines according to the fourth lane edge feature, where the Bezier curve parameters include the coordinate positions of multiple Bezier curve key points on the two-dimensional coordinate system of the lane image, and the multiple Bezier curve key points include a curve start point, a curve end point, and at least one curve control point; The classification branch unit is used to predict the existence probability of the at least two lane edge lines according to the fourth lane edge feature, so as to summarize with the Bezier curve parameters of the at least two lane edge lines to obtain the Bezier curve parameters of the lane edge lines, where the lane edge lines include a left lane edge line and a right lane edge line.

3. The lane line direction detection method according to claim 2, characterized in that The backbone network unit adopts a residual network ResNet and a RESA network.

4. The lane line direction detection method according to claim 1, wherein The lane line detection model adopts a lane line detection network that selects positions based on global image features and row anchors, where the row anchor refers to a row of multiple segmentation units horizontally arranged in the lane image.

5. The lane line direction detection method according to claim 1, characterized in that Determining the Bezier curve parameters of the lane line according to the Bezier curve parameters of the lane edge line, the left lane edge curve, the right lane edge curve, and the column position of the lane line on a certain row anchor includes: According to the column position of the lane line on a certain row anchor and the column positions of the left lane edge curve and the right lane edge curve on the certain row anchor respectively, the ratio ρ is determined according to the following formula: where d la,ls represents the distance from the column position of the lane line on the certain row anchor to the column position of the left lane edge curve on the certain row anchor, and d rs,ls represents the distance from the column position of the right lane edge curve on the certain row anchor to the column position of the left lane edge curve on the certain row anchor; According to the Bezier curve parameters of the lane edge line, the column position of the lane line on the certain row anchor, and the ratio ρ, the Bezier curve parameters of the lane line are calculated according to the following formula: wherein, P la,start represents the coordinate position of the starting point of the curve of the lane line on the two-dimensional coordinate system, P la,row represents the coordinate position of the column position of the lane line on a certain row anchor on the two-dimensional coordinate system, k represents a non-zero natural number, P la,k represents the coordinate position of the k-th curve control point of the lane line along the starting point to the end point direction on the two-dimensional coordinate system, P ls,k represents the coordinate position of the k-th curve control point of the left lane edge line along the starting point to the end point direction on the two-dimensional coordinate system, P rs,k represents the coordinate position of the k-th curve control point of the right lane edge line along the starting point to the end point direction on the two-dimensional coordinate system, P la,end represents the coordinate position of the end point of the curve of the lane line on the two-dimensional coordinate system, P ls,rs represents the intersection position of the left lane edge curve and the right lane edge curve on the two-dimensional coordinate system.

6. The lane line direction detection method according to claim 5, wherein, The certain row anchor has the largest road width among the at least one row anchor, where the road width refers to the distance from the column position of the right lane edge curve on the corresponding row anchor to the column position of the left lane edge curve on the corresponding row anchor.

7. The lane line direction detection method according to claim 1, characterized in that The multiple Bezier curve key points include a curve start point, a curve end point, and two curve control points.

8. A lane line direction detection device, characterized in that, It includes a lane image acquisition module, a curve parameter prediction module, an edge curve determination module, an information separation and processing module, a lane line detection module, a curve parameter determination module, and a lane curve determination module; The lane image acquisition module is used to acquire a lane image; The curve parameter prediction module is communicatively connected to the lane image acquisition module, and is used to input the lane image into a pre-trained Bezier curve parameter prediction model, and output the Bezier curve parameters of the lane edge lines, where the lane edge lines include a left lane edge line and a right lane edge line, the Bezier curve parameters include the coordinate positions of multiple Bezier curve key points on the two-dimensional coordinate system of the lane image, and the multiple Bezier curve key points include a curve start point, a curve end point, and at least one curve control point; The edge curve determination module is communicatively connected to the curve parameter prediction module, and is configured to determine the left lane edge curve on the two-dimensional coordinate system according to the Bezier curve parameters of the left lane edge line, and determine the right lane edge curve on the two-dimensional coordinate system according to the Bezier curve parameters of the right lane edge line; The information separation and processing module is communicatively connected to the lane image acquisition module and the edge curve determination module respectively, and is configured to perform lane information separation and processing on the lane image according to the left lane edge curve and the right lane edge curve, so as to obtain a new lane image only containing lane information, where the lane information refers to the image information between the left lane edge curve and the right lane edge curve in the lane image; The lane line detection module is communicatively connected to the information separation and processing module, and is configured to input the new lane image into a pre-trained lane line detection model, and output lane line position information, where the lane line position information includes the column positions of the lane line on at least one row anchor, and the row anchor refers to a row of multiple segmentation units horizontally arranged in the lane image, and the column position refers to a certain segmentation unit among the multiple segmentation units; The curve parameter determination module is communicatively connected to the edge curve determination module and the lane line detection module respectively, and is configured to determine the Bezier curve parameters of the lane line according to the Bezier curve parameters of the lane edge line, the left lane edge curve, the right lane edge curve, and the column position of the lane line on a certain row anchor; The lane curve determination module is communicatively connected to the curve parameter determination module, and is configured to determine the lane curve on the two-dimensional coordinate system according to the Bezier curve parameters of the lane line, so as to obtain the trend of the lane line.

9. A computer device, characterized in that, It includes a memory, a processor, and a transceiver that are communicatively connected in sequence, where the memory is used to store computer programs, the transceiver is used to send and receive messages, and the processor is used to read the computer programs and execute the lane line trend detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that , Instructions are stored on the computer-readable storage medium, and when the instructions are run on a computer, the lane line trend detection method according to any one of claims 1 to 7 is executed.

Images