Methods, systems and media for lane intersection detection

Abstract

The present application provides a method, system and medium for lane line intersection detection. The method can be applied to scenes such as ports, mines, highways or parks. The method includes: receiving an image to be detected sent by a vehicle-mounted camera device, and identifying the lane lines in the image to be detected; performing curve fitting on the identified lane lines to obtain the corresponding lane line curve; for each lane line curve, determining the lane line calibration width corresponding to each position of the lane line curve; according to each lane line curve and the lane line calibration width corresponding to each position of each lane line curve, extracting features from the image to be detected to obtain the corresponding feature image; inputting the feature image into the intersection detection model, and determining the intersection of the lane lines in the image to be detected according to the result output by the intersection detection model. The method of the present application reduces the installation cost of the vehicle driving system and improves the stability of the system detection.

Description

Technical Field

[0001] The present application belongs to the field of autonomous driving technology, and in particular relates to a method, system, and medium for detecting lane intersections, which can be applied to scenarios such as ports, highways, logistics, mines, closed parks, or urban transportation. Background Art

[0002] In autonomous driving technology, lane line detection function is an important foundation for various advanced driver assistance systems and autonomous driving systems, and can help realize vehicle lane keeping, vehicle positioning and other functions.

[0003] In conventional road structures, lane lines are approximately parallel to each other, but in areas such as highways and ramps, different lanes merge and separate, so their lane lines intersect with each other. The correct identification of lane line intersections is conducive to improving the understanding of lane line topology, improving the stability and accuracy of lane line detection, and providing a basis for decision-making in downstream modules, thereby improving driving safety. In the existing technology, lane line intersection detection is usually combined with high-precision maps, that is, the vehicle driving system first obtains the current positioning information of the vehicle through a high-precision positioning module such as GPS, and then combines it with the high-precision map module set in the driving system to determine whether there is an intersection in the lane line in front of the driving direction of the current positioning information. However, this detection method requires the loading of a high-precision positioning module and a high-precision map module in the vehicle driving system, resulting in high installation costs for the vehicle driving system. In addition, the detection relies on high-precision positioning, and the detection of lane line intersections cannot be completed in areas with poor positioning effects, resulting in poor detection stability of the driving system.

[0004] Therefore, a solution for lane intersection detection with low installation cost and high detection stability is needed for a vehicle driving system. Summary of the Invention

[0005] The present application provides a method, system and medium for lane line intersection detection, which are used to solve the technical problems of high installation cost and poor detection stability of vehicle driving systems.

[0006] In a first aspect, the present application provides a method for detecting lane intersections, comprising:

[0007] Receiving an image to be detected sent by a vehicle-mounted camera device, and identifying lane lines in the image to be detected;

[0008] Perform curve fitting on the identified lane line to obtain the corresponding lane line curve;

[0009] For each lane line curve, determining the lane line calibration width corresponding to each position of the lane line curve;

[0010] Performing feature extraction on the image to be detected based on each lane line curve and the lane line calibrated width corresponding to each position of each lane line curve to obtain a corresponding feature image;

[0011] The feature image is input into an intersection detection model, and the intersection of the lane lines in the image to be detected is determined according to the result output by the intersection detection model.

[0012] In this embodiment, after receiving the image to be detected uploaded by the vehicle-mounted camera device, the lane lines in the image to be detected can be first identified to determine the approximate area where the lane lines are located in the image. Afterwards, curve fitting can be performed on the identified lane lines to obtain the corresponding lane line curve, thereby further determining the exact position of the lane lines. Since lane lines have actual widths while curves do not have widths, after obtaining the lane line curves, it is also necessary to determine the lane line calibration widths corresponding to each position of the lane line curve. By using the lane line curves and the lane line calibration widths corresponding to each position of the lane line curves, the corresponding lane lines can be fully and accurately characterized, thereby determining the exact position of the lane lines in the image. After determining the exact position of the lane lines in the image, the lane line pixel features at the corresponding positions can be extracted to obtain the corresponding feature image. The feature image can then be input into the intersection detection model. Based on the results output by the intersection detection model, the intersection of the lane lines in the image to be detected can be accurately obtained. Through such a setting, when detecting lane intersections, the vehicle driving system does not need to load a high-precision positioning module and a high-precision map module. It only needs an on-board camera device to obtain images during the driving process and use the intersection detection model to achieve accurate detection of intersections. This not only reduces the installation cost of the vehicle driving system, but also no longer relies on high-precision positioning information, thereby improving the stability of the driving system's intersection detection.

[0013] In a possible implementation, determining the lane line calibrated width corresponding to each position of the lane line curve specifically includes:

[0014] Determine the maximum width and minimum width of the lane line corresponding to the lane line curve in the image to be detected;

[0015] Determining the vanishing point coordinates of the lane line corresponding to the lane line curve in the image to be detected according to a preset coordinate system corresponding to the image to be detected;

[0016] The lane line calibrated width corresponding to each position of the lane line curve is determined according to the maximum width value and the minimum width value, and the vanishing point position coordinates.

[0017] In this implementation, since lane lines have actual widths, while curves do not, in order to subsequently extract the true features of the lane lines, after obtaining the lane line curve, it is necessary to determine the lane line calibrated widths corresponding to each position on the lane line curve. Due to the characteristics of the field of view in an image, lane lines that are originally of uniform width will appear to gradually narrow in the image, and multiple lane lines will intersect at vanishing points. Therefore, based on the maximum and minimum widths of the lane lines in the image to be detected, as well as the coordinates of the vanishing points of the lane lines, the lane line calibrated widths corresponding to each position on the lane line curve can be accurately determined.

[0018] In a possible implementation, determining the lane line calibrated width corresponding to each position of the lane line curve based on the maximum width value and the minimum width value and the vanishing point position coordinates specifically includes:

[0019] Use the following formula to determine the lane line calibration width corresponding to each position of the lane line curve:

[0020]

[0021] Wherein, the W i represents the lane line calibration width corresponding to the ordinate position of the i-th pixel of the lane line curve, and the W min represents the minimum width of the lane line corresponding to the lane line curve in the image to be detected, and the W max represents the maximum width of the lane line corresponding to the lane line curve in the image to be detected, and the r i represents the vertical coordinate of the i-th pixel in the preset coordinate system, and the vp y represents the vertical coordinate corresponding to the vanishing point position in the preset coordinate system, Height represents the image height corresponding to the image to be detected, and γ represents the lane width ratio coefficient.

[0022] In this embodiment, generally speaking, the vertical coordinate corresponding to the vanishing point position is at the highest point in the image, that is, the vertical coordinate corresponding to each position of the lane line curve is smaller than the vertical coordinate corresponding to the vanishing point position. In this case, the lane line calibration width corresponding to each position in this situation can be obtained based on the vertical coordinate corresponding to each position and the corresponding calculation formula (12). However, there may be special cases, such as a downhill road, in which case the vertical coordinate corresponding to the vanishing point position may be smaller than the vertical coordinate corresponding to some positions. In this case, the lane line calibration width corresponding to each position in this situation can be obtained according to the calculation formula (11). Through this setting, the size relationship between the vertical coordinate corresponding to the vanishing point position and the vertical coordinates corresponding to each position can be used to obtain the lane line calibration width corresponding to each position using different calculation methods, so that the lane line calibration width is more consistent with the actual situation and the accuracy of the lane line calibration width calculation is improved.

[0023] In a possible implementation, performing feature extraction on the image to be detected based on each lane curve and the lane line calibrated width corresponding to each position of each lane curve to obtain a corresponding feature image specifically includes:

[0024] Determine the RGB image corresponding to the image to be detected;

[0025] For each lane line curve, extracting corresponding lane line pixel features in the RGB image according to the position of the lane line curve and the lane line calibrated width corresponding to each position of the lane line curve;

[0026] A feature image corresponding to the image to be detected is generated based on the lane line pixel features corresponding to each lane line curve.

[0027] In this embodiment, the lane line corresponding to the lane line curve and the lane line calibration width corresponding to each position of the lane line curve can be used to fully and accurately characterize the lane line, thereby determining the exact position of the lane line in the image. After determining the exact position of the lane line in the image, the lane line pixel features (i.e., pixels) at the corresponding position can be extracted accordingly to obtain the lane line pixel features corresponding to the lane line curve. Based on the lane line pixel features corresponding to each lane line curve, a feature image corresponding to the image to be detected can be generated, and the feature image can include comprehensive and real features of the lane line. Through such a setting, the feature image corresponding to the image to be detected can be accurately obtained by using each lane line curve and the lane line calibration width corresponding to each position of each lane line curve, so as to facilitate the subsequent detection of lane line intersections, improve the detection accuracy, and greatly reduce false detections caused by interference from information such as ground text.

[0028] In a possible implementation, before generating a feature image corresponding to the image to be detected based on lane line pixel features corresponding to each lane line curve, the method further includes:

[0029] Scaling the lane line pixel features corresponding to each lane line curve according to a preset width value to obtain scaled pixel features;

[0030] splicing the scaled pixel features according to row correspondence to obtain target lane line pixel features corresponding to each lane line curve;

[0031] Accordingly, generating a feature image corresponding to the image to be detected based on the lane line pixel features corresponding to each lane line curve includes:

[0032] A feature image corresponding to the image to be detected is generated based on the target lane line pixel features corresponding to each lane line curve.

[0033] In this embodiment, after obtaining the lane line pixel features corresponding to the lane line curve based on the position of the lane line curve and the calibrated lane line width corresponding to each position of the lane line curve, the lane line pixel features are the features of the lane line in the image, which are different from the lane line features in the actual situation. For example, the width of the lane line in the actual situation is consistent. Therefore, in order to make the lane line pixel features more consistent with the actual situation and better represent the real features, the lane line pixel features need to be scaled to a fixed width and spliced ​​to obtain the target lane line pixel features. Through this setting, the target lane line pixel features can be made more consistent with the actual lane line situation, making the feature image generated based on this more realistic and accurate, and achieving alignment of the data feature dimensions.

[0034] In one possible implementation, inputting the feature image into an intersection detection model, and determining the intersection of lane lines in the image to be detected based on a result output by the intersection detection model, specifically includes:

[0035] Inputting the feature image into an intersection detection model to obtain detection results of a preset number of areas corresponding to the feature image;

[0036] According to the detection results of the preset number of areas, the intersection of the lane lines in the image to be detected is determined, and the intersection of the lane lines in the image to be detected is output to a downstream module.

[0037] In this implementation, a multi-region classification intersection detection model can be used to perform intersection detection on feature images. The classification results for each region are used to determine whether there are bifurcations within that region. Combining the detection results from all regions can determine whether lane intersections exist in the image to be detected, as well as their specific locations and number. Using this multi-region classification intersection detection model effectively simplifies the output dimensionality. End-to-end training significantly streamlines the detection process, significantly reducing network convergence speed and complexity, thereby lowering the computational resource requirements and improving algorithm efficiency.

[0038] In one possible implementation, the detection result includes a probability of existence of an intersection and a probability of each intersection type. Accordingly, determining the intersection of lane lines in the image to be detected based on the detection results of the preset number of areas specifically includes:

[0039] For each of the regions,

[0040] Determining whether there is a lane intersection in the area based on the probability of the intersection corresponding to the area;

[0041] If there is a lane intersection in the area, determining the intersection type corresponding to the lane intersection according to the probabilities of each intersection type corresponding to the lane intersection;

[0042] Determining the lane line intersections in the image to be detected according to the lane line intersections corresponding to each of the regions and the intersection types corresponding to the lane line intersections;

[0043] The intersection types include: right-side outgoing type, right-side incoming type, left-side outgoing type, and left-side incoming type.

[0044] In this embodiment, the detection results may include the probability of existence of intersections and the probabilities of each intersection type. Accordingly, the above-mentioned step S402 of determining the intersections of lane lines in the image to be detected based on the detection results of a preset number of areas may include: for each area, determining whether there is a lane line intersection in the area based on the probability of existence of the intersection corresponding to the area; if there is a lane line intersection in the area, determining the intersection type corresponding to the lane line intersection based on the probabilities of each intersection type corresponding to the lane line intersection; determining the intersection of lane lines in the image to be detected based on the lane line intersection corresponding to each area and the intersection type corresponding to the lane line intersection; wherein the intersection types include: right-side outgoing type, right-side incoming type, left-side outgoing type, and left-side incoming type.

[0045] In a second aspect, the present application provides a vehicle driving system, comprising:

[0046] A receiving module, configured to receive an image to be detected sent by a vehicle-mounted camera device and identify lane lines in the image to be detected;

[0047] A processing module is used to perform curve fitting on the identified lane lines to obtain corresponding lane line curves; for each lane line curve, determine the lane line calibration width corresponding to each position of the lane line curve; based on each lane line curve and the lane line calibration width corresponding to each position of each lane line curve, perform feature extraction on the image to be detected to obtain a corresponding feature image; input the feature image into an intersection detection model, and determine the intersection of the lane lines in the image to be detected based on the results output by the intersection detection model.

[0048] In a third aspect, the present application provides a vehicle driving system, comprising: a processor, and a memory communicatively connected to the processor;

[0049] The memory stores computer-executable instructions;

[0050] The processor executes the computer-executable instructions stored in the memory to implement the method of the first aspect above.

[0051] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to implement the method of the first aspect when executed by a processor.

[0052] In a fifth aspect, the present application provides a computer program product, comprising a computer program, which implements the method of the first aspect when executed by a processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a structural diagram of a vehicle driving system;

[0054] Figure 2 This is a system architecture diagram of an embodiment of the present application;

[0055] Figure 3 This is a flowchart of a method for lane intersection detection according to an embodiment of the present application;

[0056] Figure 4 This is a schematic diagram of the lane line recognition process according to an embodiment of the present application;

[0057] Figure 5 A flowchart of a method for lane intersection detection according to another embodiment of the present application;

[0058] Figure 6 A schematic diagram of the vanishing point position and width according to an embodiment of the present application;

[0059] Figure 7 This is a flowchart of a method for lane intersection detection according to another embodiment of the present application;

[0060] Figure 8 This is a flowchart of a method for lane intersection detection according to another embodiment of the present application;

[0061] Figure 9 This is a schematic diagram of the intersection detection result according to an embodiment of the present application;

[0062] Figure 10 This is a schematic structural diagram of a vehicle driving system according to an embodiment of the present application;

[0063] Figure 11 This is a schematic structural diagram of a vehicle driving system according to another embodiment of the present application. DETAILED DESCRIPTION

[0064] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0065] The lane line intersection detection method, system and medium of the present application can be applied to scenarios such as ports, highways, logistics, mines, closed parks, or urban traffic, or scenarios involving vehicle autonomous driving, and the lane line intersection detection method of the present application can be applied.

[0066] In autonomous driving technology, lane detection is a crucial foundation for various advanced driver assistance systems and autonomous driving systems, enabling functions such as lane keeping and vehicle positioning. On conventional roads, lanes are approximately parallel. However, on highways and at merging ramps, lanes merge and diverge, causing them to intersect. Accurately identifying lane intersections improves understanding of lane topology, enhances the stability and accuracy of lane detection, and provides a basis for decision-making in downstream modules, thereby enhancing driving safety.

[0067] Lane intersection detection is usually combined with high-precision maps. That is, the vehicle driving system first obtains the vehicle's current positioning information through a high-precision positioning module such as GPS, and then combines it with the high-precision map module set in the driving system to determine whether there is an intersection in the lane line in front of the driving direction of the current positioning information.

[0068] For example, Figure 1It is a structural diagram of a vehicle driving system in the prior art, such as Figure 1 As shown, as the vehicle travels, the positioning module 11 in the vehicle driving system 1 will obtain the vehicle's positioning information in real time and send it to the processing module 13. The processing module 13 will obtain the corresponding road information from the map module 12 based on the positioning information to determine whether there is a lane line intersection in front of the vehicle.

[0069] However, this detection method requires loading a high-precision positioning module and a high-precision map module into the vehicle driving system, resulting in high installation costs for the vehicle driving system. In addition, the detection relies on high-precision positioning, and the detection of lane intersections cannot be completed in areas with poor positioning effects, resulting in poor detection stability of the driving system.

[0070] Based on this technical problem, the inventive concept of this application is: how to provide a method for lane intersection detection that can reduce the installation cost of a vehicle driving system and improve the detection stability of the system.

[0071] Specifically, the method can receive an image to be detected sent by a vehicle-mounted camera device, and identify the lane lines in the image to be detected; perform curve fitting on the identified lane lines to obtain corresponding lane line curves; for each lane line curve, determine the lane line calibration width corresponding to each position of the lane line curve; perform feature extraction on the image to be detected based on each lane line curve and the lane line calibration width corresponding to each position of each lane line curve to obtain a corresponding feature image; input the feature image into an intersection detection model, and determine the intersection of the lane lines in the image to be detected based on the result output by the intersection detection model. The method of the present application, after receiving the image to be detected uploaded by the vehicle-mounted camera device, can first identify the lane lines in the image to be detected to determine the approximate area where the lane lines are located in the image. Afterwards, curve fitting can be performed on the identified lane lines to obtain corresponding lane line curves, thereby further determining the exact position of the lane lines. Because lane lines have actual widths, while curves do not, after obtaining the lane line curve, it is necessary to determine the calibrated lane line widths corresponding to each position on the lane line curve. Using the lane line curve and the calibrated lane line widths corresponding to each position on the lane line curve, the corresponding lane line can be fully and accurately represented, thereby determining the exact position of the lane line in the image. After determining the exact position of the lane line in the image, the lane line pixel features at the corresponding position can be extracted to obtain the corresponding feature image. This feature image is then input into the intersection detection model. Based on the output of the intersection detection model, the intersection of the lane lines in the image to be detected can be accurately determined. With this setup, when detecting lane line intersections, the vehicle driving system does not need to load a high-precision positioning module or a high-precision map module. Instead, it only requires an on-board camera to capture images during driving and utilize the intersection detection model to accurately detect intersections. This not only reduces the installation cost of the vehicle driving system but also eliminates reliance on high-precision positioning information, thereby improving the stability of the driving system's intersection detection.

[0072] The following specific embodiments describe in detail the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0073] Figure 2 This is a system architecture diagram of an embodiment of the present application, such as Figure 2As shown, the vehicle driving system 1 does not need to be provided with a positioning module and a mapping module. The on-board camera device 2 captures the road image in front of the vehicle in the driving direction in real time as the vehicle travels, and sends it to the vehicle driving system 1 as the image to be detected. After receiving the image, the vehicle driving system 1 identifies the lane lines in the image to be detected and performs curve fitting on the identified lane lines to obtain the corresponding lane line curve; for each lane line curve, the vehicle driving system determines the lane line calibration width corresponding to each position of the lane line curve; based on each lane line curve and the lane line calibration width corresponding to each position of each lane line curve, the image to be detected is subjected to feature extraction to obtain a corresponding feature image; the feature image is input into the intersection detection model, and based on the result output by the intersection detection model, the intersection of the lane lines in the image to be detected is determined.

[0074] Example 1

[0075] Figure 3 This is a flow chart of a method for detecting lane intersections according to an embodiment of the present application. This embodiment describes the method for detecting lane intersections by taking the vehicle driving system as the execution subject. Figure 3 As shown, the lane intersection detection method may include the following steps:

[0076] S101: Receive an image to be detected sent by a vehicle-mounted camera device, and identify lane lines in the image to be detected.

[0077] In this embodiment, the vehicle-mounted camera device can be a device capable of capturing images, such as a vehicle-mounted camera. The vehicle-mounted camera device can capture images of the road in front of the vehicle's driving direction in real time as the vehicle moves, and upload the captured images as images to be detected to the vehicle driving system.

[0078] In this embodiment, the vehicle driving system receives an image to be detected and can identify lane lines from multiple objects in the image to facilitate the subsequent instantiation of lane lines. The specific lane line recognition process can refer to existing technologies, for example, existing lane line detection algorithms such as LaneNet can be used for recognition, and will not be described in detail here.

[0079] For example, Figure 4 This is a schematic diagram of the lane line recognition process according to an embodiment of the present application. Figure 4 As shown, the image to be detected can be input into a lane line detection network loaded with a lane line detection algorithm to obtain the identified lane lines.

[0080] S102: Perform curve fitting on the identified lane line to obtain a corresponding lane line curve.

[0081] In this embodiment, when performing curve fitting on the identified lane lines, for example, a coordinate system can first be established for the image to be detected based on a preset coordinate system (such as the world coordinate system or the vehicle coordinate system, which can be flexibly configured by those skilled in the art) to obtain the coordinates of each pixel in the image. Subsequently, for each identified lane line, the coordinates of each position corresponding to the lane line can be determined, and curve fitting is performed based on the coordinates of each position to obtain an instantiated lane line curve (a curve represented by a mathematical equation).

[0082] S103: For each lane line curve, determine the lane line calibration width corresponding to each position of the lane line curve.

[0083] In this embodiment, the lane line curve is only a curve expressed in the form of a mathematical equation. It does not have information such as width and can only represent the position information of the lane line. However, in reality, the lane line has width. Therefore, it is also necessary to determine the lane line calibration width corresponding to each position of the lane line curve to more realistically and accurately represent the lane line.

[0084] S104: performing feature extraction on the image to be detected based on each lane line curve and the lane line calibrated width corresponding to each position of each lane line curve to obtain a corresponding feature image.

[0085] In this embodiment, the lane line curve can represent the position information of the lane line. After obtaining the lane line calibration width corresponding to each position of the lane line curve, feature extraction is performed on the image to be detected based on this, and the features corresponding to the lane line can be obtained realistically and accurately, so as to facilitate subsequent intersection recognition.

[0086] S105: Input the feature image into the intersection detection model, and determine the intersection of the lane lines in the image to be detected based on the result output by the intersection detection model.

[0087] In this embodiment, the intersection detection model may be a detection model in the prior art, and no limitation is imposed herein as long as it can perform intersection detection.

[0088] In this embodiment, based on the detection probability output by the intersection detection model, it is possible to determine whether there is an intersection in the lane line in the image to be detected, thereby determining the topological structure of the lane line, so as to output accurate driving prompt information or perform safe autonomous driving.

[0089] In this embodiment, the image to be detected sent by the vehicle-mounted camera device can be received, and the lane lines in the image to be detected can be identified; the identified lane lines can be curve-fitted to obtain the corresponding lane line curve; for each lane line curve, the lane line calibration width corresponding to each position of the lane line curve can be determined; based on each lane line curve and the lane line calibration width corresponding to each position of each lane line curve, features can be extracted from the image to be detected to obtain a corresponding feature image; the feature image can be input into the intersection detection model, and based on the results output by the intersection detection model, the intersection of the lane lines in the image to be detected can be determined. The method of the present application, after receiving the image to be detected uploaded by the vehicle-mounted camera device, can first identify the lane lines in the image to be detected to determine the approximate area where the lane lines are located in the image. Afterwards, curve fitting can be performed on the identified lane lines to obtain the corresponding lane line curve, thereby further determining the exact position of the lane lines. Because lane lines have actual widths, while curves do not, after obtaining the lane line curve, it is necessary to determine the calibrated lane line widths corresponding to each position on the lane line curve. Using the lane line curve and the calibrated lane line widths corresponding to each position on the lane line curve, the corresponding lane line can be fully and accurately represented, thereby determining the exact position of the lane line in the image. After determining the exact position of the lane line in the image, the lane line pixel features at the corresponding position can be extracted to obtain the corresponding feature image. This feature image is then input into the intersection detection model. Based on the output of the intersection detection model, the intersection of the lane lines in the image to be detected can be accurately determined. With this setup, when detecting lane line intersections, the vehicle driving system does not need to load a high-precision positioning module or a high-precision map module. Instead, it only requires an on-board camera to capture images during driving and utilize the intersection detection model to accurately detect intersections. This not only reduces the installation cost of the vehicle driving system but also eliminates reliance on high-precision positioning information, thereby improving the stability of the driving system's intersection detection.

[0090] The following describes in detail the specific implementation method of determining the lane line calibration width corresponding to each position of the lane line curve in step S103 of the above embodiment using Example 2.

[0091] Example 2

[0092] Figure 5 This is a flow chart of a method for detecting lane intersections according to another embodiment of the present application. This embodiment describes the method for detecting lane intersections by taking the vehicle driving system as the execution subject. Figure 5 As shown, the lane intersection detection method may include the following steps:

[0093] S201: Determine the maximum width value and the minimum width value of the lane line corresponding to the lane line curve in the image to be detected.

[0094] In this embodiment, after identifying the lane line in the image to be detected, the maximum width value and the minimum width value of the lane line can be determined. The maximum width value can be the maximum width of the lane line in the horizontal direction, and the minimum width value can be the minimum width of the lane line in the horizontal direction.

[0095] S202: Determine the vanishing point coordinates of the lane line corresponding to the lane line curve in the image to be detected according to a preset coordinate system corresponding to the image to be detected.

[0096] In this embodiment, the preset coordinate system can be flexibly set by those skilled in the art. For example, the preset coordinate system can be a world coordinate system, a vehicle coordinate system, or a default coordinate system of the driving system. No restrictions are imposed here.

[0097] In this embodiment, the vanishing point of a lane line can be the point in the image where the lane line intersects with other lane lines. Typically, multiple lane lines in an image intersect at a single point, known as the vanishing point. After identifying the lane lines in the image to be detected, the vanishing point location can be determined. Using a pre-defined coordinate system, the vanishing point coordinates can be determined.

[0098] For example, Figure 6 This is a schematic diagram of the vanishing point position and width according to an embodiment of the present application. Figure 6 As shown, Figure 6 A in the middle represents the vanishing point where the lane lines intersect. The multiple curves that intersect at position A are the lane line curves obtained by fitting. Figure 6 The width of the intersection of the middle horizontal line and the lane curve is the width of the position corresponding to the horizontal line on the lane line. The maximum value of the width is the maximum width value. Figure 6 The lowest position in the , the minimum width is the minimum width value, Figure 6 The position close to the vanishing point A.

[0099] S203: Determine the lane line calibration width corresponding to each position of the lane line curve according to the maximum width value, the minimum width value, and the vanishing point position coordinates.

[0100] In one possible implementation, step S203 may include determining the lane line calibrated width corresponding to each position of the lane line curve based on the maximum width value, the minimum width value, and the vanishing point coordinates. The determination may include:

[0101] Use the following formula (1) to determine the lane line calibration width corresponding to each position of the lane line curve:

[0102]

[0103] Among them, W iIndicates the lane line calibration width corresponding to the ordinate position of the i-th pixel of the lane line curve, W min Indicates the minimum width of the lane line corresponding to the lane line curve in the image to be detected, W ma x represents the maximum width of the lane line corresponding to the lane line curve in the image to be detected, r i Indicates the vertical coordinate of the i-th pixel in the preset coordinate system, vp y It represents the vertical coordinate of the vanishing point in the preset coordinate system, Height represents the image height of the image to be detected, and γ represents the lane width ratio coefficient.

[0104] In this embodiment, generally speaking, the vertical coordinate corresponding to the vanishing point position is at the highest point in the image, that is, the vertical coordinate corresponding to each position of the lane line curve is smaller than the vertical coordinate corresponding to the vanishing point position. In this case, the lane line calibration width corresponding to each position in this situation can be obtained based on the vertical coordinate corresponding to each position and the corresponding calculation formula (12). However, there may be special cases, such as a downhill road, in which case the vertical coordinate corresponding to the vanishing point position may be smaller than the vertical coordinate corresponding to some positions. In this case, the lane line calibration width corresponding to each position in this situation can be obtained according to the calculation formula (11). Through this setting, the size relationship between the vertical coordinate corresponding to the vanishing point position and the vertical coordinates corresponding to each position can be used to obtain the lane line calibration width corresponding to each position using different calculation methods, so that the lane line calibration width is more consistent with the actual situation and the accuracy of the lane line calibration width calculation is improved.

[0105] In this embodiment, since lane lines have actual widths, while curves do not, in order to subsequently extract the true features of the lane lines, after obtaining the lane line curve, it is necessary to determine the lane line calibrated widths corresponding to each position on the lane line curve. Due to the characteristics of the field of view in an image, lane lines that are originally of uniform width will appear to gradually narrow in the image, and multiple lane lines will intersect at vanishing points. Therefore, based on the maximum and minimum widths of the lane lines in the image to be detected, as well as the coordinates of the vanishing points of the lane lines, the lane line calibrated widths corresponding to each position on the lane line curve can be accurately determined.

[0106] The following is a detailed description of the specific implementation method of performing feature extraction on the image to be detected to obtain the corresponding feature image based on each lane line curve and the lane line calibration width corresponding to each position of each lane line curve in step S104 of the above embodiment.

[0107] Example 3

[0108] Figure 7This is a flow chart of a method for detecting lane intersections according to another embodiment of the present application. This embodiment describes the method for detecting lane intersections by taking the vehicle driving system as the execution subject. Figure 7 As shown, the lane intersection detection method may include the following steps:

[0109] S301: Determine the RGB image corresponding to the image to be detected.

[0110] In this embodiment, the existing image processing technology can be used to convert and obtain the RGB image corresponding to the image to be detected, and the relevant existing technology will not be described in detail here.

[0111] S302: For each lane line curve, extract corresponding lane line pixel features from the RGB image based on the position of the lane line curve and the lane line calibration width corresponding to each position of the lane line curve.

[0112] In this embodiment, the lane line pixel features may be the positions of the lane line curve and the pixel points corresponding to the lane line calibrated width corresponding to each position.

[0113] S303: Generate a feature image corresponding to the image to be detected based on the lane line pixel features corresponding to each lane line curve.

[0114] In one possible embodiment, before the above-mentioned step S303 generates a feature image corresponding to the image to be detected based on the lane line pixel features corresponding to each lane line curve, it may also include: scaling the lane line pixel features corresponding to each lane line curve according to a preset width value to obtain scaled pixel features; and splicing the scaled pixel features according to the row correspondence to obtain the target lane line pixel features corresponding to each lane line curve.

[0115] Accordingly, the above-mentioned step S304 generates a feature image corresponding to the image to be detected according to the lane line pixel features corresponding to each lane line curve, which may include: generating a feature image corresponding to the image to be detected according to the target lane line pixel features corresponding to each lane line curve.

[0116] In this embodiment, the preset width value can be flexibly set by those skilled in the art according to actual conditions, and no limitation is imposed here. For example, the preset width value can be set according to the proportional relationship between the actual width of the lane line and the image in real situations.

[0117] In this embodiment, after obtaining the lane line pixel features corresponding to the lane line curve based on the position of the lane line curve and the calibrated lane line width corresponding to each position of the lane line curve, the lane line pixel features are the features of the lane line in the image, which are different from the lane line features in the actual situation. For example, the width of the lane line in the actual situation is consistent. Therefore, in order to make the lane line pixel features more consistent with the actual situation and better represent the real features, the lane line pixel features need to be scaled to a fixed width and spliced ​​to obtain the target lane line pixel features. Through this setting, the target lane line pixel features can be made more consistent with the actual lane line situation, making the feature image generated based on this more realistic and accurate, and achieving alignment of the data feature dimensions.

[0118] In this embodiment, the lane line corresponding to the lane line curve and the lane line calibration width corresponding to each position of the lane line curve can be used to fully and accurately characterize the lane line, thereby determining the exact position of the lane line in the image. After determining the exact position of the lane line in the image, the lane line pixel features (i.e., pixels) at the corresponding position can be extracted accordingly to obtain the lane line pixel features corresponding to the lane line curve. Based on the lane line pixel features corresponding to each lane line curve, a feature image corresponding to the image to be detected can be generated, and the feature image can include comprehensive and real features of the lane line. Through such a setting, the feature image corresponding to the image to be detected can be accurately obtained by using each lane line curve and the lane line calibration width corresponding to each position of each lane line curve, so as to facilitate the subsequent detection of lane line intersections, improve the detection accuracy, and greatly reduce false detections caused by interference from information such as ground text.

[0119] The following fourth embodiment describes in detail the specific implementation method of inputting the feature image into the intersection detection model in step S105 of the above embodiment and determining the intersection of the lane lines in the image to be detected based on the results output by the intersection detection model.

[0120] Example 4

[0121] Figure 8 This is a flowchart of a method for detecting lane intersections provided in yet another embodiment of the present application. This embodiment illustrates the method for detecting lane intersections with a vehicle driving system as the executing entity.

[0122] like Figure 8 As shown, the lane intersection detection method may include the following steps:

[0123] S401: Inputting the feature image into an intersection detection model to obtain detection results of a preset number of regions corresponding to the feature image.

[0124] In this embodiment, the intersection detection model can be a model for detecting lane intersections in the prior art, and is not limited here. Preferably, the intersection detection model can be designed using a classification network structure to divide the feature image into multiple bins (regions), and the classification result of each bin is used to determine whether there is a bifurcation point in the region.

[0125] In this embodiment, those skilled in the art may flexibly set the preset number of regions corresponding to the characteristic image. For example, the characteristic image may be divided into 8 regions or 16 regions, and no limitation is imposed herein.

[0126] S402: Determine the intersection of lane lines in the image to be detected based on the detection results of a preset number of areas, and output the intersection of lane lines in the image to be detected to a downstream module.

[0127] In this embodiment, the detection results for a single region can be used to determine whether there are any intersections in that region. By combining the detection results for all regions, it is possible to determine whether there are any lane intersections in the image to be detected, as well as their specific locations, number, and other information. After determining the lane intersections in the image to be detected, this information can be directly output to downstream modules.

[0128] In one possible embodiment, the above-mentioned detection results may include the probability of existence of intersections and the probabilities of each intersection type. Accordingly, the above-mentioned step S402 of determining the intersections of lane lines in the image to be detected based on the detection results of a preset number of areas may include: for each area, determining whether there is a lane line intersection in the area based on the probability of existence of the intersection corresponding to the area; if there is a lane line intersection in the area, determining the intersection type corresponding to the lane line intersection based on the probabilities of each intersection type corresponding to the lane line intersection; determining the intersection of lane lines in the image to be detected based on the lane line intersection corresponding to each area and the intersection type corresponding to the lane line intersection; wherein the intersection types include: right-side outgoing type, right-side incoming type, left-side outgoing type, and left-side incoming type.

[0129] For example, Figure 9 This is a schematic diagram of the intersection detection result of an embodiment of the present application, as shown in FIG. Figure 9 As shown, the feature image can be input into the intersection detection model. The size of the feature image can be 128×32. The intersection detection model undergoes 4× downsampling, 8× downsampling, 16× downsampling and 16×32 full connection in sequence, and outputs the detection results of each area respectively. The detection result can be an X×6 matrix, where X represents the number of areas, the first two columns are the probability of the existence of the intersection (the probability of yes and the probability of no), and the last four columns are the probabilities of the four intersection types.

[0130] In this embodiment, the detection results can include the probability of intersection existence and the probability of each intersection type. When outputting the detection results for a region, not only can the presence of a lane intersection in the region be determined based on the intersection probability, but the specific intersection type can also be determined based on the probability of each intersection type. This approach can provide more comprehensive and accurate information about lane intersections in the image to be detected.

[0131] In this embodiment, a multi-region classification intersection detection model can be used to perform intersection detection on the feature image. The classification results of each region are used to determine whether there are bifurcations in that region. By combining the detection results of all regions, it is possible to determine whether there are lane intersections in the image to be detected, as well as the specific location and number of intersections. Using this multi-region classification intersection detection model effectively simplifies the output dimensionality. End-to-end training significantly streamlines the detection process, greatly reducing the network convergence speed and complexity, thereby reducing the computing resource requirements and improving the efficiency of the algorithm.

[0132] The following describes a specific embodiment of the lane intersection detection method of the present application.

[0133] Example 5

[0134] In a specific embodiment, a driver drives a vehicle on a highway under the real-time guidance of a vehicle driving system. To generate a road topology to guide the driver to drive safely, the vehicle driving system detects lane intersections on the highway in real time. The specific lane intersection detection process is as follows:

[0135] In the first step, the on-board camera captures the road image in front of the vehicle in real time as the vehicle moves, and sends it to the vehicle driving system as the image to be detected.

[0136] In the second step, after receiving the image, the vehicle driving system identifies the lane lines in the image to be detected and performs curve fitting on the identified lane lines to obtain the corresponding lane line curve.

[0137] In the third step, for each lane line curve, the vehicle driving system determines the lane line calibration width corresponding to each position of the lane line curve.

[0138] In the fourth step, the vehicle driving system extracts features from the image to be detected based on each lane line curve and the lane line calibration width corresponding to each position of each lane line curve to obtain a corresponding feature image.

[0139] In the fifth step, the vehicle driving system inputs the feature image into the intersection detection model to obtain detection results of a preset number of areas corresponding to the feature image.

[0140] In the sixth step, the vehicle driving system determines the intersection of the lane lines in the image to be detected based on the detection results of a preset number of areas, and outputs the intersection of the lane lines in the image to be detected to the display interface of the vehicle terminal.

[0141] Figure 10 This is a structural diagram of a vehicle driving system according to an embodiment of the present application. Figure 10 As shown, the vehicle driving system includes: a receiving module 11 for receiving an image to be detected sent by a vehicle-mounted camera device and identifying lane lines in the image to be detected; a processing module 12 for performing curve fitting on the identified lane lines to obtain corresponding lane line curves; for each lane line curve, determining the lane line calibration width corresponding to each position of the lane line curve; based on each lane line curve and the lane line calibration width corresponding to each position of each lane line curve, performing feature extraction on the image to be detected to obtain a corresponding feature image; inputting the feature image into an intersection detection model, and determining the intersection of the lane lines in the image to be detected based on the results output by the intersection detection model. In one embodiment, the description of the specific functions implemented by the vehicle driving system can be found in the various method steps in Examples 1 to 4, and will not be repeated here.

[0142] Figure 11 This is a structural diagram of a vehicle driving system according to an embodiment of the present application. Figure 11 As shown, the vehicle driving system includes: a processor 101, and a memory 102 communicatively connected to the processor 101; the memory 102 stores computer-executable instructions; the processor 101 executes the computer-executable instructions stored in the memory 102 to implement the steps of the lane intersection detection method in the above-mentioned method embodiments.

[0143] In the above-mentioned vehicle driving system, the memory 102 and the processor 101 are electrically connected directly or indirectly to realize data transmission or interaction. For example, these elements can be electrically connected to each other via one or more communication buses or signal lines, such as a bus connection. The memory 102 stores computer-executable instructions for implementing the data access control method, including at least one software function module that can be stored in the memory 102 in the form of software or firmware. The processor 101 executes various functional applications and data processing by running the software programs and modules stored in the memory 102.

[0144] The memory 102 may be, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc. The memory 102 is used to store programs, and the processor 101 executes the programs after receiving execution instructions. Furthermore, the software programs and modules in the memory 102 may also include an operating system, which may include various software components and / or drivers for managing system tasks (e.g., memory management, storage device control, power management, etc.), and may communicate with various hardware or software components to provide an operating environment for other software components.

[0145] The processor 101 can be an integrated circuit chip with signal processing capabilities. The processor 101 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor.

[0146] An embodiment of the present application further provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the steps of each method embodiment of the present application.

[0147] An embodiment of the present application further provides a computer program product, including a computer program, which implements the steps of each method embodiment of the present application when executed by a processor.

[0148] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of this application and include common knowledge or customary techniques in the art that are not disclosed herein.

[0149] It should be understood that the present application is not limited to the exact structure described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for detecting lane intersections, characterized in that: include: Receiving an image to be detected sent by a vehicle-mounted camera device, and identifying lane lines in the image to be detected; Perform curve fitting on the identified lane line to obtain the corresponding lane line curve; For each lane line curve, determining the lane line calibration width corresponding to each position of the lane line curve; According to each lane line curve and the lane line calibration width corresponding to each position of each lane line curve, feature extraction is performed on the image to be detected to obtain a corresponding feature image; Inputting the feature image into an intersection detection model to obtain detection results of a preset number of areas corresponding to the feature image; Determining the intersection of the lane lines in the image to be detected according to the detection results of the preset number of areas; The detection result includes the probability of existence of the intersection and the probability of each intersection type. Accordingly, the determining the intersection of the lane lines in the image to be detected according to the detection results of the preset number of areas specifically includes: For each of the regions, Determining whether there is a lane line intersection in the area according to the probability of existence of the intersection corresponding to the area; If there is a lane line intersection in the area, determining the intersection type corresponding to the lane line intersection according to the probability of each intersection type corresponding to the lane line intersection; Determine the intersection of lane lines in the image to be detected according to the lane line intersection corresponding to each of the regions and the intersection type corresponding to the lane line intersection; The intersection types include: right-side export type, right-side import type, left-side export type, and left-side import type.

2. The method according to claim 1, characterized in that: The determining of the lane line calibration width corresponding to each position of the lane line curve specifically includes: Determine the maximum width value and the minimum width value of the lane line corresponding to the lane line curve in the image to be detected; Determine the vanishing point position coordinates of the lane line corresponding to the lane line curve in the image to be detected according to a preset coordinate system corresponding to the image to be detected; The lane line calibration width corresponding to each position of the lane line curve is determined according to the maximum width value and the minimum width value and the vanishing point position coordinates.

3. The method according to claim 2, characterized in that Determining the lane line calibration width corresponding to each position of the lane line curve according to the maximum width value and the minimum width value and the vanishing point position coordinates specifically includes: The lane line calibration width corresponding to each position of the lane line curve is determined using the following formula: Among them, the W i represents the lane line calibration width corresponding to the ordinate position of the i-th pixel of the lane line curve, and W min represents the minimum width value of the lane line corresponding to the lane line curve in the image to be detected, and the W max represents the maximum width value of the lane line corresponding to the lane line curve in the image to be detected, and the r i represents the ordinate of the i-th pixel in the preset coordinate system, and the vp y represents the ordinate corresponding to the vanishing point position in the preset coordinate system, the Height represents the image height corresponding to the image to be detected, and the γ represents the lane width ratio coefficient.

4. The method according to claim 3, characterized in that The step of extracting features from the image to be detected according to each lane line curve and the lane line calibration width corresponding to each position of each lane line curve to obtain a corresponding feature image specifically includes: Determine the RGB image corresponding to the image to be detected; For each lane line curve, extracting corresponding lane line pixel features in the RGB image according to the position of the lane line curve and the lane line calibration width corresponding to each position of the lane line curve; A feature image corresponding to the image to be detected is generated according to the lane line pixel features corresponding to each lane line curve.

5. The method according to claim 4, characterized in that Before generating the feature image corresponding to the image to be detected according to the lane line pixel features corresponding to each lane line curve, the method further includes: Scaling the lane line pixel features corresponding to each lane line curve according to a preset width value to obtain scaled pixel features; splicing the scaled pixel features according to row correspondence to obtain a target lane line pixel feature corresponding to each lane line curve; Accordingly, generating a feature image corresponding to the image to be detected according to the lane line pixel features corresponding to each lane line curve includes: A feature image corresponding to the image to be detected is generated according to the target lane line pixel features corresponding to each lane line curve.

6. The method according to any one of claims 1 to 5, characterized in that The method further includes: outputting the intersection of the lane lines in the image to be detected to a downstream module.

7. A vehicle driving system, characterized in that: include: A receiving module, used to receive the image to be detected sent by the vehicle-mounted camera device, and identify the lane line in the image to be detected; A processing module, configured to perform curve fitting on the identified lane lines to obtain corresponding lane line curves; for each lane line curve, determine the lane line calibration width corresponding to each position of the lane line curve; perform feature extraction on the image to be detected according to each lane line curve and the lane line calibration width corresponding to each position of each lane line curve to obtain a corresponding feature image; input the feature image into an intersection detection model, and determine the intersection of the lane lines in the image to be detected according to the result output by the intersection detection model; The processing module is specifically used for: Inputting the feature image into an intersection detection model to obtain detection results of a preset number of regions corresponding to the feature image, wherein the detection results include the probability of existence of an intersection and the probability of each intersection type; For each of the regions, Determining whether there is a lane line intersection in the area according to the probability of existence of the intersection corresponding to the area; If there is a lane line intersection in the area, determining the intersection type corresponding to the lane line intersection according to the probability of each intersection type corresponding to the lane line intersection; Determine the intersection of lane lines in the image to be detected according to the lane line intersection corresponding to each of the regions and the intersection type corresponding to the lane line intersection; The intersection types include: right-side export type, right-side import type, left-side export type, and left-side import type.

8. A vehicle driving system, characterized in that: comprising a processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 6 when executed by a processor.

10. A computer program product, characterized in that The computer program product stores a computer program, and when the computer program is executed by a processor, the computer program is used to implement the method according to any one of claims 1 to 6.

Images