Lane line detection method and device, vehicle-mounted terminal and medium

Abstract

The invention is suitable for the technical field of automobiles, and provides a lane line detection method and device, a vehicle-mounted terminal and a computer readable storage medium, and the method comprises the steps: obtaining the point cloud data and road information of a target road, and an image which comprises a lane line and is collected when a vehicle runs on the target road; inputting the image into a trained lane line detection model for processing to obtain first lane line information of the target road; determining second lane line information of the target road according to the point cloud data and the road information; and determining a lane line detection result of the target road according to the first lane line information and the second lane line information. Compared with the prior art that the lane line is detected only through a deep learning mode, the method has the advantages that after the first lane line information detected by the lane line detection model is obtained, the point cloud data and the second lane line information determined by the road information are combined, so that the final lane line detection result is determined; therefore, the detection accuracy of the lane line is improved.

Description

Technical Field

[0001] The present application belongs to the field of automobile technology, and in particular relates to a lane line detection method, device, vehicle-mounted terminal and medium. Background Art

[0002] In the autonomous driving perception system, in order to ensure the normal operation of the entire vehicle functions, it is necessary to identify and understand the road environment and detect lane lines and road edges.

[0003] When a vehicle is driving, the existing technology usually detects lane lines in real time based on deep learning, that is, the collected images containing lane lines are input into the deep learning model for processing to obtain lane line information. However, the existing technology is not ideal in some specific scenarios. The camera is a passive sensor and has weak anti-interference ability against light sources and shadows. For example, in the dark, when the light changes when entering and exiting a tunnel, and in the morning and evening, the detection of lane lines is prone to errors, which reduces the accuracy of lane line detection. Summary of the invention

[0004] The embodiments of the present application provide a lane line detection method, device, vehicle-mounted terminal and medium, which improve the detection accuracy of lane lines.

[0005] In a first aspect, an embodiment of the present application provides a lane line detection method, including:

[0006] Acquire point cloud data of a target road, road information, and an image containing lane lines collected when a vehicle is traveling on the target road;

[0007] Inputting the image into a trained lane line detection model for processing to obtain first lane line information of the target road;

[0008] Determining second lane line information of the target road according to the point cloud data and the road information;

[0009] A lane line detection result of the target road is determined according to the first lane line information and the second lane line information.

[0010] Optionally, the point cloud data includes height information of each frame of point cloud; and determining the second lane line information of the target road according to the point cloud data and the road information includes:

[0011] Determining a road surface point cloud of the target road according to the height information of each frame of the point cloud and the road information;

[0012] The second lane line information is determined according to the road surface point cloud and the road information.

[0013] Optionally, the road information includes a curb feature of the target road, a lane line point cloud reflectivity, and a ground reflectivity; and determining the second lane line information according to the road surface point cloud and the road information includes:

[0014] Determining a road boundary of the target road according to the roadside feature;

[0015] The lane line point cloud of the target road is determined according to the road surface point cloud, the road boundary, and the difference information between the lane line point cloud reflectivity and the ground reflectivity to obtain the second lane line information.

[0016] Optionally, the first lane line information refers to first coordinate information of a lane line in the target road in a vehicle coordinate system; the second lane line information refers to second coordinate information of a lane line in the target road in a global coordinate system; and determining a lane line detection result of the target road according to the first lane line information and the second lane line information includes:

[0017] Performing coordinate conversion on the second coordinate information to obtain third coordinate information corresponding to the second coordinate information in the vehicle coordinate system;

[0018] The first coordinate information and the third coordinate information are matched, and the lane line detection result is determined according to the matching result.

[0019] Optionally, performing coordinate conversion on the second coordinate information to obtain third coordinate information corresponding to the second coordinate information in the vehicle coordinate system includes:

[0020] Acquire the position information of the vehicle at the shooting time corresponding to the image;

[0021] Determining a driving direction of the vehicle according to the position information;

[0022] Determine the detection length of the lane line according to the first coordinate information;

[0023] Determine a lane line detection range of the target road according to the driving direction and the detection length;

[0024] Coordinate transformation is performed on fourth coordinate information in the second coordinate information that is within the lane line detection range to obtain the third coordinate information.

[0025] Optionally, the first lane line information includes coordinate information of a plurality of lane lines, and the second lane line information includes coordinate information of a plurality of lane lines; and determining the lane line detection result of the target road according to the first lane line information and the second lane line information includes:

[0026] If the coordinate information of at least one lane line in the coordinate information of the multiple lane lines included in the first lane line information fails to match the coordinate information of the multiple lane lines included in the second lane line information, then determining that the lane line detection result is a false detection;

[0027] If the coordinate information of at least one lane line in the coordinate information of the multiple lane lines included in the second lane line information fails to match the coordinate information of the multiple lane lines included in the first lane line information, then determining that the lane line detection result is a missed detection;

[0028] If the coordinate information of the multiple lane lines included in the first lane line information matches the coordinate information of the multiple lane lines included in the second lane line information successfully one by one, the lateral errors between the coordinate information of the successfully matched lane lines in the first lane line information and the second lane line information are calculated respectively, and the lane line detection result is determined based on the lateral errors.

[0029] Optionally, determining the lane line detection result according to the lateral error includes:

[0030] If the lateral error is greater than or equal to a set threshold, adjusting the coordinate information of the multiple lane lines included in the first lane line information according to the coordinate information of the multiple lane lines included in the second lane line information to obtain the lane line detection result;

[0031] If the lateral error is less than the set threshold, the coordinate information of the multiple lane lines included in the first lane line information is determined as the lane line detection result.

[0032] In a second aspect, an embodiment of the present application provides a lane line detection device, including:

[0033] A first acquisition unit, used to acquire point cloud data of a target road, road information, and an image containing lane lines collected when a vehicle travels on the target road;

[0034] A processing unit, used for inputting the image into a trained lane line detection model for processing to obtain first lane line information of the target road;

[0035] A first information determining unit, configured to determine second lane line information of the target road according to the point cloud data and the road information;

[0036] The first result determination unit is used to determine the lane line detection result of the target road according to the first lane line information and the second lane line information.

[0037] In a third aspect, an embodiment of the present application provides a vehicle-mounted terminal, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the lane line detection method as described in any one of the first aspects above is implemented.

[0038] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the lane line detection method as described in any one of the above-mentioned first aspects is implemented.

[0039] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on a vehicle-mounted terminal, enables the vehicle-mounted terminal to execute the lane line detection method described in any one of the above-mentioned first aspects.

[0040] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0041] The embodiment of the present application provides a lane line detection method, which obtains point cloud data of the target road, road information, and an image containing lane lines collected when the vehicle is driving on the target road; inputs the image into a trained lane line detection model for processing to obtain the first lane line information of the target road; determines the second lane line information of the target road based on the point cloud data and road information; and determines the lane line detection result of the target road based on the first lane line information and the second lane line information. Compared with the prior art that detects lane lines only by deep learning, the method of the present application, after obtaining the first lane line information detected by the lane line detection model, will also determine the second lane line information by combining the point cloud data and the road information. Since the laser radar has a stronger anti-interference ability than the camera, it can collect accurate point cloud data in scenes such as night and in and out of tunnels, so that the lane line detection result finally determined based on the first lane line information and the second lane line information is more accurate, thereby improving the detection accuracy of the lane line. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0043] Figure 1 is a flow chart of an implementation of a lane line detection method provided in an embodiment of the present application;

[0044] Figure 2is a flow chart of an implementation of a lane line detection method provided by another embodiment of the present application;

[0045] Figure 3 is a flow chart of an implementation of a lane line detection method provided in yet another embodiment of the present application;

[0046] Figure 4 is a flow chart of an implementation of a lane line detection method provided by another embodiment of the present application;

[0047] Figure 5 is a flow chart of an implementation of a lane line detection method provided by another embodiment of the present application;

[0048] Figure 6 is a structural schematic diagram of a lane line detection device provided in an embodiment of the present application;

[0049] Figure 7 It is a structural diagram of a vehicle-mounted terminal provided in one embodiment of the present application. DETAILED DESCRIPTION

[0050] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0051] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0052] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0053] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0054] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0055] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0056] See also Figure 1 , Figure 1 This is a flow chart of a lane line detection method provided in an embodiment of the present application. In the embodiment of the present application, the execution subject of the lane line detection method is a vehicle-mounted terminal.

[0057] like Figure 1 As shown, the lane line detection method provided in an embodiment of the present application may include S101 to S104, which are described in detail as follows:

[0058] In S101, point cloud data of a target road, road information, and an image containing lane lines collected when a vehicle is traveling on the target road are obtained.

[0059] In an embodiment of the present application, when a vehicle is traveling on a target road, in order to improve the detection accuracy of lane lines on the target road, the vehicle-mounted terminal can obtain in real time point cloud data of the target road, road information, and images containing lane lines collected when the vehicle is traveling on the target road.

[0060] In one implementation of the embodiment of the present application, the vehicle-mounted terminal is provided with a laser radar, and the laser radar is used to collect point cloud data of the target road, wherein the point cloud data includes lane line point cloud data of the target road.

[0061] It should be noted that after the vehicle terminal obtains the above point cloud data through the laser radar, it can classify and store the obtained point cloud data according to the collection time and set time carried by each set of point cloud data. Among them, the set time can be determined according to actual needs and is not limited here.

[0062] For example, assuming that the collection time of the first set of point cloud data is 12:00:00 and the set duration is 5 minutes, the vehicle-mounted terminal can classify the point cloud data with a collection time between 12:00:00 and 12:05:00 into one category for storage.

[0063] Alternatively, after the vehicle terminal obtains the above point cloud data through the laser radar, it can classify and store the obtained point cloud data according to the coordinate information in each group of point cloud data and the set mileage. Among them, the set mileage can be determined according to actual needs and is not limited here.

[0064] Alternatively, after the vehicle terminal obtains the above point cloud data through the laser radar, it can classify and store the obtained point cloud data according to the size of each group of point cloud data and the set capacity of each file. The set capacity can be determined according to actual needs and is not limited here.

[0065] In another implementation of the embodiment of the present application, the vehicle terminal is provided with a vehicle camera, and the vehicle camera can shoot the road within its shooting range during the driving of the vehicle to obtain a road video of the target road.

[0066] Based on this, the vehicle-mounted terminal can obtain the point cloud data of the target road in real time through the above-mentioned laser radar, and obtain the above-mentioned road video in real time through the above-mentioned camera device.

[0067] In an embodiment of the present application, the vehicle-mounted terminal can perform frame processing on the above-mentioned road video to obtain multiple frames of video images, and perform feature extraction on the video images to obtain road information of the target road.

[0068] In S102, the image is input into a trained lane line detection model for processing to obtain first lane line information of the target road.

[0069] It should be noted that in the embodiment of the present application, the vehicle-mounted terminal pre-stores a trained lane line detection model.

[0070] The lane line detection model is used to detect lane lines in an image. The lane line detection model may be an existing detection model based on deep learning.

[0071] Based on this, in an embodiment of the present application, the vehicle-mounted terminal can input the above-mentioned image containing the lane line into a trained lane line detection model for processing to obtain the first lane line information.

[0072] In some possible embodiments, in combination with S101, after acquiring the image of the lane line, the vehicle-mounted terminal can input the image into a trained lane line detection model in real time through the vehicle-mounted camera for processing to obtain the first lane line information.

[0073] In some other possible embodiments, in combination with S101, after acquiring the image of the lane line, the vehicle-mounted terminal can input the image into a trained lane line detection model for processing in an offline state through a backend server to obtain the first lane line information.

[0074] It should be noted that the first lane line information is used to describe the first position information of the lane line on the target road, wherein the first position information can be expressed by coordinates or by fitting curve polynomial coefficients (usually a cubic polynomial fitting equation).

[0075] In S103, second lane line information of the target road is determined according to the point cloud data and the road information.

[0076] In the embodiment of the present application, point cloud data refers to a set of vectors formed after the laser radar scans the environment. Each frame includes multiple point clouds, and each point cloud contains a three-dimensional coordinate and reflection intensity information.

[0077] It should be noted that the road information includes but is not limited to the curb features of the target road, the lane line point cloud reflectivity, and the ground reflectivity.

[0078] Combined with S101, since the point cloud data has been stored in segments, the vehicle-mounted terminal can process each frame of the stored point cloud data in turn through the laser radar in an offline state, and combine it with the road information to obtain multiple lane line information, and integrate the multiple lane line information to obtain the second lane line information of the target road.

[0079] Based on this, in one embodiment of the present application, the vehicle-mounted terminal can specifically be Figure 2 Steps S201 to S202 shown in the figure obtain the second lane line information of the target road, which are described in detail as follows:

[0080] In S201, a road surface point cloud of the target road is determined according to the height information of each frame of the point cloud and the road information.

[0081] In this embodiment, since the point cloud data includes multiple point clouds and each point cloud contains a three-dimensional coordinate, the vehicle-mounted terminal can obtain the road surface point cloud of the target road in the point cloud data based on the height information in the three-dimensional coordinates in each point cloud and the ground reflectivity and curb features in the road information.

[0082] In S202, the second lane line information is determined based on the road surface point cloud and the road information.

[0083] In this embodiment, after obtaining the road surface point cloud, the vehicle-mounted terminal can determine multiple lane line point clouds in the road surface point cloud based on the road surface point cloud and road information, so as to obtain the second lane line information of the target road based on the lane line point clouds.

[0084] In one embodiment of the present application, the vehicle-mounted terminal may obtain the second lane line information of the target road through the following steps, which are described in detail as follows:

[0085] Determining a road boundary of the target road according to the roadside feature;

[0086] The lane line point cloud of the target road is determined according to the difference information among the road surface point cloud, the road boundary, the lane line point cloud reflectivity and the ground point cloud reflectivity to obtain the second lane line information.

[0087] In this embodiment, in order to improve the accuracy of determining the second lane line information, the vehicle-mounted terminal may first determine the road boundary of the target road according to the curb features.

[0088] Since the reflectivities received on different objects scanned by the laser radar are different, and there is an obvious difference in reflectivity between the lane line and the ground, in this embodiment, the vehicle-mounted terminal can determine multiple lane line point clouds of the target road from the road surface point cloud based on the road surface point cloud, road boundaries, and the difference information between the lane line point cloud reflectivity and the ground reflectivity, and obtain the second lane line information of the target road based on these lane line point clouds.

[0089] It should be noted that the above difference information can be the difference between the lane line point cloud reflectivity and the ground reflectivity, or the difference information between the lane line point cloud reflectivity and the ground reflectivity, which is not limited here.

[0090] In S104, a lane line detection result of the target road is determined according to the first lane line information and the second lane line information.

[0091] In the embodiment of the present application, since the second lane line information is obtained from the point cloud data of the target road, that is, the second lane line information is ground truth (GT) data, the vehicle-mounted terminal can evaluate the first lane line information obtained by the vehicle-mounted terminal through the lane line detection model through the second lane line information to determine the lane line detection result of the target road, that is, to evaluate the accuracy of the first lane line information obtained by the vehicle-mounted terminal through the lane line detection model.

[0092] It should be noted that since the lane line point cloud data is processed offline, in order to improve the detection accuracy, multiple frames of point clouds can be dedistorted and spliced ​​to obtain point cloud data in the mapping coordinate system, that is, point cloud data in the global coordinate system. Therefore, the second lane line information of the target road is the second coordinate information of the lane line in the target road in the global coordinate system.

[0093] Since the image containing the lane line acquired by the vehicle-mounted terminal is captured by the vehicle-mounted camera and subjected to single-frame image detection processing, the first lane line information of the target road is the first coordinate information in the vehicle coordinate system.

[0094] Based on this, in one embodiment of the present application, the vehicle-mounted terminal can specifically be Figure 3 S301 to S302 shown in the figure obtain the lane line detection result, which is described in detail as follows:

[0095] In S301, coordinate transformation is performed on the second coordinate information to obtain third coordinate information corresponding to the second coordinate information in the vehicle coordinate system.

[0096] In one embodiment of the present application, since the vehicle-mounted camera can only capture images of lane lines within its shooting range, the first lane line information is only information about a portion of the lane lines of the target road, and the point cloud data is point cloud data of the complete target road, so that the second lane line information is information about the complete lane lines of the complete target road. Therefore, in order to improve the processing efficiency of the vehicle-mounted terminal and reduce the amount of calculation, the vehicle-mounted terminal can specifically perform the following steps: Figure 4 S401 to S405 shown in the figure obtain the third coordinate information corresponding to the second coordinate information in the vehicle coordinate system, which is described in detail as follows:

[0097] In S401, the position information of the vehicle at the shooting time corresponding to the image is obtained.

[0098] It should be noted that when the vehicle-mounted terminal obtains an image containing a lane line, the image carries its shooting time, so the vehicle-mounted terminal can determine the shooting time of the image.

[0099] In this embodiment, since the vehicle-mounted terminal stores the driving trajectory of the vehicle in real time, the vehicle-mounted terminal can determine the position information of the vehicle at the shooting time corresponding to the above image based on the driving trajectory.

[0100] In S402, the driving direction of the vehicle is determined according to the position information.

[0101] In this embodiment, since the posture information includes posture and position, the vehicle-mounted terminal can determine the body direction of the vehicle according to the posture information, and further determine the specific driving direction of the vehicle.

[0102] In S403, the detection length of the lane line is determined according to the first coordinate information.

[0103] In this embodiment, the first coordinate information is used to describe the position of the lane line in the image acquired by the vehicle-mounted terminal, so that the vehicle-mounted terminal can determine the length of the lane line in the above image based on the first coordinate information, and thus can determine the detection length of the lane line of the target road that can be captured by the vehicle-mounted camera based on the length.

[0104] In S404, a lane line detection range of the target road is determined according to the driving direction and the detection length.

[0105] In S405, coordinate transformation is performed on fourth coordinate information in the second coordinate information that is within the lane line detection range to obtain the third coordinate information.

[0106] In this embodiment, in order to improve the processing efficiency of the vehicle-mounted terminal and reduce the amount of calculation, the vehicle-mounted terminal may only perform coordinate conversion on the fourth coordinate information in the second coordinate information that is within the above-mentioned lane line detection range, thereby obtaining the third coordinate information in the vehicle coordinate system, whose detection length is the same as the length of the lane line formed by the first coordinate information.

[0107] In S302, the first coordinate information and the third coordinate information are matched, and the lane line detection result is determined according to the matching result.

[0108] In this embodiment, after obtaining the third coordinate information, the vehicle-mounted terminal may match the first coordinate information with the third coordinate information to determine the lane line detection result.

[0109] Specifically, the vehicle-mounted terminal may first construct a first lane line according to the first coordinate information, and construct a second lane line according to the third coordinate information.

[0110] Afterwards, the vehicle-mounted terminal may detect whether the first lane line and the second lane line match, that is, detect whether the position range of the first lane line and the position range of the second lane line match.

[0111] In one embodiment of the present application, when the vehicle-mounted terminal detects that the difference between the position range of the first lane line and the position range of the second lane line is less than or equal to the first difference threshold, it can be determined that the first lane line matches the second lane line successfully, that is, the first coordinate information and the third coordinate information match successfully. Therefore, the vehicle-mounted terminal can determine the first coordinate information as the lane line detection result.

[0112] In another embodiment of the present application, when the vehicle-mounted terminal detects that the difference between the position range of the first lane line and the position range of the second lane line is greater than the first difference threshold, it can be determined that the first lane line fails to match the second lane line, that is, the first coordinate information fails to match the third coordinate information. Therefore, the vehicle-mounted terminal can adjust the first coordinate information according to the third coordinate information, that is, adjust the horizontal coordinate in the first coordinate information that is the same as the longitudinal coordinate in the third coordinate information to the horizontal coordinate corresponding to the above-mentioned longitudinal coordinate in the third coordinate information, and determine the adjusted first coordinate information as the lane line detection result.

[0113] From the above, it can be seen that the lane line detection method provided by the embodiment of the present application obtains the point cloud data of the target road, the road information and the image containing the lane line collected when the vehicle is driving on the target road; the image is input into the trained lane line detection model for processing to obtain the first lane line information of the target road; the second lane line information of the target road is determined according to the point cloud data and the road information; the lane line detection result of the target road is determined according to the first lane line information and the second lane line information. Compared with the prior art that detects lane lines only by deep learning, the method of the present application will determine the second lane line information in combination with the point cloud data and the road information after obtaining the first lane line information detected by the lane line detection model. Since the laser radar has a stronger anti-interference ability than the camera, it can collect accurate point cloud data in scenes such as night and in and out of tunnels, so that the lane line detection result finally determined according to the first lane line information and the second lane line information is more accurate, thereby improving the detection accuracy of the lane line.

[0114] In practical applications, since a road usually includes multiple lane lines, the first lane line information may include coordinate information of the multiple lane lines, and the second lane line information may include coordinate information of the multiple lane lines.

[0115] In one implementation of the embodiment of the present application, the vehicle-mounted terminal can match the coordinate information of a lane line in the first lane line information whose difference is less than or equal to a second difference threshold with the coordinate information of a lane line in the second lane line information based on the coordinate information of the multiple lane lines included in the first lane line information and the coordinate information of the multiple lane lines included in the second lane line information. Thereafter, the vehicle-mounted terminal can determine the lane line detection result based on the result after the above matching.

[0116] In another implementation method of the embodiment of the present application, the vehicle-mounted terminal can sequentially match the coordinate information of multiple lane lines of the first lane line with the coordinate information of multiple lane lines included in the second lane line information, and then the vehicle-mounted terminal can determine the lane line detection result based on the result after the above matching.

[0117] Based on this, see Figure 5 , Figure 5 is a lane line detection method provided by another embodiment of the present application. Figure 1 Corresponding to the embodiment, S104 in this embodiment may specifically include S501 to S503, which are described in detail as follows:

[0118] In S501, if the coordinate information of at least one lane line among the coordinate information of multiple lane lines included in the first lane line information fails to match the coordinate information of multiple lane lines included in the second lane line information, it is determined that the lane line detection result is a false detection.

[0119] In this embodiment, when the on-board terminal detects that among the coordinate information of multiple lane lines included in the first lane line information, there is at least one lane line whose coordinate information fails to match the coordinate information of multiple lane lines included in the second lane line information, it means that there is a first remaining lane line in the first lane line information that cannot be matched with any lane line in the second lane line information. Since the second lane line information is determined by the point cloud data of the target road, that is, the multiple lane lines in the second lane line information are the real lane lines on the target road, and the first lane line information contains the above-mentioned first remaining lane line, at this time, the on-board terminal can determine that the lane line detection result is a false detection, that is, the above-mentioned first remaining lane line is a redundant lane line. At this time, the on-board terminal can determine the other lane lines in the first lane line information except the above-mentioned first remaining lane line as the target lane lines of the target road.

[0120] In S502, if the coordinate information of at least one lane line in the coordinate information of the multiple lane lines included in the second lane line information fails to match the coordinate information of the multiple lane lines included in the first lane line information, it is determined that the lane line detection result is a missed detection.

[0121] In this embodiment, when the on-board terminal detects that among the coordinate information of multiple lane lines included in the second lane line information, there is at least one lane line whose coordinate information fails to match the coordinate information of multiple lane lines included in the first lane line information, it means that there is a second remaining lane line in the second lane line information that cannot be matched with any lane line in the first lane line information. Since the second lane line information is determined by the point cloud data of the target road, that is, the multiple lane lines in the second lane line information are the real lane lines existing on the target road, and the second remaining lane lines exist in the second lane line information, at this time, the on-board terminal can determine that the lane line detection result is a missed detection, that is, the second remaining lane lines are the lane lines missed by the on-board terminal. At this time, the on-board terminal can determine all lane lines in the first lane line information and the second remaining lane lines in the second lane line information as the target lane lines of the target road.

[0122] In S503, if the coordinate information of the multiple lane lines included in the first lane line information and the coordinate information of the multiple lane lines included in the second lane line information are successfully matched one by one, the lateral errors between the coordinate information of the successfully matched lane lines in the first lane line information and the second lane line information are calculated respectively, and the lane line detection result is determined based on the lateral errors.

[0123] In this embodiment, when the vehicle-mounted terminal detects that the coordinate information of multiple lane lines included in the first lane line information and the coordinate information of multiple lane lines included in the second lane line information are successfully matched one by one, it means that each lane line in the first lane line information has a lane line in the second lane line information that matches it. Therefore, in order to further improve the detection accuracy of the lane lines of the target road, the vehicle-mounted terminal can respectively calculate the lateral error between the coordinate information of the successfully matched lane lines in the first lane line information and the second lane line information, and determine the lane line detection result based on the lateral error.

[0124] Specifically, for any lateral error, the vehicle-mounted terminal may compare the lateral error with a set threshold value, wherein the set threshold value may be set according to actual needs and is not limited here.

[0125] In one embodiment of the present application, when the vehicle-mounted terminal detects that the above-mentioned lateral error is greater than or equal to the set threshold, it means that the coordinate information of a lane line in the first lane line information calculated to obtain the lateral error and the coordinate information of a lane line in the second lane line information that successfully matches it are quite different and do not meet the requirements, that is, the lane line in the first lane line information is quite different from its position in the actual environment. Therefore, the vehicle-mounted terminal may adjust the coordinate information of the lane line in the first lane line information according to the above-mentioned second lane line information including the coordinate information of a lane line that matches it, and determine the multiple lane lines in the adjusted first lane line information as the target lane lines, thereby obtaining the lane line detection result.

[0126] In another embodiment of the present application, when the vehicle-mounted terminal detects that the above-mentioned lateral error is less than the set threshold, it indicates that the coordinate information of a lane line in the first lane line information for calculating the lateral error and the coordinate information of a lane line in the second lane line information that is successfully matched with it are not much different, and meet the requirements, that is, the lane line in the first lane line information is consistent with its position in the actual environment. Therefore, the vehicle-mounted terminal can directly determine multiple lane lines in the first lane line information as target lane lines, that is, obtain the lane line detection result.

[0127] As can be seen from the above, the lane line detection method provided in this embodiment determines that the lane line detection result is a false detection when it is detected that among the coordinate information of multiple lane lines included in the first lane line information, the coordinate information of at least one lane line fails to match the coordinate information of multiple lane lines included in the second lane line information; when it is detected that among the coordinate information of multiple lane lines included in the second lane line information, the coordinate information of at least one lane line fails to match the coordinate information of multiple lane lines included in the first lane line information, the lane line detection result is determined to be a missed detection; when it is detected that the coordinate information of multiple lane lines included in the first lane line information matches the coordinate information of multiple lane lines included in the second lane line information successfully, the lateral error between the coordinate information of the successfully matched lane lines in the first lane line information and the second lane line information is calculated respectively, and the lane line detection result is determined according to the lateral error. The method provided in this embodiment can accurately determine whether the lane line is missed or falsely detected when the above lane line matching fails. At the same time, when it is detected that the above lane lines are successfully matched one by one, the lane line detection result can be further determined according to the lateral error between the coordinate information of the successfully matched lane lines, thereby improving the detection accuracy of the lane line.

[0128] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0129] Corresponding to the lane line detection method described in the above embodiment, Figure 6 A schematic diagram of the structure of a lane line detection device provided in an embodiment of the present application is shown. For the sake of convenience, only the part related to the embodiment of the present application is shown. Figure 6 The lane line detection device 600 includes: a first acquisition unit 61, a processing unit 62, a first information determination unit 63 and a first result determination unit 64. Among them:

[0130] The first acquisition unit 61 is used to acquire point cloud data of a target road, road information, and an image containing lane lines collected when a vehicle travels on the target road.

[0131] The processing unit 62 is used to input the image into a trained lane line detection model for processing to obtain the first lane line information of the target road.

[0132] The first information determining unit 63 is used to determine the second lane line information of the target road according to the point cloud data and the road information.

[0133] The first result determination unit 64 is used to determine the lane line detection result of the target road according to the first lane line information and the second lane line information.

[0134] In one embodiment of the present application, the point cloud data includes height information of each frame of point cloud; the first information determination unit 63 specifically includes: a point cloud determination unit and a second information determination unit. Among them:

[0135] The point cloud determination unit is used to determine the road surface point cloud of the target road according to the height information of each frame of the point cloud and the road information.

[0136] The second information determination unit is used to determine the second lane line information according to the road surface point cloud and the road information.

[0137] In one embodiment of the present application, the road information includes the roadside features, lane line point cloud reflectivity and ground reflectivity of the target road; the second information determination unit specifically includes: a boundary determination unit and a third information determination unit. Among them:

[0138] The boundary determination unit is used to determine the road boundary of the target road according to the roadside feature.

[0139] The third information determination unit is used to determine the lane line point cloud of the target road based on the road surface point cloud, the road boundary, and the difference information between the lane line point cloud reflectivity and the ground reflectivity to obtain the second lane line information.

[0140] In one embodiment of the present application, the first lane line information refers to the first coordinate information of the lane line in the target road in the vehicle coordinate system; the second lane line information refers to the second coordinate information of the lane line in the target road in the global coordinate system; the first result determination unit 64 specifically includes: a first conversion unit and a second result determination unit. Among them:

[0141] The first conversion unit is used to perform coordinate conversion on the second coordinate information to obtain third coordinate information corresponding to the second coordinate information in the vehicle coordinate system.

[0142] The second result determination unit is used to match the first coordinate information and the third coordinate information, and determine the lane line detection result according to the matching result.

[0143] In one embodiment of the present application, the first conversion unit specifically includes: a second acquisition unit, a direction determination unit, a length determination unit, a range determination unit and a second conversion unit.

[0144] The second acquisition unit is used to acquire the position information of the vehicle at the shooting time corresponding to the image.

[0145] The direction determination unit is used to determine the driving direction of the vehicle according to the position information.

[0146] The length determination unit is used to determine the detection length of the lane line according to the first coordinate information.

[0147] The range determination unit is used to determine the lane line detection range of the target road according to the driving direction and the detection length.

[0148] The second conversion unit is used to perform coordinate conversion on fourth coordinate information in the second coordinate information that is within the lane line detection range to obtain the third coordinate information.

[0149] In one embodiment of the present application, the first lane line information includes coordinate information of multiple lane lines, and the second lane line information includes coordinate information of multiple lane lines; the first result determination unit 64 specifically includes: a third result determination unit, a fourth result determination unit, and a fifth result determination unit. Among them:

[0150] The third result determination unit is used to determine that the lane line detection result is a false detection if the coordinate information of at least one lane line among the coordinate information of multiple lane lines included in the first lane line information fails to match the coordinate information of multiple lane lines included in the second lane line information.

[0151] The fourth result determination unit is used to determine that the lane line detection result is a missed detection if the coordinate information of at least one lane line among the coordinate information of the multiple lane lines included in the second lane line information fails to match the coordinate information of the multiple lane lines included in the first lane line information.

[0152] The fifth result determination unit is used to calculate the lateral errors between the coordinate information of the successfully matched lane lines in the first lane line information and the second lane line information respectively if the coordinate information of the multiple lane lines included in the first lane line information matches the coordinate information of the multiple lane lines included in the second lane line information successfully one by one, and determine the lane line detection result based on the lateral errors.

[0153] In one embodiment of the present application, the fifth result determination unit specifically includes: an adjustment unit and a sixth result determination unit. Among them:

[0154] The adjustment unit is used to adjust the coordinate information of multiple lane lines included in the first lane line information according to the coordinate information of multiple lane lines included in the second lane line information if the lateral error is greater than or equal to a set threshold, so as to obtain the lane line detection result.

[0155] The sixth result determination unit is used to determine the coordinate information of multiple lane lines included in the first lane line information as the lane line detection result if the lateral error is less than the set threshold.

[0156] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0157] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0158] Figure 7 This is a schematic diagram of the structure of a vehicle-mounted terminal provided in one embodiment of the present application. Figure 7 As shown, the vehicle terminal 7 of this embodiment includes: at least one processor 70 ( Figure 7 Only one is shown in the figure) a processor, a memory 71, and a computer program 72 stored in the memory 71 and executable on the at least one processor 70, and when the processor 70 executes the computer program 72, the steps in any of the above-mentioned lane line detection method embodiments are implemented.

[0159] The vehicle-mounted terminal may include, but is not limited to, a processor 70 and a memory 71. Those skilled in the art will appreciate that Figure 7 It is only an example of the vehicle-mounted terminal 7 and does not constitute a limitation on the vehicle-mounted terminal 7. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components, for example, it may also include input and output devices, network access devices, etc.

[0160] The processor 70 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0161] In some embodiments, the memory 71 may be an internal storage unit of the vehicle-mounted terminal 7, such as the memory of the vehicle-mounted terminal 7. In other embodiments, the memory 71 may also be an external storage device of the vehicle-mounted terminal 7, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the vehicle-mounted terminal 7. Further, the memory 71 may also include both the internal storage unit of the vehicle-mounted terminal 7 and an external storage device. The memory 71 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program, etc. The memory 71 may also be used to temporarily store data that has been output or is to be output.

[0162] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0163] An embodiment of the present application provides a computer program product. When the computer program product runs on a vehicle-mounted terminal, the vehicle-mounted terminal can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0164] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device capable of carrying the computer program code to the vehicle-mounted terminal, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electric carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0165] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0166] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A lane line detection method, characterized in that: include: Acquire point cloud data of a target road, road information, and an image containing lane lines collected when a vehicle is traveling on the target road; Inputting the image into a trained lane line detection model for processing to obtain first lane line information of the target road; Determining second lane line information of the target road according to the point cloud data and the road information; A lane line detection result of the target road is determined according to the first lane line information and the second lane line information.

2. The lane line detection method according to claim 1, characterized in that: The point cloud data includes height information of each frame of point cloud; and determining the second lane line information of the target road according to the point cloud data and the road information includes: Determining a road surface point cloud of the target road according to the height information of each frame of the point cloud and the road information; The second lane line information is determined according to the road surface point cloud and the road information.

3. The lane line detection method according to claim 2, characterized in that: The road information includes the curb features of the target road, the lane line point cloud reflectivity, and the ground reflectivity; and determining the second lane line information according to the road surface point cloud and the road information includes: Determining a road boundary of the target road according to the roadside feature; The lane line point cloud of the target road is determined according to the road surface point cloud, the road boundary, and the difference information between the lane line point cloud reflectivity and the ground reflectivity to obtain the second lane line information.

4. The lane line detection method according to claim 1, characterized in that: The first lane line information refers to the first coordinate information of the lane line in the target road in the vehicle coordinate system; the second lane line information refers to the second coordinate information of the lane line in the target road in the global coordinate system; The determining, according to the first lane line information and the second lane line information, a lane line detection result of the target road includes: Performing coordinate conversion on the second coordinate information to obtain third coordinate information corresponding to the second coordinate information in the vehicle coordinate system; The first coordinate information and the third coordinate information are matched, and the lane line detection result is determined according to the matching result.

5. The lane line detection method according to claim 4, characterized in that: The performing coordinate conversion on the second coordinate information to obtain third coordinate information corresponding to the second coordinate information in the vehicle coordinate system includes: Acquire the position information of the vehicle at the shooting time corresponding to the image; Determining a driving direction of the vehicle according to the position information; Determine the detection length of the lane line according to the first coordinate information; Determine a lane line detection range of the target road according to the driving direction and the detection length; Coordinate transformation is performed on fourth coordinate information in the second coordinate information that is within the lane line detection range to obtain the third coordinate information.

6. The lane line detection method according to any one of claims 1 to 5, characterized in that: The first lane line information includes coordinate information of a plurality of lane lines, and the second lane line information includes coordinate information of a plurality of lane lines; and determining a lane line detection result of the target road according to the first lane line information and the second lane line information includes: If the coordinate information of at least one lane line in the coordinate information of the multiple lane lines included in the first lane line information fails to match the coordinate information of the multiple lane lines included in the second lane line information, then determining that the lane line detection result is a false detection; If the coordinate information of at least one lane line in the coordinate information of the multiple lane lines included in the second lane line information fails to match the coordinate information of the multiple lane lines included in the first lane line information, then determining that the lane line detection result is a missed detection; If the coordinate information of the multiple lane lines included in the first lane line information matches the coordinate information of the multiple lane lines included in the second lane line information successfully one by one, the lateral errors between the coordinate information of the successfully matched lane lines in the first lane line information and the second lane line information are calculated respectively, and the lane line detection result is determined based on the lateral errors.

7. The lane line detection method according to claim 6, characterized in that: The determining the lane line detection result according to the lateral error includes: If the lateral error is greater than or equal to a set threshold, adjusting the coordinate information of the multiple lane lines included in the first lane line information according to the coordinate information of the multiple lane lines included in the second lane line information to obtain the lane line detection result; If the lateral error is less than the set threshold, the coordinate information of the multiple lane lines included in the first lane line information is determined as the lane line detection result.

8. A lane line detection device, characterized in that: include: A first acquisition unit, used to acquire point cloud data of a target road, road information, and an image containing lane lines collected when a vehicle travels on the target road; A processing unit, used for inputting the image into a trained lane line detection model for processing to obtain first lane line information of the target road; A first information determining unit, configured to determine second lane line information of the target road according to the point cloud data and the road information; The first result determination unit is used to determine the lane line detection result of the target road according to the first lane line information and the second lane line information.

9. A vehicle-mounted terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the lane line detection method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the lane line detection method according to any one of claims 1 to 7 is implemented.

Images