Lane identification method and device, vehicle and storage medium

By acquiring and analyzing lane images and map information, determining the type of road where the vehicle is located, the problem of the navigation system not being able to identify errors in time when the map data is updated, and driving safety and user experience are improved.

CN119091404APending Publication Date: 2024-12-06GREAT WALL MOTOR CO LTD
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202411039431.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-31
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

When the map data is not updated in time, it is difficult to accurately identify the lane where the vehicle is located, resulting in driving safety hazards.

Method used

By obtaining the lane image in front of the target vehicle, the number of target lanes in front of the road where the target vehicle is located is determined, and combining the number of at least two reference lanes in the map information, the type of road where the vehicle is located is determined from the candidate road type to avoid error prompts in the navigation system.

Benefits of technology

It improves the vehicle's accurate identification of the current road type, enhances driving safety and user experience, and reduces the possibility of error prompts in the navigation system.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119091404A_ABST
    Figure CN119091404A_ABST
Patent Text Reader

Abstract

The invention provides a lane recognition method and device, a vehicle and a storage medium, and the method is applied to the field of vehicles, and comprises the steps: obtaining a lane image in front of a target vehicle; determining the number of target lanes in front of a road where the target vehicle is located based on the lane image; under the condition that roads of at least two candidate road types exist in the position where a target vehicle is located, determining the road type of the road where the target vehicle is located from the at least two candidate road types based on the number of the target lanes and the number of at least two reference lanes in map information, different candidate road types correspond to different reference lane numbers. According to the method, the road type of the road where the target vehicle is currently located can be determined based on the number of the target lanes and the at least two road types in the map information, the navigation system incorrectly prompts the road type of the road where the target vehicle is currently located, and better driving experience is brought to a user.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of vehicles, and more specifically, to a lane recognition method, device, vehicle and storage medium in the field of vehicles. Background Art

[0002] With the development of vehicle autonomous driving technology, vehicles need to determine the lane they are currently in for lane planning. There are main lanes and secondary lanes on a certain level of lanes. Since the main and secondary roads have different driving speeds and road properties, it is necessary to accurately identify the lane the vehicle is currently in to ensure driving safety.

[0003] In the related art, the road on which the vehicle is located is identified through a navigation system, but the navigation system relies on high-precision map data. If the map data is not updated in a timely manner, the navigation system may identify the wrong lane, thereby posing a driving safety hazard to the user. Summary of the invention

[0004] The present application provides a lane recognition method, device, vehicle and storage medium. The method can determine the road type of the target vehicle's current road based on the number of target lanes and at least two road types in map information. The navigation system error prompts the road type of the target vehicle's current road, thereby bringing a better driving experience to the user.

[0005] In a first aspect, a lane recognition method is provided, the method comprising:

[0006] Acquire the lane image in front of the target vehicle;

[0007] Based on the lane image, determine the number of target lanes ahead of the road where the target vehicle is located;

[0008] In the case where there are at least two candidate road types at the location of the target vehicle, the road type of the road where the target vehicle is located is determined from the at least two candidate road types based on the target lane number and at least two reference lane numbers in the map information, and different candidate road types correspond to different reference lane numbers.

[0009] Through the above scheme, the lane image in front of the target vehicle is obtained, and based on the lane image, the number of target lanes in front of the road where the target vehicle is located is determined. Since different road types correspond to different traffic rules, in order to ensure the safe driving of the target vehicle, it is necessary to determine the road type where the target vehicle is currently located. In the case where there are at least two candidate road types at the location of the target vehicle, based on the number of target lanes and at least two reference lane numbers in the map information, the road type of the road where the target vehicle is located is determined from at least two candidate road types, and different candidate road types correspond to different reference lane numbers. That is, in the case where the number of target lanes is the same as the number of at least two reference lanes in the map information, the road type of the road where the target vehicle is located is determined from at least two candidate road types, so that the target vehicle determines the road type of the current road, avoiding the navigation system from incorrectly prompting the road type of the current road where the target vehicle is located, and bringing a better driving experience to the user.

[0010] In combination with the first aspect, in some possible implementations, the lane image is input into a lane line detection model to obtain the number of lane lines in the lane image;

[0011] The lane image is input into the lane line classification model to obtain the lane line category corresponding to each lane line in the lane image;

[0012] The number of target lanes is determined based on the number of lane lines in the lane image and the lane line categories corresponding to each lane line.

[0013] In combination with the first aspect, in certain possible implementations, a lane image is input into a lane line classification model, and the lane line classification model divides the lane image into a preset number of partial lane images based on the number of lane lines in the lane image and the coordinates of each lane line, where the preset number is the same as the number of lane lines in the lane image, and the partial lane images are images corresponding to each lane line in the lane image;

[0014] The lane line classification model is used to determine the lane line category corresponding to the lane line in each local lane image.

[0015] In combination with the first aspect, in some possible implementations, image features of each local lane image are extracted by using a lane line classification model to obtain image features of each local lane image;

[0016] Mapping the image features of each local lane image to the probability corresponding to each candidate lane line category;

[0017] For any partial lane image among the multiple partial lane images, the candidate lane line category with the highest probability among the candidate lane line categories is determined as the lane line category corresponding to the lane line in the partial lane image.

[0018] In combination with the first aspect, in some possible implementations, it is determined whether there is a partial lane image in which the lane line category is not recognized in each partial lane image;

[0019] In the case where there is a partial lane image in which the lane line category is not identified, data enhancement processing is performed on the partial image in which the lane line category is not identified;

[0020] Input the local lane image after data enhancement processing into the lane classification model;

[0021] The lane line classification model is used to identify the lane lines in the local lane image whose lane line categories have not been identified, and the lane line categories corresponding to the lane lines in the local lane image are obtained.

[0022] In combination with the first aspect, in some possible implementations, determining whether there is a reference lane line in the lane image, the reference lane line being used to indicate that the lane line is a lane boundary;

[0023] In the case where there are at least two reference lane lines in the lane image, determining the number of reference lane lines between the reference lane lines based on a difference in lane line orders corresponding to the at least two reference lane lines;

[0024] The number of target lanes is determined based on the number of reference lane lines between the reference lanes.

[0025] In combination with the first aspect, in some possible implementations, it is determined whether there is a first reference lane line in each lane line to the left of the vehicle head position in the lane image, where the first reference lane line is the first boundary lane line to the left of the vehicle head position;

[0026] In the case where a first reference lane line exists in each lane line to the left of the vehicle head position in the lane image, determining whether a second reference lane line exists in each lane line to the right of the vehicle head position in the lane image, the second reference lane line being the first boundary lane line to the right of the vehicle head position;

[0027] In a case where there are at least two reference lane lines in the lane image, determining the number of lane lines between the reference lane lines based on a difference in lane line orders corresponding to the at least two reference lane lines includes:

[0028] When there is a second reference lane line in each lane line to the right of the vehicle head position in the lane image, the difference between the lane line order corresponding to the first reference lane line and the lane line order corresponding to the second reference lane line is determined as the number of reference lane lines between the reference lane lines.

[0029] In combination with the first aspect, in some possible implementations, when there is no reference lane line in the lane image, the number of lane lines in the lane image is reduced by one to obtain the number of reference lane lines.

[0030] In combination with the first aspect, in some possible implementations, when the number of target lanes is equal to any one of at least two reference lane numbers in the map information, the road type of the road where the target vehicle is located is determined from at least two candidate road types.

[0031] In combination with the first aspect, the at least two reference lane numbers in the map information include the number of main road lanes and the number of auxiliary road lanes, and the candidate road types include main road lanes and auxiliary road lanes. In certain possible implementations, when the number of target lanes is equal to the number of main road lanes in the map information, it is determined that the road type of the road where the target vehicle is located is a main road lane;

[0032] Alternatively, when the number of target lanes is equal to the number of auxiliary road lanes in the map information, it is determined that the road type of the road where the target vehicle is located is an auxiliary road lane.

[0033] In combination with the first aspect, the candidate road types include main road lanes and auxiliary road lanes. In some possible implementations, when the number of target lanes is equal to any one of the at least two numbers of reference lanes in the map information, a target lane line category in the lane image is determined, and the target lane line category is a lane line category of a road where the target vehicle is currently located;

[0034] When the target lane line category is the same as the lane line category in the map information, the road type of the road where the target vehicle is located is determined from at least two candidate road types.

[0035] In combination with the first aspect, in some possible implementations, the coordinates of each lane line in the lane image and the position information of the road where the target vehicle is located are obtained;

[0036] Based on the coordinates of each lane line in the lane image and the position information of the road where the target vehicle is located, the lane line coordinates of the road where the target vehicle is located are determined;

[0037] The target lane line category is determined based on the lane line category corresponding to each lane line in the lane image and the lane line coordinates of the road where the target vehicle is located.

[0038] In combination with the first aspect, the map information further includes at least two reference lane line categories corresponding to at least two reference lane numbers. In some possible implementations, when the target lane line category is the same as the main road lane line category in the map information, it is determined that the road type of the target vehicle is the main road lane;

[0039] Alternatively, when the number of target lanes is the same as the category of the auxiliary road lane line in the map information, it is determined that the road type of the road where the target vehicle is located is an auxiliary road lane.

[0040] In a second aspect, a lane recognition device is provided, the device comprising:

[0041] An acquisition module, used for acquiring a lane image in front of a target vehicle;

[0042] A module for determining the number of target lanes, used for determining the number of target lanes ahead of the road where the target vehicle is located based on the lane image;

[0043] A road type determination module is used to determine the road type of the target vehicle's location from at least two candidate road types based on the target lane number and at least two reference lane numbers in the map information when there are at least two candidate road types at the target vehicle's location, wherein different candidate road types correspond to different reference lane numbers.

[0044] In a third aspect, a vehicle is provided, comprising a memory and a processor. The memory is used to store executable program code, and the processor is used to call and run the executable program code from the memory, so that the vehicle executes the method executed by the lane recognition method.

[0045] In a fourth aspect, a computer program product is provided, which includes: a computer program code, which, when executed on a computer, enables the computer to execute the method executed by the lane recognition method described above.

[0046] In a fifth aspect, a computer-readable storage medium is provided, which stores a computer program code. When the computer program code runs on a computer, the computer executes the method executed by the above-mentioned lane recognition method. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 is a schematic diagram of an implementation environment of a lane recognition method provided in an embodiment of the present application;

[0048] Figure 2 is a schematic flow chart of a lane recognition method provided in an embodiment of the present application;

[0049] Figure 3 is a schematic flow chart of another lane recognition method provided in an embodiment of the present application;

[0050] Figure 4 is a schematic diagram of a lane image provided in an embodiment of the present application;

[0051] Figure 5is a schematic flow chart of another lane recognition method provided in an embodiment of the present application;

[0052] Figure 6 is a structural schematic diagram of a lane recognition device provided in an embodiment of the present application;

[0053] Figure 7 It is a structural schematic diagram of a vehicle provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] The technical solution in the present application will be described clearly and in detail below in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B: "and / or" in the text is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0055] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features.

[0056] Figure 1 It is a schematic diagram of an implementation environment of a lane recognition method provided in an embodiment of the present application.

[0057] For example, Figure 1 As shown, the implementation environment includes an image sensor 101 and a vehicle device 102 .

[0058] The image sensor 101 is used to collect data outside the vehicle. In some embodiments, the image sensor 101 collects lane images of the vehicle while it is traveling.

[0059] The vehicle machine device 102 is used to process the data of the vehicle in motion. In some embodiments, the vehicle machine device 102 controls the image sensor 101 to collect lane images of the vehicle in motion. In some embodiments, the vehicle machine device 102 processes the lane images collected by the image sensor 101.

[0060] Figure 2 It is a schematic flow chart of a lane recognition method provided in an embodiment of the present application.

[0061] For example, Figure 2 As shown, taking the execution subject as a vehicle machine device as an example, the method 200 includes:

[0062] Step 201, acquiring a lane image in front of a target vehicle.

[0063] The target vehicle is a vehicle traveling on the road. The front of the target vehicle is the front of one end of the current driving direction of the target vehicle. That is, the front of one end of the head direction of the target vehicle. The lane image is an image of the target vehicle traveling on the corresponding lane. For example, the lane image includes the lane and the lane line.

[0064] Step 202: Determine the number of target lanes ahead of the road where the target vehicle is located based on the lane image.

[0065] The road where the target vehicle is located is the road where the target vehicle is currently traveling, the front of the road where the target vehicle is located is one end of the front direction of the target vehicle, the number of target lanes is the number of lanes in the lane where the target vehicle is located, and the number of target lanes is the number of lanes in the front direction of the target vehicle. For example, if the lane image includes 7 lanes and the number of roads in the current driving direction of the target vehicle is 5, then the number of target lanes is determined to be 5.

[0066] It should be understood that since the lane image is an image of the lane in front of the target vehicle, the accuracy of determining the number of target lanes in front of the road where the target vehicle is located based on the lane image is relatively high.

[0067] Step 203, when there are at least two candidate road types at the location of the target vehicle, based on the number of target lanes and at least two reference lane numbers in the map information, determine the road type of the target vehicle from the at least two candidate road types, and different candidate road types correspond to different reference lane numbers.

[0068] It should be understood that in actual applications, the target vehicle cannot do without the navigation system during its driving process. However, the navigation system requires accurate lane information. There are many types of roads on which the target vehicle is traveling. If the road type of the road on which the target vehicle is traveling is different from the road on which the target vehicle is traveling in the navigation system, the navigation system cannot provide accurate navigation services for the target vehicle. For example, if the road on which the target vehicle is traveling is a non-motorized vehicle lane, and the road on which the target vehicle is traveling in the navigation system is a motor vehicle lane, the navigation system cannot accurately plan a driving path for the target vehicle. Therefore, in order for the navigation system to provide an accurate driving path for the target vehicle, it is necessary to determine the type of road on which the target vehicle is located.

[0069] Among them, the location of the target vehicle is the location where the target vehicle is currently traveling on the road. The candidate road type is the lane type divided according to road traffic safety regulations. For example, the candidate road types include motor vehicle lanes, non-motor vehicle lanes, sidewalks, main road lanes, and auxiliary road lanes. Map information is information indicating geographic information. For example, map information includes information about the road on which the target vehicle is currently traveling or information such as the route planning and predicted time of the target vehicle from the departure point to the destination. The number of reference lanes is the number of lanes corresponding to the candidate road type. For example, the candidate road types are main road lanes and auxiliary road lanes, and the number of at least two reference lanes in the map information is 3 for the main road lanes and 1 for the auxiliary road lane.

[0070] It should be understood that after the road type of the target vehicle's road has been determined, the road type of the target vehicle's road can be determined by at least two reference lane numbers and the target lane number in the map information. For example, when the candidate road types are main road lanes and auxiliary road lanes, and the at least two reference lane numbers in the map information are 3 main road lanes and 1 auxiliary road lane, and the target lane number is determined to be 3, the road type of the target vehicle's current road is determined to be a main road lane.

[0071] The embodiment of the present application provides a schematic flow chart of a lane recognition method, which obtains a lane image in front of a target vehicle, and based on the lane image, determines the number of target lanes in front of the road where the target vehicle is located. Since different road types correspond to different traffic rules, it is necessary to determine the road type where the target vehicle is currently located in order to ensure the safe driving of the target vehicle. In the case where there are at least two candidate road types at the location of the target vehicle, based on the number of target lanes and at least two reference lane numbers in the map information, the road type of the road where the target vehicle is located is determined from at least two candidate road types, and different candidate road types correspond to different reference lane numbers. That is, in the case where the number of target lanes is the same as the number of at least two reference lanes in the map information, the road type of the road where the target vehicle is located is determined from at least two candidate road types, so that the target vehicle determines the road type of the road it is currently on, avoiding the navigation system from incorrectly prompting the road type of the road where the target vehicle is currently located, and bringing a better driving experience to the user.

[0072] Figure 3 It is a schematic flowchart of another lane recognition method provided in an embodiment of the present application.

[0073] It should be noted that the above steps 201-203 are a simple description of a lane recognition method provided in an embodiment of the present application. The following will provide a more detailed description of a lane recognition method provided in an embodiment of the present application in combination with some examples. Figure 3Taking the execution subject as a vehicle-mounted device as an example, the method includes the following steps.

[0074] Step 301, acquiring a lane image in front of a target vehicle.

[0075] Among them, an image sensor is installed on the target vehicle. The image sensor is used in a variety of systems such as the replica driving system or parking assistance system in the target vehicle. For example, the image sensor is used for reversing images, panoramic parking, lane departure warning, pedestrian detection, etc.

[0076] In some embodiments, the lane image is a digital video image (Digital Video Recorder, DVR), wherein the DVR frame image can provide higher image quality and better data integrity.

[0077] In some embodiments, the vehicle device acquires the lane image in front of the target vehicle from the image sensor. In some embodiments, the lane image of the vehicle traveling in front of the current vehicle is acquired based on a camera, a laser radar, or a ground sensor.

[0078] For example, when a target vehicle is driving on a lane, the camera on the target vehicle will automatically capture image information in the current direction of the target vehicle's front.

[0079] like Figure 4 As shown, Figure 4 It is a schematic diagram of a lane image provided in an embodiment of the present application.

[0080] In some embodiments, the acquired lane image in front of the target vehicle includes the lane and lane lines.

[0081] For example, Figure 4 As shown, the lane image includes four lanes.

[0082] In this embodiment, the lane image in front of the target vehicle is acquired through an image sensor, that is, the lane image in front of the target vehicle can be acquired in real time and continuously, thereby increasing the amount of information acquired in the lane image, and the cost of the image sensor is low, and the cost of acquiring the lane image in front of the target vehicle is also low.

[0083] Step 302: Determine the number of target lanes ahead of the road where the target vehicle is located based on the lane image.

[0084] It should be understood that different types of roads and lanes have different traffic rules, and different lanes have different speed limits and vehicle type restrictions. If the target vehicle does not follow the traffic rules, it is easy to cause a traffic accident. Therefore, in order to accurately know the road type of the lane where the target vehicle is currently traveling, the number of target lanes ahead of the road where the target vehicle is located is determined based on the lane image.

[0085] In a possible implementation, the lane image is input into a lane line detection model to obtain the number of lane lines in the lane image. The lane image is input into a lane line classification model to obtain the lane line category corresponding to each lane line in the lane image. The number of target lanes is determined based on the number of lane lines in the lane image and the lane line category corresponding to each lane line.

[0086] The lane line detection model is used to determine the number of lane lines in the lane image. In some embodiments, the lane line detection model is a cross layer optimization network (Cross Layer Refinement Network, CLR) model.

[0087] In order to explain the above-mentioned implementation mode, the above-mentioned implementation mode will be explained in several parts below.

[0088] First, the method of determining the number of lane lines in the above embodiment is described.

[0089] In a possible implementation, before the lane image is input into the lane line detection model, the lane image is preprocessed to obtain a lane image with clearer and more visible lane lines.

[0090] It should be understood that the CLR model is a deep learning model for lane line detection. The CLR model can improve the accuracy of lane line detection. In order to make the number of lane lines obtained by the lane detection model more accurate, the lane image is preprocessed before being input into the lane line detection model to remove unnecessary interference in the lane image.

[0091] In some embodiments, the lane image is processed to enhance contrast and clarity. For example, the lane image is grayed, the brightness is adjusted, the contrast is adjusted, etc. to improve the visual effect of the lane image, so that the lane line is more clearly visible.

[0092] It should be understood that since there is noise in the lane image, such as image sensor noise, motion blur, etc., in order to reduce noise interference, the lane image is filtered to remove the noise of the lane image to reduce interference with the number of detected lane lines.

[0093] It should also be understood that since different lane images have different resolutions and sizes, in order to make the input lane image have a uniform size and ratio to facilitate processing by the lane line detection model, the lane image is cropped and scaled.

[0094] For example, if the lane line detection model is a CLR model, the input size of the lane image is set to (320, 800).

[0095] In some embodiments, the preprocessed lane image is subtracted from the first mean parameter of the lane line detection model and divided by the first variance parameter of the lane line detection model, and the lane image is normalized, and then the normalized lane image is input into the lane line detection model.

[0096] Under this implementation, before the lane image is input into the lane line detection model, the lane image is preprocessed to obtain a lane image with clearer lane lines. That is, the lane image is data enhanced to make the data in the lane image clearer, making it easier for the lane line detection model to identify the number of lane lines in the lane image.

[0097] In a possible implementation, the lane image is identified using a lane line detection model to obtain the number of lane lines in the lane image.

[0098] In some embodiments, the normalized lane image is subjected to multi-layer convolution, pooling and other operations to extract lane image features. It is usually combined with FPN (Feature Pyramid Networks) to fuse feature maps of different levels, extract local features of lane line anchor points in lane image features, integrate global context information through attention mechanism, and enhance lane line feature representation.

[0099] In some embodiments, starting from the high-level feature map, feature fusion and updating are gradually performed on the lower-level feature maps. For multiple iterations of the lane line anchor points in the lane image features, each iteration updates the position and shape information of the anchor points according to the feature map of the current level, calculates the loss function of the lane line detection model, and outputs the position and shape information of the lane line.

[0100] In some embodiments, the number of lane lines in the lane image and the coordinates of each lane line are obtained through a lane line detection model.

[0101] For example, Figure 4 As shown, the lane line coordinates are [lane line 1: (x0, y0), (x1, y1), (x2, y2), ..., (x m ,y m )], [Lane 2: (x0, y0), (x1, y1), (x2, y2), ..., (x m ,y m )], [Lane 3: (x0, y0), (x1, y1), (x2, y2), ..., (x m ,y m )], [Lane 4: (x0, y0), (x1, y1), (x2, y2), ..., (x m ,y m)], [Lane 5: (x0, y0), (x1, y1), (x2, y2), ..., (x m ,y m )].

[0102] In some embodiments, the coordinates of the lane lines are sorted.

[0103] It is understandable that the order and distribution order can be from left to right. For example, the first lane line is lane line 1 (order 1), the second lane line is lane line 2 (order 2), the third lane line is lane line 3 (order 3), the fourth lane line is lane line 4 (order 4), and the fifth lane line is lane line 5 (order 5).

[0104] It is also understood that the order and distribution order can be from right to left. For example, the first lane line is lane line 5 (order 1), the second lane line is lane line 4 (order 2), the third lane line is lane line 3 (order 3), the fourth lane line is lane line 2 (order 4), and the fifth lane line is lane line 1 (order 5).

[0105] It should be understood that since the process of the lane line detection model identifying the number of lane lines is relatively complicated, in order to ensure the accuracy of the lane line detection model in identifying the number of lane lines, a threshold for the number of lane lines identified by the lane line detection model is set.

[0106] For example, if the lane line detection model is a CLR model, the maximum number of lane lines that the CLR model can recognize is 8.

[0107] In this implementation, the lane image is input into the lane line detection model to obtain the number of lane lines in the lane image, that is, the total number of lane lines in the lane image is obtained based on the lane line detection model.

[0108] The following is an explanation of the method for training the lane line detection model in the above implementation.

[0109] It should be understood that in order to determine the number of lane lines through the lane line detection model, the lane line detection model must be trained to obtain a trained lane line detection model.

[0110] In a possible implementation, a sample lane image for training a lane line detection model is obtained, and the lane line detection model is trained based on the sample lane image.

[0111] In some embodiments, a sample lane image of a training lane line detection model is obtained, the sample lane image is input into the lane line detection model, and feature extraction is performed on the sample lane image to obtain features of the sample lane lines in the sample lane image.

[0112] In some embodiments, the lane detection model starts from the high-level feature map and gradually fuses and updates the lower-level features to obtain the position and shape information of the updated points of the feature map.

[0113] It should be understood that there are deviations between the sample lane images. In order to eliminate the deviations in each lane line and standardize the data in the sample lane images, the variance and mean of the image features of multiple sample lane images are calculated.

[0114] In some embodiments, the geometric mean of the image features in the sample image is calculated to obtain a first mean parameter of the lane line detection model.

[0115] In some embodiments, since the variance is the average of the squares of the differences between each data point in the sample image and the mean, in order to normalize the image features in the sample image, a first variance parameter of the image features of the plurality of sample lane images is calculated.

[0116] In this implementation, a sample lane image for training the lane line detection model is obtained, and the lane line detection model is trained based on the sample lane image, thereby obtaining a trained lane line detection model, a first mean parameter, and a first variance parameter.

[0117] The following is an explanation of the method for determining the lane line category in the above implementation.

[0118] In a possible implementation, before the lane image is input into the lane line classification model, the lane image is preprocessed to obtain a lane image with clearer and more visible lane lines.

[0119] It should be understood that in order to make the lane line classification model recognize the lane line category with higher accuracy, the lane image needs to be preprocessed before inputting it into the lane line classification model.

[0120] It can be understood that the specific implementation method of preprocessing the lane image before inputting the lane image into the lane line classification model can be referred to the aforementioned description of preprocessing the lane image before inputting the lane image into the lane line detection model, and will not be repeated here.

[0121] In a possible implementation, the lane image is identified by a lane line classification model to obtain a lane line category corresponding to each lane line in the lane image.

[0122] The lane line classification model is used to determine the lane line category corresponding to each lane line in the lane image. For example, the lane line classification model is a MobileNetV2 model.

[0123] Lane markings are used to control and guide traffic, so lane markings are lines, arrows, words, raised road signs and contour marks on the road surface that convey traffic information such as guidance, restriction and warning to traffic participants. Lane marking categories include single dashed line, single solid line, guide line, double dashed line, double solid line, left solid right dashed line, left dashed right solid line, variable guide lane line, longitudinal deceleration lane line, roadside line, etc.

[0124] Single dashed line: There is only one dashed line between the two lanes.

[0125] The lane lines that appear in guide lines, lane bifurcations, and lane intersections are also single dashed lines; the dashed lines on both sides of the turning waiting area are also single dashed lines.

[0126] Guide line: A solid line composed of white V-shaped lines or twill lines.

[0127] Single solid line: There is only one solid line between two lanes.

[0128] Double dashed line: There are only two dashed lines between two lanes, generally used to distinguish lanes in different directions.

[0129] Double solid lines: There are only two solid lines between two lanes, generally used to distinguish lanes in different directions.

[0130] Solid left and dashed right line: A solid line and dashed lines appear in the middle of two lanes at the same time, side by side as a group, with the solid line on the left and the dashed line on the right.

[0131] Left dotted line and right solid line: A solid line and a dotted line appear in the middle of two lanes at the same time, and are grouped together, with the dotted line on the left and the solid line on the right.

[0132] Variable guide lane line: Ignore zigzag modification and mark a single line.

[0133] Longitudinal deceleration lane line: used to mark a single solid line / single dashed line with a deceleration line, ignoring the deceleration modification and marking a single solid line / single dashed line.

[0134] Roadside: The connection between the guardrails / curbs on both sides of the lane where the collection vehicle is located and the ground.

[0135] Curb: A stone marker set between the road surface and other structures.

[0136] Others: warning signs or buildings on both sides of the road.

[0137] In some embodiments, the preprocessed lane image is input into the lane classification model, and the parameters of the lane classification model are configured, for example, a size parameter, a learning rate, a second mean parameter, and a second variance parameter.

[0138] For example, if the lane line classification model is a MobileNetV2 model, the lane line classification model size parameter is (100, 300).

[0139] It should be understood that a lane image has multiple lane lines. In order to avoid interference between the multiple lane lines and obtain a lane line category with a more accurate recognition result, the lane image is cropped into multiple lane lines.

[0140] In some embodiments, the lane image is input into a lane line classification model, and the lane line classification model is used to segment the lane image into a preset number of local lane images based on the number of lane lines in the lane image and the coordinates of each lane line; the lane line classification model is used to determine the lane line category corresponding to the lane line in each local lane image.

[0141] The preset number is the same as the number of lane lines in the lane image, and the local lane image is an image corresponding to each lane line in the lane image.

[0142] In some embodiments, if the Xth lane line is clipped, a partial lane image of the Xth lane line is determined based on the starting point coordinates and the ending point coordinates of the X-1th, X, and X+1th lane lines.

[0143] For example, if the second lane line is clipped, a partial lane image of the second lane line is determined according to the starting point coordinates and the ending point coordinates of the first lane line, the second lane line, and the third lane line.

[0144] It should be understood that in order to avoid the lane lines at the image boundary in the lane image being unable to be cropped, the left boundary of the lane image is determined to be the 0th lane line, and the right boundary of the lane image is determined to be the N+1th lane line, where N is the number of lane lines in the lane image determined based on the lane line detection model.

[0145] For example, Figure 4 As shown, when the first lane line is clipped, a local lane image of the first lane line is determined according to the starting point coordinates and the ending point coordinates of lane line 0, the first lane line, and the second lane line.

[0146] For example, Figure 4 As shown, when the fifth lane line is clipped, a partial lane image of the fifth lane line is determined according to the starting point coordinates and the ending point coordinates of the fourth lane line, the fifth lane line, and the sixth lane line.

[0147] In some embodiments, based on the number of lane lines in the lane image and the coordinates of each lane line, local lane images having the same number as the number of lane lines are obtained.

[0148] It should be understood that in order to obtain more accurate lane line categories through the lane line classification model, a preset number of local lane images are scaled to within the size parameter range of the lane line classification model.

[0149] For example, if the lane line classification model is a MobileNetV2 model, the lane line classification model size parameter is (100, 300).

[0150] In some embodiments, lane lines in a preset number of partial lane images are identified through a lane line classification model to obtain lane line categories corresponding to the lane lines in the partial lane images.

[0151] In some embodiments, after a preset number of partial lane images are normalized by subtracting the second mean and dividing by the second variance, the partial lane images are input into the lane line classification model.

[0152] In some embodiments, each partial lane image is identified by a lane line classification model to obtain a lane line category of the lane line in each partial lane image.

[0153] In some embodiments, image features of each local lane image are extracted through a lane line classification model to obtain image features of each local lane image.

[0154] In some embodiments, the image features of each local lane image are mapped to the probability corresponding to each candidate lane line category.

[0155] Among them, the candidate lane line categories include single dashed line, single solid line, guide line, double dashed line, double solid line, left solid and right dashed line, left dashed and right solid line, variable guide lane line, longitudinal deceleration lane line, and roadside line.

[0156] In some embodiments, the image features of each local lane image are extracted through a convolution layer, and the image features of the local lane image are average pooled, that is, the image features are compressed into a vector of a preset length.

[0157] In some embodiments, the image features of each partial lane image are fully connected, so that the feature vectors of the image features of each partial lane image are mapped to the probability distribution of each category.

[0158] It should be understood that if the lane line classification model directly outputs the similarity between the image features of the local lane image and the candidate lane line categories, the output similarity ranges from negative infinity to positive infinity. In order to avoid the inability to determine the similarity probability between the image features of the local lane image and the candidate lane line categories, the softmax function is used to sum the probability values ​​of the image features of each local lane image output to 1, thereby obtaining the lane line category with the highest probability.

[0159] For example, if there are 8 partial lane images, the number of lane lines in the lane image is also 8, and the sum of the probability values ​​of the image features of the output 8 partial lane images is 1.

[0160] In some embodiments, after fully connecting the image features of each local lane image, the output lane line category is converted into a probability distribution through a softmax function, so that the output of each lane line category is a probability value.

[0161] In some embodiments, for any partial lane image among the multiple partial lane images, the candidate lane line category with the highest probability among the candidate lane line categories is determined as the lane line category corresponding to the lane line in the partial lane image.

[0162] It should be understood that when the probability that the image features of each local lane image correspond to the features of each candidate lane line category is greater than the preset probability, it is determined that the lane line in the local lane image is similar to each candidate lane line category, and the candidate lane line category with the highest probability among the candidate lane line categories is determined as the lane line category corresponding to the lane line in the local lane image.

[0163] In this implementation, each local lane image is identified through a lane line classification model to obtain the lane line category of the lane line in each local lane image with high accuracy.

[0164] It should be understood that in order to ensure that the confidence of the lane line category identified based on the lane line classification model is high, the confidence of each lane line category in the lane line classification model is calculated. Among them, the confidence is the probability value assigned by the lane line classification model to each lane line category. The confidence is used to describe the possibility that each local lane image calculated by the lane line classification model belongs to a lane line category.

[0165] In some embodiments, when the confidence level is greater than a preset confidence level, it is determined that the possibility of the lane line category output by the lane line classification model is higher.

[0166] For example, when the confidence level is greater than 90%, it is determined that the possibility of the lane line category output by the lane line classification model is high.

[0167] It should be understood that in order to increase the accuracy of the lane line classification model in identifying the lane line in the lane image, it is determined in multiple consecutive frames whether the confidence is greater than the preset confidence.

[0168] In some embodiments, if the lane line categories obtained based on lane images of at least 70% of the frames in a plurality of consecutive frames are consistent, then it is determined that the possibility of the lane line category output by the lane line classification model is high.

[0169] For example, if the results of at least 7 frames of images out of 10 consecutive lane image frames are consistent, it is determined that the possibility of the lane line category output by the lane line classification model is high.

[0170] In this implementation, the lane image is identified by the lane classification model to obtain the lane category corresponding to each lane line in the lane image. That is, each lane line in the lane image is classified to obtain the lane category corresponding to each lane line.

[0171] The following is an explanation of the method for training the lane classification model in the above implementation.

[0172] In a possible implementation, a sample lane image for training a lane classification model is obtained, and the lane classification model is trained based on the sample lane image.

[0173] In some embodiments, the sample lane image is preprocessed, the preprocessed sample lane image is input into a lane line classification model, a convolution layer, a normalization layer and an activation function of the lane line classification model are constructed, and features are extracted from the sample lane image.

[0174] In some embodiments, image features of the lane image are extracted through a convolutional layer of the lane classification model, and parameters of the lane classification model are updated through back propagation and gradient descent.

[0175] In some embodiments, a second mean parameter and a second variance parameter of the lane classification model are determined based on the lane image.

[0176] It can be understood that the second mean parameter and the second variance parameter of the lane line classification model can be determined by referring to the first mean parameter and the first variance parameter of the aforementioned lane line detection model.

[0177] In this implementation, a sample lane image for training the lane line classification model is obtained, and the lane line classification model is trained based on the sample lane image, thereby obtaining a trained lane line classification model, a second mean parameter, and a second variance parameter.

[0178] Finally, the method of determining the number of target lanes in the above implementation is described.

[0179] In a possible implementation, the number of target lanes is determined based on the number of lane lines in the lane image and the lane line categories corresponding to the respective lane lines.

[0180] Among them, the target lane number is the lane in front of the road where the target vehicle is currently located.

[0181] In some embodiments, the number of lanes in the lane image can be determined based on the number of lane lines in the lane image and the lane line categories corresponding to the respective lane lines.

[0182] It should be understood that two lane lines define one lane.

[0183] For example, if the left lane line category is a solid yellow line and the right lane line is a dashed white line, that is, the solid yellow line and the dashed white line determine one lane.

[0184] In some embodiments, when the number of lanes in the lane image is determined, the lane in the driving direction of the target vehicle is clarified.

[0185] It should be understood that since different lane lines have different functions, the driving direction of the target vehicle is determined based on the lane lines.

[0186] For example, solid yellow lines and dashed yellow lines are used to distinguish lanes in different directions. If the left lane line is a solid yellow line and the right lane line is a dashed white line, the lane in the current driving direction of the target vehicle is determined.

[0187] Under this implementation, the number of target lanes is determined based on the number of lane lines in the lane image and the lane line categories corresponding to each lane line. Since two lane lines determine one lane, the number of lanes on the road where the target vehicle is currently located can be determined based on the number of lane lines in the lane image and the lane line categories corresponding to each lane line, and the accuracy of determining the target lane number is high.

[0188] It should be understood that since there are many lane lines in the lane image, in order to ensure that the lane line categories corresponding to all lane lines in the lane image are identified, it is necessary to determine whether there are partial lane images with unidentified lane line categories in each partial lane image.

[0189] In a possible implementation, it is determined whether there is a partial lane image in which no lane line category is recognized among the partial lane images.

[0190] In some embodiments, when there is no partial lane image in which the lane line category is not identified, the number of target lanes is determined based on the lane line category and the number of lane lines in the partial lane image.

[0191] It should be understood that in the absence of a partial lane image in which the lane line category is not identified, the lane line category corresponding to each lane line in the lane image is determined and identified, and the number of target lanes can be determined based on the number of lane lines and the lane line category corresponding to each lane line.

[0192] Under this embodiment, in the absence of a partial lane image in which the lane line category is not identified, the target lane number is determined based on the lane line category and the number of lane lines in the partial lane image. That is, one lane is determined based on the number of two lane lines and the lane line category corresponding to the lane lines. Since the target lane number is the lane in front of the road where the target vehicle is currently located, the target lane number can be accurately determined based on the lane line category and the number of lane lines in the partial lane image.

[0193] In some embodiments, when there is a partial lane image in which the lane line category is not recognized, data enhancement processing is performed on the partial image in which the lane line category is not recognized.

[0194] It should be understood that in the case where there are partial lane images whose lane line categories are not recognized, it is determined that there are partial lane images in each partial lane image whose parameters do not conform to the lane line classification model. In order to accurately identify the partial lane images whose lane line categories are not recognized, data enhancement processing is performed on the lane images.

[0195] In some embodiments, the local lane image is processed to enhance contrast and clarity. For example, the local lane image is grayed, the brightness is adjusted, the contrast is adjusted, etc. to improve the visual effect of the local lane image, so that the lane line is more clearly visible.

[0196] It should be understood that since there is noise in the local lane image, such as image sensor noise, motion blur, etc., in order to reduce noise interference, the local lane image is filtered to remove the noise of the local lane image to reduce interference in identifying the lane line category.

[0197] Under this embodiment, in the presence of a partial lane image in which the lane line category is not identified, data enhancement processing is performed on the partial image in which the lane line category is not identified, so that the lane lines in the partial image are more clearly visible, thereby making it easier and more accurate to identify the lane line category in the partial image.

[0198] In a possible implementation, the local lane image after data enhancement processing is input into a lane classification model. The lane line classification model is used to identify the lane lines in the local lane image whose lane line categories have not been identified, and obtain the lane line categories corresponding to the lane lines in the local lane image.

[0199] It can be understood that the specific implementation method of identifying the lane lines in the local lane image whose lane line categories have not been identified through the lane line classification model to obtain the lane line categories corresponding to the lane lines in the local lane image can refer to the aforementioned steps of identifying the lane lines in a preset number of local lane images through the lane line classification model to obtain the relevant description of the lane line categories corresponding to the lane lines in the local lane image.

[0200] In this implementation, the partial lane image after data enhancement processing is input into the lane classification model; the lane lines in the partial lane image whose lane line categories are not identified are identified through the lane line classification model, and the lane line categories corresponding to the lane lines in the partial lane image are obtained. That is, in the case of a partial lane image whose lane line categories are not identified, the partial lane image is preprocessed so that the data in the partial lane image is more suitable for the lane line classification model, so that the partial lane image is identified and the lane line categories corresponding to all the lane lines in the lane image are obtained.

[0201] It should be understood that since different lanes correspond to different traffic rules, in order to determine the number of target lanes on the road where the target vehicle is currently located, it is determined whether there are reference lane lines in the lane image. When there are at least two reference lane lines, the number of target lanes on the road where the target vehicle is currently located is determined based on the at least two reference lane lines.

[0202] In one possible implementation, it is determined whether there is a reference lane line in the lane image, the reference lane line being used to indicate that the lane line is a lane boundary; when there are at least two reference lane lines in the lane image, the number of reference lane lines between the reference lane lines is determined based on the difference in lane line order corresponding to the at least two reference lane lines; and the number of target lanes is determined based on the number of reference lane lines between the reference lanes.

[0203] The reference lane line is a lane line indicating a lane boundary, for example, a solid yellow line, a dotted yellow line, a curb, and a guide line.

[0204] Solid yellow lines and dashed yellow lines are used to distinguish lanes in different directions. In other words, solid yellow lines and dashed yellow lines are used to distinguish lanes in the direction of the target vehicle. Curbs are used to distinguish motor vehicle lanes from non-motor vehicle lanes. Guide lines are used to distinguish main road lanes from auxiliary road lanes. Among them, the main road lane is the lane where there is no fork in the current lane. The auxiliary road lane is the lane with a fork in the road, and the auxiliary road lane has lane dividing lines, guide lines and lane dividing guide piles.

[0205] It should be understood that, when there are at least two reference lane lines in the lane image, the number of reference lane lines between the at least two reference lane lines can be determined based on the at least two reference lane lines.

[0206] For example, Figure 4 As shown, the lane image includes the first lane line (order 1), the second lane line (order 2), the third lane line (order 3), the fourth lane line (order 4), and the fifth lane line (order 5). If the second lane line is a solid yellow line, then the second lane line is the reference lane line, and if the fifth lane line is a curb, then the fifth lane line is the reference lane line. Based on the order of the second lane line and the fifth lane line, it is determined that the number of reference lane lines between the two reference lane lines is 2, and the lane lines between the second lane line and the fifth lane line are the third lane line and the fourth lane line.

[0207] It should be understood that since two lane lines determine one lane, there are three lanes between the second lane line and the fifth lane line, so the number of target lanes is determined based on the number of reference lane lines between the reference lanes.

[0208] For example, based on the difference between the order of the second lane line and the order of the fifth lane line, the number of reference lane lines between the two reference lane lines is determined to be 2. The second lane line and the third lane line determine a lane, the third lane line and the fourth lane line determine a lane, and the fourth lane line and the fifth lane line determine a lane. Therefore, there are 3 lanes between the second lane line and the fifth lane line, that is, the target lane number is 3.

[0209] In this implementation, it is determined whether there is a reference lane line in the lane image, and the reference lane line is used to indicate that the lane line is a lane boundary; when there are at least two reference lane lines in the lane image, the number of reference lane lines between the reference lane lines is determined based on the difference in the lane line order corresponding to the at least two reference lane lines; the number of target lanes is determined based on the number of reference lane lines between the reference lanes. That is, since two lane lines determine one lane, the number of lane lines ahead of the lane where the target vehicle is currently located is determined based on at least two reference lane lines, and the number of target lanes is determined based on the number of reference lane lines, so that the number of lanes ahead of the lane where the target vehicle is currently located is accurately determined.

[0210] It should be understood that since there are multiple boundary lane lines in the lane image, if all of the multiple boundary lane lines are determined as reference lane lines, the number of target lanes in front of the road where the target vehicle is currently located cannot be accurately determined. Therefore, the number of reference lanes in the current driving direction of the target vehicle is determined based on the front position of the vehicle in the lane image.

[0211] In a possible implementation, when there are at least two reference lane lines in the lane image, the number of reference lanes in the current driving direction of the target vehicle is determined based on the vehicle head position in the lane image.

[0212] In some embodiments, it is determined whether a first reference lane line exists in each lane line to the left of the vehicle head position in the lane image.

[0213] The first reference lane line is the first boundary lane line to the left of the vehicle head position.

[0214] It should be understood that since there are multiple reference lane lines in the lane image, these multiple reference lane lines are distributed in different areas of the lane image. Therefore, in order to determine the number of target lanes on the road where the target vehicle is located, it is determined whether there are reference lane lines on both sides of the front position of the target vehicle.

[0215] It should also be understood that since the reference lane line is a boundary lane line, if the position of the vehicle head in the lane image is determined, the driving direction of the target vehicle can be determined, thereby determining the target lane in the driving direction of the target vehicle.

[0216] In some embodiments, the vehicle head position in the lane image is 1 / 2 of the lane image, and whether there is a first reference lane line among the lane lines to the left of the vehicle head position is determined from the vehicle head position, that is, the first boundary lane line of the road where the target vehicle is located is determined from the vehicle head position, thereby facilitating the determination of the target lane number of the target vehicle.

[0217] In some embodiments, when there is a first reference lane line among the lane lines to the left of the vehicle head position in the lane image, it is determined whether there is a second reference lane line among the lane lines to the right of the vehicle head position in the lane image, and the second reference lane line is the first boundary lane line to the right of the vehicle head position.

[0218] For example, Figure 4 As shown, searching to the left from the vehicle head position in the lane image, it is determined that the second lane line among the lane lines to the left of the vehicle head position is a solid yellow line, and the second lane line is the first reference lane line, that is, the second lane line is the first boundary lane line.

[0219] It should be understood that according to traffic regulations, when a first reference lane line exists among the lane lines to the left of the target vehicle, the driving direction of the target vehicle can be determined, so there is no need to continue searching for a reference lane line to the left.

[0220] In some embodiments, the lane image is searched to the right from the vehicle head position to determine if a second reference lane line exists in each lane line to the right of the vehicle head position.

[0221] For example, Figure 4 As shown, by searching to the right from the vehicle head position in the lane image, it is determined that the fifth lane line among the lane lines to the right of the vehicle head position is the curb. The fifth lane line can be used to determine the boundary lane line of the road where the target vehicle is currently located, so there is no need to continue searching to the right for the reference lane line.

[0222] In some embodiments, when there is a second reference lane line in each lane line to the right of the vehicle head position in the lane image, the difference between the lane line order corresponding to the first reference lane line and the lane line order corresponding to the second reference lane line is determined as the number of reference lane lines between the reference lane lines.

[0223] It should be understood that since two lane lines determine one lane, when determining the situation that a first reference lane line exists in each lane line to the left of the vehicle head position in the lane image, and a second reference lane line exists in each lane line to the right of the vehicle head position in the lane image, therefore, based on the difference between the lane line order corresponding to the first reference lane line and the lane line order corresponding to the second reference lane line, the number of reference lane lines between the first reference lane line and the second reference lane line can be determined, and thus based on the number of reference lane lines, the number of target lanes at the target vehicle's position can be determined.

[0224] For example, Figure 4 As shown, the lane image includes the first lane line (order 1), the second lane line (order 2), the third lane line (order 3), the fourth lane line (order 4), and the fifth lane line (order 5). If the second lane line is a solid yellow line, then the second lane line is the reference lane line, and if the fifth lane line is a curb, then the fifth lane line is the reference lane line. Based on the order of the second lane line and the fifth lane line, it is determined that the number of reference lane lines between the two reference lane lines is 2, and the lane lines between the second lane line and the fifth lane line are the third lane line and the fourth lane line.

[0225] For another example, based on the order of the second lane line and the fifth lane line, the number of reference lane lines between the two reference lane lines is determined to be 2. The second lane line and the third lane line determine one lane, the third lane line and the fourth lane line determine one lane, and the fourth lane line and the fifth lane line determine one lane. Therefore, there are 3 lanes between the second lane line and the fifth lane line, that is, the number of target lanes is 3.

[0226] In a possible implementation, when there is at least one reference lane line in the lane image, the number of reference lane lines is determined based on the at least one reference lane line.

[0227] It should be understood that the lane image is the lane image in front of the target vehicle. There may be only one reference lane line in the lane image, that is, there is only one boundary lane line in the lane image. Therefore, the number of reference lane lines is determined based on the at least one reference lane line.

[0228] In some embodiments, it is determined whether a first reference lane line exists in each lane line to the left of the lane position in the lane image.

[0229] In some embodiments, when a first reference lane line exists in each lane line to the left of the lane position in the lane image, it is determined whether a second reference lane line exists in each lane line to the right of the lane position in the lane image.

[0230] In some embodiments, when there is no second reference lane line in the lane lines to the right of the vehicle head position in the lane image, the rightmost lane line to the right of the vehicle head position in the lane image is determined as the boundary lane line in the lane image.

[0231] The rightmost lane line in the lane image is obtained by sorting the lane lines in the lane image from left to right.

[0232] In some embodiments, the order of the rightmost lane lines in the lane image is the order corresponding to the number of lane lines N in the lane image. The method for determining the number of lane lines N in the lane image refers to the related description of inputting the lane image into the lane line detection model in the aforementioned step 302.

[0233] For example, Figure 4 As shown, the lane image includes the first lane line (order 1), the second lane line (order 2), the third lane line (order 3), the fourth lane line (order 4), and the fifth lane line (order 5). The first reference lane line to the left of the vehicle head position is the second lane line, the second lane line is a solid yellow line, there is no boundary lane line to the right of the vehicle head position, and the fifth lane line is a white dotted line, so the fifth lane line is not a reference lane line, and because the fifth lane line in the lane image is the rightmost lane line in the lane image, the fifth lane line is determined as the rightmost lane line.

[0234] In some embodiments, the number of reference lane lines is determined based on a difference between an order corresponding to the first reference lane line and an order corresponding to the rightmost lane line.

[0235] For example, based on the order of the second lane line and the fifth lane line, the number of reference lane lines between the two reference lane lines is determined to be 2, and the lane lines between the second lane line and the fifth lane line are the third lane line and the fourth lane line.

[0236] In some embodiments, two lane lines determine one lane, and the number of target lanes is determined based on the number of reference lane lines.

[0237] For example, based on the order of the second lane line and the fifth lane line, the number of reference lane lines between the two reference lane lines is determined to be 2. The second lane line and the third lane line determine one lane, the third lane line and the fourth lane line determine one lane, and the fourth lane line and the fifth lane line determine one lane. Therefore, there are 3 lanes between the second lane line and the fifth lane line, that is, the number of target lanes is 3.

[0238] In some embodiments, when there is no first reference lane line among the lane lines to the left of the vehicle head position in the lane image and there is a second reference lane line among the lane lines to the right of the vehicle head position in the lane image, it is determined whether there is a second reference lane line among the lane lines to the right of the vehicle head position in the lane image.

[0239] In some embodiments, when there is no first reference lane line among the lane lines to the left of the vehicle head position in the lane image and there is a second reference lane line among the lane lines to the right of the vehicle head position in the lane image, the number of reference lane lines is determined based on the second reference lane line.

[0240] It should be understood that the lane image is the lane image in front of the target vehicle. There may be only one reference lane line in the lane image, that is, there is only one boundary lane line in the lane image. Therefore, the number of reference lane lines is determined based on the at least one reference lane line.

[0241] In some embodiments, when there is no first reference lane line in the lane lines to the left of the vehicle head position in the lane image, the leftmost lane line to the left of the vehicle head position in the lane image is determined as the boundary lane line in the lane image.

[0242] The leftmost lane line in the lane image is obtained by sorting the lane lines in the lane image from left to right.

[0243] In some embodiments, the order of the leftmost lane lines in the lane image is the order corresponding to the first lane lines in the lane image.

[0244] For example, Figure 4 As shown, the lane image includes the first lane line (order 1), the second lane line (order 2), the third lane line (order 3), the fourth lane line (order 4), and the fifth lane line (order 5). If there is no boundary lane line in the lane lines to the left of the vehicle head position, the first lane line is determined as the boundary lane line, and the order corresponding to the first lane line is 1. If the fifth lane line is a curb, the fifth lane line is the second reference lane line, and the order corresponding to the fifth lane line is 5.

[0245] In some embodiments, the number of reference lane lines is determined based on a difference between an order of the leftmost lane lines and an order corresponding to the second reference lane lines.

[0246] In some embodiments, the target lane number is determined based on the reference lane number.

[0247] For example, based on the order corresponding to the first lane line and the order of the fifth lane line, the number of reference lane lines between the two reference lane lines is determined to be 3. The first lane line and the second lane line determine a lane, the second lane line and the third lane line determine a lane, the third lane line and the fourth lane line determine a lane, and the fourth lane line and the fifth lane line determine a lane. Therefore, there are 4 lanes between the second lane line and the fifth lane line, that is, the target lane number is 4.

[0248] In this implementation, even if there is at least one reference lane line in the lane image, the number of reference lane lines can be determined based on the order of lane lines on both sides of the lane image, thereby determining the number of target lanes.

[0249] In a possible implementation, when there is no reference lane line in the lane image, the number of lane lines in the lane image is reduced by one to obtain the number of reference lane lines.

[0250] It should be understood that when there is no reference lane line in the lane image, the number of lane lines in the lane image is determined. Since two lane lines determine one lane, the number of lane lines in the lane image is reduced by one, and the number of lane lines between the leftmost lane line and the rightmost lane line in the lane image can be obtained, thereby determining the number of target lanes based on the number of reference lane lines.

[0251] For example, Figure 4 As shown, the lane image includes the first lane line (order 1), the second lane line (order 2), the third lane line (order 3), the fourth lane line (order 4), and the fifth lane line (order 5). There is no reference lane line in the lane image. The lane image includes 5 lane lines. Subtracting 5-1, the number of lane lines between the leftmost lane and the rightmost lane in the lane image is 4, and the number of target lanes is determined to be 4.

[0252] In this embodiment, even if there is no reference lane line in the lane image, the number of reference lane lines can be determined based on the order of lane lines on both sides of the lane image, thereby determining the number of target lanes.

[0253] Step 303: Obtain the numbers of at least two reference lanes in the map information.

[0254] The at least two reference lane numbers in the map information are lane numbers corresponding to at least two lane types.

[0255] For example, if the at least two lane types are main road lanes and auxiliary road lanes, then the at least two reference lane numbers in the corresponding map information are the number of lanes corresponding to the main road lanes and the number of lanes corresponding to the auxiliary road lanes.

[0256] In some embodiments, the map information corresponding to the road where the target vehicle is located is determined, that is, the range in which the map information needs to be obtained is determined.

[0257] In some embodiments, after the road where the target vehicle is located is determined, map information corresponding to the road where the target vehicle is currently located is obtained through a global positioning system (GPS).

[0258] In some embodiments, the map information includes information such as terrain height, landform, water system, road, building, etc. In some embodiments, the map information also includes attribute information, wherein the attribute information is the attribute corresponding to the terrain data of the map information.

[0259] For example, when the map information includes roads, it is determined that the attribute information corresponding to the map information is a road type, wherein the road type corresponding to the road is a main road lane and a secondary road lane.

[0260] In some embodiments, the map information also includes lane information. For example, the lane information includes lane line category and lane line number.

[0261] In this implementation manner, the numbers of at least two reference lanes in the map information are obtained through GPS with high accuracy.

[0262] Step 304, when there are at least two candidate road types at the location of the target vehicle, determine the road type of the target vehicle from the at least two candidate road types based on the number of target lanes and the number of at least two reference lanes in the map information.

[0263] It should be understood that since different road types correspond to different traffic rules, if the target vehicle travels on the wrong road at the wrong speed, it is easy to cause a traffic accident. Therefore, when there are at least two candidate road types at the location of the target vehicle, the road type of the road where the target vehicle is located is determined.

[0264] The following describes a method for determining the road type of the road where the target vehicle is located based on the number of target lanes and the numbers of at least two reference lanes in the map information in the above implementation manner.

[0265] In a possible implementation, when the number of target lanes is equal to any one of at least two numbers of reference lanes in the map information, the road type of the road where the target vehicle is located is determined from at least two candidate road types.

[0266] It should be understood that since the target lane number is the number of lanes corresponding to the road where the target vehicle is currently located, when the target lane number is equal to any one of the at least two reference lane numbers in the map information, it means that the target lane number is the same as the road type corresponding to the at least one reference lane number.

[0267] In some embodiments, the number of lanes of at least two candidate road types corresponds to at least two reference lane numbers, wherein the at least two reference lane numbers in the map information include the number of main road lanes and the number of secondary road lanes, and the candidate road types include main road lanes and secondary road lanes.

[0268] For example, if the at least two candidate road types are main road lanes and secondary road lanes, the at least two reference lane numbers are the number of main road lanes and the number of secondary road lanes.

[0269] In some embodiments, when the number of target lanes is equal to the number of main road lanes in the map information, the road type of the road where the target vehicle is located is determined to be a main road lane.

[0270] For example, if the number of target lanes is 3, the number of main road lanes in the map information is 3, and the number of auxiliary road lanes is 1, then the road type of the road where the target vehicle is located is determined to be a main road lane.

[0271] In some embodiments, when the number of target lanes is equal to the number of auxiliary road lanes in the map information, the road type of the road where the target vehicle is located is determined to be an auxiliary road lane.

[0272] For example, if the number of target lanes is 1, the number of main road lanes in the map information is 3, and the number of auxiliary road lanes is 1, then the road type of the road where the target vehicle is located is determined to be an auxiliary road lane.

[0273] In this embodiment, when the number of target lanes is equal to any one of the numbers of at least two reference lanes in the map information, the road type of the road where the target vehicle is located is determined from the at least two candidate road types. That is, based on the number of target lanes and the number of reference lanes in the map information, the road type of the road where the target vehicle is located can be determined, and even if the map information incorrectly conveys the road type of the current location of the target vehicle, the road type of the road where the target vehicle is located can be re-determined based on the number of target lanes, thereby improving the accuracy of determining the road type of the road where the target vehicle is located.

[0274] The following describes a method for determining the road type of the target vehicle based on the target lane line category and the lane line category in the map information in the above implementation manner.

[0275] It should be understood that since the map information also includes the lane line category corresponding to each lane line, the road type of the road where the current target vehicle is located can be determined based on the lane line category in the map information.

[0276] In a possible implementation, when the target lane number is equal to any one of at least two reference lane numbers in the map information, the target lane line category in the lane image is determined, and the target lane line category is the lane line category of the road where the target vehicle is currently located; when the target lane line category is the same as the lane line category in the map information, the road type of the road where the target vehicle is located is determined from at least two candidate road types.

[0277] The target lane line category is the lane line category of the first lane line to the left of the vehicle head position and the lane line category of the first lane line to the right of the vehicle head position.

[0278] For example, Figure 4 As shown, searching from the front position of the target vehicle to the left, determining the first lane line to the left of the front position is the second lane line, and the lane line category of the second lane line is a solid yellow line; searching from the front position of the target vehicle to the right, determining the first lane line to the right of the front position is the third lane line, and the lane line category of the third lane line is a dotted white line, thereby determining the target lane line category in the lane image.

[0279] In some embodiments, the lane line categories in the map information include solid yellow lines, dashed yellow lines, solid white lines, dashed white lines, etc. When the road types of at least two reference lanes in the map information are main road lanes and auxiliary road lanes, the road type of the road where the target vehicle is located is determined from at least two candidate road types.

[0280] Among them, the lane line categories of the main road lane include solid yellow line, yellow dashed line, solid white line, and white dashed line, and the lane line categories of the auxiliary road lane include guide line, white dashed line, and curb. That is, the auxiliary road lane is distinguished by the guide line and the curb.

[0281] For example, if the target lane line categories of the target vehicle are solid yellow lines and dashed white lines, and the main road lanes in the map information are solid yellow lines and dashed white lines, then the road type of the target vehicle is determined to be a main road lane.

[0282] Under this implementation, when the target lane number is equal to any one of at least two reference lane numbers in the map information, the target lane line category in the lane image is determined, and the target lane line category is the lane line category of the road where the target vehicle is currently located; when the target lane line category is the same as the lane line category in the map information, the road type of the road where the target vehicle is located is determined from at least two candidate road types, that is, based on the lane line category of the road where the target vehicle is located and the lane line category in the map information, the road type of the road where the target vehicle is located is accurately identified.

[0283] In one possible implementation, the coordinates of each lane line in the lane image and the position information of the road where the target vehicle is located are obtained; based on the coordinates of each lane line in the lane image and the position information of the road where the target vehicle is located, the lane line coordinates of the road where the target vehicle is located are determined; based on the lane line category corresponding to each lane line in the lane image and the lane line coordinates of the road where the target vehicle is located, the target lane line category is determined.

[0284] In some embodiments, the coordinates of each lane line and the position information of the road where the target vehicle is located are obtained from the lane image.

[0285] In some embodiments, the position information of the road where the target vehicle is located is obtained from the map information. That is, the lane position of the road where the target vehicle is located in the lane image is determined. That is, the position information of the road where the target vehicle is located is determined.

[0286] For example, Figure 4 As shown, the target vehicle is in the second lane in the lane image.

[0287] In some embodiments, the coordinates of the lane line to the left of the front position of the target vehicle and the coordinates of the lane line to the right of the front position of the target vehicle are determined, thereby determining the lane line coordinates of the road where the target vehicle is located.

[0288] In some embodiments, the lane line category corresponding to the lane line coordinates in the lane image is determined based on the lane line coordinates of the road where the target vehicle is located, thereby determining the target lane line category.

[0289] In some embodiments, when the target lane line category is the same as the main road lane line category in the map information, the road type of the target vehicle is determined to be the main road lane.

[0290] For example, if the target lane line categories of the target vehicle are solid yellow lines and dashed white lines, and the main road lanes in the map information are solid yellow lines and dashed white lines, then the road type of the target vehicle is determined to be a main road lane.

[0291] In some embodiments, when the number of target lanes is the same as the category of the auxiliary road lane line in the map information, the road type of the road where the target vehicle is located is determined to be an auxiliary road lane.

[0292] For example, if the target lane line category of the target vehicle is a guide line and a curb, and the main road lane in the map information is a solid yellow line and a dashed white line, then the road type of the road where the target vehicle is located is determined to be an auxiliary road lane.

[0293] In this implementation, the coordinates of each lane line in the lane image and the position information of the road where the target vehicle is located are obtained; based on the coordinates of each lane line in the lane image and the position information of the road where the target vehicle is located, the lane line coordinates of the road where the target vehicle is located are determined; based on the lane line category corresponding to each lane line in the lane image and the lane line coordinates of the road where the target vehicle is located, the target lane line category is determined, that is, based on the position information of the road where the target vehicle is located and the coordinates of each lane line, the target lane line category is accurately identified, thereby obtaining the road type of the road where the target vehicle is located.

[0294] The embodiment of the present application provides a lane recognition method, which obtains a lane image in front of a target vehicle, determines the number of target lanes in front of the road where the target vehicle is located based on the lane image, obtains at least two reference lane numbers in the map information, and in the case where there are at least two candidate road types at the location of the target vehicle, determines the road type of the road where the target vehicle is located from the at least two candidate road types based on the number of target lanes and the number of at least two reference lanes in the map information. That is, in the case where there are at least two candidate road types at the location of the target vehicle, in order to avoid the target vehicle driving on a road of the wrong road type in an incorrect driving manner, the road type of the road where the target vehicle is currently located is determined based on the number of target lanes and the number of at least two reference lanes in the map information, thereby avoiding the target vehicle from driving on a road of the wrong road type at a low speed or a high speed, and avoiding traffic accidents involving the target vehicle.

[0295] Figure 5 It is a schematic flowchart of another lane recognition method provided in an embodiment of the present application.

[0296] It should be noted that the above steps 201-203 are a simple description of a lane recognition method provided in an embodiment of the present application. The following will provide a more detailed description of a lane recognition method provided in an embodiment of the present application in combination with some examples. Figure 5 Taking the execution subject as a vehicle-mounted device as an example, the method includes the following steps.

[0297] Step 501, acquiring a lane image in front of a target vehicle.

[0298] It can be understood that the specific implementation of obtaining the lane image in front of the target vehicle can refer to the relevant description of the aforementioned step 201 and step 301, which will not be repeated here.

[0299] Step 502: input the lane image into a lane line detection model to obtain the number of lane lines in the lane image and the lane line coordinates corresponding to each lane line.

[0300] It can be understood that the specific implementation method of inputting the lane image into the lane line detection model to obtain the number of lane lines in the lane image and the lane line coordinates corresponding to each lane line can be found in the relevant description of the aforementioned step 302, which will not be repeated here.

[0301] Step 503: sort the lane lines in order from left to right.

[0302] It can be understood that the specific implementation of sorting the lane lines in order from left to right can be found in the relevant description of the aforementioned step 302, which will not be repeated here.

[0303] Step 504 : based on the number of lane lines in the lane image and the coordinates of each lane line, the lane image is divided into a preset number of partial lane images.

[0304] It can be understood that the specific implementation of dividing the lane image into a preset number of partial lane images based on the number of lane lines in the lane image and the coordinates of each lane line can be found in the relevant description of the aforementioned step 302, which will not be repeated here.

[0305] Step 505: Input the multiple local lane images into a lane line classification model to obtain a lane line category corresponding to each lane line.

[0306] It can be understood that the specific implementation method of inputting multiple local lane images into the lane line classification model to obtain the lane line category corresponding to each lane line can be referred to the relevant description of the aforementioned 302, which will not be repeated here.

[0307] Step 506: Determine the target lane number of the target vehicle based on the lane line category and the lane line number.

[0308] It can be understood that the specific implementation of determining the target lane number of the target vehicle based on the lane line category and the lane line number can be found in the relevant descriptions of the aforementioned steps 202 and 302, which will not be repeated here.

[0309] Step 507 : Determine the road type of the road where the target vehicle is located from at least two candidate road types based on the target lane number and at least two reference lane numbers in the map information.

[0310] It can be understood that the specific implementation method of determining the road type of the road where the target vehicle is located from at least two candidate road types based on the number of target lanes and the number of at least two reference lanes in the map information can be found in the relevant descriptions of the aforementioned steps 203 and 304, which will not be repeated here.

[0311] The embodiment of the present application provides a lane recognition method, which obtains a lane image in front of a target vehicle, determines the number of target lanes in front of the road where the target vehicle is located based on the lane image, obtains at least two reference lane numbers in the map information, and in the case where there are at least two candidate road types at the location of the target vehicle, determines the road type of the road where the target vehicle is located from the at least two candidate road types based on the number of target lanes and the number of at least two reference lanes in the map information. That is, in the case where there are at least two candidate road types at the location of the target vehicle, in order to avoid the target vehicle driving on a road of the wrong road type in an incorrect driving manner, the road type of the road where the target vehicle is currently located is determined based on the number of target lanes and the number of at least two reference lanes in the map information, thereby avoiding the target vehicle from driving on a road of the wrong road type at a low speed or a high speed, and avoiding traffic accidents involving the target vehicle.

[0312] Figure 6 It is a structural schematic diagram of a lane recognition device provided in an embodiment of the present application.

[0313] For example, Figure 6 As shown, the device 600 includes:

[0314] An acquisition module 601 is used to acquire a lane image in front of a target vehicle;

[0315] A target lane number determination module 602 is used to determine the number of target lanes ahead of the road where the target vehicle is located based on the lane image;

[0316] The road type determination module 603 is used to determine the road type of the target vehicle's location from at least two candidate road types based on the number of target lanes and at least two reference lane numbers in the map information when there are at least two candidate road types at the target vehicle's location, and different candidate road types correspond to different reference lane numbers.

[0317] In a possible implementation manner, the device 600 further includes:

[0318] An input module, used for inputting a lane image into a lane line detection model to obtain the number of lane lines in the lane image;

[0319] An input module, used to input the lane image into the lane line classification model to obtain the lane line category corresponding to each lane line in the lane image;

[0320] The module 602 for determining the number of target lanes is used to determine the number of target lanes based on the number of lane lines in the lane image and the lane line categories corresponding to the lane lines.

[0321] In a possible implementation manner, the device 600 further includes:

[0322] An input module, used for inputting a lane image into a lane line classification model, and dividing the lane image into a preset number of partial lane images based on the number of lane lines in the lane image and the coordinates of each lane line through the lane line classification model. The preset number is the same as the number of lane lines in the lane image, and the partial lane images are images corresponding to each lane line in the lane image;

[0323] The lane line category determination module is used to determine the lane line category corresponding to the lane line in each local lane image through a lane line classification model.

[0324] In a possible implementation manner, the device 600 further includes:

[0325] An extraction module is used to extract image features of each local lane image through a lane line classification model to obtain image features of each local lane image;

[0326] A mapping module, used to map the image features of each local lane image to the probability corresponding to each candidate lane line category;

[0327] The lane line category determination module is used to determine, for any partial lane image among the multiple partial lane images, the candidate lane line category with the highest probability among the candidate lane line categories as the lane line category corresponding to the lane line in the partial lane image.

[0328] In a possible implementation manner, the device 600 further includes:

[0329] A partial lane image determination module is used to determine whether there is a partial lane image whose lane line category is not recognized in each partial lane image;

[0330] A processing module, for performing data enhancement processing on the local lane image where the lane line category is not identified, when there is a local lane image where the lane line category is not identified;

[0331] An input module, used for inputting the local lane image after data enhancement processing into the lane classification model;

[0332] The recognition module is used to recognize lane lines in a partial lane image whose lane line categories have not been recognized through a lane line classification model, and obtain lane line categories corresponding to the lane lines in the partial lane image.

[0333] In a possible implementation manner, the device 600 further includes:

[0334] A reference lane line determination module is used to determine whether there is a reference lane line in the lane image, and the reference lane line is used to indicate that the lane line is a lane boundary;

[0335] A module for determining the number of reference lane lines, for determining the number of reference lane lines between the reference lane lines based on the difference in lane line orders corresponding to the at least two reference lane lines when there are at least two reference lane lines in the lane image;

[0336] The target lane number determination module 602 is configured to determine the target lane number based on the number of reference lane lines between the reference lanes.

[0337] In a possible implementation manner, the device 600 further includes:

[0338] A first reference lane line determination module is used to determine whether there is a first reference lane line among the lane lines to the left of the vehicle head position in the lane image, where the first reference lane line is the first boundary lane line to the left of the vehicle head position;

[0339] A second reference lane line determination module is used to determine whether there is a second reference lane line among the lane lines to the right of the vehicle head position in the lane image, when there is a first reference lane line among the lane lines to the left of the vehicle head position in the lane image, wherein the second reference lane line is the first boundary lane line to the right of the vehicle head position;

[0340] The device 600 further includes:

[0341] A module for determining the number of reference lane lines is used to determine the difference between the lane line order corresponding to the first reference lane line and the lane line order corresponding to the second reference lane line as the number of reference lane lines between the reference lane lines when there is a second reference lane line in each lane line to the right of the vehicle head position in the lane image.

[0342] In a possible implementation manner, the device 600 further includes:

[0343] The module for determining the number of reference lane lines is used to reduce the number of lane lines in the lane image by one to obtain the number of reference lane lines when there are no reference lane lines in the lane image.

[0344] In a possible implementation manner, the device 600 further includes:

[0345] The road type determination module is used to determine the road type of the road where the target vehicle is located from at least two candidate road types when the number of target lanes is equal to any one of the numbers of at least two reference lanes in the map information.

[0346] In a possible implementation manner, the at least two reference lane numbers in the map information include the number of main road lanes and the number of auxiliary road lanes, the candidate road types include main road lanes and auxiliary road lanes, and the device 600 further includes:

[0347] A main road lane determination module, used for determining the road type of the target vehicle's location as a main road lane when the number of target lanes is equal to the number of main road lanes in the map information;

[0348] The auxiliary road lane determination module is used to determine that the road type of the target vehicle is an auxiliary road lane when the number of target lanes is equal to the number of auxiliary road lanes in the map information.

[0349] In a possible implementation, the candidate road types include a main road lane and a secondary road lane, and the device 600 further includes:

[0350] A module for determining a target lane line category, used to determine a target lane line category in a lane image when the number of target lanes is equal to any one of at least two reference lane numbers in the map information, the target lane line category being the lane line category of the road where the target vehicle is currently located;

[0351] The road type determination module is used to determine the road type of the road where the target vehicle is located from at least two candidate road types when the target lane line type is the same as the lane line type in the map information.

[0352] In a possible implementation manner, the device 600 further includes:

[0353] A position information acquisition module is used to acquire the coordinates of each lane line in the lane image and the position information of the road where the target vehicle is located;

[0354] A target lane line coordinate determination module is used to determine the lane line coordinates of the road where the target vehicle is located based on the coordinates of each lane line in the lane image and the position information of the road where the target vehicle is located;

[0355] The target lane line category determination module is used to determine the target lane line category based on the lane line category corresponding to each lane line in the lane image and the lane line coordinates of the road where the target vehicle is located.

[0356] In a possible implementation manner, the map information further includes at least two reference lane line categories corresponding to at least two reference lane numbers, the candidate road types include main road lanes and auxiliary road lanes, and the device 600 further includes:

[0357] A main road lane determination module, used to determine that the road type of the target vehicle is the main road lane when the target lane line type is the same as the main road lane line type in the map information;

[0358] The auxiliary road lane determination module is used to determine that the road type of the target vehicle is an auxiliary road lane when the number of target lanes is the same as the auxiliary road lane line category in the map information.

[0359] Figure 7 It is a structural schematic diagram of a vehicle provided in an embodiment of the present application.

[0360] For example, Figure 7 As shown, the vehicle 700 includes: a memory 701 and a processor 702, wherein the memory 701 stores an executable program code 703, and the processor 702 is used to call and execute the executable program code 703 to perform a lane recognition method.

[0361] In addition, an embodiment of the present application also protects a device, which may include a memory and a processor, wherein the memory stores an executable program code, and the processor is used to call and execute the executable program code to execute a lane recognition method provided by an embodiment of the present application.

[0362] In this embodiment, the functional modules of the device can be divided according to the above method example. For example, each functional module can be corresponded, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware. It should be noted that the division of modules in this embodiment is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0363] In the case of dividing each functional module according to each function, the device may also include a main road lane determination module, an auxiliary road lane determination module, and a target lane line category determination module, etc. It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module, which will not be repeated here.

[0364] It should be understood that the device provided in this embodiment is used to execute the above-mentioned lane recognition method, and thus can achieve the same effect as the above-mentioned implementation method.

[0365] In the case of an integrated unit, the device may include a processing module and a storage module. When the device is applied to a vehicle, the processing module may be used to control and manage the actions of the vehicle. The storage module may be used to support the vehicle in executing related program codes, etc.

[0366] The processing module may be a processor or a controller, which may implement or execute various exemplary logic blocks, modules and circuits shown in conjunction with the disclosure of the present application. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc. The storage module may be a memory.

[0367] In addition, the device provided in the embodiments of the present application may specifically be a chip, a component or a module, and the chip may include a connected processor and a memory; wherein the memory is used to store instructions, and when the processor calls and executes the instructions, the chip can execute a lane recognition method provided in the above embodiments.

[0368] This embodiment also provides a computer-readable storage medium, in which a computer program code is stored. When the computer program code is executed on a computer, the computer executes the above-mentioned related method steps to implement a lane recognition method provided by the above-mentioned embodiment.

[0369] This embodiment also provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the above-mentioned related steps to implement a lane recognition method provided by the above-mentioned embodiment.

[0370] Among them, the device, computer-readable storage medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be repeated here.

[0371] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0372] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0373] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A lane recognition method, It is characterized in that The method comprises: Acquire the lane image in front of the target vehicle; Based on the lane image, determining the number of target lanes ahead of the road where the target vehicle is located; In the case where there are at least two candidate road types at the location of the target vehicle, the road type of the road where the target vehicle is located is determined from the at least two candidate road types based on the target lane number and at least two reference lane numbers in the map information, and different candidate road types correspond to different reference lane numbers.

2. The method according to claim 1, It is characterized in that The determining, based on the lane image, the number of target lanes ahead of the road where the target vehicle is located, comprises: Inputting the lane image into a lane line detection model to obtain the number of lane lines in the lane image; Inputting the lane image into a lane line classification model to obtain a lane line category corresponding to each lane line in the lane image; The number of target lanes is determined based on the number of lane lines in the lane image and the lane line categories corresponding to the respective lane lines.

3. The method according to claim 2, It is characterized in that Inputting the lane image into a lane line classification model to obtain a lane line category corresponding to each lane line in the lane image includes: Inputting the lane image into a lane line classification model, and using the lane line classification model, based on the number of lane lines in the lane image and the coordinates of each lane line, dividing the lane image into a preset number of partial lane images, where the preset number is the same as the number of lane lines in the lane image, and the partial lane images are images corresponding to each lane line in the lane image; The lane line classification model is used to determine the lane line category corresponding to the lane line in each local lane image.

4. The method according to claim 3, It is characterized in that Determining the lane line category corresponding to the lane line in each local lane image by using the lane line classification model includes: By using the lane line classification model, image features are extracted from each of the local lane images to obtain image features of each of the local lane images; Mapping the image features of each of the local lane images to the probability corresponding to each candidate lane line category; For any partial lane image among the multiple partial lane images, the candidate lane line category with the highest probability among the candidate lane line categories is determined as the lane line category corresponding to the lane line in the partial lane image.

5. The method according to claim 3, It is characterized in that After determining the lane line category corresponding to the lane line in each local lane image by the lane line classification model, the method further includes: Determining whether there is a partial lane image in which the lane line category is not recognized among the partial lane images; In the case where there is a partial lane image in which the lane line category is not recognized, performing data enhancement processing on the partial image in which the lane line category is not recognized; Inputting the local lane image after data enhancement processing into a lane classification model; The lane line classification model is used to identify the lane line in the partial lane image whose lane line category has not been identified, so as to obtain the lane line category corresponding to the lane line in the partial lane image.

6. The method according to claim 1, It is characterized in that The determining, based on the lane image, the number of target lanes ahead of the road where the target vehicle is located, comprises: Determining whether there is a reference lane line in the lane image, the reference lane line being used to indicate that the lane line is a lane boundary; In a case where there are at least two reference lane lines in the lane image, determining the number of reference lane lines between the reference lane lines based on a difference in lane line orders corresponding to the at least two reference lane lines; The number of target lanes is determined based on the number of reference lane lines between the reference lanes.

7. The method according to claim 6, It is characterized in that The determining whether there is a reference lane line in the lane image includes: Determine whether there is a first reference lane line among the lane lines to the left of the vehicle head position in the lane image, where the first reference lane line is the first boundary lane line to the left of the vehicle head position; In the case where a first reference lane line exists in each lane line to the left of the vehicle head position in the lane image, determining whether a second reference lane line exists in each lane line to the right of the vehicle head position in the lane image, wherein the second reference lane line is the first boundary lane line to the right of the vehicle head position; In a case where there are at least two reference lane lines in the lane image, determining the number of lane lines between the reference lane lines based on a difference in lane line orders corresponding to the at least two reference lane lines includes: When there is a second reference lane line in each lane line to the right of the vehicle head position in the lane image, the difference between the lane line order corresponding to the first reference lane line and the lane line order corresponding to the second reference lane line is determined as the number of reference lane lines between the reference lane lines.

8. The method according to claim 6, It is characterized in that The method further comprises: When there is no reference lane line in the lane image, the number of lane lines in the lane image is reduced by one to obtain the number of reference lane lines.

9. The method according to claim 1, It is characterized in that In the case where there are at least two candidate road types at the location of the target vehicle, determining the road type of the road where the target vehicle is located from the at least two candidate road types based on the target lane number and at least two reference lane numbers in the map information, comprises: In a case where the target lane number is equal to any one of at least two reference lane numbers in the map information, a road type of the road where the target vehicle is located is determined from at least two candidate road types.

10. The method according to claim 9, It is characterized in that The at least two reference lane numbers in the map information include a main road lane number and a secondary road lane number, the candidate road types include a main road lane and a secondary road lane, and when the target lane number is equal to any one of the at least two reference lane numbers in the map information, determining the road type of the road where the target vehicle is located from the at least two candidate road types includes: When the number of the target lanes is equal to the number of the main road lanes in the map information, determining that the road type of the road where the target vehicle is located is a main road lane; Alternatively, when the number of the target lanes is equal to the number of auxiliary road lanes in the map information, it is determined that the road type of the road where the target vehicle is located is an auxiliary road lane.

11. The method according to claim 9, It is characterized in that The candidate road types include main road lanes and auxiliary road lanes, and when the number of target lanes is equal to any one of the at least two reference lane numbers in the map information, determining the road type of the road where the target vehicle is located from the at least two candidate road types includes: When the number of target lanes is equal to any one of at least two numbers of reference lanes in the map information, determining a target lane line category in the lane image, the target lane line category being a lane line category of a road where the target vehicle is currently located; In a case where the target lane line category is the same as the lane line category in the map information, the road type of the road where the target vehicle is located is determined from at least two candidate road types.

12. The method according to claim 11, It is characterized in that When the number of the target lanes is equal to any one of the numbers of at least two reference lanes in the map information, determining the category of the target lane line in the lane image includes: Obtaining the coordinates of each lane line in the lane image and the position information of the road where the target vehicle is located; Determine the lane line coordinates of the road where the target vehicle is located based on the coordinates of each lane line in the lane image and the position information of the road where the target vehicle is located; The target lane line category is determined based on the lane line category corresponding to each lane line in the lane image and the lane line coordinates of the road where the target vehicle is located.

13. The method according to claim 11, It is characterized in that The map information also includes at least two reference lane line categories corresponding to at least two reference lane numbers, and when the target lane line category is the same as the lane line category in the map information, determining the road type of the road where the target vehicle is located from at least two candidate road types includes: When the target lane line category is the same as the main road lane line category in the map information, determining that the road type of the target vehicle is located at a main road lane; Alternatively, when the number of the target lanes is the same as the category of the auxiliary road lane line in the map information, it is determined that the road type of the road where the target vehicle is located is an auxiliary road lane.

14. A lane recognition device, It is characterized in that The device comprises: An acquisition module, used for acquiring a lane image in front of a target vehicle; A module for determining the number of target lanes, used to determine the number of target lanes ahead of the road where the target vehicle is located based on the lane image; A road type determination module is used to determine the road type of the road where the target vehicle is located from the at least two candidate road types based on the target lane number and at least two reference lane numbers in the map information when there are at least two candidate road types at the location of the target vehicle, and different candidate road types correspond to different reference lane numbers.

15. A vehicle, It is characterized in that The vehicle comprises: A memory for storing executable program codes; A processor, configured to call and run the executable program code from the memory so that the vehicle executes the method according to any one of claims 1 to 13.

16. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed, the method according to any one of claims 1 to 13 is implemented.

Citation Information

Patent Citations

  • Road positioning and road switching judgment methods and device, equipment and storage medium

    CN109425350A

  • Vehicle positioning method and device, electronic equipment and computer readable storage medium

    CN111380536A

  • Vehicle positioning method and device, electronic equipment and computer readable storage medium

    CN111381269A

  • Lane number recognition method, device and equipment and storage medium

    CN111814724A

  • Lane positioning method, terminal and computer storage medium

    CN118189984A