Lane Position Obtaining Method, Device, Computer Device, and Storage Medium

Abstract

The present application relates to a method, apparatus, computer device, and storage medium for obtaining a lane position. The method is applied to the traffic field and includes: obtaining a current road image, extracting features from the current road image to obtain road image features, and based on the road image features, extracting lane information to obtain corresponding lane line features and road surface features. Feature fusion is performed based on the road image features, lane line features, and road surface features to obtain road fusion features, and the lane position where the vehicle is currently located is obtained according to the road fusion features. By using this method, the lane position where the current vehicle is located can be accurately determined in different actual scenarios, improving the efficiency of obtaining the lane position, so as to avoid repeatedly performing recognition operations or using the method of additionally adding a high-precision map, thereby reducing resource consumption in the process of obtaining the lane position and further improving the vehicle navigation accuracy.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and particularly to a method, device, computer device, and storage medium for obtaining lane positions. Background Art

[0002] With the development of artificial intelligence technology and the wide application of navigation technology, to ensure safe travel, more and more users use vehicle-mounted devices or mobile terminal devices equipped with navigation functions to guide their travel. Among them, vehicle-mounted devices or mobile terminal devices with navigation functions mostly install navigation application programs to online load multiple routes between the current departure place and the destination, or offline search for multiple routes that meet the requirements in the local map for display, for the user to select, and indicate the current vehicle to drive according to the route selected by the user.

[0003] However, in the traditional way of guiding vehicle driving by navigation application programs, to ensure the safety of the current vehicle's intended driving, it is necessary to produce a high-precision map to match with the current vehicle to determine the road position where the current vehicle is located, and then provide a corresponding navigation plan. However, producing a high-precision map requires consuming a large amount of human and material resources, and the production cost is relatively high, resulting in the problem that the traditional navigation plan still consumes too many resources. Summary of the Invention

[0004] Based on this, to solve the above technical problems, it is necessary to provide a method, device, computer device, and storage medium for obtaining lane positions that can reduce resource consumption and improve the accuracy of vehicle navigation.

[0005] A method for obtaining lane positions, the method includes:

[0006] Obtain a current road image, and perform feature extraction on the current road image to obtain road image features;

[0007] Based on the road image features, perform lane information extraction to obtain corresponding lane line features and road surface features;

[0008] Based on the road image features, lane line features, and road surface features, perform feature fusion to obtain road fusion features;

[0009] Obtain the lane position where the vehicle is currently located according to the road fusion features.

[0010] A device for obtaining lane positions, the device includes:

[0011] A road image feature extraction module, configured to obtain a current road image, and perform feature extraction on the current road image to obtain road image features;

[0012] A lane information extraction module, configured to extract lane information based on the road image features to obtain corresponding lane line features and road surface features;

[0013] A feature fusion module, configured to perform feature fusion based on the road image features, lane line features, and road surface features to obtain road fusion features;

[0014] A lane position acquisition module, configured to acquire the lane position where the vehicle is currently located according to the road fusion features.

[0015] A computer device, comprising a memory and a processor, where the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0016] Acquire a current road image, and perform feature extraction on the current road image to obtain road image features;

[0017] Based on the road image features, perform lane information extraction to obtain corresponding lane line features and road surface features;

[0018] Based on the road image features, lane line features, and road surface features, perform feature fusion to obtain road fusion features;

[0019] Acquire the lane position where the vehicle is currently located according to the road fusion features.

[0020] A computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0021] Acquire a current road image, and perform feature extraction on the current road image to obtain road image features;

[0022] Based on the road image features, perform lane information extraction to obtain corresponding lane line features and road surface features;

[0023] Based on the road image features, lane line features, and road surface features, perform feature fusion to obtain road fusion features;

[0024] Acquire the lane position where the vehicle is currently located according to the road fusion features.

[0025] A computer program product, comprising a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0026] Acquire a current road image, and perform feature extraction on the current road image to obtain road image features;

[0027] Based on the road image features, perform lane information extraction to obtain corresponding lane line features and road surface features;

[0028] Perform feature fusion based on the road image features, lane line features, and road surface features to obtain road fusion features;

[0029] Obtain the lane position where the vehicle is currently located according to the road fusion features.

[0030] In the above method, device, computer equipment, and storage medium for obtaining the lane position, by acquiring the current road image and performing feature extraction on the current road image, road image features are obtained. Further, based on the road image features, lane information is extracted to obtain corresponding lane line features and road surface features. By performing feature fusion based on the road image features, lane line features, and road surface features, road fusion features are obtained, so as to identify and obtain the lane position based on the road fusion features, accurately determine the lane position where the vehicle is currently located. Furthermore, the lane position where the current vehicle is located can be accurately determined in different actual scenarios, improving the efficiency of obtaining the lane position, so as to avoid repeatedly performing recognition operations or adopting the method of additionally adding a high-precision map, thereby reducing the resource consumption in the process of obtaining the lane position and further improving the vehicle navigation accuracy. Description of the Drawings

[0031] Figure 1 It is an application environment diagram of the lane position acquisition method in an embodiment;

[0032] Figure 2 It is a flowchart of the lane position acquisition method in an embodiment;

[0033] Figure 3 It is a flowchart of obtaining corresponding lane line features and road surface features in an embodiment;

[0034] Figure 4 It is a schematic diagram of the actual scenario of lane information extraction and processing in an embodiment;

[0035] Figure 5 It is a flowchart of the lane information extraction and processing in an embodiment;

[0036] Figure 6 It is a flowchart of obtaining road fusion features in an embodiment;

[0037] Figure 7 It is a flowchart of obtaining the lane position where the vehicle is currently located according to the road fusion features in an embodiment;

[0038] Figure 8 It is a flowchart of obtaining the boundary distance feature between the in-vehicle camera of the current vehicle and the current road boundary in an embodiment;

[0039] Figure 9Schematic diagram of a current road image after inverse perspective transformation processing in an embodiment;

[0040] Figure 10 Schematic flow chart of a lane position acquisition method in another embodiment;

[0041] Figure 11 Schematic diagram of the overall flow of a lane position acquisition method in an embodiment;

[0042] Figure 12 Block diagram of the structure of a lane position acquisition device in an embodiment;

[0043] Figure 13 Internal structure diagram of a computer device in an embodiment. Detailed implementation manners

[0044] In order to make the objectives, technical solutions and advantages of the present application clearer and more understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0045] The lane position acquisition method provided by the present application is applied to the traffic field and involves intelligent transportation systems and artificial intelligence technologies. Among them, an intelligent traffic system (ITS), also known as an intelligent transportation system, effectively integrates advanced scientific and technological means (information technology, computer technology, data communication technology, sensor technology, electronic control technology, automatic control theory, operations research, artificial intelligence, etc.) into transportation, service control, and vehicle manufacturing, strengthens the connection among vehicles, roads, and users, and thus forms a comprehensive transportation system that ensures safety, improves efficiency, improves the environment, and saves energy.

[0046] Artificial Intelligence (AI) is a theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a way similar to human intelligence. Artificial intelligence also studies the design principles and implementation methods of various intelligent machines, enabling machines to have the functions of perception, reasoning, and decision-making. Artificial intelligence technology is an interdisciplinary subject with a wide range of fields involved, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation / interaction systems, and mechatronics. The software technologies of artificial intelligence mainly include several major directions such as computer vision technology, speech processing technology, natural language processing technology, as well as machine learning / deep learning, autonomous driving, and intelligent transportation. Autonomous driving technology usually includes technologies such as high-precision maps, environmental perception, behavior decision-making, path planning, and motion control. Autonomous driving technology has broad application prospects.

[0047] With the research and progress of artificial intelligence technology, artificial intelligence technology has been studied and applied in multiple fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, driverless, autonomous driving, drones, robots, intelligent healthcare, intelligent customer service, vehicle networking, autonomous driving, and intelligent transportation. It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0048] The lane position acquisition method provided in this application can be applied to, for example, Figure 1In the application environment shown. Among them, the terminal device 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or placed on the cloud or other network servers. The server 104 obtains the current road image collected by the on-vehicle camera set by the current vehicle, extracts features from the current road image to obtain road image features, and then based on the road image features, extracts lane information to obtain corresponding lane line features and road surface features. The server 104 performs feature fusion based on the road image features, lane line features, and road surface features to obtain road fusion features, and then can obtain the lane position where the vehicle is currently located according to the road fusion features. Further, the server 104 can feedback the lane position to the terminal device 102 corresponding to the current vehicle, where the terminal device 102 can be a smart phone, a tablet computer, a laptop computer, an in-vehicle terminal, a smart TV, etc., but is not limited thereto. The server 104 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal 102 and the server 104 can be directly or indirectly connected through wired or wireless communication methods, and this application does not make any restrictions here.

[0049] In one embodiment, as Figure 2 shown, a method for obtaining a lane position is provided. Taking the server in Figure 1 as an example for illustration, the method includes the following steps:

[0050] Step S202, obtain the current road image, and extract features from the current road image to obtain road image features.

[0051] Specifically, the server obtains the current road image collected by the on-vehicle camera of the current vehicle, and performs an encoding (encoder) operation on the current road image to achieve feature extraction, and obtains the road image features corresponding to the current road image.

[0052] Further, a convolutional neural network can be specifically used to extract features from the current road image. The convolutional neural network includes a convolutional layer (Convolution layer), a normalization layer (Batch Normalization layer, i.e., BN layer), and an activation layer (Relu layer). The convolutional layer is used to extract basic features such as the edge texture of the current road image. The normalization layer is used to normalize the basic features extracted by the convolutional layer according to the normal distribution, filter out the noise features in the features, and make the training of the neural network model converge more quickly. The activation layer is responsible for performing a non-linear mapping on the features extracted by the convolutional layer to enhance the generalization ability of the neural network model.

[0053] In one embodiment, Resnet, that is, Residual Neural Network, can be specifically used to extract features from the current road image to obtain road image features corresponding to the current road image. Among them, other convolutional neural networks can also be used to implement the extraction of road image features, and the specific convolutional neural network is not limited in this application.

[0054] In one embodiment, it is necessary to determine the preliminary road image area of the current road image to be recognized according to the driving direction of the current vehicle, that is, in the current road image, it is necessary to determine the preliminary road image area in the same direction as the driving direction of the current vehicle, and for the determined preliminary road image area in the same direction as the driving direction of the current vehicle, further feature recognition and extraction are performed, while other image areas in the reverse direction of the driving direction of the current vehicle do not need to be further recognized. Step S204, based on the road image features, lane information is extracted to obtain corresponding lane line features and road surface features.

[0055] Specifically, a deconvolution operation is performed based on the road image features to generate a road image feature map of the same size as the current road image, and class prediction is performed based on each image pixel point included in the road image feature map, where the pixel categories included in the image pixel point can include lane line feature points, road surface feature points, and background points.

[0056] Further, after determining the pixel category to which each pixel point belongs, for example, specifically whether it belongs to the pixel category of lane line feature points or road surface features, clustering processing is performed on the features of each pixel point under this pixel category to obtain clustering clusters formed by pixel points under the same category, including clustering clusters under different categories such as lane line feature points, road surface feature points, and background points, and then lane information can be extracted to extract the corresponding lane line features and road surface features.

[0057] Step S206, feature fusion is performed based on the road image features, lane line features, and road surface features to obtain road fusion features.

[0058] Specifically, by obtaining the boundary distance feature between the in-vehicle camera of the current vehicle and the current road boundary, and fusing the road image features, lane line features, road surface features, and boundary distance feature, road fusion features can be obtained. Among them, adding the boundary distance feature to the features to be fused can use the actual distance between the current vehicle and the current road boundary as a consideration factor for subsequent lane position recognition and acquisition. By adding this consideration factor of the actual distance between the current vehicle and the current road boundary, the accuracy of lane position recognition and acquisition can be further improved to effectively solve the problem of incorrect lane position recognition in scenarios such as shadows and tunnels as shown in the following figure.

[0059] In one embodiment, the features to be fused may further include performing a deconvolution operation based on the road image features to generate a road image feature map of the same size corresponding to the current road image; performing feature fusion based on the road image features, lane line features, and road surface features to obtain a road fusion feature, including:

[0060] Performing feature fusion on the road image features, lane line features, road surface features, boundary distance features, and the road image feature map of the same size corresponding to the current road image to generate a road fusion feature.

[0061] Among them, the road image feature map of the same size corresponding to the current road image obtained by performing the deconvolution operation is also used as a consideration factor for subsequent lane position recognition and acquisition. Therefore, by adding this consideration factor of the road image feature map of the same size corresponding to the current road image, the current road image can be recognized and feature extracted at different levels, including different levels such as high-level semantic information and low-level semantic information, further improving the accuracy of lane position recognition and acquisition, and effectively solving the problem of incorrect lane position recognition in scenarios such as shadows and tunnels as shown in the following figure.

[0062] Step S208, obtaining the lane position where the vehicle is currently located according to the road fusion feature.

[0063] Specifically, performing feature transformation on the road fusion feature according to a convolutional neural network. Specifically, it is possible to use the pooling layer (average Pooline) and fully connected layer (dense) included in the convolutional neural network to perform feature transformation on the road fusion feature to obtain a feature vector of a first preset dimension, and further perform vector transpose processing on the feature vector of the first preset dimension to obtain a feature vector of a second preset dimension. Among them, the number of dimensions of the first preset dimension is greater than the number of dimensions of the second preset dimension.

[0064] Among them, the feature vector of the second preset dimension includes a first feature vector, and the first feature vector corresponds to a first boundary. By generating first confidence data corresponding to different lane positions based on the first feature vector, and performing lane position prediction based on the first confidence data, the lane position of the current vehicle with reference to the first boundary is generated.

[0065] Furthermore, the feature vector of the second preset dimension further includes a second feature vector, and the second feature vector corresponds to a second boundary. By generating second confidence data corresponding to different lane positions based on the second feature vector, and then performing lane position prediction based on the second confidence data, the lane position of the current vehicle with reference to the second boundary is generated.

[0066] Among them, the first boundary can be determined according to the current road image and the driving direction defined by the current road. The road boundary can include the double yellow line for distinguishing the driving direction of vehicles and the actual road boundary. According to the driving direction defined by the road, the double yellow line can be determined as the first boundary, that is, the left boundary, and the actual road boundary can be determined as the second boundary, that is, the right boundary, or according to the driving direction defined by the road, the actual road boundary can be determined as the first boundary, that is, the left boundary, and the double yellow line can be determined as the second boundary, that is, the right boundary.

[0067] In one embodiment, after obtaining the lane position where the vehicle is currently located according to the road fusion feature, it further includes:

[0068] Perform projection adsorption processing on the current road image to obtain the corresponding road adsorption positioning result, and generate the corresponding lane-level navigation data according to the lane position and the road adsorption positioning result, and then indicate the current vehicle to drive based on the lane-level navigation data.

[0069] Specifically, by performing projection adsorption processing on the current road image, that is, inverse perspective mapping (IPM), the lane lines in the image coordinate system are projected onto the camera coordinate system to generate the road adsorption positioning result.

[0070] Among them, during the driving process of the vehicle, in the current road image captured by the on-vehicle camera, due to the existence of the perspective effect, things that are originally parallel are shown as intersecting in the image. Therefore, it is necessary to use inverse perspective mapping to eliminate the perspective effect, obtain accurate current road image data, and determine the absolute position of the current vehicle on the road.

[0071] Furthermore, by combining the lane position and the road adsorption result, the lane position where the current vehicle is located can be more accurately determined, and the lane-level navigation data can be generated. In the case where a lane is added on the current road or a lane change is required during the current driving process, the current vehicle can be accurately guided to drive.

[0072] In the above method for obtaining the lane position, by acquiring the current road image and performing feature extraction on the current road image, road image features are obtained. Further, based on the road image features, lane information is extracted to obtain corresponding lane line features and road surface features. By performing feature fusion based on the road image features, lane line features, and road surface features, road fusion features are obtained, and based on the road fusion features, the lane position is recognized and obtained, accurately determining the lane position where the vehicle is currently located. Furthermore, the lane position where the current vehicle is located can be accurately determined in different actual scenarios, improving the efficiency of obtaining the lane position, so as to avoid repeatedly performing recognition operations or adopting the method of additionally adding a high-precision map, thereby reducing the resource consumption in the process of obtaining the lane position and further improving the vehicle navigation accuracy.

[0073] In one embodiment, as Figure 3 shown, the step of obtaining the corresponding lane line features and road surface features, that is, the step of extracting lane information based on the road image features to obtain the corresponding lane line features and road surface features, specifically includes:

[0074] Step S302, perform a deconvolution operation based on the road image features to generate a road image feature map with the same size as the current road image.

[0075] Specifically, use a convolutional neural network to perform a deconvolution (decoder) operation on the road image features to generate a road image feature map with the same size as the current road image.

[0076] Step S304, perform semantic segmentation processing on the road image feature map to generate a corresponding pixel point category prediction result.

[0077] Specifically, after performing a deconvolution operation on the road image features to obtain a road image feature map with the same size as the current road image, it is understood that the obtained is the first road image feature map, and based on this first road image feature map, a share encoder operation is performed, that is, an encoding operation to extract higher-level semantic information. After obtaining a road image feature map including higher-level semantic information, it is understood that the obtained is the second road image feature map.

[0078] Furthermore, based on this second road image feature map, semantic segmentation processing (segmentation tranch processing) is performed. By predicting the pixel point category to which each pixel point on the second road image feature map belongs, that is, determining which pixel point category each pixel point on the second road image feature map belongs to, a corresponding pixel point category prediction result is obtained.

[0079] Among them, the pixel categories may include different categories such as lane line feature points, road surface feature points, and background points.

[0080] Step S306: Perform instance segmentation processing on the road image feature map to generate a corresponding lane pixel attribute prediction result.

[0081] Specifically, after performing a deconvolution operation on the road image features to obtain a first road image feature map of the same size as the current road image, and further performing a share encoder operation based on this first road image feature map to obtain a second road image feature map including higher-level semantic information, perform instance segmentation processing (i.e., embedding tranch processing) based on the second road image feature map to obtain the corresponding lane pixel attribute prediction result.

[0082] Among them, the embedding tranch processing is used to perform an embedded representation on each pixel point on the second road image feature map to train the corresponding embedding vector (embedded vector) to achieve subsequent clustering processing. Among them, the embedding vector can be used to distinguish which lane each pixel point on the second road image feature map specifically belongs to, that is, after distinguishing whether each pixel point belongs to the pixel category corresponding to the lane line feature points, further determine which lane line each pixel point belongs to respectively.

[0083] Similarly, for the pixel category corresponding to the road surface feature points, the corresponding pixel points can also be determined to specifically belong to which area of the road surface by using the embedding vector.

[0084] Step S308: Perform feature clustering based on the pixel point category prediction result and the lane pixel attribute prediction result to obtain clustering clusters corresponding to different pixel point categories.

[0085] Specifically, according to the pixel point category prediction result, that is, after determining which pixel category each pixel point belongs to, combined with the lane pixel attribute prediction result, that is, under the pixel category corresponding to the lane line feature points, determine which lane line these pixel points specifically belong to, and then, for each pixel category, cluster the corresponding pixel points respectively to obtain clustering clusters corresponding to different pixel point categories.

[0086] Step S310: Determine the lane line features and road surface features corresponding to the road image feature map according to the clustering clusters corresponding to different pixel point categories.

[0087] Specifically, for the pixel categories corresponding to the lane line feature points, based on the prediction results of the lane phase pixel attributes of different pixel points, that is, determining the specific lane lines to which each pixel point belongs, clustering processing can be performed, and the obtained clustering clusters can be used to represent which lane the pixel points in the corresponding category specifically belong to, obtaining more accurate lane line features.

[0088] Similarly, for the pixel point categories corresponding to the road surface feature points, based on the prediction results of the lane phase pixel attributes of different pixel points, it can be determined which area of the road surface the corresponding pixel points specifically belong to. Based on the determined road surface results, clustering processing is performed, and the obtained clustering clusters can be used to represent which lane of the road surface the corresponding pixel points specifically belong to, obtaining more accurate road surface features.

[0089] Furthermore, after obtaining the lane line features and road surface features corresponding to the road image feature map, the specific number of lanes, the corresponding lane lines, and the positions of the road surface can be accurately identified on the finally processed feature map.

[0090] In one embodiment, as Figure 4 shown, a practical scenario of lane information extraction and processing is provided. Referring to Figure 4 it can be seen that by acquiring the current road image collected by the in-vehicle camera of the current vehicle, as shown in Figure (a) in Figure 5 , and performing lane information extraction based on the current road image shown in Figure (a), a lane line and a road surface feature map as shown in Figure (b) in Figure 4 can be obtained.

[0091] In one embodiment, as Figure 5 shown, a schematic flow diagram of lane information extraction and processing is provided. Referring to Figure 5 it can be seen that based on the road image features for lane information extraction, it specifically includes the following parts:

[0092] P1: Perform a deconvolution operation based on the road image features to generate a first road image feature map of the same size corresponding to the current road image.

[0093] P2: Perform a shared encoder operation based on the first road image feature map to extract higher-level semantic information and obtain a second road image feature map.

[0094] P3: Perform an embedding tranch processing based on the second road image feature map to obtain Pixelembeddings (embedded vectors).

[0095] P4: Perform segmentation tranch processing on the second road image feature map to obtain binary lane segmentation (the pixel point categories to which the road image features belong).

[0096] P5: Perform clustering processing based on Pixel embeddings and binary lane segmentation to obtain clustering (clustering clusters) corresponding to different pixel point categories.

[0097] P6: Determine lane line features and road surface features based on the clustering clusters corresponding to different pixel point categories.

[0098] In this embodiment, by performing a deconvolution operation on the road image features, a road image feature map of the same size as the current road image is generated, and semantic segmentation processing and instance segmentation processing are performed based on the road image feature map to generate corresponding pixel point category prediction results and lane pixel attribute prediction results. Furthermore, based on the pixel point category prediction results and lane pixel attribute prediction results, feature clustering is performed to obtain clustering clusters corresponding to different pixel point categories, and based on the clustering clusters corresponding to different pixel point categories, lane line features and road surface features corresponding to the road image feature map are determined. The accurate segmentation and recognition processing of the road image features are realized to obtain accurate lane line features and road surface features, so as to further improve the correct rate of subsequent lane position recognition, avoid the need to repeatedly distinguish lane lines and road surfaces, and thus improve the lane position acquisition efficiency.

[0099] In one embodiment, as Figure 6 shown, the steps of obtaining the road fusion features, that is, the steps of fusing road image features, lane line features, road surface features, and boundary distance features to obtain road fusion features, specifically include:

[0100] Step S602: Perform a first feature dimension mapping on the road image features according to the dimension data corresponding to the lane line features and road surface features to obtain a first splicing feature.

[0101] Specifically, through the feature dimension of the road image features, as well as the dimension data corresponding to the lane line features and road surface features, and according to the dimension data corresponding to the lane line features and road surface features, perform a first feature dimension mapping on the road image features to obtain the corresponding first splicing feature.

[0102] In one embodiment, the feature dimension of the road image feature is W1×H1×D, where W1×H1 represents the size of the road image feature, and D represents the number of channels of the feature. The feature dimensions of the lane line feature and the road surface feature are W2×H2×1, where W2×H2 represents the size of the road image feature, and the number of channels is 1.

[0103] Further, according to the dimension data corresponding to the lane line feature and the road surface feature, perform a first feature dimension mapping on the road image feature, map the road image feature to the W2×H2 dimension, and obtain a first spliced feature.

[0104] Step S604: Splice the first spliced feature, the lane line feature, and the road surface feature to obtain a second spliced feature.

[0105] Specifically, splice the first spliced feature mapped from the road image feature, the lane line feature, and the road surface feature to obtain a second spliced feature. Among them, the first spliced feature mapped from the road image feature and the second spliced feature have a feature dimension of the W2×H2 dimension, which is consistent with the feature dimension corresponding to the lane line feature and the road surface feature.

[0106] Step S606: Perform a second feature dimension mapping on the second spliced feature according to a preset dimension mapping function to obtain a third spliced feature.

[0107] Specifically, perform a second feature dimension mapping on the second spliced feature according to the preset dimension mapping function and the feature dimension corresponding to the preset dimension mapping function to obtain a third spliced feature.

[0108] Among them, the feature dimension corresponding to the preset dimension mapping function and the feature dimension of the third spliced feature can be the N×1 dimension, so the second spliced feature can be mapped from the W2×H2 dimension to the N×1 dimension to obtain the third spliced feature.

[0109] Step S608: Splice the boundary distance feature and the third spliced feature to obtain a road fusion feature.

[0110] Specifically, obtain the boundary distance feature between the in-vehicle camera of the current vehicle and the current road boundary, and splice the boundary distance feature and the third spliced feature to obtain a road fusion feature.

[0111] Among them, the boundary distance feature includes 2 dimensions, that is, the first distance from the first boundary and the second distance from the second boundary. By splicing the first distance, the second distance, and the third spliced feature, a road fusion feature is obtained. Among them, the feature dimension of the road fusion feature is (N + 2)×1.

[0112] In one embodiment, the following formula (1) is specifically used to perform feature fusion processing to obtain the road fusion feature C:

[0113]

[0114] Wherein, A is the road image feature, and the feature dimension of A is W1×H1×D. Among them, W1×H1 represents the size of the road image feature, and D represents the number of channels of the feature. B represents the lane line feature and the road surface feature, and the feature dimension of B is W2×H2×1. Among them, W2×H2 represents the size of the road image feature, and the number of channels is 1.

[0115] The α function is used to perform the first feature dimension mapping on the road image feature A, map the road image feature to the W2×H2 dimension, and obtain the first splicing feature And the first splicing feature and the feature B are spliced to obtain the second splicing feature

[0116] The function represents a dimension mapping function, which is used to perform the second feature dimension mapping on the second splicing feature to obtain the third splicing feature

[0117] d1 represents the first distance between the first boundaries, d2 represents the second distance between the second boundaries, and the concat function is used to realize the splicing of the boundary distance features d1, d2 and the third splicing feature to obtain the road fusion feature C.

[0118] In this embodiment, according to the dimension data corresponding to the lane line feature and the road surface feature, the first feature dimension mapping is performed on the road image feature to obtain the first splicing feature, and the first splicing feature, the lane line feature and the road surface feature are spliced to obtain the second splicing feature. Further, according to the preset dimension mapping function, the second feature dimension mapping is performed on the second splicing feature to obtain the third splicing feature, and the boundary distance feature and the third splicing feature are spliced to obtain the road fusion feature. It realizes feature extraction from different angles and multi-angle feature fusion, so that the finally obtained road fusion feature integrates multi-angle features. When subsequently following up the road fusion feature for lane position recognition and acquisition, the accuracy of the corresponding recognition and acquisition operations can be improved.

[0119] In one embodiment, as Figure 7 shown, the steps of obtaining the lane position where the vehicle is currently located according to the road fusion feature specifically include:

[0120] Step S702: Perform feature transformation on the road fusion features according to the convolutional neural network to obtain a feature vector of a first preset dimension.

[0121] Specifically, perform feature transformation on the road fusion features according to the pooling layer (average Pooline) and the fully connected layer (dense) of the convolutional neural network to obtain a feature vector of a first preset dimension.

[0122] Among them, the first preset dimension can be set and adjusted according to actual needs without specific limitation. In this embodiment, an example of the first preset dimension being 1×10 dimension is used for illustration.

[0123] Step S704: Transpose the feature vector of the first preset dimension to obtain a corresponding feature vector of a second preset dimension. The feature vector of the second preset dimension includes a first feature vector corresponding to the first boundary and a second feature vector corresponding to the second boundary.

[0124] Specifically, transpose the feature vector of the first preset dimension to obtain a corresponding feature vector of the second preset dimension. Specifically, by transposing the 1×10 dimension feature vector, two 1×5 dimension feature vectors are obtained, including a first feature vector corresponding to the first boundary and a second feature vector corresponding to the second boundary.

[0125] Step S706: Determine the first confidence data corresponding to different lane positions according to the first feature vector.

[0126] Specifically, based on the first feature vector, that is, one of the 1×5 dimension feature vectors, obtain the specific values of the first feature vector and determine them as the first confidence data corresponding to different lane positions.

[0127] Among them, the first feature vector corresponds to the first boundary. In this embodiment, taking the double yellow line as the first boundary according to the driving direction of the current vehicle, obtain a set of values of the first feature vector, specifically [0.1, 0.2, 0.6, 0.1, 0.0], as the first confidence data for different lane positions.

[0128] Step S708: Perform lane position prediction based on the first confidence data to generate the lane position of the current vehicle with reference to the first boundary.

[0129] Specifically, perform lane position prediction based on the first confidence data, screen out the confidence data with the largest value, and determine the current lane position according to the largest confidence data.

[0130] For example, the first eigenvector [0.1, 0.2, 0.6, 0.1, 0.0] is determined as the first confidence data for different lane positions. Among them, the confidence data with the largest value is 0.6. Then, further based on the confidence data 0.6 with the largest value, it can be determined that the current lane position is the third lane with reference to the first boundary.

[0131] Step S710: Determine the second confidence data corresponding to different lane positions according to the second eigenvector.

[0132] Specifically, based on the second eigenvector, that is, one of the 1×5 - dimensional eigenvectors, obtain the specific values of the second eigenvector and determine them as the second confidence data corresponding to different lane positions.

[0133] Furthermore, the second eigenvector corresponds to the second boundary. In this embodiment, according to the driving direction of the current vehicle, the actual road boundary is used as the second boundary for description. One set of values of the second eigenvector is obtained, specifically [0.2, 0.5, 0.1, 0.2, 0.0], as the second confidence data for different lane positions.

[0134] Step S712: Perform lane position prediction based on the second confidence data to generate the lane position of the current vehicle with reference to the second boundary.

[0135] Specifically, perform lane position prediction based on the second confidence data, screen out the confidence data with the largest value, and determine the current lane position according to the largest confidence data.

[0136] For example, the second eigenvector [0.2, 0.5, 0.1, 0.2, 0.0] is determined as the second confidence data for different lane positions. Among them, the confidence data with the largest value is 0.5. Then, further based on the confidence data 0.5 with the largest value, it can be determined that the current lane position is the second lane with reference to the second boundary.

[0137] Furthermore, taking the above - mentioned first eigenvector [0.1, 0.2, 0.6, 0.1, 0.0] and second eigenvector [0.2, 0.5, 0.1, 0.2, 0.0] as examples, it is determined that the lane position where the current vehicle is located is the third lane with reference to the first boundary and also the second lane with reference to the second boundary. Then, it can be simply understood as the third lane from the left and the second lane from the right, and it can be known that there are a total of four lanes on the current road.

[0138] Among them, when the current vehicle changes lanes or the current road adds lanes, by recognizing the current road image and obtaining the lane position, the driving situation of the current vehicle and the actual changes of the current road can be monitored in a timely manner to ensure the safe travel of the vehicle. In this embodiment, the road fusion features are transformed according to the convolutional neural network to obtain a feature vector of a first preset dimension, and the vector transpose of the feature vector of the first preset dimension is performed to obtain a first feature vector and a second feature vector of a corresponding second preset dimension. Furthermore, according to the first feature vector, the first confidence data corresponding to different lane positions is determined, and the lane position prediction is performed based on the first confidence data to generate the lane position of the current vehicle with the first boundary as a reference. Similarly, according to the second feature vector, the second confidence data corresponding to different lane positions can be determined, and the lane position prediction is performed based on the second confidence data to generate the lane position of the current vehicle with the second boundary as a reference. It realizes that only by performing feature transformation on the road fusion features of the current road image to obtain feature vectors, the lane position where the current vehicle is located can be accurately identified, and at the same time, the overall number of lanes on the road can be obtained, without the need to additionally draw or add a high-precision map for matching and identification, reducing resource consumption while improving the identification accuracy of the lane position.

[0139] In one embodiment, as Figure 8 shown, the steps of obtaining the boundary distance feature between the in-vehicle camera of the current vehicle and the current road boundary specifically include:

[0140] Step S802, determining the rotation matrix of the in-vehicle camera according to the installation angle of the in-vehicle camera of the current vehicle.

[0141] Specifically, by obtaining the installation angle of the in-vehicle camera of the current vehicle, such as the installation angle of the in-vehicle camera set when the vehicle leaves the factory or the installation angle of the in-vehicle camera obtained by real-time online calibration, the rotation matrix R of the vehicle camera is calculated according to the installation angle of the in-vehicle camera.

[0142] Step S804, generating the position data of the road boundary of the current road in the current road image according to the current road image and the road boundary information of the current road.

[0143] Specifically, by obtaining the current road image and the boundary information of the current road, the boundary information of the current road can be understood as, for example, whether the first boundary and the second boundary are double yellow lines or the actual road boundary, and according to the current road image and the boundary information of the current road, the position data of the road boundary of the current road in the current road image is generated.

[0144] Step S806: Obtain the internal parameters of the in-vehicle camera, the ground height of the in-vehicle camera from the current road, and the translation distance between the in-vehicle camera and the current road.

[0145] Specifically, obtain the internal parameters K of the in-vehicle camera, the ground height h of the in-vehicle camera from the current road, and the translation distance t between the in-vehicle camera and the current road. Among them, the internal parameters may include parameters related to the characteristics of the camera itself, such as the focal length and pixel size of the camera.

[0146] Step S808: Calculate the characteristic point coordinates of the road boundary based on the position data, rotation matrix, internal parameters, ground height, and translation distance.

[0147] Specifically, calculate the characteristic point coordinates (X, Y) of the road boundary based on the position data (u, v), rotation matrix R, internal parameters K, ground height h, and translation distance t.

[0148] Further, specifically use the following formula (2) to calculate the characteristic point coordinates (X, Y):

[0149]

[0150] Among them, Z is the scale factor of the in-vehicle camera, P uv corresponds to the position data (u, v), ZP uv corresponds to the characteristic point coordinates (X, Y), K is the internal parameter of the in-vehicle camera, R is the rotation matrix corresponding to the in-vehicle camera, P w corresponds to the position of the three-dimensional point in the world coordinate system, t is the translation distance between the in-vehicle camera and the current road. By setting the origin of the world coordinate system to the ground directly below the camera, the ground height of the in-vehicle camera from the current road is h.

[0151] Step S810: Determine the boundary distance feature between the in-vehicle camera of the current vehicle and the current road boundary based on the characteristic point coordinates of the road boundary.

[0152] Specifically, based on the calculated characteristic point coordinates (X, Y) of the road boundary, and taking the point directly below the in-vehicle camera as the origin (0, 0, 0) of the world coordinate system. Assuming the road is a plane, the three-dimensional coordinates of the point on the ground are (X, Y, 0). Then, based on the three-dimensional coordinates (X, Y, 0) of the point on the ground, use the following formula (3) to calculate the distance between the characteristic point of the road boundary and the in-vehicle camera:

[0153]

[0154] Among them, d is the distance between the feature points of the road boundary and the in-vehicle camera, including the first distance d1 corresponding to the first boundary and the second distance d2 corresponding to the second boundary. X and Y are the coordinates of the feature points of the road boundary, and h is the height of the in-vehicle camera from the ground of the current road.

[0155] In one embodiment, after obtaining the lane position where the vehicle is currently located according to the road fusion feature, it further includes:

[0156] Performing projection adsorption processing on the current road image according to the installation angle of the in-vehicle camera, the ground height, and the calibration parameters between the current vehicle and the in-vehicle camera to obtain the corresponding road adsorption positioning result;

[0157] Generating corresponding lane-level navigation data according to the lane position and the road adsorption positioning result;

[0158] Indicating the current vehicle to travel based on the lane-level navigation data.

[0159] Specifically, according to the installation angle of the in-vehicle camera, the ground height, and the calibration parameters between the current vehicle and the in-vehicle camera, where the calibration parameters between the current vehicle and the in-vehicle camera are parameters in the world coordinate system, such as the position and rotation direction of the camera, perform projection adsorption processing on the current road image.

[0160] Among them, specifically, perform inverse perspective transformation processing (IPM, Inverse perspective mapping) on the current road image to project the lane lines in the image coordinate system into the camera coordinate system to generate the road adsorption positioning result.

[0161] Furthermore, during the vehicle driving process, in the current road image captured by the in-vehicle camera, due to the existence of the perspective effect, things that are originally parallel are shown as intersecting in the image. Therefore, it is necessary to use inverse perspective transformation processing to eliminate the perspective effect, obtain accurate current road image data, and determine the absolute position of the current vehicle on the road. Furthermore, corresponding lane-level navigation data can be generated according to the lane position and the road adsorption positioning result, and the current vehicle can be indicated to travel based on the lane-level navigation data.

[0162] In one embodiment, as Figure 9 shown, a current road image after inverse perspective transformation processing is provided. Referring to Figure 9 it can be seen that after performing inverse perspective transformation processing on the current road image (c) captured by the in-vehicle camera, the image (d) after IPM projection is obtained. In the image (d) after IPM projection, the perspective effect is eliminated, and a schematic diagram of accurate lane lines and the road surface is obtained, and there is no intersecting situation caused by the perspective effect.

[0163] In this embodiment, according to the installation angle of the on-vehicle camera of the current vehicle, the rotation matrix of the on-vehicle camera is determined, and according to the current road image and the road boundary information of the current road, the position data of the road boundary of the current road in the current road image is generated. Further, by obtaining the built-in parameters of the on-vehicle camera, the ground height of the on-vehicle camera from the current road, and the translation distance between the on-vehicle camera and the current road, the characteristic point coordinates of the road boundary are calculated according to the position data, the rotation matrix, the built-in parameters, the ground height, and the translation distance. According to the characteristic point coordinates of the road boundary, the boundary distance characteristic between the on-vehicle camera of the current vehicle and the current road boundary can be determined. It is realized to accurately determine the boundary distance characteristic between the current road boundary and the on-vehicle camera of the vehicle according to the road image collected by the on-vehicle camera of the current vehicle and the road boundary information. When performing subsequent feature fusion, the boundary distance characteristic is used as one of the considerations for feature fusion. By adding the actual distance between the current vehicle and the current road boundary as a consideration factor, the accuracy of lane position recognition and acquisition can be further improved, so as to effectively solve the problem of incorrect lane position recognition in scenarios such as shadows and tunnels as shown in the following figure.

[0164] In one embodiment, as Figure 10 shown, a method for obtaining lane position is provided. Referring to Figure 10 it can be seen that the method for obtaining lane position specifically includes the following steps:

[0165] Step S1001: Obtain the current road image and perform feature extraction on the current road image to obtain road image features.

[0166] Step S1002: Perform a deconvolution operation based on the road image features to generate a road image feature map of the same size corresponding to the current road image.

[0167] Step S1003: Perform semantic segmentation processing based on the road image feature map to generate a corresponding pixel point category prediction result.

[0168] Step S1004: Perform instance segmentation processing based on the road image feature map to generate a corresponding lane pixel attribute prediction result.

[0169] Step S1005: Perform feature clustering according to the pixel point category prediction result and the lane pixel attribute prediction result to obtain clustering clusters corresponding to different pixel point categories.

[0170] Step S1006: Determine the lane line feature and the road surface feature corresponding to the road image feature map according to the clustering clusters corresponding to different pixel point categories.

[0171] Step S1007, obtain the boundary distance feature between the on-vehicle camera of the current vehicle and the current road boundary.

[0172] Step S1008, perform a first feature dimension mapping on the road image feature according to the dimension data corresponding to the lane line feature and the road surface feature, to obtain a first spliced feature.

[0173] Step S1009, splice the first spliced feature, the lane line feature, and the road surface feature, to obtain a second spliced feature.

[0174] Step S1010, perform a second feature dimension mapping on the second spliced feature according to a preset dimension mapping function, to obtain a third spliced feature.

[0175] Step S1011, splice the boundary distance feature and the third spliced feature, to obtain a road fusion feature.

[0176] Step S1012, perform a feature transformation on the road fusion feature according to a convolutional neural network, to obtain a feature vector of a first preset dimension.

[0177] Step S1013, perform a vector transpose on the feature vector of the first preset dimension, to obtain a corresponding feature vector of a second preset dimension, where the feature vector of the second preset dimension includes a first feature vector corresponding to a first boundary and a second feature vector corresponding to a second boundary.

[0178] Step S1014, determine first confidence data corresponding to different lane positions according to the first feature vector.

[0179] Step S1015, perform a lane position prediction based on the first confidence data, to generate a lane position of the current vehicle with reference to the first boundary.

[0180] Step S1016, determine second confidence data corresponding to different lane positions according to the second feature vector.

[0181] Step S1017, perform a lane position prediction based on the second confidence data, to generate a lane position of the current vehicle with reference to the second boundary.

[0182] Step S1018, perform a projection adsorption process on the current road image according to the installation angle of the on-vehicle camera, the ground height, and the calibration parameters between the current vehicle and the on-vehicle camera, to obtain a corresponding road adsorption positioning result.

[0183] Step S1019, generate corresponding lane-level navigation data according to the lane position and the road adsorption positioning result.

[0184] Step S1020, indicate the current vehicle to travel based on the lane-level navigation data.

[0185] In the above lane position acquisition method, by acquiring the current road image and extracting features from the current road image, road image features are obtained. Further, based on the road image features, lane information is extracted to obtain corresponding lane line features and road surface features. By performing feature fusion based on the road image features, lane line features, and road surface features, road fusion features are obtained, and based on the road fusion features, the recognition and acquisition of the lane position are performed to accurately determine the lane position where the vehicle is currently located. Furthermore, the lane position where the current vehicle is located can be accurately determined in different actual scenarios, improving the efficiency of lane position acquisition, so as to avoid repeatedly performing recognition operations or adopting the method of additionally adding a high-precision map, thereby reducing the resource consumption in the process of lane position acquisition and further improving the vehicle navigation accuracy.

[0186] In one embodiment, as Figure 11 shown, an overall process of a lane position acquisition method is provided. Referring to Figure 11 it can be known that this lane position acquisition method specifically includes:

[0187] S1: Acquisition and obtaining of the current road image.

[0188] S2: Feature extraction of the road image:

[0189] 1) Perform an encoder operation (encoding operation) on the current road image to obtain a road image feature feature corresponding to the current road image.

[0190] 2) Perform a decoder operation (deconvolution operation) on the road image feature feature. After obtaining a road image feature map with the same size as the current road image, further perform a segment process (segmentation process) on the road image feature map, including semantic segmentation process (segmentation tranch process) and instance segmentation process (embedding tranch process), to obtain lane line features and road surface features.

[0191] S3: Feature fusion:

[0192] 1) Obtain the boundary distance feature between the in-vehicle camera of the current vehicle and the current road boundary, that is, photogrammetry information.

[0193] 2) Concatenate (concat) the road image feature feature, lane line features, road surface features, boundary distance feature, and road image feature map to obtain road fusion features.

[0194] S4: Lane position acquisition:

[0195] The road fusion features are identified and classified by a classification head to generate the lane positions corresponding to the first boundary (which can be considered as the left boundary) and the lane positions corresponding to the second boundary (which can be considered as the right boundary).

[0196] In the above method for obtaining lane positions, the current road image is acquired, and features of the current road image are extracted to obtain road image features. Further, based on the road image features, lane information is extracted to obtain corresponding lane line features and road surface features. By performing feature fusion based on the road image features, lane line features, and road surface features, road fusion features are obtained, and based on the road fusion features, the lane positions are identified and obtained, accurately determining the lane position where the vehicle is currently located. Furthermore, the lane position where the current vehicle is located can be accurately determined in different actual scenarios, improving the efficiency of obtaining lane positions, so as to avoid repeatedly performing recognition operations or adopting the method of additionally adding high-precision maps, thereby reducing resource consumption in the process of obtaining lane positions and further improving the vehicle navigation accuracy.

[0197] This application also provides an application scenario that applies the above method for obtaining lane positions. Specifically, the application of the method for obtaining lane positions in this application scenario is as follows:

[0198] The method for obtaining lane positions can be integrated into mobile terminal devices and in-vehicle terminal devices. Among them, mobile terminal devices can include smartphones, smart bracelets, tablet computers, laptop computers, and smart TVs, etc. By integrating with mobile terminal devices or in-vehicle terminal devices, data interaction can be performed with the server in real time, and the corresponding lane positions and corresponding lane-level navigation data are fed back to each mobile terminal device or in-vehicle terminal device for display and to guide the current vehicle to travel.

[0199] It should be understood that although the steps in the respective flowcharts involved in the above embodiments are displayed in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the respective flowcharts involved in the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0200] In one embodiment, as Figure 12As shown in the figure, a lane position acquisition device is provided. This device can be a software module, a hardware module, or a combination of both to form a part of a computer device. Specifically, the device includes: a road image feature extraction module 1202, a lane information extraction module 1204, a feature fusion module 1206, and a lane position acquisition module 1208, where:

[0201] The road image feature extraction module 1202 is used to acquire the current road image and extract features from the current road image to obtain road image features.

[0202] The lane information extraction module 1204 is used to extract lane information based on the road image features to obtain corresponding lane line features and road surface features.

[0203] The feature fusion module 1206 is used to perform feature fusion based on the road image features, lane line features, and road surface features to obtain road fusion features.

[0204] The lane position acquisition module 1208 is used to obtain the lane position where the vehicle is currently located according to the road fusion features.

[0205] In the above lane position acquisition device, by acquiring the current road image and extracting features from the current road image, road image features are obtained. Further, based on the road image features, lane information is extracted to obtain corresponding lane line features and road surface features. By performing feature fusion based on the road image features, lane line features, and road surface features, road fusion features are obtained, so as to identify and acquire the lane position based on the road fusion features, and accurately determine the lane position where the vehicle is currently located. Furthermore, the lane position where the current vehicle is located can be accurately determined in different actual scenarios, improving the efficiency of lane position acquisition, avoiding repeated execution of recognition operations or using the method of additionally adding a high-precision map, thereby reducing resource consumption in the process of lane position acquisition and further improving the accuracy of vehicle navigation.

[0206] In one embodiment, a lane position acquisition device is provided, which further includes a boundary distance feature acquisition module for:

[0207] Acquiring the boundary distance feature between the in-vehicle camera of the current vehicle and the current road boundary.

[0208] In one embodiment, the boundary distance feature acquisition module is further used for:

[0209] Determine the rotation matrix of the vehicle-mounted camera based on the installation angle of the vehicle-mounted camera of the current vehicle; generate the position data of the road boundary of the current road in the current road image according to the current road image and the road boundary information of the current road; obtain the internal parameters of the vehicle-mounted camera, the ground height of the vehicle-mounted camera from the current road, and the translation distance between the vehicle-mounted camera and the current road; calculate the coordinate of the feature point of the road boundary according to the position data, the rotation matrix, the internal parameters, the ground height, and the translation distance; determine the boundary distance feature between the vehicle-mounted camera of the current vehicle and the current road boundary according to the coordinate of the feature point of the road boundary.

[0210] In one embodiment, the feature fusion module is further configured to: fuse the road image feature, the lane line feature, the road surface feature, and the boundary distance feature to obtain the road fusion feature.

[0211] In one embodiment, the feature fusion module is further configured to:

[0212] Perform a first feature dimension mapping on the road image feature according to the dimension data corresponding to the lane line feature and the road surface feature to obtain a first splicing feature; splice the first splicing feature, the lane line feature, and the road surface feature to obtain a second splicing feature; perform a second feature dimension mapping on the second splicing feature according to a preset dimension mapping function to obtain a third splicing feature; splice the boundary distance feature and the third splicing feature to obtain the road fusion feature.

[0213] In one embodiment, the lane information extraction module is further configured to:

[0214] Perform a deconvolution operation based on the road image feature to generate a road image feature map of the same size corresponding to the current road image; perform semantic segmentation processing on the road image feature map to generate a corresponding pixel point class prediction result; perform instance segmentation processing on the road image feature map to generate a corresponding lane pixel attribute prediction result; perform feature clustering according to the pixel point class prediction result and the lane pixel attribute prediction result to obtain clustering clusters corresponding to different pixel point classes; determine the lane line feature and the road surface feature corresponding to the road image feature map according to the clustering clusters corresponding to different pixel point classes.

[0215] In one embodiment, the lane position acquisition module is further configured to:

[0216] Perform feature transformation on the road fusion features according to the convolutional neural network to obtain a feature vector of a first preset dimension; perform vector transposition on the feature vector of the first preset dimension to obtain a corresponding feature vector of a second preset dimension; the feature vector of the second preset dimension includes a first feature vector corresponding to a first boundary; determine first confidence data corresponding to different lane positions according to the first feature vector; perform lane position prediction based on the first confidence data to generate a lane position of the current vehicle with reference to the first boundary.

[0217] In one embodiment, the lane position acquisition module is further configured to:

[0218] Determine second confidence data corresponding to different lane positions according to the second feature vector; perform lane position prediction based on the second confidence data to generate a lane position of the current vehicle with reference to the second boundary.

[0219] In one embodiment, a lane position acquisition device is provided, further including:

[0220] A road adsorption positioning result generation module, configured to perform projection adsorption processing on the current road image according to the installation angle of the vehicle-mounted camera, the ground height, and the calibration parameters between the current vehicle and the vehicle-mounted camera to obtain a corresponding road adsorption positioning result;

[0221] A lane-level navigation data generation module, configured to generate corresponding lane-level navigation data according to the lane position and the road adsorption positioning result;

[0222] A vehicle navigation module, configured to instruct the current vehicle to travel based on the lane-level navigation data.

[0223] For the specific limitations of the lane position acquisition device, reference may be made to the limitations on the lane position acquisition method in the foregoing text, which will not be elaborated here. Each module in the above lane position acquisition device can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor of the computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above modules.

[0224] In one embodiment, a computer device is provided. The computer device can be a server, and its internal structure diagram can be as Figure 13As shown. The computer device includes a processor, a memory, and a network interface connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of the computer device is used to store data such as road image features, lane line features, road surface features, road fusion features, boundary distance features, first confidence data, second confidence data, and lane positions. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, it implements a method for obtaining lane positions.

[0225] Those skilled in the art can understand that Figure 13 the structure shown in is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0226] In one embodiment, a computer device is further provided, including a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, the steps in the above method embodiments are implemented.

[0227] In one embodiment, a computer-readable storage medium is provided, storing a computer program. When the computer program is executed by the processor, the steps in the above method embodiments are implemented.

[0228] In one embodiment, a computer program product or a computer program is provided. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the steps in the above method embodiments.

[0229] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in this application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in this application can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.

[0230] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered to be within the scope described in this specification.

[0231] The above embodiments only represent several implementation manners of this application. The description is relatively specific and detailed, but it should not be construed as a limitation on the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of this application, several modifications and improvements can still be made, and these all belong to the protection scope of this application. Therefore, the protection scope of the patent of this application should be subject to the appended claims.

Claims

1. A method for obtaining lane position, characterized in that, The method includes: Obtain a current road image, and perform feature extraction on the current road image to obtain road image features; Based on the road image features, perform lane information extraction to obtain corresponding lane line features and road surface features; Based on the road image features, lane line features, and road surface features, perform feature fusion to obtain road fusion features; Obtain the lane position where the vehicle is currently located according to the road fusion features; The obtaining the lane position where the vehicle is currently located according to the road fusion features includes: performing feature transformation on the road fusion features according to a convolutional neural network to obtain a feature vector of a first preset dimension; performing vector transposition on the feature vector of the first preset dimension to obtain a corresponding feature vector of a second preset dimension; the feature vector of the second preset dimension includes a first feature vector corresponding to a first boundary; the first boundary is determined according to the current road image and the driving direction defined by the current road; according to the first feature vector, determine first confidence data corresponding to different lane positions; based on the first confidence data, perform lane position prediction to generate the lane position of the current vehicle with reference to the first boundary.

2. The method according to claim 1, wherein The method further includes: obtaining boundary distance features between an in-vehicle camera of the current vehicle and the current road boundary; The performing feature fusion based on the road image features, lane line features, and road surface features to obtain road fusion features includes: fusing the road image features, lane line features, road surface features, and boundary distance features to obtain road fusion features.

3. The method according to claim 1, wherein The performing lane information extraction based on the road image features to obtain corresponding lane line features and road surface features includes: Perform a deconvolution operation based on the road image features to generate a road image feature map of the same size as the current road image; Perform semantic segmentation processing on the road image feature map to generate a corresponding pixel point category prediction result; Perform instance segmentation processing on the road image feature map to generate a corresponding lane pixel attribute prediction result; According to the pixel point category prediction result and the lane pixel attribute prediction result, perform feature clustering to obtain clustering clusters corresponding to different pixel point categories; According to the clustering clusters corresponding to different pixel point categories, determine the lane line features and road surface features corresponding to the road image feature map.

4. The method according to claim 2, wherein The fusing the road image features, lane line features, road surface features, and boundary distance features to obtain road fusion features includes: According to the dimension data corresponding to the lane line features and road surface features, perform a first feature dimension mapping on the road image features to obtain a first splicing feature; Splice the first splicing feature, lane line features, and road surface features to obtain a second splicing feature; According to a preset dimension mapping function, perform a second feature dimension mapping on the second splicing feature to obtain a third splicing feature; Splice the boundary distance feature and the third splicing feature to obtain road fusion features.

5. The method according to claim 1, wherein The feature vector of the second preset dimension further includes a second feature vector corresponding to the second boundary; the method further includes: Determining second confidence data corresponding to different lane positions according to the second feature vector; Performing lane position prediction based on the second confidence data to generate the lane position of the current vehicle with reference to the second boundary.

6. The method according to claim 2, wherein The obtaining of the boundary distance feature between the in-vehicle camera of the current vehicle and the current road boundary includes: Determining the rotation matrix of the in-vehicle camera according to the installation angle of the in-vehicle camera of the current vehicle; Generating position data of the road boundary of the current road in the current road image according to the current road image and the road boundary information of the current road; Obtaining the internal parameters of the in-vehicle camera, the ground height of the in-vehicle camera from the current road, and the translation distance between the in-vehicle camera and the current road; Calculating the coordinate of the feature point of the road boundary according to the position data, the rotation matrix, the internal parameters, the ground height, and the translation distance; Determining the boundary distance feature between the in-vehicle camera of the current vehicle and the current road boundary according to the coordinate of the feature point of the road boundary.

7. The method according to claim 6, characterized in that The method further includes: Performing projection adsorption processing on the current road image according to the installation angle of the in-vehicle camera, the ground height, and the calibration parameters between the current vehicle and the in-vehicle camera to obtain a corresponding road adsorption positioning result; Generating corresponding lane-level navigation data according to the lane position and the road adsorption positioning result; Instructing the current vehicle to travel based on the lane-level navigation data.

8. A lane position acquisition device, characterized in that, The device includes: A road image feature extraction module, configured to obtain a current road image and perform feature extraction on the current road image to obtain road image features; A lane information extraction module, configured to perform lane information extraction based on the road image features to obtain corresponding lane line features and road surface features; A feature fusion module, configured to perform feature fusion based on the road image features, lane line features, and road surface features to obtain road fusion features; A lane position acquisition module, configured to acquire the lane position where the vehicle is currently located according to the road fusion features; The lane position acquisition module is further configured to: perform feature transformation on the road fusion features according to a convolutional neural network to obtain a feature vector of a first preset dimension; perform vector transposition on the feature vector of the first preset dimension to obtain a corresponding feature vector of a second preset dimension; the feature vector of the second preset dimension includes a first feature vector corresponding to a first boundary; the first boundary is determined according to the current road image and the driving direction defined by the current road; determining first confidence data corresponding to different lane positions according to the first feature vector; performing lane position prediction based on the first confidence data to generate the lane position of the current vehicle with reference to the first boundary.

9. The device according to claim 8, characterized in that, The device further includes a boundary distance feature acquisition module, configured to: acquire the boundary distance feature between the in-vehicle camera of the current vehicle and the current road boundary; The feature fusion module is further configured to: fuse the road image feature, the lane line feature, the road surface feature, and the boundary distance feature to obtain a road fusion feature.

10. The device according to claim 8, characterized in that, The lane information extraction module is further configured to: perform a deconvolution operation based on the road image feature to generate a road image feature map with the same size as the current road image; perform semantic segmentation processing on the road image feature map to generate a corresponding pixel point category prediction result; perform instance segmentation processing on the road image feature map to generate a corresponding lane pixel attribute prediction result; perform feature clustering according to the pixel point category prediction result and the lane pixel attribute prediction result to obtain clustering clusters corresponding to different pixel point categories; determine the lane line feature and the road surface feature corresponding to the road image feature map according to the clustering clusters corresponding to different pixel point categories.

11. The device according to claim 9, characterized in that, The feature fusion module is further configured to: perform a first feature dimension mapping on the road image feature according to the dimension data corresponding to the lane line feature and the road surface feature to obtain a first splicing feature; splice the first splicing feature, the lane line feature, and the road surface feature to obtain a second splicing feature; perform a second feature dimension mapping on the second splicing feature according to a preset dimension mapping function to obtain a third splicing feature; splice the boundary distance feature and the third splicing feature to obtain a road fusion feature.

12. The device according to claim 8, characterized in that, The feature vector of the second preset dimension further includes a second feature vector corresponding to a second boundary; the lane position acquisition module is further configured to: determine second confidence data corresponding to different lane positions according to the second feature vector; perform lane position prediction based on the second confidence data to generate the lane position of the current vehicle with reference to the second boundary.

13. The device according to claim 9, characterized in that, The boundary distance feature acquisition module is further configured to: determine the rotation matrix of the on-vehicle camera according to the installation angle of the on-vehicle camera of the current vehicle; generate position data of the road boundary of the current road in the current road image according to the current road image and the road boundary information of the current road; obtain the internal parameters of the on-vehicle camera, the ground height of the on-vehicle camera from the current road, and the translation distance between the on-vehicle camera and the current road; calculate the feature point coordinates of the road boundary according to the position data, the rotation matrix, the internal parameters, the ground height, and the translation distance; determine the boundary distance feature between the on-vehicle camera of the current vehicle and the current road boundary according to the feature point coordinates of the road boundary.

14. The device according to claim 13, wherein The device further includes: a road adsorption positioning result generation module, configured to perform projection adsorption processing on the current road image according to the installation angle of the on-vehicle camera, the ground height, and the calibration parameters between the current vehicle and the on-vehicle camera to obtain a corresponding road adsorption positioning result; a lane-level navigation data generation module, configured to generate corresponding lane-level navigation data according to the lane position and the road adsorption positioning result. A vehicle navigation module for instructing the current vehicle to travel based on the lane-level navigation data.

15. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

16. A computer-readable storage medium stores a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images