Vehicle positioning method, device, equipment, medium, program product and autonomous vehicle

Abstract

The invention provides a vehicle positioning method, device and equipment, a medium, a program product and an automatic driving vehicle, relates to the field of artificial intelligence, in particular to the technical fields of computer vision, deep learning, large models and the like, and can be applied to scenes of automatic driving, automatic parking and the like. The vehicle positioning method comprises the following steps: performing feature extraction on a collected road image to obtain image features; lane line sorting detection is carried out based on the image features, lane line sorting information is obtained, and the lane line sorting information represents the sorting order of at least one lane line in the road image; lane line detection is carried out based on the lane line sorting features and the image features, lane line information is obtained, and the lane line sorting features represent the lane line sorting information; and carrying out vehicle positioning based on the lane line information to obtain target lane line information related to the target vehicle.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence, and in particular to technical fields such as computer vision, deep learning, and large models, and can be applied to scenarios such as autonomous driving and automatic parking. Background Art

[0002] With the rapid development of science and technology, vehicles can use road maps to achieve autonomous driving functions. In the process of realizing autonomous driving functions, vehicle positioning technology can help vehicles navigate and make decisions, which is an important part of autonomous driving. Summary of the invention

[0003] The present disclosure provides a vehicle positioning method, device, electronic device, storage medium, program product and autonomous driving vehicle.

[0004] According to one aspect of the present disclosure, a vehicle positioning method is provided, including: performing feature extraction on a collected road image to obtain image features; performing lane line sorting detection based on the image features to obtain lane line sorting information, wherein the lane line sorting information represents an arrangement order of at least one lane line in the road image; performing lane line detection based on lane line sorting features and image features to obtain lane line information, wherein the lane line sorting features represent the lane line sorting information; and performing vehicle positioning based on the lane line information to obtain target lane line information related to the target vehicle.

[0005] According to another aspect of the present disclosure, a vehicle positioning device is provided, including: an extraction module, used to extract features from a collected road image to obtain image features; a sorting module, used to perform lane line sorting detection based on the image features to obtain lane line sorting information, wherein the lane line sorting information represents the arrangement order of at least one lane line in the road image; a detection module, used to perform lane line detection based on lane line sorting features and image features to obtain lane line information, wherein the lane line sorting features represent the lane line sorting information; and a positioning module, used to perform vehicle positioning based on the lane line information to obtain target lane line information related to the target vehicle.

[0006] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method provided according to an embodiment of the present disclosure.

[0007] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method provided according to an embodiment of the present disclosure.

[0008] According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein the computer program implements the method described above when executed by a processor.

[0009] According to another aspect of the present disclosure, an autonomous driving vehicle is provided, comprising the electronic device provided by an embodiment of the present disclosure.

[0010] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0012] Figure 1 An exemplary system architecture to which the vehicle positioning method and device according to an embodiment of the present disclosure can be applied is schematically shown;

[0013] Figure 2 A flow chart of a vehicle positioning method according to an embodiment of the present disclosure is schematically shown;

[0014] Figure 3 A schematic diagram of a principle diagram of lane line detection based on lane line sorting features and image features according to an embodiment of the present disclosure is shown;

[0015] Figure 4 A schematic diagram showing a principle diagram of obtaining multiple lane line information according to multiple lane line fusion features according to an embodiment of the present disclosure;

[0016] Figure 5 A schematic diagram showing a principle diagram of fusing image features and map features to obtain fused image features according to an embodiment of the present disclosure;

[0017] Figure 6 A schematic diagram showing vehicle positioning using a multi-branch network according to an embodiment of the present disclosure is exemplified;

[0018] Figure 7 A vehicle positioning solution according to an embodiment of the present disclosure is schematically shown;

[0019] Figure 8 A block diagram schematically shows a vehicle positioning device according to an embodiment of the present disclosure; and

[0020] Fig. 9 A block diagram of an electronic device suitable for implementing a vehicle positioning method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0021] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0022] In the technical solution disclosed in the present invention, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations, necessary confidentiality measures are taken, and do not violate public order and good morals.

[0023] In autonomous driving, vehicle positioning can provide the vehicle with key information such as its position and posture on the road, helping the vehicle to perceive the environment, make decisions, and plan paths. However, the inventors found that vehicle positioning still has the problem of low positioning accuracy, making it difficult to achieve high-precision vehicle positioning.

[0024] The disclosed embodiments provide a vehicle positioning method, device, electronic device, storage medium, program product and autonomous driving vehicle. The vehicle positioning method includes: extracting features from a collected road image to obtain image features; performing lane line sorting detection based on the image features to obtain lane line sorting information, wherein the lane line sorting information represents the arrangement order of at least one lane line in the road image; performing lane line detection based on the lane line sorting features and the image features to obtain lane line information, wherein the lane line sorting features represent the lane line sorting information; and performing vehicle positioning based on the lane line information to obtain target lane line information related to the target vehicle.

[0025] According to the embodiments of the present disclosure, lane line sorting detection is performed by extracting image features from road images, so that the detected lane line sorting information can more accurately characterize the number of lane lines in the traffic area and the arrangement order between multiple lane lines. Lane line detection is performed based on lane line sorting features and image features, and the semantic attributes that characterize the arrangement order of lane lines can be fully integrated with image features as prior semantic information to perform lane line detection, so that the lane line information detected based on the prior semantic information can more accurately characterize the multiple lane lines in the traffic area, and then the target vehicle can be accurately positioned according to the lane line information, thereby improving the vehicle positioning accuracy. In addition, the lane line sorting information can also be used as prior information to provide basic data for the subsequent automatic driving strategy formulation, path planning, etc. of the target vehicle, to ensure that the target vehicle can perform automatic driving functions based on accurate lane line-level positioning during driving, thereby improving the driving safety of the target vehicle.

[0026] Figure 1An exemplary system architecture to which the vehicle positioning method and device according to an embodiment of the present disclosure can be applied is schematically shown.

[0027] It should be noted that Figure 1 The examples shown are only examples of system architectures to which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but do not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios. For example, in another embodiment, an exemplary system architecture to which the vehicle positioning method and apparatus can be applied may include a terminal device, but the terminal device may implement the vehicle positioning method and apparatus provided by the embodiments of the present disclosure without interacting with a server.

[0028] like Figure 1 As shown, the system architecture 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a vehicle 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0029] The user can use the first terminal device 101 and the second terminal device 102 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101 and the second terminal device 102, such as knowledge reading applications, web browser applications, search applications, instant messaging tools, email clients and / or social platform software, etc. (only as examples).

[0030] The first terminal device 101 and the second terminal device 102 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0031] The vehicle 103 may be equipped with a vehicle-side sensor, which may include an image sensor, but is not limited thereto. The vehicle-side sensor may also be a laser radar, a millimeter-wave radar, or other types of sensors. The vehicle 103 may be any type of vehicle, such as a car, a truck, etc. The vehicle 103 may also be equipped with a communication device, and the vehicle 103 may transmit information to and from the server 105 through the network 104 based on the communication device.

[0032] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for the content browsed by the user using the first terminal device 101 and the second terminal device 102. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0033] The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server can also be a server of a distributed system, or a server combined with blockchain.

[0034] It should be noted that the vehicle positioning method provided in the embodiment of the present disclosure can generally be executed by the vehicle 103. Accordingly, the vehicle positioning device provided in the embodiment of the present disclosure can generally be set in the vehicle 103. Alternatively, the vehicle positioning method provided in the embodiment of the present disclosure can generally be executed by the first terminal device 101 and the second terminal device 102. Accordingly, the vehicle positioning device provided in the embodiment of the present disclosure can generally be set in the first terminal device 101 and the second terminal device 102.

[0035] Alternatively, the vehicle positioning method provided in the embodiment of the present disclosure may also be generally performed by the server 105. Accordingly, the vehicle positioning device provided in the embodiment of the present disclosure may generally be arranged in the server 105. The vehicle positioning method provided in the embodiment of the present disclosure may also be performed by a server or server cluster that is different from the vehicle 103 and can communicate with the first terminal device 101, the second terminal device 102, the vehicle 103 and / or the server 105. Accordingly, the vehicle positioning device provided in the embodiment of the present disclosure may also be arranged in a server or server cluster that is different from the vehicle 103 and can communicate with the first terminal device 101, the second terminal device 102, the vehicle 103 and / or the server 105.

[0036] It should be understood that Figure 1 The number of terminal devices, vehicles, networks and servers in the embodiment is only for illustration. Any number of terminal devices, vehicles, networks and servers may be provided as required.

[0037] Figure 2 The flowchart of the vehicle positioning method according to the embodiment of the present disclosure is schematically shown.

[0038] like Figure 2 As shown, the vehicle positioning method 200 includes operations S210 to S240.

[0039] In operation S210 , feature extraction is performed on the collected road image to obtain image features.

[0040] In operation S220, lane line order detection is performed based on the image features to obtain lane line order information.

[0041] In operation S230, lane line detection is performed based on the lane line sorting feature and the image feature to obtain lane line information, and the lane line sorting feature represents the lane line sorting information.

[0042] In operation S240, vehicle positioning is performed based on the lane line information to obtain target lane line information related to the target vehicle.

[0043] According to an embodiment of the present disclosure, a road image can represent a road area, for example, it can be a single-frame or multi-frame image obtained by capturing images of the road area. The road image can include any type of image, such as a color image, a grayscale image, etc. The road image can be captured by an image acquisition device installed on a vehicle, and the image acquisition device can include, for example, a monocular camera, a surround camera, a depth camera laser radar, a millimeter-wave radar, an ultrasonic radar, etc. Exemplarily, a road image can represent various types of roads, such as highways (such as expressways and ordinary roads), closed roads (such as test tracks, etc.), internal roads (such as communities, parking lots, industrial parks, etc.), urban roads, rural roads, etc.

[0044] According to an embodiment of the present disclosure, a road image can be acquired by a monocular camera. Compared with laser radar or depth camera-based acquisition of road images, acquisition of road images based on a monocular camera can be based on the relatively simple hardware structure of the monocular camera and the hardware performance that is easy to install and deploy to conveniently adapt to different vehicles and road conditions, thereby reducing the difficulty of acquiring road images.

[0045] According to the embodiments of the present disclosure, road images can represent traffic elements such as lane lines, vehicles, pedestrians, obstacles, traffic signs and signal lights. Road images can also represent the geometric characteristics of the road (such as width, length, slope, shape, curvature, etc.), and can also represent the traffic conditions of the road (such as whether it is congested) and environmental conditions (such as lighting conditions, weather conditions), etc.

[0046] According to an embodiment of the present disclosure, feature extraction can be performed on the collected road image through a feature extraction network to obtain image features. For example, feature extraction can be performed on the road image based on any type of network algorithm such as a convolutional neural network algorithm, a feature pyramid network, etc. to obtain image features, which are not specifically limited here. The image features can represent the semantic information of the collected road image, so that the lane line sorting information of the lane lines in the road passage area can be represented according to the image features.

[0047] According to an embodiment of the present disclosure, a multi-branch network can also be used to process road images to obtain image features. A multi-branch network can be understood as a network architecture that connects multiple network branches together to collaboratively complete complex tasks. Compared with a single-branch network, a multi-branch network can have higher flexibility and adaptability and can handle more complex tasks. By focusing different network branches on different subtasks, a multi-branch network can achieve better performance when processing comprehensive tasks. For example, the main branch network can be responsible for the extraction of global features, while the secondary branch network can be responsible for the resolution of specific subtasks. Depending on the task, each branch network can adopt a different network structure.

[0048] According to another embodiment of the present disclosure, feature extraction can be performed on a road image through a feature extraction branch to obtain a first image sub-feature. Image segmentation can be performed on the road image through an image segmentation branch to obtain a second image sub-feature. The first image sub-feature and the second image sub-feature are fused to obtain an image feature. Image segmentation can include semantic segmentation and / or instance segmentation. For example, image segmentation can be performed on a road image based on any type of network algorithm such as a fully convolutional neural network algorithm to obtain a second image sub-feature, which is not specifically limited here.

[0049] In one example, the first image sub-feature and the second image sub-feature may be fused, which may include processing the first image sub-feature and the second image sub-feature based on a fusion algorithm. The fusion algorithm may include, for example, matrix multiplication, splicing, attention algorithm, etc. The embodiments of the present disclosure do not limit the specific algorithm type of the fusion algorithm.

[0050] According to an embodiment of the present disclosure, lane lines can be used to indicate information such as driving paths, lane boundaries, travel directions, traffic rules, etc., and can include various types of lane lines, such as solid lines, dashed lines, double yellow lines, edge lines, dividing lines, stop lines, crosswalk lines, etc.

[0051] According to an embodiment of the present disclosure, lane line sorting information can be used to characterize the arrangement order of at least one lane line in a road image. Lane line sorting information can more accurately characterize the number of lane lines in the traffic area and the arrangement order of at least one lane line. Lane line sorting information helps to more accurately determine the specific position of the target vehicle on the road, further improving the accuracy of vehicle positioning. Lane line sorting information can be used as prior information, combined with road image information, road map information, etc. to perform lane line detection, achieve lane-level precise positioning of the target vehicle, and improve vehicle positioning accuracy. Lane line sorting information can also be used as prior information to provide basic data for subsequent strategy formulation, path planning, etc., to ensure the coordination and consistency of the entire system.

[0052] Exemplarily, the at least one lane line mentioned above may include all lane lines in the road image, and may also include a target lane line related to the target vehicle in the road image. For example, the target vehicle may include the ego vehicle, and the target lane line may include lane lines adjacent to both sides of the target vehicle, a lane line where the target vehicle is located, and the like.

[0053] According to an embodiment of the present disclosure, lane line sorting information may represent the number of lane lines in a road image and the order of arrangement between multiple lane lines. For example, the road image includes 5 lane lines, which are lane line 1, lane line 2, lane line 3, lane line 4, and lane line 5 in order. For example, the order of lane line numbers may be from left to right or from right to left, etc., which is not limited here.

[0054] According to another embodiment of the present disclosure, the lane line sorting information may also represent the order of the lane lines where the target vehicle (such as the vehicle) is located in the road image. For example, the target vehicle is between the second lane line and the third lane line, or the target vehicle is on the fifth lane line (such as riding on the lane line or pressing the lane line).

[0055] According to another embodiment of the present disclosure, the lane line sorting information may also characterize the number of all lane lines in the road image, the arrangement order of each lane line, and the arrangement order of the lane lines where the target vehicle (such as the ego vehicle) is located.

[0056] According to an embodiment of the present disclosure, lane line sorting detection can be performed on image features through a lane line sorting branch to obtain lane line sorting information. For example, lane line sorting detection can be performed on image features based on any type of network algorithm such as a convolutional neural network algorithm to obtain lane line sorting information, which is not specifically limited here.

[0057] According to the embodiments of the present disclosure, lane line detection can be performed based on lane line sorting features and image features to obtain lane line information. Exemplarily, the lane line information can represent the shape, position, direction, type, traffic sign, topological relationship, etc. of at least one lane line. For example, lane line detection can be performed based on any type of network algorithm such as a convolutional neural network algorithm to obtain lane line information, which is not specifically limited here.

[0058] According to an embodiment of the present disclosure, vehicle positioning can be performed based on lane line information to obtain target lane line information related to the target vehicle. Exemplarily, the target lane line information may include the order of arrangement of the target lane lines, the distance from the target vehicle, the shape, position, direction, type, traffic signs, topological relationship, etc. For example, the target lane lines related to the target vehicle may include the lane lines adjacent to the target vehicle on both sides, the lane lines where the target vehicle is located, etc., and the target vehicle may include the self-vehicle.

[0059] According to the embodiments of the present disclosure, lane line sorting detection is performed by extracting image features from road images, so that the detected lane line sorting information can more accurately characterize the number of lane lines in the traffic area and the arrangement order between multiple lane lines. Lane line detection is performed based on lane line sorting features and image features, and the semantic attributes that characterize the arrangement order of lane lines can be fully integrated with image features as prior semantic information to perform lane line detection, so that the lane line information detected based on the prior semantic information can more accurately characterize the multiple lane lines in the traffic area, and then the target vehicle can be accurately positioned according to the lane line information, thereby improving the vehicle positioning accuracy. In addition, the lane line sorting information can also be used as prior information to provide basic data for the subsequent automatic driving strategy formulation, path planning, etc. of the target vehicle, to ensure that the target vehicle can perform automatic driving functions based on accurate lane line-level positioning during driving, thereby improving the driving safety of the target vehicle.

[0060] According to the embodiments of the present disclosure, lane line sorting features can be used as prior information and combined with image features to perform lane line detection, so that the lane line information can more accurately represent multiple lane lines in the traffic area, and then the lane-level precise positioning of the target vehicle can be achieved based on the lane line information, thereby improving the accuracy of vehicle positioning.

[0061] As an example, the lane line detection branch can be used to extract features from the lane line sorting information, and then lane line detection can be performed in combination with image features to obtain lane line information. As another example, the lane line detection branch can be used to perform lane line detection based on hidden features of the lane line sorting branch and image features to determine lane line information.

[0062] According to an embodiment of the present disclosure, the lane line information includes at least one of the following: a vector line, a lane line type, and a lane line topological relationship.

[0063] According to the embodiments of the present disclosure, vector lines can describe information such as the position, shape, and direction of traffic of lane lines, as well as traffic rules such as merging methods, in the form of vector data. Lane line types can include, for example, solid lines, dotted lines, double yellow lines, edge lines, dividing lines, stop lines, guide arrows, crosswalks, etc. Lane line topological relationships can include, for example, convergence, bifurcation, and connection relationships between lane lines.

[0064] According to an embodiment of the present disclosure, lane line detection is performed based on lane line sorting features and image features to obtain lane line information, including: fusing lane line sorting features and image features to obtain fused features; performing feature extraction on the fused features to obtain lane line fused features corresponding to each of the multiple lane lines; and performing detection based on the lane line fused features to obtain lane line information.

[0065] According to an embodiment of the present disclosure, a lane line sorting feature can represent lane line sorting information, for example, it can represent semantic information of the arrangement order of lane lines. As an example, the lane line sorting feature can include a global sorting feature and a local sorting feature. The global sorting feature can represent the number of lane lines in a road image and the arrangement order relationship between multiple lane lines, and the local sorting feature can represent the arrangement order number of selected lane lines in a road image.

[0066] According to an embodiment of the present disclosure, the lane line sorting feature and the image feature may be fused to obtain a fused feature. The fused feature may be used to perform lane line detection through a lane line detection branch to obtain lane line information. The fused feature may detect the lane line more accurately by fusing the image information and the lane line sorting information, which helps to further improve the accuracy of vehicle positioning.

[0067] Figure 3 The schematic diagram shows a principle diagram of lane line detection based on lane line sorting features and image features according to an embodiment of the present disclosure.

[0068] like Figure 3 As shown, the image feature 301 can be input into the lane line sorting detection branch 302, and the lane line sorting feature 303 can be output. The image feature 301 and the lane line sorting feature 303 can be fused to obtain a fused feature 304. The fused feature 304 can be input into the lane line detection branch 305, and the lane line information 306 can be output.

[0069] According to an embodiment of the present disclosure, the fusion of lane line sorting features and image features may include processing the lane line sorting features and image features based on a fusion algorithm. The fusion algorithm may include, for example, matrix multiplication, splicing, attention algorithm, etc. The embodiment of the present disclosure does not limit the specific algorithm type of the fusion algorithm.

[0070] According to an embodiment of the present disclosure, the fusion feature may include lane line fusion features of each of the multiple lane lines in the traffic area. For example, the fusion feature may include lane line fusion features of each of N lane lines, where N is a positive integer.

[0071] According to an embodiment of the present disclosure, the fusion feature can represent the lane line information of each of the multiple lane lines, and the fusion feature can be extracted to obtain the lane line fusion feature corresponding to each of the multiple lane lines. Further, detection can be performed based on the lane line fusion feature of each of the multiple lane lines to obtain the lane line information of each of the multiple lane lines.

[0072] As an example, the i-th lane line fusion feature corresponding to the i-th lane line can be input into the lane line detection branch, and the i-th lane line information is output, where i is a positive integer. As another example, N lane line fusion features can be input into the lane line detection branch in the order of the N lane lines, and the lane line information of each of the N lane lines is output.

[0073] According to an embodiment of the present disclosure, by extracting features from fused features, the semantic information of multiple lane lines and the semantic information of the order of multiple lane lines can be captured, and the lane line fusion features of each of the multiple lane lines can be obtained. The lane line fusion features can represent the lane line semantic information and the lane line arrangement order semantic information corresponding to the lane lines, and the lane line fusion features can be detected to obtain the lane line information corresponding to the lane lines. Exemplarily, the fusion features can represent the global lane line information, and the lane line fusion features can represent the lane line information corresponding to the lane lines.

[0074] According to an embodiment of the present disclosure, a global feature can be obtained based on the fused feature, and the global feature can represent the global semantic information of the lane line. The fused feature can be extracted to obtain a middle lane fused feature corresponding to the lane line, and the middle lane fused feature can represent the local semantic information of the lane line. The middle lane line fused feature can be fused with the global feature to obtain a lane line fused feature corresponding to the lane line, so that the lane line fused feature can more fully represent the global semantic information and local semantic information of the lane line, thereby improving the accuracy of lane line detection, which is conducive to further improving the accuracy of vehicle positioning.

[0075] According to an embodiment of the present disclosure, the lane line information is obtained by detecting based on the lane line fusion feature, including: based on the attention mechanism, processing multiple lane line fusion features according to the arrangement order between the multiple lane line fusion features, and obtaining multiple lane line information

[0076] According to an embodiment of the present disclosure, the arrangement order between multiple lane line fusion features is determined according to the lane line sorting information.

[0077] According to an embodiment of the present disclosure, the lane line sorting information may represent the arrangement order between the multiple lane lines. The arrangement order between the multiple lane line fusion features may be determined according to the arrangement order between the multiple lane lines represented by the lane line sorting information. For example, the arrangement order between the multiple lane line fusion features is the same as the arrangement order between the multiple lane lines.

[0078] Figure 4 The schematic diagram shows a principle diagram of obtaining multiple lane line information according to multiple lane line fusion features according to an embodiment of the present disclosure.

[0079] like Figure 4As shown, the lane line detection branch 401 may include, for example, a long short-term memory model 401a and a convolutional neural network 401b. A plurality of lane line fusion features 402 may be input into the lane line detection branch 401 according to the arrangement order of the plurality of lane line fusion features to obtain a plurality of lane line information 403.

[0080] As an example, multiple lane line fusion features can be arranged in order, and multiple lane line fusion features arranged in order can be input into the lane line detection branch to obtain multiple lane line information. As another example, multiple lane line fusion features can be input into the lane line detection branch in order of arrangement, and multiple lane line information can be obtained accordingly. It should be noted that the lane line detection branch can also include other network algorithms with attention mechanisms, such as converters and convolutional neural networks, which are not limited here.

[0081] According to an embodiment of the present disclosure, fusing lane line sorting features and image features to obtain fused features includes: fusing image features and map features based on an attention mechanism to obtain image fusion features, wherein the map features are determined based on a road map related to the road image; and fusing image fusion features and lane line sorting features to obtain fusion features.

[0082] According to the embodiments of the present disclosure, the image features may be fused with the map features to obtain the image fusion features. The lane line sorting branch may be used to perform lane line sorting detection on the image fusion features to obtain lane line sorting information. The image fusion features and the lane line sorting features may be fused to obtain the fusion features.

[0083] According to the embodiments of the present disclosure, the image fusion feature can more accurately detect lane lines in the road image by fusing information from the road image and the road map, which helps to further improve the accuracy of vehicle positioning. In addition, the image fusion feature can also reduce the errors and uncertainties that may be introduced by a single road image through the complementarity and redundancy of multi-source information, thereby ensuring that lane lines can be accurately detected under various environmental conditions, thereby helping to improve the accuracy and robustness of vehicle positioning.

[0084] According to an embodiment of the present disclosure, fusing image features and map features may include processing image features and map features based on a fusion algorithm. The fusion algorithm may include, for example, matrix multiplication, splicing, attention algorithm, etc. The embodiment of the present disclosure does not limit the specific algorithm type of the fusion algorithm.

[0085] Exemplarily, map features can be obtained by extracting features from a road map associated with a road image. Map features can characterize the semantic information of the road map, so that lane lines in the road traffic area can be characterized based on the map features. Preferably, the road map can be a lane-level map, which can supplement and provide detailed map information such as lane line structure, lane line position, lane line arrangement order, and lane line topological relationship. These map information can complement the road image to improve the accuracy and robustness of vehicle positioning in complex road conditions such as multiple lanes, obstructed lanes, and lack of obvious lane lines.

[0086] According to the embodiments of the present disclosure, image features and map features can be fused using, for example, a cross-attention mechanism, which can reduce errors that may be introduced into road images in complex situations such as multiple lanes and occlusions, thereby enabling image fusion features to more accurately and comprehensively learn the semantic information of lane lines in the traffic area.

[0087] Figure 5 The schematic diagram shows a principle diagram of fusing image features and map features to obtain fused image features according to an embodiment of the present disclosure.

[0088] like Figure 5 As shown, feature extraction can be performed on a road image 501 to obtain image features 502. Feature extraction can be performed on a road map 503 related to the road image to obtain map features 504. Image features 502 and map features 504 can be fused based on an attention network 505 to obtain fused image features 506. Exemplarily, the attention network can be constructed based on, for example, a cross attention network algorithm.

[0089] According to an embodiment of the present disclosure, performing vehicle positioning based on lane line information includes: performing vehicle positioning based on at least one of the first attribute and the second attribute, as well as the lane line information.

[0090] According to an embodiment of the present disclosure, a first attribute characterizes that a first lane line among multiple lane lines is obscured, and the first attribute is determined based on the following operations: performing object detection based on image features or image fusion features to obtain object detection results, the image fusion features are determined based on a road image and a road map related to the road image, and the object detection result characterizes objects around the target vehicle; when the object detection result characterizes that an object obscures the first lane line among multiple lane lines, the first attribute is determined.

[0091] According to an embodiment of the present disclosure, the first attribute may characterize that the first lane line among multiple lane lines is blocked. The first lane line may be understood as a lane line that is blocked among multiple lane lines. For example, the first lane line may include a target lane line that is blocked among multiple lane lines. For another example, the first lane line may include each blocked lane line among multiple lane lines. Exemplarily, the blocking may include left side blocking, right side blocking, both sides blocking, and the like.

[0092] According to an embodiment of the present disclosure, the vehicle can be positioned based on the first attribute and the lane line information. For example, when the first attribute indicates that the first lane line is blocked, the lane line information of the blocked area may not be used directly for positioning, such as by calling map data, combining detection of roadside equipment or other markers, etc., thereby ensuring the safety and accuracy of downstream strategies.

[0093] According to an embodiment of the present disclosure, the second attribute represents that the target vehicle is located on both sides of a second lane line among the multiple lane lines. The second attribute is obtained by detecting the lane-riding attribute of the target vehicle based on image features or image fusion features.

[0094] According to an embodiment of the present disclosure, the second lane line can be understood as the lane line that is "ridden" by the target vehicle. The vehicle "riding" can include, for example, the wheels of the vehicle pressing on or approaching the lane line, or a part of the vehicle body crossing the lane line (the target vehicle is located on both sides of the second lane line). For example, the second attribute can be determined based on the positional relationship or distance between the target vehicle and the lane line.

[0095] According to an embodiment of the present disclosure, vehicle positioning can be performed based on the second attribute and lane line information. For example, when the second attribute indicates that the target vehicle is located on both sides of the second lane line, a positioning result indicating that the target vehicle is riding the lane line can be output, or a positioning result indicating that the target vehicle is riding the second lane line can be output, thereby assisting the formulation of downstream strategies.

[0096] According to an embodiment of the present disclosure, vehicle positioning can also be performed based on the first attribute, the second attribute, and the lane line information. For example, when the first attribute indicates that the first lane line is obscured, the lane line information of the obscured area may not be directly used for positioning, such as by calling map data, combining the detection of roadside equipment or other markers, etc. Positioning can be performed. When the second attribute indicates that the target vehicle is located on both sides of the second lane line, a positioning result indicating that the target vehicle is riding the line can be output, or a positioning result indicating that the target vehicle is riding the second lane line can be output. By performing vehicle positioning based on the first attribute, the second attribute and the lane line information, the vehicle can be positioned more comprehensively and accurately in a complex road environment, thereby ensuring the safety and accuracy of downstream strategies.

[0097] According to an embodiment of the present disclosure, the first attribute can be used to determine the occlusion of the lane line, and the second attribute can be used to determine whether the target vehicle is at the edge of the lane line or crosses the lane line. The first attribute and / or the second attribute can be used to assist in the formulation of downstream strategies to ensure that the vehicle can make safer and more accurate decisions in complex road environments.

[0098] According to an embodiment of the present disclosure, object detection can be performed based on image features or image fusion features to obtain object detection results. As an example, image features can be input into an attribute detection branch to perform object detection to obtain object detection results. As another example, image fusion features can be input into an attribute detection branch to perform object detection to obtain object detection results.

[0099] According to an embodiment of the present disclosure, the object detection result may characterize the objects around the target vehicle. The objects around the target vehicle may include, for example, lane lines, vehicles, pedestrians, obstacles, signs, etc. Exemplarily, the object detection result may include, for example, a detection bounding box and category information, and the detection bounding box may characterize the object's location, size, shape, and other attribute information. For example, based on the detection bounding box and lane line information represented by the object detection result, it may be determined whether the object blocks the lane line, thereby determining the first attribute.

[0100] According to an embodiment of the present disclosure, the target vehicle may be subjected to line-riding attribute detection based on image features or image fusion features to obtain a second attribute. As an example, the image features may be input into an attribute detection branch to perform line-riding attribute detection to obtain a second attribute. As another example, the image fusion features may be input into an attribute detection branch to perform line-riding attribute detection to obtain a second attribute. The second attribute may characterize a situation where the target vehicle approaches a lane line or crosses a lane line, and the second attribute may include attribute information indicating that the target vehicle is riding or pressing a lane line, etc. For example, the second attribute may be determined based on the positional relationship between the target vehicle and the lane line or the distance from the lane line.

[0101] According to an embodiment of the present disclosure, a multi-branch network may be used to process a road image and a road map to obtain target lane line information related to a target vehicle.

[0102] Figure 6 A schematic diagram of vehicle positioning using a multi-branch network according to an embodiment of the present disclosure is exemplified.

[0103] According to the embodiments of the present disclosure, Figure 6 As shown, the multi-branch network 600 may include, for example, a lane line sorting detection branch 610 , a lane line detection branch 620 , and an attribute prediction branch 630 .

[0104] According to the embodiments of the present disclosure, feature extraction can be performed on a road image and a road map to obtain image features and map features, and image features and map features can be fused to obtain image fusion features.

[0105] According to an embodiment of the present disclosure, the lane line sorting detection branch 610 can be used to perform lane line sorting detection on the image fusion feature to obtain the lane line sorting feature. The image fusion feature and the lane line sorting feature can be fused to obtain the fusion feature. The lane line detection branch 620 can be used to perform lane line detection on the fusion feature to obtain lane line information. The attribute detection branch 630 can be used to perform object detection and lane-riding attribute detection on the image feature or the image fusion feature to obtain the first attribute and the second attribute. The vehicle can be positioned based on the first attribute, the second attribute and the lane line information to obtain the target lane line information.

[0106] According to an embodiment of the present disclosure, a vehicle positioning solution is also provided, which aims to improve the accuracy and robustness of vehicle positioning by continuously optimizing and iterating the model.

[0107] Figure 7 A vehicle positioning solution according to an embodiment of the present disclosure is schematically shown.

[0108] like Figure 7 As shown, the vehicle positioning solution may include the following road test verification link 701, data analysis link 702, data annotation link 703 and model training link 704. The following will combine the exemplary implementation of each link.

[0109] 1) Road test verification phase 701: Test the vehicle in a large number of real road environments and continuously collect operating data. Through these road test data, we can capture instances in various real driving scenarios, that is, extreme or special situations. These instance data are crucial to identifying challenges and problems that the model may encounter in actual applications, and help to fully understand the performance of the model.

[0110] 2) Data analysis stage 702: First, the collected instance data is systematically analyzed, using data mining and statistical analysis methods to summarize the shortcomings of the current model in specific scenarios. The analysis then focuses on determining in which scenarios or conditions the model performs poorly, such as in complex traffic environments, severe weather conditions, or specific geographical areas. Finally, through this analysis, the specific direction and goals of the model that need to be improved can be clarified.

[0111] 3) Data labeling stage 703: First, conduct special data collection for scenarios where the model performs poorly in the analysis. Ensure that the data covers various complex situations and instances to increase the diversity and representativeness of the data. Secondly, accurately label the collected data to ensure the high quality of the training data to ensure the accuracy and consistency of the data. High-quality labeled data is the basis for improving the model training effect and can directly affect the final performance of the model.

[0112] 4) Model training stage 704: Based on the results of data analysis and labeled data, improve the model structure and optimize the model parameters to make it more suitable for the needs of specific scenarios. Use the newly added special data, especially the data for the instance, to iteratively train the model. Through continuous training and verification, gradually improve the accuracy and robustness of the model. During the training process, verification methods such as cross-validation and test set verification can be used to ensure the consistency and reliability of the deep learning model in the training set and actual application.

[0113] In one example, the autonomous driving operation data of the vehicle in the actual road environment can be collected in the road test verification link 701. The operation data can be analyzed in the data analysis link 702 to summarize the current scenes or conditions that need to be improved (such as complex traffic environment, bad weather conditions, specific geographical areas, etc.). According to the scenes that need to be improved, targeted data collection and data annotation can be carried out in the data annotation link 703, and the annotated data can be used for training and iteration of models (such as multi-branch networks). The model can be trained using the annotated data in the model training link 704, so that the model can better adapt to the needs of specific scenes or improve the performance of the model in specific scenes. The autonomous driving operation data of the vehicle in the actual road environment can be collected in the road test verification link 701, and the trained model can be verified. Through the above links, a complete closed-loop process can be realized to ensure the continuous optimization and iteration of the model in the vehicle positioning with lane-level accuracy. The vehicle positioning closed-loop solution provided in the embodiment of the present disclosure can improve the overall performance and reliability of the model, and enable the model to have the ability to quickly respond to and adapt to the changing road environment and driving scenes, thereby improving the accuracy and robustness of vehicle positioning.

[0114] It should be noted that the above embodiments of the present disclosure are only exemplary embodiments, but are not limited thereto, and may also include other vehicle positioning methods known in the art as long as lane-level positioning can be achieved.

[0115] According to an embodiment of the present disclosure, a vehicle positioning device is also provided.

[0116] Figure 8 A block diagram of a vehicle positioning device according to an embodiment of the present disclosure is schematically shown.

[0117] like Figure 8 As shown, the vehicle positioning device 800 may include an extraction module 810 , a sorting module 820 , a detection module 830 and a positioning module 840 .

[0118] An extraction module 810 is used to extract features from the collected road image to obtain image features;

[0119] A sorting module 820 is used to perform lane line sorting detection based on image features to obtain lane line sorting information, where the lane line sorting information represents the arrangement order of at least one lane line in the road image;

[0120] A detection module 830 is used to perform lane line detection based on lane line sorting features and image features to obtain lane line information, where the lane line sorting features represent the lane line sorting information;

[0121] The positioning module 840 is used to perform vehicle positioning based on the lane line information to obtain target lane line information related to the target vehicle.

[0122] According to an embodiment of the present disclosure, the detection module 830 may include a fusion submodule, an extraction submodule and a detection submodule.

[0123] The fusion submodule is used to fuse the lane line sorting features and image features to obtain fused features.

[0124] The extraction submodule is used to extract the fused features to obtain lane line fusion features corresponding to each of the multiple lane lines.

[0125] The detection submodule is used to perform detection based on lane line fusion features to obtain lane line information.

[0126] According to an embodiment of the present disclosure, the detection submodule may include a detection unit.

[0127] The detection unit is used to process multiple lane line fusion features based on an attention mechanism according to the arrangement order between the multiple lane line fusion features to obtain multiple lane line information, wherein the arrangement order between the multiple lane line fusion features is determined according to the lane line sorting information.

[0128] According to an embodiment of the present disclosure, the fusion submodule may include a first fusion unit and a second fusion unit.

[0129] The first fusion unit is used to fuse the image feature and the map feature based on the attention mechanism to obtain the image fusion feature, wherein the map feature is determined based on a road map related to the road image.

[0130] The second fusion unit is used to fuse the image fusion feature and the lane line sorting feature to obtain a fusion feature.

[0131] According to an embodiment of the present disclosure, the positioning module 840 may include a positioning submodule.

[0132] The positioning submodule is used to perform vehicle positioning based on at least one of a first attribute and a second attribute, and lane line information, wherein the first attribute represents that a first lane line among multiple lane lines is blocked, and the second attribute represents that the target vehicle is located on both sides of a second lane line among the multiple lane lines.

[0133] According to an embodiment of the present disclosure, the first attribute is determined based on the following operations: performing object detection based on image features or image fusion features to obtain object detection results, the image fusion features are determined based on a road image and a road map related to the road image, and the object detection results characterize objects around the target vehicle; when the object detection result characterizes that the object occludes a first lane line among multiple lane lines, the first attribute is determined; wherein the second attribute characterizes that the target vehicle is located on both sides of a second lane line among the multiple lane lines, and the second attribute is obtained by performing line riding attribute detection on the target vehicle based on the image features or the image fusion features.

[0134] It should be noted that the vehicle positioning device part in the embodiment of the present disclosure corresponds to the vehicle positioning method part in the embodiment of the present disclosure. The description of the vehicle positioning device part specifically refers to the vehicle positioning method part, which will not be repeated here.

[0135] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0136] According to an embodiment of the present disclosure, an electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method provided by the embodiment of the present disclosure.

[0137] According to an embodiment of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute the method provided by the embodiment of the present disclosure.

[0138] According to an embodiment of the present disclosure, a computer program product includes a computer program, and when the computer program is executed by a processor, the method provided by the embodiment of the present disclosure is implemented.

[0139] According to an embodiment of the present disclosure, the present disclosure also provides an autonomous driving vehicle, comprising: an electronic device provided by an embodiment of the present disclosure and capable of executing the method provided by an embodiment of the present disclosure.

[0140] Fig. 9A schematic block diagram of an example electronic device 900 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0141] like Fig. 9 As shown, the device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 902 or a computer program loaded from a storage unit 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the device 900 can also be stored. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0142] A number of components in the device 900 are connected to the I / O interface 905, including: an input unit 906, such as a keyboard, a mouse, etc.; an output unit 907, such as various types of displays, speakers, etc.; a storage unit 908, such as a disk, an optical disk, etc.; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the device 900 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0143] The computing unit 901 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 901 performs the various methods and processes described above, such as a vehicle positioning method. For example, in some embodiments, the vehicle positioning method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the vehicle positioning method described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to perform the vehicle positioning method in any other appropriate manner (e.g., by means of firmware).

[0144] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0145] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0146] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0147] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0148] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0149] A computer system may include a client and a server. The client and the server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises through computer programs running on respective computers and having a client-server relationship to each other. The server may be a cloud server, a server in a distributed system, or a server combined with a blockchain.

[0150] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0151] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A vehicle positioning method, include: Perform feature extraction on the collected road images to obtain image features; Performing lane line sorting detection based on the image features to obtain lane line sorting information, wherein the lane line sorting information represents an arrangement order of at least one lane line in the road image; Performing lane line detection based on a lane line sorting feature and the image feature to obtain lane line information, wherein the lane line sorting feature represents the lane line sorting information; Vehicle positioning is performed based on the lane line information to obtain target lane line information related to the target vehicle.

2. The method according to claim 1, in, The performing lane line detection based on the lane line sorting feature and the image feature to obtain lane line information includes: Fusing the lane line sorting feature and the image feature to obtain a fused feature; Extracting features from the fused features to obtain lane line fused features corresponding to each of the plurality of lane lines; and Detection is performed based on the lane line fusion feature to obtain the lane line information.

3. The method according to claim 2, in, The detecting based on the lane line fusion feature to obtain the lane line information includes: Based on the attention mechanism, the multiple lane line fusion features are processed according to the arrangement order between the multiple lane line fusion features to obtain multiple lane line information, wherein the arrangement order between the multiple lane line fusion features is determined according to the lane line sorting information.

4. The method according to claim 2, in, The fusion of the lane line sorting feature and the image feature to obtain the fusion feature includes: fusing the image features and the map features based on an attention mechanism to obtain an image fusion feature, wherein the map feature is determined based on a road map associated with the road image; and The image fusion feature and the lane line sorting feature are fused to obtain the fusion feature.

5. The method according to claim 1, in, The vehicle positioning based on the lane line information includes: The vehicle is positioned based on at least one of the first attribute and the second attribute and the lane line information, wherein the first attribute represents that a first lane line among the plurality of lane lines is blocked, and the first attribute is determined based on the following operations: Performing object detection based on the image feature or the image fusion feature to obtain an object detection result, wherein the image fusion feature is determined based on the road image and a road map related to the road image, and the object detection result represents an object around the target vehicle; In a case where the object detection result indicates that the object occludes a first lane line among the plurality of lane lines, determining the first attribute; The second attribute represents that the target vehicle is located at both sides of a second lane line among the multiple lane lines, and the second attribute is obtained by performing lane-riding attribute detection on the target vehicle based on the image feature or the image fusion feature.

6. The method according to claim 1, in, The lane line information includes at least one of the following: Vector lines, lane line types, and lane line topological relationships.

7. A vehicle positioning device, include: An extraction module is used to extract features from the collected road images to obtain image features; A sorting module, configured to perform lane line sorting detection based on the image features to obtain lane line sorting information, wherein the lane line sorting information represents an arrangement order of at least one lane line in the road image; A detection module, configured to perform lane line detection based on a lane line sorting feature and the image feature to obtain lane line information, wherein the lane line sorting feature represents the lane line sorting information; The positioning module is used to perform vehicle positioning based on the lane line information to obtain target lane line information related to the target vehicle.

8. The device according to claim 7, in, The detection module comprises: A fusion submodule, used for fusing the lane line sorting feature and the image feature to obtain a fusion feature; an extraction submodule, configured to extract features from the fused features to obtain lane line fusion features corresponding to each of the plurality of lane lines; and The detection submodule is used to perform detection based on the lane line fusion feature to obtain the lane line information.

9. The method according to claim 8, in, The detection submodule comprises: A detection unit is used to process the multiple lane line fusion features based on an attention mechanism according to the arrangement order between the multiple lane line fusion features to obtain multiple lane line information, wherein the arrangement order between the multiple lane line fusion features is determined according to the lane line sorting information.

10. The method according to claim 8, in, The fusion submodule includes: a first fusion unit, configured to fuse the image feature and the map feature based on an attention mechanism to obtain an image fusion feature, wherein the map feature is determined based on a road map related to the road image; and The second fusion unit is used to fuse the image fusion feature and the lane line sorting feature to obtain the fusion feature.

11. The method according to claim 7, in, The positioning module comprises: A positioning submodule is used to perform vehicle positioning based on at least one of a first attribute and a second attribute and the lane line information, wherein the first attribute represents that a first lane line among the plurality of lane lines is blocked, and the first attribute is determined based on the following operations: Performing object detection based on the image feature or the image fusion feature to obtain an object detection result, wherein the image fusion feature is determined based on the road image and a road map related to the road image, and the object detection result represents an object around the target vehicle; In a case where the object detection result indicates that the object occludes a first lane line among the plurality of lane lines, determining the first attribute; The second attribute represents that the target vehicle is located at both sides of a second lane line among the multiple lane lines, and the second attribute is obtained by performing lane-riding attribute detection on the target vehicle based on the image feature or the image fusion feature.

12. An electronic device, include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.

13. A non-transitory computer-readable storage medium storing computer instructions, in, The computer instructions are used to cause the computer to execute the method according to any one of claims 1-6.

14. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 6.

15. An autonomous driving vehicle, include: The electronic device according to claim 12.

Images