Method for fusing lane lines and electronic device

Abstract

The embodiment of the invention provides a method for fusing lane lines and electronic equipment. The method comprises the steps that at least two sets of lane line data sets are acquired, each set of lane line data set in the at least two sets of lane line data sets comprises a plurality of points represented by point sets, each point has a three-dimensional coordinate value and an attribute tag about a traffic sign line, and the attribute tag indicates the type of the point belonging to the traffic sign line; a point set in the lane line data set is screened to obtain a screened point set meeting a predetermined condition, and a solid line of the traffic sign line is represented by a plurality of continuous points spaced by a predetermined distance in the screened point set; based on the screening point sets corresponding to the at least two groups of lane line data sets respectively, establishing at least two lane lines having a matching relationship with each other; and based on the at least two lane lines having the matching relationship, determining a fused lane line. According to the method disclosed by the embodiment of the invention, the matching accuracy and generalization of the model are improved, so that the reliability of lane line fusion is improved.

Description

Technical Field

[0001] Example embodiments of the present disclosure generally relate to the field of autonomous driving, and more particularly to a method for fusing lane lines and an electronic device. Background Art

[0002] When producing high-precision maps, it is necessary to collect road images, as well as data such as the vehicle's real-time tracking (RTK) and inertial measurement unit (IMU), and realize map production through visual 3D reconstruction technology. In order to further improve the completeness and accuracy of the map, it is necessary to collect and reconstruct data on the same road multiple times and fuse the maps produced multiple times. Since there are inevitably accuracy errors in multiple productions, the maps cannot completely overlap, so lane line matching and fusion technology is required to match and fuse the multiple reconstructed maps. Summary of the invention

[0003] In a first aspect of the present disclosure, a method for merging lane lines is provided. The method comprises: obtaining at least two sets of lane line data sets, each of the at least two sets of lane line data sets comprises a plurality of points represented by a point set, each of the plurality of points has a three-dimensional coordinate value and an attribute label about a traffic sign line, and the attribute label indicates that the point belongs to the type of traffic sign line; filtering the point set in the lane line data set to obtain a filtering point set that meets a predetermined condition, wherein the solid line of the traffic sign line is represented by a plurality of points in the filtering point set that are continuous and spaced apart by a predetermined distance; establishing at least two lane lines having a matching relationship with each other based on the filtering point sets respectively corresponding to the at least two sets of lane line data sets; and determining a fused lane line based on the at least two lane lines having a matching relationship.

[0004] In some embodiments, the types of traffic sign lines include at least solid lines, dashed lines and arrow lines, and the screening point set that meets the predetermined conditions also includes at least the midpoint on the dashed line and the center point of the arrow line.

[0005] In some embodiments, the plurality of points representing the solid line are determined by projecting at least one of the midpoint on the dashed line and the center point of the arrow line onto the intersection point on the solid line.

[0006] In some embodiments, establishing at least two lane lines having a matching relationship with each other includes: inputting the screening point set into a matching model to establish at least two lane lines having a matching relationship with each other.

[0007] In some embodiments, establishing at least two lane lines that have a matching relationship with each other also includes: determining a pre-matching matrix based on at least two sets of lane line data sets; determining a pre-matching matrix pair between any two sets of lane line data sets in the at least two sets of lane line data sets based on the pre-matching matrix, wherein the spatial distance between the pre-matching matrix pair and other surrounding pre-matching matrix pairs is less than a predetermined threshold in terms of point-to-point spatial distance; and determining at least two lane lines based on the pre-matching matrix pair.

[0008] In some embodiments, determining the fused lane line includes: determining the ratio of the screening points corresponding to the two lane lines to be fused to form a pre-matching matrix pair; in response to determining that the ratio is greater than a threshold ratio, determining that the two lane lines to be fused form a matching relationship; and determining the fused lane line based on the two lane lines to be fused that form a matching relationship.

[0009] In some embodiments, the method further includes: training the matching model based on real road matching data and simulation data and using a echelon descent optimization algorithm.

[0010] In a second aspect of the present disclosure, an electronic device is provided. The electronic device includes: at least one processing unit; and at least one memory, which is coupled to the at least one processing unit and stores machine executable instructions, and when the instructions are executed by the at least one processing unit, the device performs an action, the action including: obtaining at least two sets of lane line data sets, each of the at least two sets of lane line data sets includes a plurality of points represented by a point set, each of the plurality of points has a three-dimensional coordinate value and an attribute label about a traffic sign line, and the attribute label indicates that the point belongs to the type of traffic sign line; filtering the point set in the lane line data set to obtain a filtering point set that meets a predetermined condition, wherein the solid line of the traffic sign line is represented by a plurality of points in the filtering point set that are continuous and spaced apart by a predetermined distance; establishing at least two lane lines having a matching relationship with each other based on the filtering point sets respectively corresponding to the at least two sets of lane line data sets; and determining a fused lane line based on the at least two lane lines having the matching relationship.

[0011] In a third aspect of the present disclosure, a computer-readable storage medium is provided, wherein a computer program is stored on the computer-readable storage medium, and the computer program can be executed by a processor to implement the method of the first aspect.

[0012] In a fourth aspect of the present disclosure, a computer program product is provided, which includes computer executable instructions, and when the computer executable instructions are executed by a processor, the method of the first aspect is implemented.

[0013] It should be understood that the contents described in this content section are not intended to limit the key features or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The above and other objects, features and advantages of the present disclosure will become more apparent through a more detailed description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, wherein like reference numerals generally represent like components throughout the exemplary embodiments of the present disclosure.

[0015] Figure 1 A simplified schematic diagram showing a scenario to which the method according to an embodiment of the present disclosure can be applied;

[0016] Figure 2 An exemplary flow chart of a method provided according to an embodiment of the present disclosure is shown;

[0017] Figure 3 A schematic diagram showing a method for screening data points according to an embodiment of the present disclosure is shown;

[0018] Figure 4 A schematic diagram of a flow chart of matching screening data points in a method according to an embodiment of the present disclosure is shown;

[0019] Figure 5 A schematic diagram showing two sets of lane lines having a matching relationship obtained according to the method of an embodiment of the present disclosure;

[0020] Figure 6 A schematic diagram showing a method according to an embodiment of the present disclosure of obtaining fused lane lines by fusing multiple lane line data sets;

[0021] Figure 7 A flowchart of a method according to an embodiment of the present disclosure is shown; and

[0022] Figure 8 A schematic block diagram of an electronic device suitable for implementing an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0023] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0024] It should be noted that the titles of any sections / subsections provided herein are not restrictive. Various embodiments are described throughout this article, and any type of embodiment may be included under any section / subsection. In addition, the embodiments described in any section / subsection may be combined in any manner with any other embodiments described in the same section / subsection and / or different sections / subsections.

[0025] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may be included below. The terms "first", "second", etc. may refer to different or the same objects. Other explicit and implicit definitions may be included below.

[0026] As used herein, the term "model" can learn the association between the corresponding input and output from the training data, so that after the training is completed, the corresponding output can be generated for a given input. The generation of the model can be based on machine learning technology. Deep learning is a machine learning algorithm that processes inputs and provides corresponding outputs by using multiple layers of processing units. In this article, "model" may also be referred to as "machine learning model", "machine learning network" or "network", which are used interchangeably in this article. A model may also include different types of processing units or networks.

[0027] As used herein, a "unit", "operating unit" or "subunit" may consist of a machine learning model or network of any suitable structure. As used herein, a group of elements or similar expressions may include one or more such elements.

[0028] The embodiments of the present disclosure may involve user data, data acquisition and / or use, etc. These aspects are subject to the corresponding laws, regulations and relevant provisions. In the embodiments of the present disclosure, all data collection, acquisition, processing, processing, forwarding, use, etc. are carried out on the premise that the user knows and confirms. Accordingly, when implementing each embodiment of the present disclosure, the type, scope of use, usage scenario, etc. of the data or information that may be involved should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with the relevant laws and regulations. The specific notification and / or authorization method can vary according to the actual situation and application scenario, and the scope of the present disclosure is not limited in this respect.

[0029] As briefly mentioned above, in the process of map production, a large amount of data needs to be collected first, including image information of the road and the positioning and posture data of the vehicle itself. These data are mainly obtained through sensors such as real-time kinematic positioning (RTK) and inertial measurement units (IMU). RTK provides high-precision position information, while IMU records the vehicle's acceleration, angular velocity and other motion information. These data combined provide the basis for subsequent visual 3D reconstruction.

[0030] Visual 3D reconstruction technology uses 2D images taken from multiple angles to generate 3D road models and environmental maps. This is one of the core steps in map production. However, in order to improve the completeness and accuracy of the generated map, the data collected once is often not enough, and multiple data collections are required for the same road.

[0031] Due to the influence of environmental conditions and sensor errors during each data collection, there are often certain accuracy errors between the generated maps, resulting in multiple versions of maps not being able to completely overlap. In this case, the map fusion process becomes crucial.

[0032] Lane matching fusion technology is used to solve this problem. This technology identifies lane lines in different reconstruction versions and uses them as a benchmark for matching and alignment. During the fusion process, multiple reconstructed maps are compared and adjusted to ultimately generate a more complete and accurate map. In this way, errors in different versions of maps can be corrected, improving the accuracy and consistency of the final map.

[0033] At present, lane fusion usually adopts several technologies, such as spatial nearest neighbor matching technology, spatial nearest neighbor matching technology based on lateral consistency, and matching technology based on manually designed lane features. The original spatial nearest neighbor matching technology can only work when the error is small. When the error exceeds the width of a lane, the method will fail completely.

[0034] The core idea of ​​the spatial nearest neighbor matching technology based on lateral consistency is to assume that the lateral errors of the lane lines to be matched are consistent. This technology first performs spatial nearest neighbor matching to establish a preliminary matching relationship between multiple lane lines, then calculates the lateral distance of the matching lane lines, and then iteratively adjusts until the lateral distance similarity of all matching lane lines reaches the highest, thereby obtaining the final matching result. Although this method effectively solves the problem of large lateral errors, it cannot deal with longitudinal errors and has high requirements for map integrity.

[0035] The matching technology based on manually designed lane features designs lane features and corresponding loss functions, performs iterative optimization until the loss value is minimized and obtains the matching result. However, due to the limited expression ability of manually designed features, this method has poor generalization ability when processing abnormal data.

[0036] In order to solve or at least partially solve the above-mentioned problems or other potential problems, an embodiment of the present disclosure provides a method for fusing lane lines, which first obtains at least two sets of lane line data sets containing three-dimensional coordinates and attribute labels, and filters them to extract point sets with specific attributes, wherein solid lines are represented by multiple points that are continuous and spaced at a predetermined distance. Then, matching lane lines are established based on the filtered point sets, and finally these matching lane lines are fused to generate unified fused lane lines. This method uses key points to express lane lines, simplifies line matching into a point matching task, and based on the attention-based matching model, improves the generalization ability of matching through data-driven. In addition, this method also uses a local consistency algorithm to improve the accuracy of matching.

[0037] Figure 1 A simplified schematic diagram of a scenario to which the method according to an embodiment of the present disclosure can be applied is shown. Figure 1 In the example environment shown, a fusion device 150 (also referred to as an electronic device), such as a server, for processing lane line data may be included. The fusion device 150 may process lane line data collected in multiple ways and / or multiple times, and perform a fusion operation to ultimately obtain a fused lane line. The vehicle 110 may obtain the fused lane line determined by the cloud device in an appropriate manner, and drive or automatically drive according to the fused lane line.

[0038] Figure 2 A simplified flow chart of a method for merging lane lines performed by an electronic device is shown. Figure 2 As shown, in box 210, at least two lane line data sets to be fused are first prepared. The data sets can be represented by point sets. Each point has a 3D coordinate value and an attribute label related to the traffic sign line. The attribute label indicates the type of traffic sign line to which the point belongs, such as a solid line, a dotted line, or an arrow line.

[0039] Next, in block 220, each set of points is screened to select points that meet the predetermined conditions. For example, in some embodiments, the midpoint of the dashed line in the traffic sign line and the center point of the arrow line can be selected as the screening point set. In addition, for the solid line in the traffic sign line, points can be selected along the predetermined distance (e.g., 1m, 2m or any other appropriate value) as the screening point set.

[0040] In some embodiments, alternatively or additionally, for a solid line, at least some of the midpoints of the dashed line and the center points of the arrow line may be projected onto the intersection points on the solid line as a screening point set, and for a solid line without a projection relationship, sampling is performed at a predetermined distance interval, such as Figure 3 Finally, we get the screening point set, that is, the point set p to be matched. i ={x,y,z,c}. By using key points to express lane lines, line matching is simplified to a point matching task, which significantly reduces the amount of calculation in the lane fusion process and improves the fusion efficiency.

[0041] Then, in block 230, at least two lane lines having a matching relationship with each other are established based on the screening point sets corresponding to the at least two lane line data sets. Figure 4 The example shown is used as an example for explanation. Specifically, in some embodiments, the screening point set obtained in the previous step (i.e., the point set to be matched) Input to the matching model, such as Figure 4 shown.

[0042] Figure 4 The matching model shown in the figure consists of two core modules, namely the attention-based neural network module and the local consistency-based matching module. In the attention-based neural network module, the input data is first positionally encoded, and the neural network's perception of spatial information is enhanced by adding position information. Next, the self-attention mechanism is used, which can capture the global feature relationship within the input data, ensuring that the model can not only see the local features, but also understand the overall structure of the entire input data. On this basis, the model uses the mutual attention mechanism to capture the feature association between the two data sets and help discover similarities or related information between them. Through this mechanism, the model can understand the potential matching relationship between different data from a global perspective, laying the foundation for subsequent precise matching.

[0043] In the matching module based on local consistency, the model focuses on matching by comparing the local features of the input data. First, the similar areas between the two data sets are analyzed by local consistency to ensure that the matching process not only considers the global features, but also pays attention to the similarity of the local areas. Then, the model further uses the neighborhood matching expression method, combines the results of local consistency, refines the matching expression of the features of the neighboring areas, and finally outputs the matching results. For example, in some embodiments, in the local consistency matching module, the Hungarian algorithm can be used to calculate the matching relationship between local features to ensure that the optimal matching combination is found between local features, thereby enhancing the matching accuracy of the entire model, wherein the Hungarian matching algorithm, also known as the Kuhn-Munkres algorithm, is a classic graph theory algorithm, mainly used to solve the maximum weight matching problem of bipartite graphs. The core idea is to find a set of matches in the bipartite graph so that the weights and weights of all matching edges are maximized or minimized. This module makes up for the shortcomings of the global attention mechanism in detail matching, ensuring that the model can grasp the overall situation and handle subtle local feature differences when dealing with complex matching tasks.

[0044] In general, the matching model combines the global attention mechanism with local consistency matching to ensure that when matching, it can fully understand the global relationship of the input data and carefully process the local features, thereby improving the accuracy and stability of the matching.

[0045] Specifically, the point set to be matched first enters the attention-based neural network module, which encodes the position and upgrades the point set to a high-dimensional space through the self-attention and mutual-attention network layers, outputting the features of each key point. Then enter the local consistency based module, which first performs matrix multiplication on the two sets of features to obtain the pre-matching matrix That is, the pre-matching matrix is ​​determined based on at least two sets of lane line data sets.

[0046] Then the pre-matching pairs are obtained by the Hungarian matching algorithm. In each pre-matching pair, the local consistency algorithm calculates whether it is similar to the surrounding matching pairs in terms of spatial distance between points, such as Figure 5 That is, the electronic device determines a pre-matching matrix pair between any two lane line data sets in at least two lane line data sets based on the pre-matching matrix, wherein the spatial distance between the points of the pre-matching matrix pair and other surrounding pre-matching matrix pairs is less than a predetermined threshold (i.e., similar).

[0047] Further references Figure 5, where the left and right sides show lane lines represented by two lane line datasets, respectively, and the horizontal lines represent points with matching relationships. Figure 4 As shown in the ellipse in the figure, the distances between the four matching pairs are similar. If the distances between the pre-matching pair and the surrounding matching pairs are too different, the local consistency algorithm will reduce the pre-matching matrix. The value of the corresponding position in . Continue to iterate all matching pairs to obtain the final matching matrix S i,j , and then the final matching result is output through the Hungarian matching algorithm.

[0048] Finally, in boxes 240 and 250, the at least two lane lines can be determined based on the pre-matching matrix pair. Specifically, the electronic device will count the matching relationship of each lane line point, and if more than 90% of the key points of the other lane line form a matching relationship with the lane line, the two lane lines form a match. In other words, the electronic device will determine the ratio of the pre-matching matrix pair formed by the screening points corresponding to the two lane lines to be fused, and then if the ratio is determined to be greater than the threshold ratio (such as the 90% mentioned above), it is determined that the two lane lines to be fused form a matching relationship.

[0049] Finally, the fused lane line is determined based on the two lane lines to be fused that form a matching relationship. In some embodiments, the lane lines that form a matching relationship can be spatially averaged to obtain the final fused lane line, such as Figure 6 If there are more than two lane line data sets, two of the lane line data sets can be processed as described above to obtain fused lane lines, and then the fused lane lines and the remaining lane line data sets are processed as described above to obtain new fused lane lines, until all lane line data sets are processed as described above to finally obtain a fused lane line.

[0050] In some embodiments, if there are more than two lane line data sets, the above processing may be performed on every two of the lane line data sets until all lane line data sets are finally processed and fused lane lines are finally obtained.

[0051] In order to make the matching lane lines output by the above matching model more accurate, the above matching model needs to be trained. The training data set can include real road matching data and a large amount of simulation data. Real data can improve the accuracy of the model in the real world, while simulation data helps to increase the diversity and scale of training. In addition, simulation data allows testing of special or extreme situations, enhancing the robustness of the model.

[0052] The training loss function is Where P i,jFor correct matching in the matching matrix S i,j The score in and For the error matching in the matching matrix S i,j The loss function ensures that the model is more accurate during inference by evaluating correct matches and incorrect matches. By balancing the high scores for correct matches and the low scores for incorrect matches, the loss function can effectively drive the model to distinguish between correct and incorrect matches, thereby improving the model's discrimination ability. This loss design not only focuses on the scores of correct matches, but also strengthens the matching accuracy of the model by penalizing the scores of incorrect matches, thereby improving overall performance.

[0053] The training optimizer can use the Adam optimizer, which is a commonly used gradient descent optimization algorithm. The Adam optimizer combines the advantages of momentum and adaptive learning rate, and can converge faster and adapt to different types of data. In addition, Adam is less sensitive to hyperparameters (such as learning rate), making the training process more stable and reducing the difficulty of parameter adjustment. The learning rate is set to 10 -4 , that is, a relatively small step size, ensures that the model gradually converges. The training iteration round is 200,000 times, ensuring that the model has enough time to learn and optimize the data, thereby achieving better performance. Through large amounts of data and refined matching matrix training, the model can accurately match in a variety of different scenarios, thereby improving its performance on new data, that is, enhancing its generalization ability.

[0054] Figure 7 A flow chart of a method for merging lane lines according to an embodiment of the present disclosure is shown. In some embodiments, the method can be implemented by the electronic device mentioned above or other appropriate devices. For ease of understanding, the specific examples, numbers or values ​​mentioned in the following description are merely exemplary and are not intended to limit the scope of protection of the present disclosure.

[0055] like Figure 7 As shown, in the method executed by the electronic device, in box 710, the electronic device obtains at least two sets of lane line data sets, each of the at least two sets of lane line data sets includes multiple points represented by a point set, each point in the multiple points has a three-dimensional coordinate value and an attribute label about a traffic sign line, and the attribute label indicates that the point belongs to the type of traffic sign line.

[0056] Next, in box 720, the electronic device filters the point set in the lane line data set to obtain a filtered point set that meets a predetermined condition, wherein a solid line is represented by a plurality of points in the filtered point set that are continuous and spaced apart by a predetermined distance.

[0057] In box 730, the electronic device establishes at least two lane lines that have a matching relationship with each other based on the screening point sets corresponding to the at least two lane line data sets. Finally, in box 740, the electronic device determines a fused lane line based on the at least two lane lines that have a matching relationship.

[0058] In some embodiments, the types of traffic sign lines include at least solid lines, dashed lines and arrow lines, and the screening point set that meets the predetermined conditions also includes at least the midpoint on the dashed line and the center point of the arrow line.

[0059] In some embodiments, the plurality of points representing the solid line are determined by projecting at least one of the midpoint on the dashed line and the center point of the arrow line onto the intersection point on the solid line.

[0060] In some embodiments, establishing at least two lane lines having a matching relationship with each other may include: inputting the screening point set into a matching model to establish at least two lane lines having a matching relationship with each other.

[0061] In some embodiments, establishing at least two lane lines that have a matching relationship with each other may also include determining a pre-matching matrix based on at least two sets of lane line data sets; determining a pre-matching matrix pair between any two sets of lane line data sets in the at least two sets of lane line data sets based on the pre-matching matrix, wherein the spatial distance between the pre-matching matrix pair and other surrounding pre-matching matrix pairs is less than a predetermined threshold in terms of point-to-point spatial distance; and determining at least two lane lines based on the pre-matching matrix pair.

[0062] In some embodiments, determining the fused lane line may include determining the ratio of the screening points corresponding to the two lane lines to be fused to form a pre-matching matrix pair; in response to determining that the ratio is greater than a threshold ratio, determining that the two lane lines to be fused form a matching relationship; and determining the fused lane line based on the two lane lines to be fused that form a matching relationship.

[0063] In some embodiments, the electronic device may also train the matching model based on real road matching data and simulation data and using a echelon descent optimization algorithm.

[0064] Figure 8 Schematic block diagram of an electronic device suitable for implementing an embodiment of the present disclosure is shown. Figure 8As shown, according to an electronic device provided by the present disclosure, the electronic device 800 may be a server or other appropriate device mentioned above for communicating with a vehicle. The electronic device 800 includes a central processing unit (CPU) 801, which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) 802 or computer program instructions loaded from a storage unit into a random access memory (RAM) 803. In RAM 803, various programs and data required for device operation may also be stored. CPU 801, ROM 802, and RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0065] Multiple components in the device are connected to the I / O interface 805, including: an input unit 806, such as a touch screen, buttons, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0066] The various processes and processing described above, such as the processes mentioned above, can be performed by the processing unit 801. For example, in some embodiments, the processes 710, 720, 730 and 740 can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit. In some embodiments, part or all of the computer program can be loaded and / or installed on the device via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the CPU 801, one or more actions of the processes 710, 720, 730 and 740 described above can be performed.

[0067] Embodiments of the present disclosure relate to methods, electronic devices, and / or computer program products. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present disclosure.

[0068] Computer readable storage medium can be a tangible device that can hold and store instructions used by an instruction execution device. Computer readable storage medium can be, for example, (but not limited to) an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a convex structure in a groove on which instructions are stored, and any suitable combination thereof. The computer readable storage medium used here is not interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated by a waveguide or other transmission medium (for example, a light pulse by an optical fiber cable), or an electrical signal transmitted by a wire.

[0069] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0070] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0071] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0072] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0073] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0074] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0075] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for fusing lane lines, comprising: Acquire at least two sets of lane line data sets, each of the at least two sets of lane line data sets includes a plurality of points represented by a point set, each point in the plurality of points has a three-dimensional coordinate value and an attribute label with respect to a traffic sign line, and the attribute label indicates that the point belongs to a type of the traffic sign line; Screening the point set in the lane line data set to obtain a screening point set that meets a predetermined condition, wherein the solid line of the traffic sign line is represented by a plurality of points in the screening point set that are continuous and spaced apart by a predetermined distance; Establishing at least two lane lines having a matching relationship with each other based on the screening point sets respectively corresponding to the at least two sets of lane line data sets; as well as Based on the at least two lane lines having the matching relationship, a fused lane line is determined.

2. The method according to claim 1, wherein the types of the traffic sign lines include at least the solid line, the dotted line and the arrow line, and The screening point set that meets the predetermined condition at least includes the midpoint on the dotted line and the center point of the arrow line. 3 . The method according to claim 2 , wherein the plurality of points representing the solid line are determined by projecting at least one of a midpoint on the dotted line and a center point of the arrow line onto an intersection point on the solid line.

4. The method according to any one of claims 1 to 3, wherein establishing at least two lane lines having a matching relationship with each other comprises: The screening point set is input into a matching model to establish at least two lane lines having a matching relationship with each other.

5. The method according to claim 4, wherein establishing at least two lane lines having a matching relationship with each other further comprises: Determine a pre-matching matrix based on the at least two sets of lane line data sets; Determine a pre-matching matrix pair between any two of the at least two lane line data sets based on the pre-matching matrix, wherein the spatial distance between points of the pre-matching matrix pair and other surrounding pre-matching matrix pairs is less than a predetermined threshold; as well as The at least two lane lines are determined based on the pre-matching matrix pair.

6. The method according to claim 5, wherein determining the fused lane line comprises: Determine the ratio of the pre-matching matrix pair formed by the screening points corresponding to the two lane lines to be fused; In response to determining that the ratio is greater than a threshold ratio, determining that the two lane lines to be fused form a matching relationship; as well as The fused lane line is determined based on the two lane lines to be fused that form the matching relationship.

7. The method according to claim 4, further comprising: The matching model is trained based on real road matching data and simulation data and using an echelon descent optimization algorithm.

8. An electronic device, comprising: a memory for storing one or more computer instructions; as well as A processor, configured to execute the one or more computer instructions stored in the memory to perform the following actions: Acquire at least two sets of lane line data sets, each of the at least two sets of lane line data sets includes a plurality of points represented by a point set, each point in the plurality of points has a three-dimensional coordinate value and an attribute label with respect to a traffic sign line, and the attribute label indicates that the point belongs to a type of the traffic sign line; Screening the point set in the lane line data set to obtain a screening point set that meets a predetermined condition, wherein the solid line of the traffic sign line is represented by a plurality of points in the screening point set that are continuous and spaced apart by a predetermined distance; Establishing at least two lane lines having a matching relationship with each other based on the screening point sets respectively corresponding to the at least two sets of lane line data sets; as well as Based on the at least two lane lines having the matching relationship, a fused lane line is determined.

9. The electronic device according to claim 1, wherein the types of the traffic sign lines include at least the solid line, the dotted line and the arrow line, and The screening point set that meets the predetermined condition at least includes the midpoint on the dotted line and the center point of the arrow line. 10 . The electronic device according to claim 9 , wherein the plurality of points representing the solid line are determined by projecting at least one of a midpoint on the dotted line and a center point of the arrow line onto an intersection point on the solid line.

11. The electronic device according to any one of claims 8 to 10, wherein establishing at least two lane lines having a matching relationship with each other comprises: The screening point set is input into a matching model to establish at least two lane lines having a matching relationship with each other.

12. The electronic device according to claim 11, wherein establishing at least two lane lines having a matching relationship with each other further comprises: Determine a pre-matching matrix based on the at least two sets of lane line data sets; Determine a pre-matching matrix pair between any two of the at least two lane line data sets based on the pre-matching matrix, wherein the spatial distance between points of the pre-matching matrix pair and other surrounding pre-matching matrix pairs is less than a predetermined threshold; as well as The at least two lane lines are determined based on the pre-matching matrix pair.

13. The electronic device according to claim 12, wherein determining the fused lane line comprises: Determine the ratio of the pre-matching matrix pair formed by the screening points corresponding to the two lane lines to be fused; In response to determining that the ratio is greater than a threshold ratio, determining that the two lane lines to be fused form a matching relationship; as well as The fused lane line is determined based on the two lane lines to be fused that form the matching relationship.

14. The electronic device according to claim 11, further comprising: The matching model is trained based on real road matching data and simulation data and using an echelon descent optimization algorithm.

15. A computer-readable storage medium having one or more computer instructions stored thereon, wherein the one or more computer instructions are executed by a processor to implement the method according to any one of claims 1-7.

16. A computer program product comprising computer executable instructions, wherein the computer executable instructions, when executed by a processor, implement the method according to any one of claims 1 to 7.

Images