Combining lane lines of a vehicle

Abstract

A method for merging lane maps includes determining a road topology for a road segment. The method further includes identifying a plurality of fusion points based at least in part on the road topology and at least in part on the first lane map of the road segment and the second lane map of the road segment. The method further includes forming a fused lane map based at least in part on the plurality of fusion points. The method further includes performing a first action based at least in part on the fused lane map.

Description

Technical Field

[0001] The present disclosure relates to advanced driver assistance and automated driving systems and methods for vehicles. Background Art

[0002] To improve occupant perception and convenience, vehicles may be equipped with advanced driver assistance systems (ADAS) and / or automated driving systems (ADS). ADAS systems may use various sensors such as cameras, radar, and LiDAR to detect and identify objects around the vehicle, including other vehicles, pedestrians, road configurations, traffic signs, and road markings. ADAS systems can take action based on environmental conditions around the vehicle, such as braking or alerting vehicle occupants. However, current ADAS systems may not take into account other factors that may affect the occupant experience. ADS systems may use various sensors to detect objects in the vehicle's surroundings and control the vehicle to navigate the vehicle through the environment to a predetermined destination. However, current ADAS and ADS systems may rely on accurate interpretation of road markings (e.g., lane markings) for optimal operation.

[0003] Therefore, while ADAS and ADS systems and methods achieve their intended purposes, a need remains for new and improved systems and methods for incorporating a vehicle's lane map. Summary of the invention

[0004] According to several aspects, a method for merging lane lines is provided. The method includes determining a road topology of a road segment. The method also includes identifying a plurality of fusion points based at least in part on the road topology and at least in part on a first lane line map of the road segment and a second lane line map of the road segment. The method also includes forming a fused lane line map based at least in part on the plurality of fusion points. The method also includes performing a first action based at least in part on the fused lane line map.

[0005] In another aspect of the present disclosure, determining the road topology of the road section may also include retrieving topological data about the road section. Determining the road topology of the road section may also include determining multiple virtual side lines based at least in part on the topological data about the road section. Each virtual side line in the multiple virtual side lines is perpendicular to the road section. Multiple virtual side lines are evenly spaced along the length of the road section.

[0006] In another aspect of the present disclosure, identifying a plurality of fused points may further include determining a plurality of corresponding point pairs. Each point pair in the plurality of corresponding point pairs includes a first point and a second point. Each point pair in the plurality of corresponding point pairs corresponds to a virtual siding in the plurality of virtual sidings. Identifying a plurality of fused points may further include fusing each point pair in the plurality of corresponding point pairs to form a plurality of fused points.

[0007] In another aspect of the present disclosure, determining multiple corresponding point pairs may also include selecting a first virtual side line among multiple virtual side lines. Determining multiple corresponding point pairs may also include moving a subset of multiple points of the first lane line map toward multiple points of the second lane line map based at least in part on the first virtual side line among the multiple virtual side lines. Determining multiple corresponding point pairs may also include identifying multiple first intersections. Each of the multiple first intersections is a point among the multiple points of the first lane line map. Each of the multiple first intersections intersects with the first virtual side line among the multiple virtual side lines. Determining multiple corresponding point pairs may also include identifying multiple second intersections. Each of the multiple second intersections is a point among the multiple points of the second lane line map. Each of the multiple second intersections intersects with the first virtual side line among the multiple virtual side lines. Determining multiple corresponding point pairs may also include determining a point pair among the multiple corresponding point pairs based at least in part on the multiple first intersections and the multiple second intersections. A first point of one of the plurality of corresponding point pairs is from a plurality of first intersection points, and a second point of one of the plurality of corresponding point pairs is from a plurality of second intersection points. A distance between the first point and the second point is less than or equal to a predetermined corresponding point pair distance threshold. Determining the plurality of corresponding point pairs may also include repeating a selecting step, a moving step, an identifying step, and a determining step until each of the plurality of virtual sidings has been selected.

[0008] In another aspect of the present disclosure, moving a subset of the plurality of points of the first lane line map may also include selecting a subset of the plurality of points of the first lane line map. Each subset of the plurality of points of the first lane line map is within a circle centered on a virtual side line of the plurality of virtual side lines. Moving the subset of the plurality of points of the first lane line map may also include moving the subset of the plurality of points of the first lane line map toward the plurality of points of the second lane line map using an iterative closest point algorithm. The subset of the plurality of points of the first lane line map moves only along a horizontal axis. The horizontal axis is parallel to a virtual side line of the plurality of virtual side lines.

[0009] In another aspect of the present disclosure, determining a point pair in the plurality of corresponding point pairs may further include identifying a first point and a second point. A driving direction of the first point corresponds to a driving direction of the second point.

[0010] In another aspect of the present disclosure, identifying the first point and the second point may also include determining the driving direction of each of the plurality of first intersections. The driving direction of each of the plurality of first intersections includes one of a left driving direction and a right driving direction. Identifying the first point and the second point may also include determining the driving direction of each of the plurality of second intersections. The driving direction of each of the plurality of second intersections includes one of a left driving direction and a right driving direction. Identifying the first point and the second point may also include identifying the first point and the second point. The first point is one of the plurality of first intersections having a left driving direction, and the second point is one of the plurality of second intersections having a left driving direction.

[0011] In another aspect of the present disclosure, fusing each point pair in the plurality of corresponding point pairs may further include generating a plurality of fused points. Each fused point in the plurality of fused points is located between a first point and a second point in one of the plurality of corresponding point pairs.

[0012] In another aspect of the present disclosure, forming the fused lane line map may further include drawing a lane line between each of the plurality of fused points. The lane line is substantially parallel to the road segment.

[0013] In another aspect of the present disclosure, performing the first action may further include adjusting operation of the vehicle based at least in part on the fused lane map.

[0014] According to several aspects, a system for merging lane maps of vehicles is provided. The system also includes a camera system, a vehicle communication system, and a vehicle controller in electrical communication with the camera system and the vehicle communication system. The vehicle controller is programmed to use the camera system to generate a first lane map of a road segment in the vehicle's surroundings. The vehicle controller is also programmed to use the vehicle communication system to receive a second lane map of the road segment. The vehicle controller is also programmed to identify multiple fusion points based at least in part on the first lane map and the second lane map. The vehicle controller is also programmed to form a fused lane map based at least in part on the multiple fusion points. The vehicle controller is also programmed to perform a first action based at least in part on the fused lane map.

[0015] In another aspect of the present disclosure, in order to identify multiple fusion points, the vehicle controller is also programmed to determine multiple virtual lateral lines. Each of the multiple virtual lateral lines is perpendicular to the road segment. The multiple virtual lateral lines are evenly spaced along the length of the road segment. In order to identify multiple fusion points, the vehicle controller is also programmed to determine multiple corresponding point pairs. Each point pair in the multiple corresponding point pairs includes a first point and a second point. Each point pair in the multiple corresponding point pairs corresponds to a virtual lateral line in the multiple virtual lateral lines. In order to identify multiple fusion points, the vehicle controller is also programmed to fuse each point pair in the multiple corresponding point pairs to form multiple fusion points.

[0016] In another aspect of the present disclosure, in order to determine a point pair in a plurality of corresponding point pairs, the vehicle controller is further programmed to select a first virtual side line in a plurality of virtual side lines. In order to determine a point pair in a plurality of corresponding point pairs, the vehicle controller is further programmed to move a subset of a plurality of points of a first lane line map toward a plurality of points of a second lane line map based at least in part on a first virtual side line in a plurality of virtual side lines. In order to determine a point pair in a plurality of corresponding point pairs, the vehicle controller is further programmed to identify a plurality of first intersection points. Each of the plurality of first intersection points is a point in a plurality of points in the first lane line map. Each of the plurality of first intersection points intersects with a first virtual side line in a plurality of virtual side lines. In order to determine a point pair in a plurality of corresponding point pairs, the vehicle controller is further programmed to identify a plurality of second intersection points. Each of the plurality of second intersection points is a point in a plurality of points in the second lane line map. Each of the plurality of second intersection points intersects with a first virtual side line in a plurality of virtual side lines. To determine a point pair in the plurality of corresponding point pairs, the vehicle controller is further programmed to determine a point pair in the plurality of corresponding point pairs based at least in part on the plurality of first intersection points and the plurality of second intersection points. A first point in the point pair in the plurality of corresponding point pairs is from the plurality of first intersection points, and a second point in the point pair in the plurality of corresponding point pairs is from the plurality of second intersection points. A distance between the first point and the second point is less than or equal to a predetermined corresponding point pair distance threshold.

[0017] In another aspect of the present disclosure, in order to move a subset of the plurality of points of the first lane line map, the vehicle controller is further programmed to select a subset of the plurality of points of the first lane line map. Each subset of the plurality of points of the first lane line map is within a circle centered on one of the plurality of virtual side lines. In order to move the subset of the plurality of points of the first lane line map, the vehicle controller is further programmed to move the subset of the plurality of points of the first lane line map toward the plurality of points of the second lane line map using an iterative closest point algorithm. The subset of the plurality of points of the first lane line map moves only along a horizontal axis. The horizontal axis is parallel to one of the plurality of virtual side lines.

[0018] In another aspect of the present disclosure, in order to determine a point pair in a plurality of corresponding point pairs, the vehicle controller is also programmed to determine the driving direction of each of the plurality of first intersections. The driving direction of each of the plurality of first intersections includes one of a left driving direction and a right driving direction. In order to determine a point pair in a plurality of corresponding point pairs, the vehicle controller is also programmed to determine the driving direction of each of the plurality of second intersections. The driving direction of each of the plurality of second intersections includes one of a left driving direction and a right driving direction. In order to determine a point pair in a plurality of corresponding point pairs, the vehicle controller is also programmed to identify a first point and a second point. The first point is one of the plurality of first intersections having a left driving direction, and the second point is one of the plurality of second intersections having a left driving direction.

[0019] In another aspect of the present disclosure, to fuse each of the plurality of corresponding point pairs, the vehicle controller is further programmed to generate a plurality of fusion points. Each of the plurality of fusion points is located between a first point and a second point of one of the plurality of corresponding point pairs.

[0020] In another aspect of the present disclosure, performing the first action may also include adjusting the operation of the autonomous driving system based at least in part on the fused lane map.

[0021] According to several aspects, a system for merging lane maps of vehicles is provided. The system includes a server system. The server system includes a server communication system, and a server controller in electrical communication with the server communication system. The server controller is programmed to use the server communication system to receive a first lane map and a second lane map of a road segment. The server controller is also programmed to identify multiple fusion points based at least in part on the first lane map and the second lane map. The server controller is also programmed to form a fused lane map based at least in part on multiple fusion points. The server controller is also programmed to use the server communication system to transmit the fused lane map. The system also includes a vehicle system. The vehicle system includes a vehicle communication system, an autonomous driving system, and a vehicle controller in electrical communication with the vehicle communication system and the autonomous driving system. The vehicle controller is programmed to receive the fused lane map from the server system using the vehicle communication system. The vehicle controller is also programmed to adjust the operation of the autonomous driving system based at least in part on the fused lane map.

[0022] In another aspect of the present disclosure, in order to identify multiple fusion points, the server controller is also programmed to determine multiple virtual sidings. Each of the multiple virtual sidings is perpendicular to the road segment. The multiple virtual sidings are evenly spaced along the length of the road segment. In order to identify multiple fusion points, the server controller is also programmed to determine multiple corresponding point pairs. Each point pair in the multiple corresponding point pairs includes a first point and a second point. Each point pair in the multiple corresponding point pairs corresponds to a virtual siding in the multiple virtual sidings. In order to identify multiple fusion points, the server controller is also programmed to generate multiple fusion points. Each of the multiple fusion points is located between the first point and the second point of one of the multiple corresponding point pairs.

[0023] In another aspect of the present disclosure, in order to determine a point pair in a plurality of corresponding point pairs, the server controller is further programmed to select a first virtual side line in a plurality of virtual side lines. In order to determine a point pair in a plurality of corresponding point pairs, the server controller is further programmed to move a subset of a plurality of points of a first lane line map toward a plurality of points of a second lane line map based at least in part on a first virtual side line in a plurality of virtual side lines. In order to determine a point pair in a plurality of corresponding point pairs, the server controller is further programmed to identify a plurality of first intersection points. Each of the plurality of first intersection points is a point in a plurality of points in the first lane line map. Each of the plurality of first intersection points intersects with a first virtual side line in a plurality of virtual side lines. In order to determine a point pair in a plurality of corresponding point pairs, the server controller is further programmed to identify a plurality of second intersection points. Each of the plurality of second intersection points is a point in a plurality of points in the second lane line map. Each of the plurality of second intersection points intersects with a first virtual side line in a plurality of virtual side lines. To determine a point pair in the plurality of corresponding point pairs, the server controller is further programmed to determine a point pair in the plurality of corresponding point pairs based at least in part on the plurality of first intersection points and the plurality of second intersection points. A first point in the point pair in the plurality of corresponding point pairs is from the plurality of first intersection points, and a second point in the point pair in the plurality of corresponding point pairs is from the plurality of second intersection points. A distance between the first point and the second point is less than or equal to a predetermined corresponding point pair distance threshold.

[0024] Further areas of applicability will become apparent from the description provided herein.It should be understood that the description and specific examples are intended for purposes of illustration only and are not intended to limit the scope of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The drawings described herein are for illustration purposes only and are not intended to limit the scope of the present disclosure in any way.

[0026] Figure 1 is a schematic diagram of a system for merging lane maps of vehicles according to an exemplary embodiment;

[0027] Figure 2is a flow chart of a method for merging lane maps of vehicles according to an exemplary embodiment;

[0028] Figure 3 is an illustration of a plurality of lane line diagrams according to an exemplary embodiment;

[0029] Figure 4 is a flow chart of a method for determining a plurality of corresponding point pairs according to an exemplary embodiment;

[0030] Figure 5 is a diagram of a road segment showing a plurality of corresponding point pairs according to an exemplary embodiment;

[0031] Figure 6 is a diagram of a road segment showing a plurality of fusion points according to an exemplary embodiment; and

[0032] Figure 7 is an illustration of a fused lane map according to an exemplary embodiment. DETAILED DESCRIPTION

[0033] The following description is merely exemplary in nature and is not intended to limit the present disclosure, application, or uses.

[0034] Crowdsourcing methods can be used to collect data about lane lines on a road, such as lane maps. However, lane maps from different sources (e.g., perception measurements performed by a vehicle, aerial images of the road, vehicle telemetry data, etc.) at the same location (i.e., road segment) may be different due to variations in measurement equipment, environmental conditions, location accuracy, etc. Therefore, the present disclosure provides a new and improved system and method for merging lane maps obtained from different sources and / or at different points in time, thereby improving the performance and / or accuracy of vehicle systems (e.g., autonomous driving systems) that utilize lane maps.

[0035] Reference Figure 1 , a schematic diagram of a system 10 for merging lane lines is shown. The system 10 includes one or more vehicles 12, each of the one or more vehicles 12 includes a vehicle system 14. The system 10 also includes a server system 16.

[0036] The vehicle system 14 includes a vehicle controller 18 , a camera system 20 , an autonomous driving system 22 , and a vehicle communication system 24 .

[0037] The vehicle controller 18 is used to implement the method 100 for merging lane lines, as described below. The vehicle controller 18 includes at least one processor and a non-transitory computer-readable storage device or medium. The processor can be a custom or commercially available processor, a central processing unit (CPU), a graphics processing unit (GPU), an auxiliary processor in a plurality of processors associated with the vehicle controller 18, a semiconductor-based microprocessor (in the form of a microchip or chipset), a macroprocessor, a combination thereof, or a device generally for executing instructions. The computer-readable storage device or medium can include volatile and non-volatile storage devices such as read-only memory (ROM), random access memory (RAM), and keep-alive memory (KAM). KAM is a persistent or non-volatile memory that can be used to store various operating variables when the processor is powered off. The computer-readable storage device or medium can be implemented using a variety of storage devices, such as PROM (programmable read-only memory), EPROM (electrical PROM), EEPROM (electrically erasable PROM), flash memory, or another electrical, magnetic, optical, or combination storage device capable of storing data used by the vehicle controller 18 to control various systems of the vehicle 12, some of which represent executable instructions. The vehicle controller 18 may also be comprised of multiple controllers that are in electrical communication with one another. The vehicle controller 18 may be interconnected with additional systems and / or controllers of the vehicle 12 , allowing the vehicle controller 18 to access data such as the speed, acceleration, braking, and steering angle of the vehicle 12 .

[0038] The vehicle controller 18 is in electrical communication with the camera system 20, the autopilot system 22, and the vehicle communication system 24. In an exemplary embodiment, the electrical communication is established using, for example, a CAN network, a FLEXRAY network, a local area network (e.g., WiFi, Ethernet, etc.), a serial peripheral interface (SPI) network, etc. It should be understood that various additional wired and wireless technologies and communication protocols for communicating with the vehicle controller 18 are within the scope of the present disclosure.

[0039] The camera system 20 is used to capture images and / or video of the environment surrounding the vehicle 12. In an exemplary embodiment, the camera system 20 includes a photo and / or video camera positioned to view the environment surrounding the vehicle 12. In a non-limiting example, the camera system 20 includes a camera fixed to the interior of the vehicle 12 (e.g., fixed to the headliner of the vehicle 12) with a view through the windshield. In another non-limiting example, the camera system 20 includes a camera fixed to the exterior of the vehicle 12 (e.g., fixed to the roof of the vehicle 12) with a view of the environment in front of the vehicle 12.

[0040] In another exemplary embodiment, the camera system 20 is a surround view camera system that includes multiple cameras (also referred to as satellite cameras) arranged to provide views of the environment adjacent to all sides of the vehicle 12. In a non-limiting example, the camera system 20 includes a front-facing camera (e.g., mounted in the front grille of the vehicle 12), a rear-facing camera (e.g., mounted on the tailgate of the vehicle 12), and two side-facing cameras (e.g., mounted below each of the two side-view mirrors of the vehicle 12). In another non-limiting example, the camera system 20 also includes an additional rear-view camera mounted near the center high-mounted brake light of the vehicle 12.

[0041] It should be understood that camera systems with additional cameras and / or additional mounting locations are within the scope of the present disclosure. It should also be understood that cameras with various sensor types, including, for example, charge coupled device (CCD) sensors, complementary metal oxide semiconductor (CMOS) sensors, and / or high dynamic range (HDR) sensors, are within the scope of the present disclosure. In addition, cameras with various lens types, including, for example, wide angle lenses and / or narrow angle lenses, are also within the scope of the present disclosure.

[0042] The automated driving system 22 is used to provide assistance to the occupants to increase the occupants' awareness and / or control the behavior of the vehicle 12. Within the scope of the present disclosure, the automated driving system 22 encompasses systems that provide any level of assistance to the occupants (e.g., blind spot warnings, lane departure warnings, etc.) as well as systems that are capable of autonomously driving the vehicle 12 under some or all conditions. It should be understood that all levels of driving automation defined by, for example, SAE J3016 (i.e., SAE LEVEL 0, SAE LEVEL 1, SAE LEVEL 2, SAE LEVEL 3, SAE LEVEL 4, and SAE LEVEL 5) are within the scope of the present disclosure.

[0043] In an exemplary embodiment, the autopilot system 22 is configured to detect and / or receive information about the environment surrounding the vehicle 12 and process the information to provide assistance to the occupants. In some embodiments, the autopilot system 22 is a software module executed on the vehicle controller 18. In other embodiments, the autopilot system 22 includes a separate autopilot system controller similar to the vehicle controller 18 that is capable of processing information about the environment surrounding the vehicle 12. In an exemplary embodiment, the autopilot system 22 can operate in a manual operation mode, a partially automated operation mode, and / or a fully automated operation mode.

[0044] Within the scope of the present disclosure, manual operation mode refers to the autonomous driving system 22 providing warnings or notifications to the occupants, but not directly intervening or controlling the vehicle 12. In a non-limiting example, the autonomous driving system 22 receives information from a plurality of vehicle sensors (e.g., the camera system 20). Using techniques such as computer vision, the autonomous driving system 22 understands the environment around the vehicle 12 and provides assistance to the occupants. For example, if the autonomous driving system 22 identifies that the vehicle 12 may collide with a remote vehicle based on data from the plurality of vehicle sensors, the autonomous driving system 22 may provide a warning to the occupants.

[0045] Within the scope of the present disclosure, a partially automated mode of operation means that the autonomous driving system 22 provides warnings or notifications to the occupants and can directly intervene or control the vehicle 12 in certain circumstances. In a non-limiting example, the autonomous driving system 22 is additionally in electrical communication with components of the vehicle 12 (e.g., the braking system, propulsion system, and / or steering system of the vehicle 12) so that the autonomous driving system 22 can control the behavior of the vehicle 12. In a non-limiting example, the autonomous driving system 22 can control the behavior of the vehicle 12 by applying the brakes of the vehicle 12 to avoid an imminent collision. In another non-limiting example, the autonomous driving system 22 can control the braking system, propulsion system, and steering system of the vehicle 12 to temporarily drive the vehicle 12 toward a predetermined destination. However, occupant intervention may be required at any time. In an exemplary embodiment, the autonomous driving system 22 may include additional components, such as an eye tracking device configured to monitor the occupant's attention level and ensure that the occupant is ready to take over control of the vehicle 12.

[0046] Within the scope of the present disclosure, a fully automated mode of operation refers to the automated driving system 22 using data from multiple vehicle sensors to understand the environment and control the vehicle 12 to drive the vehicle 12 to a predetermined destination without requiring control or intervention by an occupant.

[0047] The automated driving system 22 operates using a path planning algorithm that is configured to generate a safe and efficient trajectory for the vehicle 12 to navigate the environment around the vehicle 12. In an exemplary embodiment, the path planning algorithm is a machine learning algorithm that is trained to output control signals for the vehicle 12 based on input data collected from a plurality of vehicle sensors. In another exemplary embodiment, the path planning algorithm is a deterministic algorithm that has been programmed to output control signals for the vehicle 12 based on data collected from a plurality of vehicle sensors.

[0048] In a non-limiting example, the path planning algorithm performs perception and mapping tasks to interpret data collected from multiple vehicle sensors and create a detailed map of the environment. The detailed map may include information about lane lines, road geometry, speed limits, traffic signs, and / or other relevant features. Based on the detailed map and the current state of the vehicle 12 (i.e., the position, speed, and orientation of the vehicle 12), the path planning algorithm generates a series of road points or a continuous path that the vehicle 12 should follow to reach the destination while complying with rules, regulations, and safety restrictions. It should be understood that the autonomous driving system 22 may include any software and / or hardware modules configured to operate in a manual operating mode, a partially automated operating mode, or a fully automated operating mode as described above.

[0049] The vehicle controller 18 uses the vehicle communication system 24 to communicate with other systems external to the vehicle 12. For example, the vehicle communication system 24 includes the ability to communicate with vehicle communications ("V2V" communications), infrastructure ("V2I" communications), remote systems at a remote call center (e.g., General Motors' ON-STAR), and / or personal devices. In general, the term vehicle-to-everything communications ("V2X" communications) refers to communications between the vehicle 12 and any remote system (e.g., vehicle, infrastructure, and / or remote system). In certain embodiments, the vehicle communication system 24 is a wireless communication system that is configured to communicate via a wireless local area network (WLAN) using the IEEE 802.11 standard or by using cellular data communications (e.g., using GSMA standards such as SGP.02, SGP.22, SGP.32, etc.). Therefore, the vehicle communication system 24 may also include an embedded universal integrated circuit card (eUICC) that is configured to store at least one cellular connection profile, such as an embedded subscriber identity module (eSIM) file. The vehicle communication system 24 is also configured to communicate via a personal area network (e.g., Bluetooth) and / or near field communication (NFC). However, additional or alternative communication methods such as dedicated short-range communication (DSRC) channels and / or mobile telecommunication protocols based on the third generation partnership project (3GPP) standard are also considered to be within the scope of the present disclosure. DSRC channels refer to unidirectional or bidirectional short-range to medium-range wireless communication channels designed specifically for automotive use and a set of corresponding protocols and standards. 3GPP refers to a partnership between multiple standard organizations that develop mobile telecommunication protocols and standards. The structure of the 3GPP standard is "version". Therefore, communication methods based on 3GPP versions 14, 15, 16 and / or future 3GPP versions are considered to be within the scope of the present disclosure. Therefore, the vehicle communication system 24 may include one or more antennas and / or communication transceivers for receiving and / or transmitting signals, such as collaborative sensing messages (CSM). The vehicle communication system 24 is configured to wirelessly transmit information between the vehicle 12 and another vehicle. In addition, the vehicle communication system 24 is configured to wirelessly transmit information between the vehicle 12 and infrastructure or other vehicles. It should be appreciated that the vehicle communication system 24 may be integrated with the vehicle controller 18 (eg, on the same circuit board as the vehicle controller 18 or otherwise integrated as part of the vehicle controller 18) without departing from the scope of the present disclosure.

[0050] Continue to refer to Figure 1, the server system 16 includes a server controller 26 that is in electrical communication with a graph database 28 and a server communication system 30. In a non-limiting example, the server system 16 is located in a server farm, a data center, etc., and is connected to the Internet using the server communication system 30. The server controller 26 includes at least one server processor 32 and a server non-transitory computer-readable storage device or server medium 34. The description of the types and configurations given above for the vehicle controller 18 also applies to the server controller 26. In some examples, the server controller 26 can be different from the vehicle controller 18 in that the server controller 26 can have a higher processing speed, include more memory, include more input / output, etc. In a non-limiting example, the server processor 32 and the server medium 34 of the server controller 26 are similar in structure and / or function to the processor and medium of the vehicle controller 18, as described above. The graph database 28 is used to store graph data about roads, including, for example, lane line graph data, as will be discussed in more detail below. The server communication system 30 is used to communicate with an external system (e.g., the vehicle controller 18) via the vehicle communication system 24. In a non-limiting example, the server communication system 30 is similar in structure and / or function to the vehicle communication system 24 of the vehicle system 14, as described above. In some examples, the server communication system 30 may differ from the vehicle communication system 24 in that the server communication system 30 is capable of higher power signal transmission, more sensitive signal reception, higher bandwidth transmission, additional transmission / reception protocols, etc.

[0051] Reference Figure 2 , a flow chart of a method 100 for merging lane lines of vehicles is shown. It should be understood that the method 100 can be performed by the vehicle system 14, the server system 16, or the entire system 10 (i.e., the vehicle system 14 and the server system 16 working together). Throughout this disclosure, references to method steps performed by the system 10 should be understood to refer to any combination of the vehicle system 14 and the server system 16. The method 100 begins at box 102 and proceeds to box 104. Figure 3 , showing a diagram of multiple lane line maps. Figure 3 Instructions included Figure 3 The meaning of the symbols in key 36.

[0052] Reference Figure 3 And continue to refer to Figure 2, at box 104, the system 10 receives a first lane line map 38a and a second lane line map 38b of a specific road segment 40. In a non-limiting example, the first lane line map 38a and the second lane line map 38b are crowdsourced from one or more vehicles 12. In another non-limiting example, the first lane line map 38a and / or the second lane line map 38b are provided by another source, such as aerial images, vehicle telemetry data, road survey data, etc. It should be understood that the system 10 and method 100 of the present disclosure are applicable to lane line maps obtained from any two or more sources, including but not limited to lane line maps crowdsourced from one or more vehicles 12. Within the scope of the present disclosure, lane line maps are used to describe the location, type and color of lane markings on the road. The first lane line map 38a and the second lane line map 38b include a plurality of points 42. Each point in the plurality of points 42 describes a lane line (i.e., a road marking). Each point in the plurality of points 42 includes a plurality of point characteristics. In a non-limiting example, the plurality of point characteristics include at least the geographic location of the point, the color of the lane line described by the point (e.g., yellow or white), the type of lane line described by the point (e.g., solid, dashed, and / or double), the vehicle's heading when observing the lane line described by the point, and the driving direction of the point. Within the scope of the present disclosure, the driving direction is the position of the point relative to the vehicle observing the point. The driving direction may be a left driving direction or a right driving direction. For example, a point describing a center lane line has a left driving direction because the point is observed on the left side of a vehicle traveling in both directions. For an edge lane line, the driving direction of the point depends on the vehicle's heading when observing the lane line. It should be understood that the lane line map may include additional elements, such as additional drawn markings, road edges (i.e., the boundary between a paved road surface and an unpaved road surface), virtual lane lines (i.e., lane lines inferred by a computer, which do not include drawn lane lines, such as within an intersection), and / or additional objects, markings, and characteristics of the road without departing from the scope of the present disclosure. Additionally, it should be appreciated that the system 10 and method 100 disclosed herein are applicable to additional elements of the lane map disclosed above in addition to lane markings.

[0053] It should be understood that although two lane line diagrams are discussed for the purpose of explanation, the system 10 and method 100 can also be used to merge any number of lane line diagrams within the scope of the present disclosure. In addition, for the sake of explanation, the first lane line diagram 38a and the second lane line diagram 38b shown herein include multiple points 42 describing a single lane line (e.g., a center lane line or an edge lane line). However, it should be understood that although the following explanation will focus on the example of a single lane line, the system 10 and method 100 disclosed herein are also applicable to points including descriptions of multiple lane lines (e.g., a center lane line, a left edge lane line, a right edge lane line, a shoulder lane line, a road edge line, etc.).

[0054] like Figure 3 As shown, the first lane line diagram 38a and the second lane line diagram 38b may be different. This may be caused by many factors, such as changes in global navigation satellite system (GNSS) accuracy, changes in sensor accuracy, changes in environmental conditions during data collection of the first lane line diagram 38a and / or the second lane line diagram 38b, occlusion of lane lines during data collection of the first lane line diagram 38a and / or the second lane line diagram 38b, etc.

[0055] In an exemplary embodiment, the server system 16 receives a first lane map 38a and a second lane map 38b from one or more vehicles 12 using a server communication system 30. In a non-limiting example, the first lane map 38a and / or the second lane map 38b are stored in a map database 28 of the server system 16. In another exemplary embodiment, the vehicle system 14 uses a camera system 20 to perform multiple measurements of the environment surrounding the vehicle 12 and calculate the first lane map 38a. The second lane map 38b is sent by the server system 16 or another remote vehicle and received by the vehicle system 14 using the vehicle communication system 24. It should be understood that, as described above, execution of the method 100 can occur on the vehicle system 14 using the vehicle controller 18 and / or the server system 16 using the server controller 26 without departing from the scope of the present disclosure. After frame 104, the method 100 proceeds to frame 106.

[0056] At block 106, the system 10 determines a road topology for the road segment 40. Within the scope of the present disclosure, the road topology includes information such as the geographic location of the road segment 40, the elevation of the road segment 40, the heading of the direction of travel on the road segment 40, the number of lanes on the road segment 40, the type of road surface (e.g., paved or unimproved) of the road segment 40, and the like. In an exemplary embodiment, the road topology is determined by receiving the topology data using the server communication system 30 and / or the vehicle communication system 24. In another exemplary embodiment, the topology data is stored in the graph database 28 and retrieved by the server controller 26. After block 106, the method 100 proceeds to block 108.

[0057] Refer again Figure 3 And continue to refer to Figure 2, at box 108, the system 10 determines a plurality of virtual side lines 44. Each of the plurality of virtual side lines 44 is perpendicular to the road segment 40. In an exemplary embodiment, a plurality of virtual side lines 44 are evenly spaced apart (e.g., at intervals of one meter) along the road segment 40. The term "virtual" is used because the plurality of virtual side lines 44 are not physical objects on the road segment 40. Instead, the system 10 and the method 100 use a plurality of virtual side lines 44 to represent a vertical direction relative to the road segment 40, as will be discussed in more detail below. In an exemplary embodiment, a plurality of virtual side lines 44 are determined based at least in part on the road topology determined at box 106. In a non-limiting example, a plurality of virtual side lines 44 are determined to be perpendicular to the heading of the direction of travel on the road segment 40 determined at box 106. After box 108, the method 100 proceeds to box 110.

[0058] At block 110, the system 10 determines a plurality of corresponding point pairs 46. Figure 4 , a flow chart of an exemplary embodiment 110a of block 110 is shown. Figure 5 , shows a diagram of a road segment 40 showing a plurality of corresponding point pairs 46 . Figure 5 Instructions included Figure 5 The meaning of the symbols in key 36. Figure 3 , Figure 4 and Figure 5 , the exemplary embodiment 110a of the frame 110 starts at frame 402. At frame 402, the system 10 selects one of the plurality of virtual side lines 44, hereinafter referred to as the first virtual side line 44a. After frame 402, the exemplary embodiment 110a proceeds to frame 404. At frame 404, the system 10 selects a subset of the plurality of points 42 of the first lane line map 38a. In the exemplary embodiment, each subset of the subset of the plurality of points 42 of the first lane line map 38a is within a circle 48 centered at the intersection 50 of the first virtual side line 44a and the road segment 40. In a non-limiting example, the circle 48 has a predetermined radius (e.g., three meters). After frame 404, the exemplary embodiment 110a proceeds to frame 406.

[0059] At box 406, the system 10 moves a subset of the plurality of points 42 of the first lane line diagram 38a selected at box 404 toward the plurality of points 42 of the second lane line diagram 38b. In an exemplary embodiment, an iterative closest point (ICP) algorithm is used to move the subset of the plurality of points 42 of the first lane line diagram 38a. The main goal of the ICP algorithm is to align two or more points by iteratively minimizing the difference between corresponding points. The ICP algorithm operates by establishing a correspondence between a plurality of source points (i.e., a plurality of points 42 of the second lane line diagram 38b) and a plurality of target points (i.e., a subset of the plurality of points 42 of the first lane line diagram 38a) based on a proximity measurement such as Euclidean distance. Then, the correspondence is used to estimate the transformation matrix that best aligns the plurality of source points and the plurality of target points. In a non-limiting example, the ICP algorithm is configured so that the transformation matrix only allows the movement of the plurality of target points (i.e., a subset of the plurality of points 42 of the first lane line diagram 38a) along the horizontal axis. The horizontal axis is the axis parallel to the first virtual lateral line 44a. The ICP algorithm iteratively refines the transformation by minimizing the total distance between corresponding points. The iterative process continues until convergence is achieved, which means optimal alignment between multiple source points and multiple target points. It should be understood that other methods and algorithms for aligning multiple points may be used without departing from the scope of the present disclosure. Figure 5 The corresponding point pair 46 is shown after being moved at block 406. After block 406, the exemplary embodiment 110a proceeds to blocks 408 and 410.

[0060] At block 408, the system 10 identifies a plurality of first intersection points 52a. Each of the plurality of first intersection points 52a is a point in the plurality of points 42 of the first lane line map 38a. Each of the plurality of first intersection points 52a intersects the first virtual side line 44a, such as Figure 5 As shown. Figure 5 In the exemplary embodiment shown, the plurality of first intersection points 52a includes only one point. However, it should be understood that the plurality of first intersection points 52a may include one or more points. For example, if the first lane line diagram 38a includes a plurality of lane lines, the plurality of first intersection points 52a may include at least one point for each lane line in the plurality of lane lines. After frame 408, the exemplary embodiment 110a proceeds to frame 412, as will be discussed in more detail below.

[0061] At block 410, the system 10 identifies a plurality of second intersection points 52b. Each of the plurality of second intersection points 52b is a point in the plurality of points 42 of the second lane line map 38b. Each of the plurality of second intersection points 52b intersects the first virtual side line 44a, such as Figure 5 As shown. Figure 5In the exemplary embodiment shown, the plurality of second intersection points 52b only include one point. However, it should be understood that the plurality of second intersection points 52b may include one or more points. For example, if the second lane line diagram 38b includes a plurality of lane lines, the plurality of second intersection points 52b may include at least one point for each lane line in the plurality of lane lines. After frame 410, the exemplary embodiment 110a proceeds to frame 412.

[0062] At box 412, the system 10 determines a point pair in the plurality of corresponding point pairs 46. In an exemplary embodiment, a point pair in the plurality of corresponding point pairs 46 includes a first point and a second point. The first point in the point pair in the plurality of corresponding point pairs 46 is an intersection point in the plurality of first intersection points 52a. The second point in the point pair in the plurality of corresponding point pairs 46 is an intersection point in the plurality of second intersection points 52b. In an exemplary embodiment, the distance between the first point in the point pair in the plurality of corresponding point pairs 46 and the second point in the point pair in the plurality of corresponding point pairs 46 is less than or equal to a predetermined corresponding point pair distance threshold (e.g., thirty centimeters). In another exemplary embodiment, the driving direction of the first point in the point pair in the plurality of corresponding point pairs 46 corresponds to the driving direction of the second point in the point pair in the plurality of corresponding point pairs 46. As described above, the driving direction is a point characteristic in the plurality of point characteristics. In a non-limiting example, the system 10 determines the driving direction of each intersection point in the plurality of first intersection points 52a and the driving direction of each intersection point in the plurality of second intersection points 52b. One of the plurality of corresponding point pairs 46 includes a first point of the plurality of first intersection points 52a having a left-traffic direction and a second point of the plurality of second intersection points 52b also having a left-traffic direction. After block 412 , the exemplary embodiment 110a proceeds to block 414 .

[0063] At block 414, if each of the plurality of virtual sidings 44 has not been selected at block 402, exemplary embodiment 110a returns to block 402. If each of the plurality of virtual sidings 44 has been selected at block 402, exemplary embodiment 110a ends and method 100 proceeds to block 112.

[0064] Reference Figure 6 , shows a diagram of a road segment 40 displaying a plurality of fusion points 54 . Figure 6 Instructions included Figure 6 The meaning of the symbols in key 36. Figure 2 and Figure 6 At box 112, the system 10 fuses each of the plurality of corresponding point pairs 46 to generate a plurality of fused points 54. To fuse each of the plurality of corresponding point pairs 46, the original first lane line map 38a and the second lane line map 38b are used (i.e., before the moving step during the exemplary embodiment 110a at box 406), as shown in FIG. Figure 6In an exemplary embodiment, each fusion point in the plurality of fusion points 54 is located between the first point and the second point of each point pair in the plurality of corresponding point pairs 46. In a non-limiting example, the plurality of fusion points 54 are located at a midpoint between the first point and the second point of each point pair in the plurality of corresponding point pairs 46, as shown in FIG. Figure 6 As shown. In another non-limiting example, the plurality of fused points 54 are positioned based on the confidence levels of the first lane line diagram 38a and the second lane line diagram 38b. For example, if the first lane line diagram 38a has a higher confidence level, the plurality of fused points 54 are located closer to the first lane line diagram 38a. After frame 112, the method 100 proceeds to frame 114.

[0065] Reference Figure 7 , a diagram of a fused lane line graph 56 is shown. Figure 7 Instructions included Figure 7 The meaning of the symbols in key 36. Figure 2 and Figure 7 , at box 114, the system 10 forms the fused lane map 56. In an exemplary embodiment, to form the fused lane map 56, the system 10 draws lane lines 58a, 58b, 58c between each of the plurality of fused points 54. The lane lines 58a, 58b, 58c are substantially parallel to the road segment 40. In the scope of the present disclosure, substantially parallel to the road segment 40 means that the lane lines 58a, 58b, 58c are oriented along the travel direction of the road segment 40. In other words, the lane lines 58a, 58b, 58c are substantially perpendicular to the plurality of virtual side lines 44 determined at box 108. Figure 7 An example with multiple lane markings 58a, 58b, 58c is shown. In a non-limiting example, the first lane marking 58a is a left edge lane marking, the second lane marking 58b is a center lane marking, and the third lane marking 58c is a right edge lane marking. After frame 114, the method 100 proceeds to frame 116.

[0066] At box 116, the fused lane map 56 formed at box 114 is transmitted to one or more vehicles 12. In an exemplary embodiment where the fused lane map 56 is formed using the server system 16, the server controller 26 transmits the fused lane map 56 to the one or more vehicles 12 using the server communication system 30. In an exemplary embodiment where the fused lane map 56 is formed using the vehicle system 14, the vehicle controller 18 transmits the fused lane map 56 to the one or more vehicles 12 and / or the server system 16 using the vehicle communication system 24. After box 116, the method 100 proceeds to box 118.

[0067] At box 118, the vehicle system 14 receives the fused lane map 56 transmitted at box 116 and adjusts the operation of the autonomous driving system 22 based at least in part on the fused lane map 56. In an exemplary embodiment, the path planning algorithm of the autonomous driving system 22 uses the fused lane map 56 to generate a trajectory for the vehicle 12. In another exemplary embodiment, the autonomous driving system 22 uses the fused lane map 56 to increase the accuracy of the lane departure warning feature and / or lane keeping feature of the autonomous driving system 22. After box 118, the method 100 proceeds to box 120, where it enters a standby state.

[0068] In an exemplary embodiment, the system 10 repeatedly exits the standby state 120 and restarts the method 100 at block 102. In a non-limiting example, the system 10 exits the standby state 120 and restarts the method 100 on a timer, such as once every three hundred milliseconds.

[0069] The system 10 and method 100 of the present disclosure provide multiple advantages. First, using the ICP algorithm discussed above with reference to box 406, points are moved only along a horizontal axis parallel to multiple virtual side lines. This prevents the introduction of point registration errors in the longitudinal direction. In addition, by iteratively moving along the road segment to fuse the points at each virtual side line in the multiple virtual side lines, the resulting fused lane line map is smooth between consecutive road segments. The use of the system 10 and method 100 allows different data about lane lines on the road to be fused into a single fused lane line map. The fused lane line map is then used by a vehicle system (e.g., an autonomous driving system) to improve the performance and / or accuracy of autonomous driving and / or driver assistance features.

[0070] The description of the present disclosure is merely exemplary in nature, and variations that do not depart from the gist of the present disclosure are intended to fall within the scope of the present disclosure. These variations should not be regarded as departing from the spirit and scope of the present disclosure.

Claims

1. A method for merging a lane line map, the method comprising: Determine the road topology of the road segment; identifying a plurality of fusion points based at least in part on the road topology and based at least in part on a first lane map for the road segment and a second lane map for the road segment; forming a fused lane map based at least in part on the plurality of fused points; and A first action is performed based at least in part on the fused lane map.

2. The method according to claim 1, wherein: Determining the road topology of the road segment further includes: retrieving topological data about the road segment; and A plurality of virtual sidings are determined based at least in part on topological data regarding the road segment, wherein each of the plurality of virtual sidings is perpendicular to the road segment, and wherein the plurality of virtual sidings are evenly spaced along a length of the road segment.

3. The method according to claim 2, wherein: Identifying the plurality of fusion points further comprises: determining a plurality of corresponding point pairs, wherein each of the plurality of corresponding point pairs comprises a first point and a second point, and wherein each of the plurality of corresponding point pairs corresponds to one virtual lateral line of the plurality of virtual lateral lines; and Each point pair in the plurality of corresponding point pairs is fused to form the plurality of fused points.

4. The method according to claim 3, wherein: Determining the plurality of corresponding point pairs further includes: selecting a first virtual siding line of the plurality of virtual siding lines; Based at least in part on a first virtual side line of the plurality of virtual side lines, moving a subset of the plurality of points of the first lane line map toward the plurality of points of the second lane line map; identifying a plurality of first intersection points, wherein each of the plurality of first intersection points is one of a plurality of points of the first lane line map, and wherein each of the plurality of first intersection points intersects a first virtual side line of the plurality of virtual side lines; identifying a plurality of second intersection points, wherein each of the plurality of second intersection points is one of a plurality of points of the second lane line map, and wherein each of the plurality of second intersection points intersects a first virtual side line of the plurality of virtual side lines; determining a point pair among the plurality of corresponding point pairs based at least in part on the plurality of first intersection points and the plurality of second intersection points, wherein the first point of the point pair among the plurality of corresponding point pairs is from the plurality of first intersection points, and the second point of the point pair among the plurality of corresponding point pairs is from the plurality of second intersection points, and wherein a distance between the first point and the second point is less than or equal to a predetermined corresponding point pair distance threshold; and The selecting, the moving, the identifying, and the determining are repeated until each of the plurality of virtual lateral lines is selected.

5. The method of claim 4, moving the subset of the plurality of points of the first lane line map further comprising: selecting the subset of the plurality of points of the first lane line map, wherein each of the subset of the plurality of points of the first lane line map is within a circle centered on one of the plurality of virtual side lines; and The subset of the plurality of points of the first lane line map is moved toward the plurality of points of the second lane line map using an iterative closest point algorithm, wherein the subset of the plurality of points of the first lane line map is moved only along a horizontal axis, and wherein the horizontal axis is parallel to one of the plurality of virtual side lines.

6. The method according to claim 4, wherein: Determining a point pair among the plurality of corresponding point pairs further includes: The first point and the second point are identified, wherein a driving direction of the first point corresponds to a driving direction of the second point.

7. The method according to claim 6, wherein: Identifying the first point and the second point further includes: Determine a driving direction of each of the plurality of first intersections, wherein the driving direction of each of the plurality of first intersections comprises one of a left driving direction and a right driving direction; determining a driving direction of each of the plurality of second intersections, wherein the driving direction of each of the plurality of second intersections comprises one of the left driving direction and the right driving direction; and The first point and the second point are identified, wherein the first point is one of a plurality of first intersection points having the left-traffic direction, and the second point is one of a plurality of second intersection points having the left-traffic direction.

8. The method according to claim 3, wherein: Fusion of each point pair in the plurality of corresponding point pairs further comprises: The plurality of fusion points are generated, wherein each fusion point in the plurality of fusion points is located between the first point and the second point of one of the plurality of corresponding point pairs.

9. The method according to claim 1, wherein: Forming the fused lane line map further includes: A lane line is drawn between each of the plurality of fusion points, wherein the lane line is substantially parallel to the road segment.

10. The method according to claim 1, wherein: Executing the first action also includes: Operation of the vehicle is adjusted based at least in part on the fused lane map.

Images