System and method for estimating lane boundary using slicing model

Abstract

To provide systems and methods for estimating lane boundaries using a slicing model with road data for generating maps.SOLUTION: A method includes: computing three-dimensional (3D) representations that are discretized from acquired data about road edges associated with driving lanes; deriving discrete and lateral slices of the road edges using a slicing model, the road edges being connected in a road graph that describes a mapped area; extracting features from the lateral slices individually using a neural model for forming a histogram to estimate lane boundaries about the driving lanes; and generating a map by linking the lane boundaries individually along the road edges.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The subject matter described in this specification generally relates to updating lane boundaries of a road, and more particularly to estimating lane boundaries using a slicing model together with road data to generate a map.

Background Art

[0002] A vehicle acquires data regarding a road from sensors in order to perform driving tasks. For example, the vehicle uses sensor data to facilitate recognition of other vehicles, obstacles, pedestrians, and additional aspects of the surrounding environment. In various implementations, the vehicle uses a light detection and ranging (LIDAR) sensor that emits light to scan the surrounding environment, while the logic associated with the LIDAR analyzes the acquired data to detect the presence of objects and other features of the surrounding environment. In further examples, additional / alternative sensors such as cameras acquire information regarding the surrounding environment, from which the system derives recognition regarding aspects of the surrounding environment. This sensor data can be useful in various situations to improve recognition of the surrounding environment so that a system such as an automated driving system (ADS) can recognize the indicated aspects and accurately plan and navigate accordingly.

[0003] Regarding navigation, the vehicle processes sensor data to update an outdated map. For example, the vehicle detects a sign indicating a speed reduction and infers a construction site by using GPS (Global Positioning System) data to identify that the map data is outdated. However, ADS performs complex tasks that rely on high-definition (HD) maps to ensure accuracy. In vehicles that use sensor data (such as images, GPS, etc.) to update the map, the accuracy may be insufficient for complex tasks. Thus, in one approach, the system deploys a dedicated vehicle to obtain high-level data about the road in order to update the map by manually annotating features (such as lane boundaries). The dedicated vehicle and manual assistance are costly and not effective for maintaining the HD map, which reduces the reliability of ADS operation and other driving tasks.

Summary of the Invention

Means for Solving the Problems

[0004] In one embodiment, an exemplary system and method relate to estimating lane boundaries using a slicing model along with road data to generate a map. In various implementations, systems for updating a map, particularly those involved with high-definition (HD) maps, increase in cost and complexity. For example, in a system for updating lane boundaries and lines changed by new construction and repair, image data (e.g., fleet data) obtained from vehicles is manually annotated. However, manual annotation of data is costly, inefficient, and causes a delay in real-time updates. Alternatively, systems that process sensor data within an automated mapping platform (AMP) can update an HD map, but lack the accuracy for complex driving tasks (e.g., lane tracking), thereby reducing safety. Thus, in one embodiment, an estimation system slices data obtained along a road edge so that a neural model can estimate lane boundaries by extracting features. Here, a road map defines the topology of an area having road edges that represent segments joined at nodes (e.g., intersections). The estimation system can derive horizontal slices using a slicing model that improves efficiency and scalability by discretizing data regarding the road edge. In one approach, the estimation system extracts features individually from the horizontal slices, thereby enhancing the accuracy and definition for updating lane boundaries.

[0005] In various implementations, the neural model includes a decoder that calculates a confidence value and a boundary placement of a lane boundary using a histogram of aggregated features. Further, the estimation system automatically generates a map by connecting and reconnecting lane boundaries along the road edge using the confidence value and the boundary placement. Thus, the estimation system processes the sliced data individually and connects the slices to generate a map with updated and more complete lane boundaries, thereby improving the accuracy and efficiency of map (e.g., HD map) generation.

[0006] In one embodiment, an estimation system is disclosed that estimates lane boundaries using a slicing model together with road data to generate a map. The estimation system includes a memory that stores instructions that, when executed by a processor, cause the processor to calculate a discretized 3D representation from acquired data regarding a road edge associated with a driving lane. The instructions also include instructions to derive discrete and lateral slices of the road edge using a slicing model, where the road edges are connected in a road graph that describes the mapped area. The instructions also include instructions to individually extract features from the lateral slices using a neural model to form a histogram for estimating lane boundaries regarding the driving lane. The instructions also include instructions to generate a map by individually connecting lane boundaries along the road edge.

[0007] In one embodiment, a non-transitory computer-readable medium is disclosed that estimates lane boundaries using a slicing model together with road data to generate a map and includes instructions that, when executed by a processor, cause the processor to perform one or more functions. The instructions include instructions to calculate a discretized 3D representation from acquired data regarding a road edge associated with a driving lane. The instructions also include instructions to derive discrete and lateral slices of the road edge using a slicing model, where the road edges are connected in a road graph that describes the mapped area. The instructions also include instructions to individually extract features from the lateral slices using a neural model to form a histogram for estimating lane boundaries regarding the driving lane. The instructions also include instructions to generate a map by individually connecting lane boundaries along the road edge.

[0008] In one embodiment, a method for estimating lane boundaries using a slicing model together with road data to generate a map is disclosed. In one embodiment, the method includes calculating a discretized 3D representation from acquired data regarding road edges associated with a driving lane. The method also includes deriving discrete and lateral slices of the road edge using the slicing model, where the road edges are connected in a road graph describing the mapped area. The method also includes extracting features individually from the lateral slices using a neural model to form a histogram for estimating lane boundaries regarding the driving lane. The method also includes generating a map by individually connecting the lane boundaries along the road edge.

Brief Description of the Drawings

[0009] The accompanying drawings, which are incorporated herein and constitute a part of this specification, illustrate various systems, methods, and other embodiments of the present disclosure. It will be understood that the boundaries of the elements shown in the figures (e.g., rectangles, groups of rectangles, or other shapes) represent one embodiment of the boundaries. In some embodiments, one element may be designed as multiple elements, or multiple elements may be designed as one element. In some embodiments, an element shown as an internal component of another element may be implemented as an external component, and vice versa. Further, the elements may not be drawn to the correct scale.

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

[0011] This specification discloses systems, methods, and other embodiments related to estimating lane boundaries using a slicing model along with road data to generate a map. In various implementations, systems for generating detailed maps encounter difficulties due to insufficient sensor information. For example, an autonomous driving system (ADS) relies on a detailed and accurate high-definition (HD) map to perform complex tasks such as lane tracking. Obscured lane boundaries and lane lines that are automatically updated by a map generation platform degrade the safety of ADS operation. As previously explained, manually annotating road data obtained from a dedicated map generation vehicle is economically costly and inefficient. Additionally, maps updated in this way may become outdated due to construction and lane changes due to road wear. Thus, generating an HD map with sufficient accuracy for ADS operation, including complex maneuvers such as lane tracking, incurs costs and delays, thereby reducing the reliability of the system. Accordingly, in one embodiment, an estimation system generates a map by separately extracting features from lateral slices and forming a histogram for predicting lane boundaries regarding a driving lane using a neural model. Prior to extraction, the estimation system uses a slicing model to derive lateral slices regarding discrete road edges for efficiency and scalability. In one approach, the lateral slices may be subsections of a road edge having a fixed dimension for efficient computation. The road edge represents a segment of a road map that defines the topology of an area. For example, discretization simplifies the longitudinal simultaneous localization and mapping (SLAM) calculations of the road by the estimation system and reduces the complexity of the system.

[0012] Furthermore, the histogram can aggregate and compress features with reduced dimensions through bins each associated with a horizontal slice, thereby improving efficiency. Regarding automatic annotation, the estimation system decodes the features and outputs the confidence values and boundary positions of the lane boundaries. This includes counting the compressed data within the bins to examine the relevance and correlation of the features. In one approach, the estimation system automatically recombines the lane boundaries individually along the road edge to generate a map, such as by merging adjacent horizontal slices with defined features. Thus, the present estimation system efficiently and accurately updates the lane boundaries automatically through discretization and aggregation of road data, thereby avoiding manual annotation for map generation.

[0013] In various implementations, the present estimation system generates a map by labeling features using an inverse distance relationship between the lane boundaries, confidence values, and boundary positions. Here, the estimation system can rely on the inverse distance for a simpler loss calculation (e.g., mean squared error loss) during training, thereby reducing the computational cost and system cost. As a further enhancement, the present estimation system selects features using a neural model by factoring the relationship between the distance between the compressed data within the bins and the lane boundaries. In this way, the present estimation system improves the definition of the lane boundaries and reduces the computational cost by slicing the road data that enables simpler geometric modeling and map generation.

[0014] Referring to FIG. 1, an example of a vehicle 100 is shown. As used herein, "vehicle" is any form of electric transportation means. In one or more implementations, the vehicle 100 is an automobile. Although the placement configuration is described herein with respect to automobiles, it will be understood that the embodiments are not limited to automobiles. In some implementations, the estimation system 170 benefits from the functionality discussed herein related to estimating lane boundaries using a slicing model along with road data to generate a map, and uses road-side units (RSUs), consumer electronics (CE), mobile devices, robots, drones, etc.

[0015] The vehicle 100 also includes various elements. It will be understood that in various embodiments, the vehicle 100 can have fewer elements than those shown in FIG. 1. The vehicle 100 can have any combination of the various elements shown in FIG. 1. Further, the vehicle 100 can have elements additional to those shown in FIG. 1. In some arrangements, the vehicle 100 can be implemented without having one or more of the elements shown in FIG. 1. In FIG. 1, various elements are shown as being disposed within the vehicle 100, but it will be understood that one or more of these elements can be disposed outside the vehicle 100. Further, the elements shown can be physically distant from each other.

[0016] Some of the possible elements of vehicle 100 are shown in FIG. 1 and will be described together with the subsequent figures. However, the description of many of the elements in FIG. 1 will be provided after considering FIGS. 2-5 for the purpose of making this description concise. Further, for the sake of simplicity and clarity of the figures, it will be understood that reference numerals are repeated between different figures to indicate corresponding or similar elements where appropriate. Further, a number of specific details are outlined to provide a complete understanding of the embodiments described herein. However, those skilled in the art will understand that the embodiments described herein can be implemented using various combinations of these elements. In any case, vehicle 100 includes an estimation system 170 implemented to perform the methods and other functions disclosed herein related to estimating lane boundaries using a slicing model together with road data to generate a map.

[0017] Referring to FIG. 2, an embodiment of the estimation system 170 of FIG. 1 is further shown. In this embodiment, the estimation system 170 is implemented in the vehicle 100 for online calculation, although the estimation system 170 can also be run offline on a server that generates a map. Online estimation is advantageous in vehicles 100 operating in autonomous mode and other applications that require high-resolution and accurate map data. Offline estimation has the advantage of improved accuracy and calculation speed because the calculation ability is improved compared to the vehicle 100.

[0018] The estimation system 170 is shown as including the processor 110 of the vehicle 100 in FIG. 1. Thus, the processor 110 may be part of the estimation system 170, the estimation system 170 may include a processor separate from the processor 110 of the vehicle 100, or the estimation system 170 may access the processor 110 via a data bus or another communication path. In one embodiment, the estimation system 170 includes a memory 210 that stores a detection module 220. The memory 210 is a random-access memory (RAM), read-only memory (ROM), hard disk drive, flash memory, or other suitable memory for storing the detection module 220. The detection module 220 is, for example, computer-readable instructions that cause the processor 110 to perform various functions disclosed herein when executed by the processor 110.

[0019] Referring to FIG. 2, the detection module 220 generally functions to include instructions that control the processor 110 to receive data input from one or more sensors of the vehicle 100. The input is, in one embodiment, the observation of one or more objects in the environment proximate to the vehicle 100 and / or other aspects regarding the surroundings. As shown herein, the detection module 220, in one embodiment, obtains sensor data 250 that includes at least camera images. In a further arrangement, the detection module 220 obtains sensor data 250 from additional sensors such as a radar sensor 123, a LIDAR sensor 124, and other sensors that may be suitable for identifying the vehicle and the position of the vehicle.

[0020] Accordingly, in one embodiment, the detection module 220 controls the respective sensors to provide data input in the form of sensor data 250. Further, although the detection module 220 is described as controlling various sensors to provide sensor data 250, in one or more embodiments, the detection module 220 can employ other techniques for obtaining sensor data 250, which can be either active or passive. For example, the detection module 220 can passively obtain sensor data 250 from a stream of electronic information provided to additional components within the vehicle 100 by various sensors. Further, the detection module 220 can implement various approaches for fusing data from multiple sensors when providing sensor data 250 and / or from sensor data obtained via a wireless communication link. Accordingly, in one embodiment, the sensor data 250 represents a combination of perceptual information obtained from multiple sensors.

[0021] Furthermore, in one embodiment, the estimation system 170 includes a data store 230. In one embodiment, the data store 230 is a database. The database is, in one embodiment, an electronic data structure stored in the memory 210 or another data store and is set by routines executable by the processor 110 for purposes such as analysis of the stored data, provision of the stored data, and organization of the stored data. Thus, in one embodiment, the data store 230 stores data used by the detection module 220 when performing various functions. In one embodiment, the data store 230 includes sensor data 250, for example, along with metadata characterizing various aspects of the sensor data 250. For example, the metadata can include position coordinates (e.g., longitude and latitude), relative map coordinates or tile identifiers, a time / date stamp since the individual sensor data 250 was generated, and the like. In one embodiment, the data store 230 further includes a road slice 240. The road slice 240 includes a lateral slice representing a sub-section of a road edge that can have a fixed dimension of road width (e.g., 30 meters (m)). Here, the road graph defines the topology and position context of the area using road edges that represent segments joined at nodes (e.g., intersections, breaks, etc.). However, the road graph may lack specific details regarding lanes (e.g., boundaries, lines, colors, etc.). As will be described below, the estimation system 170 can generate a map by expanding and shrinking the road longitudinally by merging lateral slices according to the extracted features in the histogram.

[0022] Next, referring to FIG. 3, an embodiment of an estimation system 170 is shown in which a neural model 300 slices data regarding a road edge to estimate a lane boundary. Here, the estimation system 170 and the detection module 220 can include instructions for the processor 110 to discretize the sensor data 250 into a 3D representation of the road edge from a road graph, lane lines, etc., similar to a sparse LiDAR representation. The road edge can be defined as a defined length on a road graph that topologically describes the mapped area. The lateral slice can be a constant distance longitudinally along the road graph. In this way, the estimation system 170 reduces the computational cost without sacrificing accuracy or detail when detecting lane boundaries through slicing.

[0023] Regarding the neural model 300, the lateral slice 302 has dimensions that define the amount of discrete information (e.g., width 300 units) to be processed by the neural model 300. In addition to the neural model 300, the estimation system 170 can also implement other data-driven models trained to extract and infer lane boundaries using the sliced data. The dimensions of the neural model 300 can factor the detection range of localities (e.g., lane boundaries with detected features of 2 m) within the sliced data. For example, the key points of the road detected by models such as heuristic and data-driven models are information that constitutes the lateral slice 302 for a road width of 30 meters (m). In the neural model 300, the detected key points have spatial positions and types, but may lack directionality regarding the objects for accurately inferring lane boundaries.

[0024] In various implementations, the horizontal slice 302 is an input having 7 channels that are processed by the encoder 304 into 16 channels using a one-dimensional (1d) convolution operation. The channels can be input layers for detecting one of, for example, the ego vehicle position, the left boundary of the ego vehicle, the right boundary of the ego vehicle, the left boundary of the next lane of the ego vehicle, the right boundary of the next lane of the ego vehicle, the left road boundary, the right road boundary, the boundary type (e.g., dashed line, solid line, color, etc.). The layer can also apply learned weights to the raw values of the learning model, and the amount of the weights is the product of the input channels, the output channels, and the kernel size. In this way, the estimation system 170 identifies the positions of the detection points existing across the relevant channels, infers the relationships, and connects the lane boundaries with high accuracy and detail.

[0025] As described above, slicing data has the advantage that segments can repeatedly solve problems that can be locally optimized. Slicing also improves the resolution of cuts or merges between inputs that affect decoding, and improves global inference as the detection accuracy between horizontal slices improves. In the case of Figure 3, neural model 300 can select channels from horizontal slices that have key points to expand the detection area and improve accuracy by applying various factorizations. For example, neural model 300 factorizes the offset of the detection channel (e.g., the distance from the vehicle) associated with slicing, past horizontal slices to fill gaps, adjacent horizontal slices to fill gaps, etc. Other factors for detection include data associated with the left line, right line, own lane, adjacent lane, road boundary, boundary type, lane offset, time, orientation, etc. In this way, encoder 304 aggregates data associated with channels and separates them into bins for feature extraction. In one approach, estimation system 170 forms a trace having a sequence of frames such that the frames have detections (e.g., key points) related to the group. In this way, encoder 304 can use the traces within the horizontal slices for each channel to identify the directionality of the lane using discrete frames, thereby improving the accuracy of the system.

[0026] In one approach, the convolution operation performs matrix multiplications and additions using the keypoints detected for each channel in order to adapt non-linearity and data compression while suppressing system cost. Here, matrix calculations factorize the kernel size and different channels for each layer in order to fill the gaps in the horizontal slices. Further, the neural model 300 can implement various amounts of channels and kernel sizes for each layer for different applications (e.g., lane tracking, night driving, etc.). For example, FIG. 3 shows an encoder 304 having a kernel size of 9 with padding 4 and a max pooling operation before another 1D convolution operation from 16 channels to 16 channels using a kernel size of 9 and padding 2. The kernel size can define a region having data for detecting prominent features. Max pooling quantifies the average presence of prominent features. Further, after performing max pooling, the encoder 304 extracts features regarding the lane boundary (LB) and the road boundary (RB) for decoding. In one approach, when showing similar features, the RB is also the lane boundary, or the RB is equal to the LB. Further, the encoder 304 can characterize by separately counting the data points of the bins of the horizontal slices (i.e., the detection points of a specific channel) that also exist across separate channels and forming a histogram accordingly. In this way, the encoder 304 factorizes the context within the horizontal slices according to the concentration of the data points across the channels in order to reduce the computational cost and complexity.

[0027] Furthermore, in one embodiment, decoder 306 or 308 performs a one-dimensional convolution process from 16 channels to 8 channels using a kernel size of 3 and padding of 1 for a specific application. Here, although neural model 300 shows implementation of two decoders, neural model 300 can also implement two-depth layers for outputting RB values and LB values. In neural model 300, a layer processes the output from 8 channels to 1 channel using a one-dimensional convolution process with a kernel size of 3 and padding of 1. In another process from 1 channel to 1 channel, a one-dimensional convolution process is used to output the confidence values and boundary arrangements of RB and LB for each horizontal slice. Such boundary positions use the inverse distance between RB / LB and the inferred features into which neural model 300 is assembled on the map. In particular, neural model 300 can use the inverse distance for a simpler loss calculation (e.g., mean squared error loss) during training, thereby reducing development costs.

[0028] Regarding map generation, the estimation system 170 can heuristically connect lane boundaries individually along the road edge using the confidence values and boundary positions output for each horizontal slice. This can include identifying the relationship between lane characteristics that satisfy the inverse distance threshold and the clarity of features along the road edge. For example, the two end horizontal slices are adjacent and have a dashed line with an increased confidence value compared to the central horizontal slice that includes missing paint. Thus, the estimation system 170 can reliably merge the horizontal slices using the dashed line over three horizontal slices if within the confidence and position thresholds.

[0029] Next, referring to FIG. 4, an example is shown of a slice image of a road and an estimation system 170 outputting estimated lane boundaries. The estimation system 170 forms a lateral slice 410 with respect to a road edge in a road graph that topologically describes a highway 420. Vision data 430 includes detected keypoints that the estimation system 170 uses to calculate a histogram of the lateral slice 410. In one approach, the estimation system 170 correlates the keypoints with GPS data to improve position accuracy. The neural model 300 can output positioned points (440) each having a confidence value for a potential lane line or boundary. The estimation system 170 can generate a map at the peak of the confidence value using the lateral line and lane boundaries. In this way, the estimation system 170 improves the definition of the lane boundaries and reduces the computational cost using the lateral slice 410 that enables simpler geometric modeling and map generation.

[0030] Next, referring to FIG. 5, a flowchart of a method 500 associated with estimating lane boundaries using a slicing model together with road data to generate a map is shown. The method 500 is described from the perspective of the estimation system 170 of FIGS. 1 and 2. Although the method 500 is described in combination with the estimation system 170, it should be understood that the method 500 is not limited to being implemented within the estimation system 170 and is an example of a system that can implement the method 500. Further, the estimation system 170 may operate online within the vehicle 100 or offline on a server. As described above, online estimation is advantageous for a vehicle 100 operating in an automatic mode that requires high-definition and accurate map data. Conversely, offline estimation has the advantage of improved accuracy and computational speed because the computational power is improved compared to the vehicle 100.

[0031] At 510, the detection module 220 calculates a discretized 3D representation of the acquired data regarding the road edge. The road graph defines a topology that includes the position context of the area represented by the road edge with segments joined by nodes (e.g., intersections, breaks, etc.). The road map may lack details regarding the lanes (e.g., boundaries, lines, colors, etc.). Here, the acquired data can include detection data associated with the left line, right line, own lane, adjacent lanes, road boundary, boundary type, lane offset, time, orientation, etc. Other detection data includes channel offsets associated with slicing (e.g., distance from the vehicle), past lateral slices for filling gaps, adjacent lateral slices for filling gaps, etc. As described above, such data can be aggregated and separated into bins according to the source channel, slice parameters, and application.

[0032] At 520, the estimation system 170 derives discrete lateral slices of the road edge using a slicing model. Here, the slicing model empirically forms lateral slices from the acquired data and information regarding the road edge. The lateral slices can represent subsections of the road edge that can have a fixed dimension of the road width (e.g., 30 meters (m)). As described above, the estimation system 170 performs scaling for generating a longitudinal map of the road by merging the lateral slices according to the extracted features in the histogram. For example, the estimation system 170 forms trace points within the lateral slices for each channel to identify the directionality of the lanes. In one approach, the estimation system 170 forms a trace having a sequence of frames such that the frames group related detections (e.g., keypoints). In one approach, the encoder 304 uses the trace points within the lateral slices for each channel to identify the directionality of the lanes using discrete frames, thereby improving the system accuracy. Through the lateral slices, the estimation system 170 resolves cuts or joins between the inputs that affect decoding, improves global inference as the detection accuracy between the lateral slices improves, and thereby improves the reliability of the system.

[0033] At 530, the estimation system 170 extracts features of the lateral slices using a neural model and a histogram to estimate the lane boundaries. In one approach, the estimation system 170 extracts features regarding LB and RB for decoding by separately counting the data points of the bins of the lateral slices that exist across separate channels. The counted data points can be organized in a histogram for further prediction. Thus, the estimation system 170 can factorize the context regarding the lateral slices according to the concentration of the data points to increase the computational efficiency.

[0034] Regarding decoding, the estimation system 170 can decode the features extracted by the encoding regarding the lane boundaries individually or separately. For example, a certain layer in the neural model processes the encoder output using an operation from 8 channels to 1 channel by 1D convolutional processing. In another processing from 1 channel to 1 channel, 1D convolutional processing is used to output the confidence values and boundary arrangements of RB and LB for each lateral slice. Such boundary positions factorize the inverse distance between RB / LB and the estimated features into which the neural model 300 is assembled in the map. As described above, the neural model 300 uses the inverse distance for a simpler loss calculation (e.g., mean squared error loss) during training, thereby reducing the system cost.

[0035] In 540, the estimation system 170 generates a map by connecting lane boundaries along the road edge. In one approach, the estimation system 170 uses heuristics to individually connect lane boundaries along the road edge. For example, the confidence values and boundary positions for each lateral slice derived from decoding are associated with potential lane boundaries. The estimation system 170 can identify the relationship between lane characteristics that satisfy the inverse distance threshold and the clarity of the features along the road edge. For example, two end lateral slices have a dashed line with a high confidence value with an adjacent central lateral slice and contain paint dropout. Therefore, the estimation system 170 can merge the lateral slices with high reliability using the dashed line spanning three lateral slices if within the confidence and position thresholds. Thus, the estimation system 170 individually processes the sliced data and by connecting the slices, generates a map with updated and more complete lane boundaries, thereby improving the accuracy and efficiency of generating a reliable map (e.g., HD map).

[0036] Here, with reference to FIG. 1, an exemplary environment in which the systems and methods disclosed herein may operate will be described in detail. In some examples, vehicle 100 is configured to selectively switch between different operation / control modes according to the direction of one or more modules / systems of vehicle 100. In one approach, the modes include the following, namely, 0, no automation, 1, driving assistance, 2, partial automation, 3, conditional automation, 4, high automation, and 5, full automation. In one or more configurations, vehicle 100 can be configured to operate in a subset of the possible modes.

[0037] In one or more embodiments, vehicle 100 is an autonomous vehicle or a self-driving vehicle. As used herein, a "self-driving vehicle" refers to a vehicle capable of operating in an autonomous mode (e.g., category 5, full automation). An "automation mode" or "autonomous mode" refers to using one or more computing systems to navigate and / or maneuver vehicle 100 along a travel route with minimal or no input from a human driver. In one or more embodiments, vehicle 100 is highly automated or fully automated. In one embodiment, vehicle 100 is configured to have one or more semi-autonomous driving modes, in which one or more computing systems perform part of the navigation and / or maneuvering of the vehicle along the travel route, and a vehicle operator (i.e., a driver) provides input to the vehicle to perform part of the navigation and / or maneuvering of vehicle 100 along the travel route.

[0038] Vehicle 100 can include one or more processors 110. In one or more arrangements, processor 110 can be the main processor of vehicle 100. For example, processor 110 can be an electronic control unit (ECU), an application-specific integrated circuit (ASIC), a microprocessor, etc. Vehicle 100 can include one or more data stores 115 for storing one or more types of data. Data store 115 can include volatile memory and / or non-volatile memory. Examples of suitable data stores 115 include RAM, flash memory, ROM, programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, magnetic disks, optical disks, and hard drives. Data store 115 can be a component of processor 110, or data store 115 can be operably connected to processor 110 such that it is used by processor 110. The term "operably connected" as used throughout this specification can include direct or indirect connections, including connections that do not involve direct physical contact.

[0039] In one or more arrangements, one or more data stores 115 can include map data 116. The map data 116 can include maps of one or more geographic regions. In some examples, the map data 116 can include information or data regarding roads, traffic control devices, road signs, structures, features, and / or landmarks within one or more geographic regions. The map data 116 can be in any suitable format. In some examples, the map data 116 can include aerial photographs of an area. In some examples, the map data 116 can include ground views of an area, including 360-degree ground views. The map data 116 can include measurements, dimensions, distances, and / or information regarding one or more items included in the map data 116 and / or with respect to other items included in the map data 116. The map data 116 can include a digital map having information regarding road shape.

[0040] In one or more arrangements, the map data 116 can include one or more topographic maps 117. The topographic maps 117 can include information regarding the terrain, roads, surfaces, and / or other features of one or more geographic regions. The topographic maps 117 can include elevation data for one or more geographic regions. The topographic maps 117 can define one or more ground surfaces, which can include paved roads, unpaved roads, land, and other things that define the ground surface.

[0041] In one or more arrangements, the map data 116 can include one or more static obstacle maps 118. The static obstacle map 118 can include information regarding one or more static obstacles located within one or more geographic regions. A "static obstacle" is a physical object whose position does not change or substantially change over a certain period of time and / or whose size does not change or substantially change over a certain period of time. Examples of static obstacles can include trees, buildings, curbs, fences, guardrails, medians, utility poles, statues, monuments, signs, benches, furniture, mailboxes, large rocks, or mounds. The static obstacle can be an object that extends above the ground level. The one or more static obstacles included in the static obstacle map 118 can have position data, size data, dimension data, material data, and / or other data associated therewith. The static obstacle map 118 can include measurements, dimensions, distances, and / or information of one or more static obstacles. The static obstacle map 118 can be of high quality and / or highly detailed. The static obstacle map 118 can be updated to reflect changes within the mapped area.

[0042] One or more data stores 115 can include sensor data 119. In this context, "sensor data" means any information regarding sensors provided in the vehicle 100 and includes capabilities and other information regarding such sensors. As will be described below, the vehicle 100 can include a sensor system 120. The sensor data 119 can be related to one or more sensors of the sensor system 120. As an example, in one or more arrangements, the sensor data 119 can include information regarding one or more LIDAR sensors 124 of the sensor system 120.

[0043] In some examples, at least a portion of the map data 116 and / or the sensor data 119 can be disposed in one or more data stores 115 mounted on the vehicle 100. Alternatively, or in addition, at least a portion of the map data 116 and / or the sensor data 119 can be disposed in one or more data stores 115 located remotely from the vehicle 100.

[0044] As described above, the vehicle 100 can include a sensor system 120. The sensor system 120 can include one or more sensors. A "sensor" means a device that can detect and / or sense something. In at least one embodiment, the one or more sensors detect and / or sense in real time. As used herein, the term "real time" means a level of processing responsiveness such that a user or system perceives that a particular process or decision is sufficiently immediate, or enables a processor to keep up with some external process.

[0045] In an arrangement where the sensor system 120 includes multiple sensors, the sensors may function independently, or two or more sensors may function in combination. The sensor system 120 and / or the one or more sensors can be operably connected to the processor 110, the data store 115, and / or another element of the vehicle 100. The sensor system 120 can generate observations regarding a portion of the environment of the vehicle 100 (e.g., nearby vehicles).

[0046] The sensor system 120 can include any suitable type of sensor. Various examples of different types of sensors are described herein. However, it will be understood that the embodiments are not limited to the specific sensors described. The sensor system 120 can include one or more vehicle sensors 121. The vehicle sensors 121 can detect information regarding the vehicle 100 itself. In one or more arrangements, the vehicle sensors 121 can be configured to detect changes in the position and orientation of the vehicle 100, for example, based on inertial acceleration. In one or more arrangements, the vehicle sensors 121 can include one or more accelerometers, one or more gyroscopes, an inertial measurement unit (IMU), a dead reckoning system, a global navigation satellite system (GNSS), a global positioning system (GPS), a navigation system 147, and / or other suitable sensors. The vehicle sensors 121 can be configured to detect one or more characteristics of the vehicle 100 and / or the manner in which the vehicle 100 is operating. In one or more arrangements, the vehicle sensors 121 can include a speedometer for determining the current speed of the vehicle 100.

[0047] Alternatively, or in addition, the sensor system 120 can include one or more environmental sensors 122 configured to obtain data regarding the environment surrounding the vehicle 100 in which the vehicle 100 is operating. "Surrounding environment data" includes data regarding the external environment in which the vehicle is located or one or more portions thereof. For example, one or more environmental sensors 122 can be configured to sense obstacles and / or data regarding such obstacles in at least a portion of the external environment of the vehicle 100. Such obstacles can be stationary objects and / or dynamic objects. One or more environmental sensors 122 can be configured to detect other things in the external environment of the vehicle 100, such as, for example, lane markers, signs, traffic lights, traffic signs, lane lines, crosswalks, curbs proximate to the vehicle 100, off-road objects, and the like.

[0048] In this specification, various examples of sensors of the sensor system 120 are described. These exemplary sensors can be part of one or more environmental sensors 122 and / or one or more vehicle sensors 121. However, it will be understood that embodiments are not limited to the specific sensors described.

[0049] As an example, in one or more arrangements, the sensor system 120 can include one or more of a radar sensor 123, a LIDAR sensor 124, a sonar sensor 125, a weather sensor, a tactile sensor, a position sensor, and / or one or more cameras 126. In one or more arrangements, one or more cameras 126 can be a high dynamic range (HDR) camera, a stereo camera, or an infrared (IR) camera.

[0050] Vehicle 100 can include an input system 130. An "input system" includes components or arrangements or groups thereof that enable various entities to input data into a machine. The input system 130 can receive inputs from the vehicle's occupants. Vehicle 100 can include an output system 135. An "output system" includes one or more components that facilitate presenting data to the vehicle's occupants.

[0051] Vehicle 100 can include one or more vehicle systems 140. Various examples of one or more vehicle systems 140 are shown in FIG. 1. However, vehicle 100 can include more, fewer, or different vehicle systems. Although specific vehicle systems are defined separately, it should be understood that any one or more of the systems or portions thereof can be combined or separated in other ways via hardware and / or software within vehicle 100. Vehicle 100 can include a propulsion system 141, a braking system 142, a steering system 143, a throttle system 144, a transmission system 145, a signal system 146, and / or a navigation system 147. Any of these systems can include one or more devices, components, and / or combinations thereof that are currently known or developed in the future.

[0052] The navigation system 147 can include one or more devices, applications, and / or combinations thereof that are currently known or developed in the future and are configured to determine the geographical location of vehicle 100 and / or to determine the driving route of vehicle 100. The navigation system 147 can include one or more map generation applications for determining the driving route of vehicle 100. The navigation system 147 can include a global positioning system, a local positioning system, or a geographical location information system.

[0053] Processor 110, estimation system 170, and / or autonomous driving module 160 can be operably connected to communicate with various vehicle systems 140 and / or their individual components. For example, returning to FIG. 1, processor 110 and / or autonomous driving module 160 can communicate to send and / or receive information from various vehicle systems 140 to control the movement of vehicle 100. Processor 110, estimation system 170, and / or autonomous driving module 160 can control some or all of vehicle system 140, and thus can be partially or fully autonomous, as defined by the Society of Automotive Engineers (SAE) levels 0-5.

[0054] Processor 110, estimation system 170, and / or autonomous driving module 160 can be operably connected to communicate with various vehicle systems 140 and / or their individual components. For example, returning to FIG. 1, processor 110, estimation system 170, and / or autonomous driving module 160 can communicate to send and / or receive information from various vehicle systems 140 to control the movement of vehicle 100. Processor 110, estimation system 170, and / or autonomous driving module 160 can control some or all of vehicle system 140.

[0055] Processor 110, estimation system 170, and / or the autonomous driving module 160 may be operable to control the navigation and operation of the vehicle 100 by controlling one or more of the vehicle system 140 and / or its components. For example, when operating in autonomous mode, processor 110, estimation system 170, and / or the autonomous driving module 160 can control the direction and / or speed of the vehicle 100. Processor 110, estimation system 170, and / or the autonomous driving module 160 can cause the vehicle 100 to accelerate, decelerate, and / or change direction. As used herein, "cause" or "causing" means to make an event or action occur, force, compel, direct, order, instruct, and / or enable, or at least put the state in which such an event or action can occur, in a direct or indirect way.

[0056] The vehicle 100 can include one or more actuators 150. The actuator 150 can be an element or combination of elements operable to change one or more of the vehicle system 140 or its components in response to receiving a signal or other input from the processor 110 and / or the autonomous driving module 160. For example, the one or more actuators 150 can include, by way of example only, motors, pneumatic actuators, hydraulic pistons, relays, solenoids, and / or piezoelectric actuators.

[0057] Vehicle 100 can include one or more modules, at least some of which are described herein. The modules can be implemented as computer-readable program code that, when executed by processor 110, performs one or more of the various processes described herein. One or more of the modules can be components of processor 110, or one or more of the modules can be executed on and / or distributed among other processing systems to which processor 110 is operably connected. The modules can include instructions (e.g., program logic) executable by one or more processors 110. Alternatively or in addition, one or more data stores 115 can include such instructions.

[0058] In one or more arrangements, one or more of the modules described herein can include artificial intelligence elements, such as neural networks, fuzzy logic, or other machine learning algorithms. Further, in one or more arrangements, one or more of the modules can be distributed among the plurality of modules described herein. In one or more arrangements, two or more of the modules described herein can be combined into a single module.

[0059] Vehicle 100 can include one or more autonomous driving modules 160. The autonomous driving module 160 can be configured to receive data from the sensor system 120 and / or any other type of system capable of capturing information related to vehicle 100 and / or the external environment of vehicle 100. In one or more arrangements, the autonomous driving module 160 can use such data to generate one or more driving scene models. The autonomous driving module 160 can determine the position and speed of vehicle 100. The autonomous driving module 160 can determine the positions of obstacles, such as traffic signs, trees, shrubs, neighboring vehicles, pedestrians, etc., or other environmental features.

[0060] The automatic driving module 160 is configured to receive, and / or determine, position information of obstacles in the external environment of the vehicle 100 so as to be used in creating a map or determining the position of the vehicle 100 with respect to map data, based on signals from a plurality of satellites, or any other data and / or signals that can be used to determine the current state of the vehicle 100 or determine the position of the vehicle 100 with respect to its environment, and to estimate the position and orientation of the vehicle 100, the vehicle position in global coordinates, by the processor 110, and / or one or more of the modules described herein.

[0061] The autonomous driving module 160 can be configured to determine a driving route, a current autonomous driving maneuver of the vehicle 100, a future autonomous driving maneuver, and / or a modification of the current autonomous driving maneuver, based on data acquired by the sensor system 120, the driving scene model, and / or data from any other suitable information source such as a determination from the sensor data 250, either independently or in combination with the estimation system 170. "Driving maneuver" means one or more actions that affect the movement of the vehicle. Examples of driving maneuvers include, among several possibilities, accelerating, decelerating, braking, turning, lateral movement of the vehicle 100, changing the driving lane, merging into a driving lane, and / or reversing. The autonomous driving module 160 can be configured to perform the determined driving maneuvers. The autonomous driving module 160 can cause such autonomous driving maneuvers to be performed, either directly or indirectly. As used herein, "cause" or "causing" means to make, order, direct, and / or enable an event or action to occur, either directly or indirectly, or to put in a state where at least such an event or action can occur. The autonomous driving module 160 can be configured to receive data, interact with, and / or control various vehicle functions and / or to transmit data to the vehicle 100 or one or more of its systems (e.g., one or more of the vehicle systems 140).

[0062] This specification discloses detailed embodiments. However, it should be understood that the disclosed embodiments are intended as examples. Accordingly, the specific structural and functional details disclosed herein are not to be construed as limiting, but rather as a representative basis for the claims and for teaching those skilled in the art to variously adopt the aspects herein in substantially any suitable detailed structure. Further, the terms and phrases used herein are not intended to be limiting, but rather to provide an understandable description of possible implementations. Although various embodiments are shown in FIGS. 1-5, the embodiments are not limited to the structures or applications shown.

[0063] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments. In this regard, a block in a flowchart or block diagram can represent a module, segment, or portion of code that includes one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may in fact be executed substantially concurrently, or depending on the functions involved, may sometimes be executed in the reverse order.

[0064] The above-described systems, components, and / or processes can be implemented in hardware, or in a combination of hardware and software, and can be implemented in a centralized manner in one processing system or in a distributed manner in which different elements span several interconnected processing systems. Any kind of processing system or another device adapted to implement the methods described herein is suitable. A typical combination of hardware and software can be a processing system having computer-usable program code that controls the processing system to implement the methods described herein when loaded and executed.

[0065] The systems, components, and / or processes can also be embedded in a computer-readable storage such as a machine-readable computer program product or other data program storage device that tangibly embodies a program of instructions executable by a machine for executing the methods and processes described herein. These elements can also be embedded in an application product that includes functions enabling the implementation of the methods described herein and that can implement these methods when loaded into a processing system.

[0066] Furthermore, the arrangement forms described in this specification can take the form of a computer program product embodied in one or more computer-readable media having, for example, stored computer-readable program code embodied therein. Any combination of one or more computer-readable media can be utilized. The computer-readable media may be a computer-readable signal medium or a computer-readable storage medium. The term "computer-readable storage medium" means a non-transitory storage medium. The computer-readable storage medium may be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer-readable storage media include portable computer diskettes, hard disk drives (HDD), solid-state drives (SSD), ROM, EPROM or flash memory, portable compact disc read-only memory (CD-ROM), digital versatile discs (DVD), optical storage devices, magnetic storage devices, or any suitable combination of the above. In the context of this document, a computer-readable storage medium may be any tangible medium that can store or retain a program for use by or in connection with an instruction execution system, apparatus, or device.

[0067] Generally, a module as used herein includes routines, programs, objects, components, data structures, etc. that perform a particular task or implement a particular data type. In a further aspect, generally, memory stores the aforementioned modules. The memory associated with a module may be a buffer or cache embedded within a processor, RAM, ROM, flash memory, or other suitable electronic storage medium. In a further aspect, the modules contemplated by the present disclosure are implemented as an ASIC, as a hardware component of a system on a chip (SoC), as a programmable logic array (PLA), or as another suitable hardware component in which a defined set of configurations (e.g., instructions) for performing the disclosed functions are embedded.

[0068] Program code embodied on a computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, fiber optic, cable, radio frequency (RF), etc., or any suitable combination of the foregoing. The computer program code for performing the operations of the aspects of the present disclosure's arrangement can be written in any combination of one or more programming languages, including object-oriented programming languages such as Java®, Smalltalk®, C++, or the like, and conventional procedural programming languages such as the "C" programming language or similar programming languages. The program code may be executed entirely on the user's computer, partially executed on the user's computer as a stand-alone software package, partially executed on the user's computer and partially executed on a remote computer, or executed entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or wide area network (WAN), or connected to an external computer (e.g., via the Internet using an Internet service provider).

[0069] As used herein, the terms "a" and "an" are defined as one or more. The term "plurality" as used herein is defined as two or more. The term "another" as used herein is defined as at least a second or more. The terms "comprising" and / or "having" as used herein are defined as including (i.e., open terms). The phrase "at least one of... and..." as used herein refers to and encompasses any and all combinations of one or more of the associated listed items. By way of example, the phrase "at least one of A, B, and C" includes A, B, C, or any combination thereof (e.g., AB, AC, BC, or ABC).

[0070] Aspects of the present specification can be embodied in other forms without departing from the spirit or essential attributes thereof. Accordingly, reference should be made to the following claims, rather than the foregoing specification, as indicating the scope of the present specification.

Claims

1. 1. An estimation system comprising: A processor; When executed by the processor, the processor computing a discretized three-dimensional (3D) representation from the acquired data relating to road edges associated with the travel lanes; deriving discrete, lateral slices of the road edges using a slicing model, the road edges being connected in a road graph that describes the mapped region; extracting features from the lateral slices individually using a neural model to form a histogram for estimating lane boundaries for the lane of travel; generating a map by individually connecting the lane boundaries along the road edges; A memory for storing instructions; An estimation system comprising:

2. 2. The estimation system of claim 1, further comprising instructions for outputting, by a decoder, confidence values ​​and boundary locations of the lane boundaries by counting compressed data within bins of the histogram, each of the bins being associated with one of the lateral slices, and the neural model comprising the decoder.

3. The estimation system of claim 2 , wherein the instructions for generating the map further comprise instructions for transforming the map using the confidence values ​​and the inverse distances from the boundary locations to the lane boundaries.

4. The instructions for extracting the features include: From the acquired data, process the detected key points of the driving lane using a matrix calculation for different channels, the matrix calculation factors a kernel size per layer of the neural model, and the different channels fill gaps in the lateral slices; fitting nonlinearities from the matrix calculation to estimate the lane boundaries; further comprising the instructions: The estimation system according to claim 1 .

5. 5. The estimation system of claim 4, wherein the layer of the neural model includes trace points of a vehicle in one or more of the lateral slices, the trace points identifying lane orientation.

6. The estimation system of claim 4 , further comprising instructions for using the neural model to select the features according to a distance between compressed data in bins of the histogram.

7. 2. The estimation system of claim 1, wherein the instructions for generating the map further comprise instructions for individually connecting the lane boundaries according to two or more of the lateral slices that are adjacent for at least one of the road edges, the two or more of the lateral slices satisfying a feature clarity of the road edges.

8. 2. The estimation system of claim 1, further comprising instructions for aggregating the features in the histogram, the features including one of adjacent slices associated with different channels from the acquired data, a confidence value, a distance between vehicles, a boundary type, a lane offset, a surface type, a time of day, and an orientation.

9. 2. The estimation system of claim 1, wherein the lateral slices are subsections of the road edge having fixed dimensions that are scalable in the longitudinal direction, and the information in the histogram varies according to the fixed dimensions and scalability.

10. A non-transitory computer-readable medium, comprising: When executed by a processor, the processor: computing a discretized three-dimensional (3D) representation from the acquired data relating to road edges associated with the travel lanes; deriving discrete, lateral slices of the road edges using a slicing model, the road edges being connected in a road graph that describes the mapped region; extracting features from the lateral slices individually using a neural model to form a histogram for estimating lane boundaries for the lane of travel; generating a map by individually connecting the lane boundaries along the road edges; A non-transitory computer-readable medium comprising instructions.

11. 11. The non-transitory computer-readable medium of claim 10, further comprising instructions for outputting, by a decoder, confidence values ​​and boundary locations of the lane boundaries by counting compressed data within bins of the histogram, the bins being each associated with one of the lateral slices, and the neural model comprising the decoder.

12. Computing a discretized three-dimensional (3D) representation from the acquired data of a road edge associated with the travel lane; deriving discrete, lateral slices of the road edges using a slicing model, the road edges being connected in a road graph that describes the mapped region; extracting features from the lateral slices individually using a neural model to form a histogram for estimating lane boundaries for the lane of travel; generating a map by individually connecting the lane boundaries along the road edges; A method comprising:

13. outputting, by a decoder, confidence values ​​and boundary locations of the lane boundaries by counting compressed data within bins of the histogram, each of the bins being associated with one of the lateral slices, the neural model including the decoder; The method of claim 12 further comprising:

14. The method of claim 13 , wherein generating the map further comprises transforming the map using the confidence values ​​and the inverse distances from the boundary locations to the lane boundaries.

15. Extracting the features comprises: From the acquired data, process the detected key points of the driving lane using a matrix calculation for different channels, the matrix calculation factors the kernel size per layer of the neural model, and the different channels fill gaps in the lateral slices; fitting nonlinearities from the matrix calculation to estimate the lane boundaries; and Further comprising: The method of claim 12.

16. The method of claim 15 , wherein the layer of the neural model includes trace points of a vehicle in one or more of the lateral slices, the trace points identifying lane orientation.

17. The method of claim 15 , further comprising: selecting the features using the neural model according to a distance between compressed data in bins of the histogram.

18. 13. The method of claim 12, wherein generating the map further comprises: individually connecting the lane boundaries according to two or more adjacent lateral slices for at least one of the road edges, the two or more lateral slices satisfying a feature clarity of the road edges.

19. 13. The method of claim 12, further comprising aggregating the features in the histogram, the features comprising one of adjacent slices associated with different channels from the acquired data, a confidence value, a distance between vehicles, a boundary type, a lane offset, a surface type, a time of day, and an orientation.

20. The method of claim 12 , wherein the lateral slices are subsections of the road edge having fixed dimensions that are longitudinally scalable, and the information in the histogram varies according to the fixed dimensions and scalability.

Images