SYSTEM AND METHOD FOR ESTIMATING THE ROAD INDEX

Abstract

The present invention provides a system (100) and a method (600) for lane index estimation in a single vehicle, which uses dynamic object data and free space data to estimate a single lane index. The system (100) is configured to process environmental attributes from sensors (102) mounted on the vehicle. These environmental attributes are fused to form a dynamic object fusion vector representing spatial and temporal relationships. Furthermore, a sequential learning model analyzes temporal dependencies and patterns in the dynamic object fusion vector to estimate the individual lane index. The model is trained using supervised and unsupervised learning to enable robust and accurate lane detection in real-time environments.In addition, the estimated individual lane index is dynamically updated as the vehicle moves, reflecting the lane the vehicle is traveling on.

Description

TECHNICAL AREA

[0001] The present invention relates to the field of autonomous and advanced driver assistance systems (ADAS) for vehicles. In particular, the present invention provides a system and method for accurately estimating the lane index, which enables a precise determination of a vehicle's lane position on roads under dynamic and real-time environmental conditions. BACKGROUND

[0002] Accurate lateral localization is essential for ensuring the safety and efficiency of autonomous driving and advanced driver assistance systems (ADAS). However, the absence or incomplete visibility of lane markings poses a significant challenge, as these markings are crucial for determining the vehicle's position within a lane. In such scenarios, while high-resolution maps can provide detailed information about lane markings, onboard sensors often fail to detect them or provide data with low accuracy. This discrepancy between map data and sensor input leads to a decrease in localization reliability and accuracy, ultimately compromising system reliability and increasing the risk of unsafe vehicle operation.

[0003] Existing lateral localization methods typically combine high-resolution maps with real-time data from sensors such as cameras, LiDAR, and radar. These systems use sensor fusion algorithms to merge multiple data sources and improve localization accuracy. High-resolution maps provide a static reference for road-level details, while sensors capture dynamic environmental information to adapt to real-time conditions. However, these methods are highly dependent on the availability and clarity of road markings, which in practice are often compromised or absent, for example, in bad weather, construction zones, or areas with faded road markings.

[0004] The reliance on sensor data brings further limitations. Cameras, for example, struggle in low-light conditions or when lane markings are obscured by rain, snow, or dirt. While LiDAR and radar offer some advantages, they often lack the necessary resolution to reliably detect subtle road features. The resulting discrepancies between map-based road information and sensor data lead to inconsistencies in the localization process. This problem is exacerbated by the probabilistic nature of most localization models, where unreliable or incomplete sensor input significantly reduces the level of confidence, which in turn leads to inaccuracies in determining the vehicle's lateral position.

[0005] Patent DE102021004525A1 discloses a method for virtual, object-modeled lane detection, in which tasks in the vicinity of a single vehicle are detected using multiple environmental sensors. The method comprises steps for vehicle detection by identifying their outlines and centers and determining virtual center-of-gravity trajectories for each sensor. These trajectories are combined to derive the path of the individual lane, assuming parallel lanes and uniform lane widths. Furthermore, the method assigns a quality measure to the virtual individual lane and the adjacent lanes, based on the sensor range, traffic density, and the motion characteristics of nearby tasks.However, the cited document does not address scenarios such as missing or only partially visible lane markings, where discrepancies between map data and sensor results can occur. Furthermore, it does not discuss improving lateral localization accuracy under conditions where sensor data is incomplete, unreliable, or unreliable.

[0006] Therefore, there is a need for a solution that overcomes these limitations and allows for a precise determination of a vehicle's position on the road, even when there are no clear lane markings. TASKS OF THE PRESENT INVENTION

[0007] A general object of the present invention is to provide a system and a method for improving the accuracy of roadway estimation by effectively combining free space and dynamic object data.

[0008] One object of the present invention is to enable robust lane detection even when there are no clear lane markings or dynamic objects in the environment.

[0009] Another objective of the present invention is to improve occlusion performance by using fused representations of dynamic tasks and free space.

[0010] Another object of the present invention is to enable adaptability to different traffic conditions by dynamically updating the road assessment in real time.

[0011] Another object of the present invention is to reduce the error rates in roadway estimation using multi-head cross-attention and sequential learning models.

[0012] Another objective of the present invention is to ensure consistent performance in roadway detection, even in unfavorable scenarios such as obstructions or missing sensor data.

[0013] Another objective of the present invention is to enable efficient end-to-end training for the simultaneous optimization of multi-head attention and sequential learning networks.

[0014] Another object of the present invention is to provide a scalable solution for lane detection in single- and multi-lane road scenarios. SUMMARY

[0015] Aspects of the present invention relate to the field of autonomous and advanced driver assistance systems (ADAS) for vehicles. In particular, the present invention provides a system and method for accurately estimating the lane index, which enables a precise determination of the lane position of an individual vehicle on roads under dynamic and real-time environmental conditions.

[0016] One aspect of the present invention relates to a system for estimating the road index in a single vehicle, comprising a controller and a memory operationally connected to the processor. The memory stores instructions which, when executed, cause the control unit to receive environmental attributes of a road from sensors mounted on the individual vehicle. The environmental attributes include dynamic task and free-space data indicating drivable areas around the individual vehicle. Furthermore, the system fuses the received dynamic object data and free-space data to generate a dynamic object fusion vector that represents spatial and temporal relationships between the individual vehicle, the dynamic tasks, and the free space, in order to improve the reliability of road detection even in complex driving scenarios.Furthermore, the control unit extracts temporal dependencies and patterns from the dynamic object fusion vector using a sequential learning model. Additionally, the control unit estimates an individual lane index based on these extracted temporal dependencies and patterns, as well as the spatial and temporal relationships between the individual vehicle and its environment. This estimated individual lane index indicates the lane on which the individual vehicle is traveling.

[0017] In some embodiments, the sequential learning model used in the system is trained with a combination of unsupervised and supervised learning techniques to ensure improved accuracy in spatial pattern recognition and lane index prediction, even in difficult or ambiguous situations.

[0018] In some embodiments, the control unit dynamically updates the individual lane index as the individual vehicle moves along the lane by taking into account real-time changes in dynamic tasks, clearance boundaries, and other environmental attributes to ensure consistent and accurate localization during travel.

[0019] In some embodiments, the system is also configured to divide the roadway into multi-lane or single-lane conditions, thereby improving the applicability of the system in different driving scenarios.

[0020] Another aspect of the present invention relates to a method for road index estimation implemented in a single vehicle, as implemented by the system.

[0021] Various tasks, features, aspects and advantages of the subject matter according to the invention will become clearer from the following detailed description of preferred embodiments together with the accompanying drawing figures, in which the same numbers represent the same components. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings serve to further understand the present invention and are an integral part of this description. The drawings illustrate exemplary embodiments of the present invention and, together with the description, serve to explain the principles of the present invention. Fig. Figure 1 shows an exemplary network architecture of the proposed system for road index estimation for a vehicle to illustrate its general operation according to an embodiment of the present invention. Fig. Figure 2 shows exemplary functional units of a control unit connected to the proposed road index estimation system for a vehicle, according to an exemplary embodiment of the present invention. Fig. Figure 3 shows a flowchart of an exemplary process for estimating a single lane index using the proposed system, in accordance with an embodiment of the present invention. Fig. Figure 4 shows a flowchart of an exemplary method for the vectorized representation of free spaces and dynamic tasks according to an embodiment of the present invention. Fig. Figure 5 shows a flowchart illustrating the steps for estimating the individual lane index using the vectorized representation of free space and the dynamic task, in accordance with an embodiment of the present invention. Fig. Figure 6 is a flowchart illustrating the proposed method for estimating the road index for a vehicle in accordance with an embodiment of the present invention. DETAILED DESCRIPTION

[0023] A detailed description of embodiments of the invention, illustrated in the accompanying drawings, follows. The embodiments are described in sufficient detail to clearly convey the invention. However, this level of detail is not intended to limit foreseeable variations of embodiments; on the contrary, it is intended to cover all modifications, equivalents, and alternatives that fall within the scope of the present invention as defined by the accompanying claims.

[0024] The embodiments described here relate to the field of autonomous and advanced driver assistance systems (ADAS) for vehicles. In particular, the present invention provides a system and a method for accurately estimating the lane index, which enables a precise determination of the lane position of an individual vehicle on roads under dynamic and real-time environmental conditions.

[0025] Various embodiments of the present invention are described in detail with reference to the Fig. 1-6.

[0026] With reference to Fig. Figure 1 discloses an exemplary network architecture of the proposed lane index estimation system for a vehicle. The system 100 comprises one or more sensors 102 and a control unit 104. The sensors 102 are mounted on the individual vehicle and may include a combination of LiDAR, radar, cameras, ultrasonic sensors, or other advanced sensor technologies. These sensors 102 are configured to detect one or more environmental attributes of a lane on which the individual vehicle is traveling. Environmental attributes include dynamic object data and free space data indicating drivable areas around the individual vehicle, as well as lane marking data, if available.

[0027] In some embodiments, the control unit 104 is configured to receive environmental attributes from the sensors 102 and fuse the dynamic task data with the free space data to generate a dynamic task fusion vector. This fusion vector encapsulates the spatial and temporal relationships between the individual vehicle, the surrounding dynamic tasks, and the identified free space, providing a unified representation of the vehicle's environment for further analysis.

[0028] In some embodiments, the control unit 104 is further configured to recognize one or more dynamic tasks from the received dynamic object data by using spatial attributes such as position and distance, and to group the recognized objects into a first set of clusters based on their spatial relationships and proximity to one another. The control unit evaluates the mean of each cluster in the first group of clusters to determine the central position of the respective clusters and calculates the distances of the recognized tasks relative to the mean of their corresponding clusters. These distances are correlated with the evaluated mean values ​​of the clusters in the first group of clusters to refine the spatial relationships and ensure the correct alignment of the tasks within their clusters.To improve accuracy, the control unit 104 adds a second set of clusters for each dynamic task located within a predefined region of interest (ROI) around the individual vehicle. Noise and irrelevant clusters are then filtered out from both the first and second sets of clusters by removing clusters that are not associated with any dynamic tasks or that are located in longitudinal directions deemed irrelevant. Furthermore, the control unit 104 embeds the mean and variance of the remaining clusters from both sets into the dynamic object fusion vector, resulting in a statistically enriched representation of the spatial distribution and variability of the clustered environment.

[0029] In some embodiments, the control unit 104 is further configured to extract temporal dependencies and patterns from the task fusion dynamic vector using a sequential learning model. Based on these extracted patterns and the spatial and temporal relationships between the individual vehicle and its environment, the control unit 104 estimates a single lane index. The control unit 104 then refines the estimated single lane index to improve accuracy and ensure that the single lane index reliably indicates the specific lane in which the individual vehicle is traveling. Additionally, the control unit 104 refines the single lane index with respect to the map by comparing the estimated lane with the map data to account for static lane features such as lane curvatures, exits, and junctions.This map-based refinement increases the context-related reliability of the individual lane index, particularly in complex traffic scenarios. Furthermore, the control unit 104 is configured to refine the estimated individual lane index based on lane marking data received from the sensors 102. This multi-source refinement process ensures robust and accurate lane detection under real-world conditions.

[0030] Furthermore, the control unit 104 is configured to dynamically update the individual lane index as the individual vehicle moves on the lane by taking into account real-time changes in dynamic tasks, clearance limits and other environmental attributes to ensure consistent and accurate localization while driving.

[0031] According to another embodiment, the system 100 can include a display device 106 that can be operationally coupled to the control unit 104. The display device 106 is configured to visually represent relevant information such as the estimated road index, the positions of surrounding vehicles, and road attributes, providing real-time feedback to the driver or the vehicle's automated system. For example, this display device 106 can include visualizations of the individual vehicle's road position, surrounding dynamic tasks, and road conditions, thus supporting both the driver's attention and the vehicle navigation system's decisions. In an exemplary embodiment, the display device 106 can be configured as a dashboard, an LED display panel, an LCD module, or a GUI module integrated into the individual vehicle.In another exemplary embodiment, the system 100 can be operationally coupled to an external device such as a laptop, a smartphone, a tablet or another mobile computing device, which itself can function as a display device 106.

[0032] In one embodiment, the control unit 104 can communicate with the sensors 104 and the display unit 106 via a network 110. Furthermore, the network 110 can be a wireless network, a wired network, or a combination thereof, and can be implemented as one of various network types, such as an intranet, local area network (LAN), wide area network (WAN), the internet, and the like. In addition, the network 110 can be either a dedicated network or a shared network. The shared network can represent a connection of different types of networks that can use a variety of protocols, such as Hypertext Transfer Protocol (HTTP), Transmission Control Protocol / Internet Protocol (TCP / IP), Wireless Application Protocol (WAP), and the like.

[0033] In one embodiment, the control unit 104 can be implemented by any or a combination of hardware and software components, such as a cloud, a server 112, a computer system, a computer device, a network device, and the like. Furthermore, the control unit 104 can interact with the sensors 102 and the display device 106 via a website or application that may reside within the proposed system 100. In one implementation, the proposed system 100 can be accessed via a website or application that can be configured with any operating system, including, but not limited to, Android™, iOS™, and the like.

[0034] With reference to Fig. Section 2 discloses exemplary functional units of the control unit 104, which is connected to the proposed road index estimation system 100 for the vehicle. The control unit 104 comprises one or more processors 202. The one or more processors 202 can be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, logic circuits, and / or any devices that process data based on operating instructions. Among other capabilities, the one or more processors 202 can be configured to retrieve and execute computer-readable instructions stored in a memory 204. The memory 204 can store one or more computer-readable instructions or routines that can be retrieved and executed to create or share the data units via a network service.The memory 204 can include any non-volatile memory device, e.g., volatile memory such as Random Access Memory (RAM) or non-volatile memory such as Erasable Programmable Read-Only Memory (EPROM), Flash memory, and the like.

[0035] In one embodiment, the control unit 104 may also include one or more interfaces 206. The interface(s) 206 may include a variety of interfaces, such as interfaces for data input and output devices, referred to as input / output devices (I / O devices), storage devices, and the like. The interface(s) 206 may provide a communication path for one or more components of the vehicle. Examples of such components include the processing machine(s) 208 and a database 210.

[0036] In one embodiment, the processing machine(s) 208 can be implemented as a combination of hardware and programming (e.g., programmable instructions) to implement one or more functions of the processing machine(s) 208. In other embodiments, the processing module(s) 208 can be implemented by electronic circuits. The database 210 can contain data that is either stored or generated as a result of functions implemented by one of the components of the processing machine(s) 208. In some embodiments, the processing module(s) 208 can include an environment data acquisition module 212, a data fusion module 214, a sequential learning module 216, a single lane index estimation module 218, and one or more other modules 220. The other module(s) 220 can implement functionalities that complement the applications / functions performed by the system.

[0037] In one embodiment, the environmental data acquisition module 212 is configured to acquire essential roadway attributes from sensors 102 mounted on the individual vehicle, including LiDAR, radar, and cameras. These sensors 102 provide dynamic object data and free-space data representing moving objects and drivable areas around the individual vehicle. Furthermore, the environmental data acquisition module 212 is configured to classify the roadway by determining whether it is a multi-lane or single-lane road. This classification is based on data received from the sensors 102, which capture lane markings, vehicle positions, and other roadway features.

[0038] In one embodiment, the data fusion module 214 is configured to process the collected sensor data and merge dynamic object data with free-space information into a unified representation called a dynamic object fusion vector. This includes clustering data to effectively organize and summarize spatial relationships. The dynamic objects detected from the dynamic object data, such as vehicles, are grouped based on their spatial proximity and relationships, forming an initial set of clusters representing distinct groups of detected objects. These clusters are formed based on spatial proximity and relationships between the objects to ensure that tasks located close to one another are grouped together.For each cluster in the first group of clusters, the data fusion module 214 calculates a mean to represent its central position and evaluates the distances of the tasks within the cluster relative to this mean. These distances are then correlated with the cluster means to refine the spatial relationships between the tasks and their corresponding clusters.

[0039] Additionally, the Data Fusion Module 214 identifies the dynamic tasks within a predefined region of interest (ROI) around the individual vehicle and creates a second set of clusters to represent these tasks. These additional clusters improve spatial resolution in critical areas near the individual vehicle, enabling a more detailed understanding of its immediate surroundings. Furthermore, the Data Fusion Module 214 filters out noise and irrelevant clusters from both the first and second sets by removing clusters that either have no associated dynamic tasks or that are located in longitudinal directions irrelevant to the individual vehicle's navigation.Furthermore, the data fusion module 214 calculates the statistical properties of the remaining clusters, including their mean and variance, and embeds these metrics in the dynamic object fusion vector. This results in a comprehensive representation that captures both the spatial distribution of the clusters and the variability within the environment, enabling the individual vehicle to effectively interpret its surroundings and make accurate decisions in real time.

[0040] In one embodiment, the sequential learning module 216 is configured to analyze the dynamic vector for task fusion to extract temporal dependencies and patterns using the sequential learning model. The sequential learning model is trained using a combination of unsupervised learning to identify spatial patterns in the fused data and supervised learning to refine lane index predictions based on labeled training data. By employing sequential learning techniques such as recurrent neural networks (RNNs) or their variants, such as long short-term memory (LSTM) or gated recurrent units (GRUs), the sequential learning module 216 captures the evolution of the environment over time, enabling the system to adapt to dynamic changes and improve the accuracy of lane index predictions.

[0041] In one embodiment, module 218 is configured to estimate the individual lane index based on the processed outputs of sequential learning module 216. Using the extracted spatial and temporal relationships, this module calculates the most probable lane on which the individual vehicle is traveling. The estimation process considers both dynamic factors, such as the trajectories of nearby vehicles, and static factors, including lane geometry and boundaries.

[0042] To increase accuracy and reliability, Module 218 for estimating the individual lane index includes additional post-processing techniques, such as confidence assessment, adjustments based on identified inconsistencies, and comparison with known lane restrictions. This validation and refinement ensures that the calculated individual lane index closely matches real-world conditions and provides a robust representation of the specific lane occupied by the individual vehicle.

[0043] In Fig. Figure 3 shows a flowchart 300 illustrating an exemplary procedure for estimating the individual lane index using the proposed system 100. The process begins with the acquisition of data from various sensors. In block 302, data from a LiDAR sensor is acquired, and in block 304, data from a radar sensor is acquired. Using this data, the corresponding free space is determined in block 308. Simultaneously, camera perception data is acquired in block 306 and further used in block 310 to identify dynamic tasks. Once the sensor data is acquired, early fusion and vectorization are performed in block 312 to represent the free space and dynamic tasks in a unified format. In block 314, a multi-head cross-selection method is applied to improve data processing.Subsequently, in Block 316, one or more sequential models, such as cascaded LSTMs and GRUs, are used to analyze temporal and spatial dependencies in the fused data. Furthermore, in Block 318, the system estimates the individual lane index, which specifies the particular lane on which each vehicle is traveling. System 100 integrates various sensor inputs and advanced modeling techniques to deliver accurate lane index predictions.

[0044] Fig. Figure 4 shows a flowchart 400 illustrating an exemplary process for the vectorized representation of free spaces and dynamic tasks using the proposed system 100. The process begins with the collection of data from LiDAR in block 302 or from radar in block 304, as shown in Fig. Figure 3 shows this data. This data is used by control unit 104 in block 402 to determine the distance to the boundaries of the drivable area. In block 404, the K-value is calculated based on the number of dynamic vehicles detected in front of each vehicle, along with the clusters identified to the left and right, which act as a distortion factor. For example, if three dynamic vehicles are detected in front of the individual vehicle and two clusters are formed on either side due to proximity and spatial orientation, the K-value integrates these counts to dynamically influence the clustering thresholds. These clusters group tasks based on proximity and movement patterns.

[0045] For example, vehicles driving close together are grouped based on their relative positions and trajectories. This calculated K-value aids in clustering and processing the identified tasks and free-space data, enabling precise visualization and analysis. By refining the clustering process, the system provides a detailed and actionable understanding of the environment and improves the reliability of subsequent lane estimation and navigation tasks.

[0046] At the same time, the system in block 310 identifies dynamic tasks based on detections by sensors such as LiDAR, radar or cameras, as in Fig. Figure 3 illustrates this process. In block 416, the control unit 104 calculates the number of dynamic tasks and ensures that one object is identified per lane. This calculated data is then forwarded to block 404 for correlation with the cluster information. In block 406, the system correlates the dynamic vehicle distances with the mean of the identified clusters. This establishes a relationship between the spatial distribution of the clusters and the detected dynamic vehicles, ensuring that the clustering process accurately represents the positions and movements of these vehicles within their environment. In block 408, the system adds additional clusters for each vehicle within its region of interest (ROI), further improving the representation of each vehicle's surroundings.In block 410, noise and irrelevant clusters, especially those in the longitudinal direction without dynamic vehicle information, are filtered out to improve data clarity. Subsequently, in block 412, the mean and variance of the remaining clusters are embedded to quantify their distribution and variability. Furthermore, in block 414, a vectorized representation of the free space is generated to obtain a structured understanding of the drivable areas around the individual vehicle.

[0047] In parallel, control unit 104 determines the positions of the dynamic vehicles detected in the vicinity in block 418. In block 420, control unit 104 calculates the distances of these dynamic vehicles from the individual vehicle and then proceeds to blocks 406 and 422. These distances are stacked from left to right relative to the individual vehicle in block 422 to establish an ordered spatial relationship. In block 424, the distances are encoded, and the detected tasks are assigned to the respective lanes. In block 426, the system generates a vectorized representation of the dynamic tasks by integrating the spatial and positional information into a format ready for further processing.

[0048] With reference to Fig. Figure 5 illustrates a flowchart 500 illustrating an exemplary process for the vectorized representation for estimating the individual lane index using the vectorized representation of free space and the dynamic task according to an embodiment of the present invention. This process begins with the receipt of inputs from the vectorized representation of free space, which is shown in block 414 in Fig. 4 is generated, and the vectorized representation of the dynamic tasks that are in block 426 in Fig. Block 4 is generated. These inputs provide structured data that describe in detail the drivable areas and the dynamic tasks around the individual vehicle. From the received vectorized representations, specific attributes are extracted in Block 314 using a multi-head cross-attention mechanism. In Block 314, the system extracts a value in Block 502, while a key is derived in Block 504. These attributes are calculated from the vectorized data in Block 414. Simultaneously, a query is extracted in Block 506 from the additional inputs received in Block 426. A score is also calculated in Block 510. This score is derived from the value extracted in Block 502 and the result of a scaled dot product operation performed in Block 508. The scaled dot product is calculated using the key and query extracted in Blocks 504 and 506, respectively.The calculated score helps in assessing the spatial and temporal relationships between the free space and the dynamic tasks.

[0049] In Block 512, a fully linked layer receives the score from the Multi-Head Attention Mechanism 314. In Block 514, a softmax function is applied to the output of Block 512 to normalize the scores and convert them into probabilities. These probabilities are then used in Block 516 to create a dynamic object lane fusion vector, which produces an output representing the individual lane index. Similarly, in Block 518, a linear transformation is applied to the output of Block 512 to refine the data, followed by Block 520, which calculates the number of lanes to provide another output.

[0050] In block 522, the control unit 104 receives the map's lane markings as input. These markings provide a reference for the lane boundaries based on pre-mapped data and help identify whether lane markings are present in the surroundings. In block 524, the control unit 104 also receives the sensor lane marking type. This input corresponds to the lane markings detected by the individual vehicle's sensors (such as LiDAR, radar, or cameras). The sensor lane marking type can include solid lines, dashed lines, or no markings at all.

[0051] Block 526 performs a coding-based embedding process, where both the map lane markings (from Block 522) and the sensor lane markings (from Block 524) are encoded with a confidence level. If the sensor data is more reliable or unambiguous (i.e., has a higher confidence level), the embedding process reflects this confidence, which can affect lane index prediction.

[0052] Furthermore, in block 316 (as in Fig. (as shown) sequential modeling is applied to the encoded data and the data received from blocks 414, 426, 516, and 520, using recurrent neural networks (RNNs), including LSTM layer 1, LSTM layer 2, and LSTM layer 3 in blocks 528-1, 528-2, and 528-3, respectively, to capture temporal dependencies and sequential patterns in the data over time. LSTM networks are adept at processing time-series data and are therefore well-suited for sequential information such as vehicle movement and changes in the environment. Additionally, GRU layer 1, GRU layer 2, and GRU layer 3 in blocks 530-1, 530-2, and 530-3 are used alongside the LSTM networks to process the dynamic task data in parallel. GRU networks can effectively process sensor data that evolves over time and help predict the trajectory of an individual vehicle.The number of LSTM and GRU layers is not limited; it can be increased or decreased.

[0053] In blocks 528-4, 528-5 and 528-6, sensor data from road markings, dynamic tasks and trusted data from LSTM layer 4, LSTM layer 5 and LSTM layer 6 respectively are combined and merged to ensure that the system integrates the sensor inputs into a unified representation that reflects both the historical context and the current sensor data.

[0054] Furthermore, in Block 512, a fully linked layer is applied to the processed data from Block 316. More complex interactions between the features are learned and refined to improve the accuracy of the predictions. Additionally, in Block 514, softmax activation is applied to predict the number of lanes the individual vehicle crosses and to create a probability distribution for the number of lanes. Finally, the individual lane index is estimated in Block 318 to represent the actual lane the individual vehicle is traveling on.

[0055] In Fig.Figure 6 shows a flowchart illustrating the proposed method 600 for estimating the road index for a single vehicle. In block 602, the method 600 comprises the step of receiving one or more environmental attributes, including dynamic object data and free space data indicating drivable areas around the individual vehicle, by a control unit 104. The environmental attributes are acquired by one or more sensors 102 positioned on the individual vehicle.

[0056] In the further course, in block 604, the procedure 600 includes the fusion of the received dynamic object data and the free space data by the controller 104 to form a dynamic object fusion vector that represents spatial and temporal relationships between the individual vehicle, the dynamic objects and the free space.

[0057] Furthermore, the procedure 600 also includes the clustering step, in which the control unit 104 identifies one or more objects from the received dynamic object data based on spatial attributes such as position and distance. These identified dynamic tasks are grouped into an initial set of clusters based on their spatial relationships and proximity to one another. For each cluster in the first group, the control unit 104 evaluates its mean to determine the cluster's central position. The control unit 104 also calculates the distances of the identified tasks relative to the mean of the corresponding clusters in the first group. These distances are correlated with the evaluated mean values ​​of the clusters in the first group to refine the spatial relationships and ensure the correct alignment of the tasks within their clusters.

[0058] To improve the representation of the environment, the control unit adds a second set of clusters for dynamic tasks located within a predefined region of interest (ROI) of the individual vehicle. Noise and irrelevant clusters are then filtered out from both the first and second sets by removing clusters that are not assigned dynamic tasks or that are located in longitudinal directions irrelevant to the individual vehicle's navigation. Furthermore, the control unit 104 embeds the mean and variance of the remaining clusters from the first and second sets into the dynamic object fusion vector, resulting in a statistically enriched representation of the clustered environment.

[0059] Subsequently, in Block 606, Procedure 600 involves the extraction of temporal dependencies and patterns from the dynamic task fusion vector by Controller 104 using a sequential learning model. The sequential learning model is trained using a combination of unsupervised learning to identify spatial patterns in the fused data and supervised learning to refine lane index predictions based on labeled training data.

[0060] Subsequently, in block 608, procedure 600 comprises the estimation of a single lane index by the control unit 104, based on the temporal dependencies and patterns as well as the spatial and temporal relationships between the individual vehicles and the environment. The estimated single lane index indicates a lane in which the individual vehicle is traveling.

[0061] Procedure 600 further includes the step of refining the predicted individual road index based on road marking data, if available.

[0062] Method 600 further includes the step of dynamically updating the individual lane index as the individual vehicle moves along the lane by incorporating real-time changes in dynamic object trajectories, free space boundaries, and one or more environment attributes.

[0063] Procedure 600 further includes the step of classifying the roadway to determine whether it is a multi-lane or single-lane environment, which increases the accuracy and reliability of the roadway estimation process.

[0064] While the foregoing describes various embodiments of the present invention, other and further embodiments of the present invention can be developed without departing from the basic scope. The scope of the present invention is defined by the following claims. The present invention is not limited to the described embodiments, versions, or examples, which are included to enable a person with ordinary technical knowledge to manufacture and use the present invention when combined with the information and knowledge available to such a person. ADVANTAGES OF THE PRESENT INVENTION

[0065] The present invention provides a system and a method for improving the accuracy of roadway estimation by effectively integrating free space data with dynamic tasks.

[0066] The present invention provides a system and a method to achieve reliable lane detection even when there are no clear lane markings or surrounding dynamic tasks.

[0067] The present invention provides a system and a method for improving occlusion performance by utilizing fused representations of free space and dynamic tasks.

[0068] The present invention provides a system and a method for enabling real-time adaptability in roadway estimation under various traffic conditions.

[0069] The present invention provides a system and a method for minimizing error rates in road estimation using multi-head cross-attention and sequential learning models.

[0070] The present invention provides a system and a method for maintaining consistent lane detection performance in difficult scenarios such as obstructions or incomplete sensor data.

[0071] The present invention provides a system and a method for facilitating efficient end-to-end training for the simultaneous optimization of multi-head attentional and sequential learning networks.

[0072] The present invention provides a system and a method that offers a scalable solution for lane detection for both single-lane and multi-lane road conditions. QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] DE 102021004525A1

[0005]

Claims

[1] Road index estimation system (100) implemented in a single vehicle, wherein the system (100) comprises: a controller (104) connected to a processor (202); and a memory (204) that is operationally coupled to the processor (202), wherein the memory stores one or more instructions which, when executed, cause the controller (104) to: from one or more sensors positioned on the individual vehicle, one or more environmental attributes of a roadway are received, wherein the one or more environmental attributes include dynamic object data and free space data indicating drivable areas around the individual vehicle; The received dynamic object data and the free space data are fused to create a dynamic object fusion vector, where the dynamic object fusion vector represents spatial and temporal relationships between the individual vehicle, the dynamic objects, and free space; Extracting temporal dependencies and patterns from the dynamic task fusion vector using a sequential learning model; and Estimation of a single lane index based on the extracted temporal dependencies and patterns and the spatial and temporal relationships between the individual vehicle and the environment, where the single lane index indicates a lane in which the individual vehicle travels. [2] System (100) according to claim 1, wherein the controller (104) is configured to refine the estimated single lane index based on lane marking data received from the one or more sensors 102. [3] System (100) according to claim 1, wherein the sequential learning model is trained using a combination of unsupervised learning to identify spatial patterns in the fused dynamic object data and the free space data and supervised learning to refine lane index predictions based on labeled training data. [4] System (100) according to claim 1, wherein the control unit (104) is configured to dynamically update the individual lane index as the individual vehicle moves on the lane by incorporating real-time changes in dynamic object trajectories, free space boundaries and one or more environment attributes. [5] System (100) according to claim 1, wherein the controller (104) is further configured such that it: Recognizing one or more objects from the received dynamic object data using spatial attributes such as position and distance, and grouping the recognized dynamic objects into an initial set of clusters based on spatial relationships and proximity to each other; Evaluate the mean of the first group of clusters to determine the central position of each cluster; Evaluate the distances of the identified one or more tasks relative to the mean of the corresponding one or more clusters from the first set of clusters; the distances of the identified task(s) correlate with the evaluated mean of the first group of clusters; Add a second set of clusters for each dynamic task that is located within a predefined region of interest (ROI) of the individual vehicle, Filtering out noise and irrelevant clusters from the first sets of clusters and the second sets of clusters by removing one or more clusters without the associated one or more dynamic tasks or in irrelevant longitudinal directions; and Embed the mean and variance of the first group of clusters and the second group of clusters into the dynamic fusion vector of the task. [6] Method (600) for estimating the road index for a single vehicle, the method comprising: Receiving (602) one or more environmental attributes of a roadway from one or more sensors positioned on the individual vehicle by a controller, wherein the one or more environmental attributes include dynamic object data and free space data indicating drivable areas around the individual vehicle; Merging (604) the received dynamic object data and free space data to form a dynamic object fusion vector, wherein the dynamic fusion vector represents spatial and temporal relationships between the individual vehicle, the dynamic tasks and free space; Extracting (606) temporal dependencies and patterns from the dynamic object fusion vector using a sequential learning model by the controller; and Estimating (608) a single lane index by the controller, based on the temporal dependencies and patterns and the spatial and temporal relationships between the single vehicle and the environment, wherein the single lane index indicates a lane in which the single vehicle is traveling. [7] The method (600) according to claim 6 further comprises the step of refining the individual road index on the basis of road marking data received from the one or more sensors. [8] Method (600) according to claim 6, wherein the sequential learning model is trained using a combination of unsupervised learning to identify spatial patterns in the fused dynamic object data and free space data and supervised learning to refine lane index predictions based on labeled training data. [9] The method (600) according to claim 6 further comprises the step of dynamically updating the individual lane index while the individual vehicle moves along the lane by incorporating real-time changes in dynamic object trajectories, free space boundaries and one or more environment attributes. [10] The method (600) according to claim 6 further comprises the clustering step, wherein the clustering comprises: The controller detects one or more objects from the received dynamic object data using spatial attributes such as position and distance, and groups the detected dynamic objects into an initial set of clusters based on spatial relationships and proximity to each other; The controller evaluates the mean of the first group of clusters to determine the central position of each cluster; Evaluation of the distances of the identified task(s) relative to the mean of the corresponding cluster(s) from the first set of clusters by the controller; Correlating the distances of the detected task(s) with the evaluated mean of the first group of clusters by the controller; The controller adds a second set of clusters for each dynamic task located within a predefined region of interest (ROI) of the individual vehicle, Filtering out noise and irrelevant clusters from the first sets of clusters and the second sets of clusters by removing, via the controller, the one or more clusters that are not associated with one or more dynamic tasks or that are located in irrelevant longitudinal directions; and Embedding the mean and variance of the first set of clusters and the second set of clusters into the dynamic object fusion vector by the controller.

Images