METHOD AND SYSTEM FOR AMBIENCE RESOLUTION AND DEVIATION REDUCTION FOR PERCEPTION AND MAPPING ASSOCIATION

Abstract

A procedure involves receiving map data with a plurality of map road edges and a plurality of perception road edges for a road segment. The procedure involves determining a probability value for orientations, each indicating whether at least one of the map road edges of the orientation should be the same road line as one of the perception road edges of the orientation. The procedure involves resolving ambiguity if no probability value of the road segment satisfies at least one orientation criterion, and determining a resolved orientation between the perception and map road edges. This involves applying multiple levels of ambiguity, each with a different ambiguity test, and determining whether a non-ambiguity count should be incremented depending on the magnitude of probabilities with respect to the orientations.The resolved alignment exists when the alignment between roadway edges has a highest non-ambiguity count.

Description

[0001] The present disclosure relates to navigation systems for a vehicle and in particular to a perception and map edge association for a vehicle.

[0002] An autonomous driving system often includes a vehicle navigation system and is a complex system encompassing many different aspects. For example, an autonomous driving system may include multiple sensors to collect perceptual data regarding the vehicle's environment. In addition to sensors, the autonomous driving system also uses map data. Perceptual data is matched with map data in association systems to perform navigation for autonomous driving systems. The present disclosure provides an improved association system.

[0003] German patent application DE 10 2016 213 782 A1 discloses a method and a device for determining the lateral position of a vehicle relative to the lanes of a roadway. The device has an image processing unit for detecting lane markings and a position determination unit for determining an initial position for the vehicle. An evaluation unit is configured to determine when the detected lane markings are insufficient to determine the vehicle's lateral position by comparing them with lane marking information from a lane geometry map. In this case, the evaluation unit determines an approximate lateral position of the vehicle using information from the lane geometry map.

[0004] German patent DE 10 2022 004 774 A1 discloses a system and method for lane determination of an ego-vehicle, in which SMPC and LRL sensors are configured to detect the boundary distance, i.e., the distance of the boundary from each side of the ego-vehicle. An SMPC boundary distance and an LRL boundary distance can be compared. The minimum distance between the SMPC boundary distance and the LRL boundary distance is used to calculate the probability that each of the lanes is an ego-vehicle lane. Furthermore, the minimum distance is fed into a Bayesian filter to enable the fusion of the second lane attributes assigned to multiple sensors, thereby facilitating lane determination of the ego-vehicle.

[0005] German patent DE 10 2022 004 633 A1 discloses a system and a method for determining the lane position of an ego-vehicle. The system and method comprise: obtaining a compatibility percentage associated with each of the first and second sets of lane and position attributes, taking into account a confusion matrix, and calculating a first and second fit rating for the first and second sets of lane and position attributes, respectively. Furthermore, the extracted set of lane and position attributes with the higher fit rating can be fed to a Bayesian filter to enable a fusion of the first and second sets of lane and position attributes, thereby enabling the ego-vehicle to determine its lane position.

[0006] German patent DE 10 2023 004 673 A1 discloses a system and a method for validating the road edge distance during ego lane estimation. It involves determining the sensor-based distance of the ego vehicle from the left / right edge of the lane. Furthermore, the sensor-based distance can be compared to a threshold distance, which is calculated by multiplying the number of lanes to the left / right of the ego lane by the lane width and then adding the distance to the left / right ego lane marking, based on the road edge distance validated together with the safety boundary, to the multiplied value. If the determined distance is less than the distance threshold, it is validated, and the ego lane is estimated accordingly. However, if the distance is greater than the distance threshold, the distance is considered overestimated and discarded.

[0007] US Patent 2024 / 0263965 A1 discloses a system for perceiving and mapping map edges for an autonomous vehicle. The system comprises one or more controllers that execute instructions to receive a plurality of perceived lane edge candidates, each representative of a perceived lane edge, and a plurality of map lane edge candidates, each representative of a map lane edge. The one or more controllers identify a discrepancy between a selected perceived lane edge candidate and a best-matching map edge candidate for a lane characteristic based on a posterior mapping probability matrix and a discrepancy marker. The one or more controllers segment the pair of inconsistent perceived and map lane edges into one or more matching segments.The one or more matching segments indicate where the perception lane edge and the map lane edge coincide based on the lane properties.

[0008] According to one implementation, a method involves receiving map data containing a plurality of map lane edges of a road segment and receiving perception data containing a plurality of perception lane edges of the road segment. The method also involves forming at least one alignment that includes at least one of the map lane edges and at least one of the perception lane edges, and determining, by at least one processor, a probability value for each of the alignments indicating whether at least one of the map lane edges of the alignment should be the same lane line as one of the perception lane edges of the alignment.The procedure involves resolving an ambiguity by at least one processor when no probability value of the at least one orientation satisfies at least one orientation criterion, and includes determining a resolved orientation between the at least one perceptual roadway edge and the at least one map roadway edge, including applying several ambiguity levels. Each ambiguity level has a different ambiguity test, and at least two of the ambiguity levels are arranged to determine whether a non-ambiguity count should be incremented depending on the magnitude of probabilities with respect to the at least one orientation.The procedure involves determining, by at least one processor, that the resolved orientation exists if the orientation between the at least one perceptual roadway edge and the at least one map roadway edge has a highest non-ambiguity count.

[0009] The alignment is a group alignment with one or more individual alignments, each individual alignment having a perceptual lane edge paired with a map lane edge, and the multiple ambiguity levels sequentially including a first low-probability decoupling level arranged to remove low-probability individual alignments, a second ego lane tracking level which considers previous individual associations between map and perceptual lane edges, and a third ego lane feature consistency level which matches lane features of the individual alignments with lane features of the previous individual associations.

[0010] In another exemplary implementation, the alignment is a group alignment with one or more individual alignments, where each individual alignment has a perceptual lane edge paired with a map lane edge, and one of the ambiguity levels is a low-probability decoupling level that ignores individual alignments with an individual probability below an ambiguity threshold of the individual alignment. The individual alignments are ignored to calculate the probability value of a group alignment.

[0011] In another exemplary implementation, the road segment is a current segment, and one of the ambiguity levels is an ego lane-tracking level that involves: generating ambiguous group orientations, each with one or more ambiguous individual orientations, and determining whether a current map or perception lane edge is sufficiently similar to the ambiguous individual orientation of a previous map or perception lane edge of a previous segment and the ambiguous individual orientation. The previous segment is located before the current segment.

[0012] In another exemplary implementation, the non-ambiguity count for each of the ego-individual orientations is incremented with a sufficient match between a current map and perception lane edge and a previous timestamp map and perception lane edge association.

[0013] In another exemplary implementation, the road segment is a current segment, and one of the ambiguity levels is an ego road feature consistency level that determines probabilities of feature matches between road line features of a current map or perception road edge of the current timestamp with a corresponding previous map or perception road edge, such that each pair of current and previous map and perception road edges forms an individual ambiguity alignment. Determining feature match probabilities is repeated for each individual ambiguity alignment within a group of ambiguity alignments with multiple individual ambiguity alignments.

[0014] In another exemplary implementation, the non-ambiguity count is incremented with each of the individual ambiguity orientations that has multiple features, each with a feature match probability above a feature match threshold.

[0015] In another exemplary implementation, the features include at least one of the following: road type, road color, road curvature, and road course.

[0016] In another exemplary implementation, the distance between perception and map road edges is not one of the features.

[0017] According to another implementation, a system includes a memory, processor circuits forming at least one processor communicatively coupled to the memory, and configured to operate by: receiving map data containing a plurality of map lane edges of a road segment, and receiving perception data containing a plurality of perception lane edges of the road segment. The processor is configured to operate by forming at least one alignment having at least one of the map lane edges and at least one of the perception lane edges, and determining a probability value for each of the alignments indicating whether at least one of the map lane edges of the alignment should be the same lane line as one of the perception lane edges of the alignment.The processor is arranged to operate by resolving an ambiguity if no probability value of the at least one orientation satisfies at least one orientation criterion, and by determining a resolved orientation between the at least one perceiving lane edge and the at least one map lane edge, including applying several levels of ambiguity. Each level of ambiguity has a different ambiguity test, and at least two of the ambiguity levels are arranged to determine whether a non-ambiguity count should be incremented depending on the magnitude of probabilities with respect to the at least one orientation. The system is also arranged to operate by determining that the resolved orientation exists if the orientation between the at least one perceiving lane edge and the at least one map lane edge has a highest non-ambiguity count.The alignment is a group alignment with one or more individual alignments, each individual alignment having a perceptual lane edge paired with a map lane edge, and the multiple ambiguity levels sequentially including a first low-probability decoupling level arranged to remove low-probability individual alignments, a second ego lane tracking level which considers previous individual associations between map and perceptual lane edges, and a third ego lane feature consistency level which matches lane features of the individual alignments with lane features of the previous individual associations.

[0018] In another exemplary implementation, the alignment is a group alignment with one or more individual alignments, each having a map roadway edge paired with a perception roadway edge. The at least one processor is arranged to operate by generating several alternative group alignments, each with individual alignments having differently paired map and perception roadway edges from group alignment to group alignment, and generating a probability value for each alternative group alignment.

[0019] In another exemplary implementation, generating the multiple alternative group orientations involves maintaining map lane edges in an arrangement relative to each other in a map, maintaining perception lane edges in an arrangement relative to each other in a perception image, and moving the map relative to the perception image to change the map and perception lane edge pairs in the individual orientations from group orientation to group orientation.

[0020] In another exemplary implementation, determining the probability value involves forming a probability for each individual orientation, including considering one or more feature match probabilities of roadway line features, without considering the distance between map and perceived roadway edges.

[0021] In another exemplary implementation, the probability value considers at least one of the following: road type, road color, road curvature, and road course to generate a selected best group alignment. The at least one processor is arranged to operate by determining a deviation correction of the selected best group alignment, including considering the distance between at least one map road edge and at least one perception road edge of the selected best group alignment.

[0022] In another exemplary implementation, the at least one processor is arranged to work by applying a confidence value to a probability of each individual alignment in order to generate the probability value of the group alignment.

[0023] In another exemplary implementation, the at least one processor is arranged to work by determining which group orientation, among several alternative group orientations, has the best alignment between map and perception lane edges, including determining whether a ratio with two highest probability values ​​is above a threshold.

[0024] In an exemplary implementation, a vehicle includes a memory and processor circuits that form at least one processor which is communicatively coupled to the memory and arranged to operate by: receiving map data which includes a plurality of map lane edges of a road segment, and receiving perception data which includes a plurality of perception lane edges of the road segment.The processor is arranged to operate by determining a probability value for each of the orientations, indicating whether at least one of the map roadway edges of the orientation should be the same roadway as one of the perception roadway edges of the orientation, and resolving any ambiguity if no probability value of the at least one orientation satisfies at least one orientation criterion, and including determining a resolved orientation between the at least one perception roadway edge and the at least one map roadway edge, including applying several levels of ambiguity. Each level of ambiguity has a different ambiguity test, and at least two of the ambiguity levels are arranged to determine whether a non-ambiguity count should be incremented depending on the magnitude of probabilities with respect to the at least one orientation.The processor is configured to operate by determining that the resolved orientation exists if the orientation between the at least one perceiving lane edge and the at least one map lane edge has a highest non-ambiguity count.

[0025] In another exemplary implementation, the at least one processor is arranged to operate by: determining map and perception non-roadway objects on and near the road segment, including segments before or after or both of the road segment; calculating a group longitudinal map deviation, including taking into account distances between map and perception non-roadway objects of multiple pairs; and applying the group longitudinal map deviation to fit map or perception roadway edge positions to the map or perception data.

[0026] In another exemplary implementation, the at least one processor is arranged to operate by: combining the group longitudinal map deviation and a lateral map deviation, determined by using the probability value and the resolved orientation to form a map deviation correction, and applying the map deviation correction to generate a group mapping between map lane edges and perception lane edges, while taking distance into account as a lane feature.

[0027] In another exemplary implementation, the at least one processor is arranged to work by using group mapping to provide a prior mapping to be used in a future ambiguity level.

[0028] The present revelation is described below in connection with the following figures. The figures are not to scale, and numbers in the figures denote identical elements, and where: Fig. 1 a schematic diagram of an exemplary vehicle with an exemplary system for performing a perception map edge mapping according to at least one of the implementations herein; Fig. 2 a schematic diagram of an exemplary perception map edge mapping system from Fig. 1 and according to at least one of the implementations herein; Fig. 3 a flowchart of an exemplary procedure for performing a perception map edge mapping according to at least one of the implementations herein; Fig.4 a flowchart of an exemplary procedure for performing a lateral perception map edge mapping according to at least one of the implementations herein; Fig. 5 is a schematic diagram of an exemplary intersection used to illustrate a perception map edge mapping according to at least one of the implementations herein; Fig. 6 is a schematic diagram of an exemplary road segment showing a lateral perception map edge mapping according to at least one of the implementations herein; Fig. 7 is a table that shows the results of the lateral correction from Fig. 6 according to at least one of the implementations shown herein; Fig. 8 is another schematic diagram of exemplary streets showing a lateral perception map edge mapping according to at least one of the implementations herein; Fig.9 a flowchart of an exemplary procedure for performing a longitudinal perception map edge mapping according to at least one of the implementations herein; Fig. 10 is a schematic diagram of an exemplary intersection used to illustrate a longitudinal perception map edge mapping according to at least one of the implementations herein; Fig. 11 is a schematic diagram of an exemplary road segment showing a longitudinal perception map edge mapping correction according to at least one of the implementations herein; Fig. 12 a flowchart of an exemplary procedure for resolving ambiguities for a perception map edge mapping according to at least one of the implementations herein; Fig. 13 a flowchart of an exemplary ambiguity resolution of the first stage of the procedure of Fig.12 according to at least one of the implementations herein; Fig. 14 is a schematic diagram of an exemplary road segment, which represents a first-stage ambiguity resolution of Fig. 13 according to at least one of the implementations herein shows; Fig. 15 a flowchart of an exemplary ambiguity resolution of the second stage of the procedure of Fig. 12 according to at least one of the implementations herein; Fig. 16 is a schematic diagram of an exemplary road segment, which represents a second-stage ambiguity resolution of Fig. 15 according to at least one of the implementations herein; Fig. 17 a flowchart of an exemplary ambiguity resolution of the third stage of the procedure of Fig. 12 according to at least one of the implementations herein; and Fig.18 is a schematic diagram of an exemplary road segment, which represents a third-stage ambiguity resolution of Fig. 17 according to at least one of the implementations herein.

[0029] An example of map error occurs when map data may not accurately represent lane counts on a road or may contain incorrect geometry. Map data can contain missing or inaccurate information for a variety of reasons, such as changes in the actual road layout that have occurred since the map data was collected as a construction example.

[0030] An autonomous driving system associates (or adapts) perceptual lane edges and map lane edges based on a one-to-one relationship for localization and scene generation purposes. However, problems can arise when the autonomous driving system attempts to associate perceptual lane edges and map lane edges due to missing, inaccurate, or inconsistent map and / or perceptual data. Furthermore, it is understood that high-resolution maps providing the level of detail of the vehicle's surroundings required for autonomous driving may not always be available, which can also create problems in associating perceptual lane edges with map lane edges.

[0031] In addition to the challenges mentioned above, environments where global navigation satellite systems (GNSS) fail (weak, obstructed, or completely unavailable) remain a challenge for perceptual map association tasks. This is because high uncertainty of a host's pose on the map can cause misalignment between perceptual lane edges and the corresponding map lane edge candidates when map lane edge candidates are projected into the perceptual frame. Thus, map lane deviation occurs when high-resolution map data is unavailable.

[0032] To solve these problems, the methods and systems disclosed herein perform a lateral alignment of perception lane edges with map lane edges of a road segment, which can be used for vehicle navigation and / or autonomous driving. The lateral alignment is achieved by determining group alignment probability values, each group alignment having one or more individual alignments or matches of perception and map lane edges. A probability of each individual alignment is considered to determine a group alignment probability value, and each individual alignment probability is generated by considering a number of edge features without taking into account the distance between paired perception and map lane edges.It should be noted that the term "road edge" can be used interchangeably here with the terms edge, roadway, road line, and line. Furthermore, the term "match" indicates an association or alignment with a probability of specifying the same road line or object, and is intended to indicate more than simply being paired for comparison and analysis, unless the context specifies otherwise.

[0033] Due to its shape, lane width mismatches and errors between perceiving lanes and map lanes can result from low-quality maps. However, it has been found that lane width mismatches can be mitigated by relaxing the boundary condition for spacing (by omitting spacing) as a restrictive alignment metric or feature during the lateral alignment process, in conjunction with maintaining the arrangement or sequence of adjacent lane edges relative to each other during alignment. In other words, the perceiving lane lines do not change their position and orientation relative to each other, and the map lane lines do not change their position and orientation relative to each other, and both maintain a global orientation such as north.

[0034] Thus, the distance is not considered for lateral alignment while a best group alignment is being determined, and until after a most probable group alignment has been selected. Once a final lateral alignment has been determined, the distance is used after errors have been removed and when a final or specified map deviation correction is applied to the map data. In this latter case, corrected map edges are mapped to their corresponding perceptual edges based on "narrow thresholds," referencing the distance, which is now taken into account as described below. Thus, the distance is finally considered with the other edge features when a specified or final deviation correction is applied to compensate for map deviation.The result is that lateral alignment, together with longitudinal alignment based on the alignment of perceived and map objects by a roadway, rather than aligning the edges or roadways themselves, leads to the removal of a map deviation.

[0035] If no group alignment is selected, an ambiguity resolution process is performed to resolve existing ambiguities, but the system cannot determine which of a number of individual perception and map-edge lane alignments are the correct matches. This may involve the use of several stages, such as three, with each stage employing a different ambiguity resolution process to determine a resolved group alignment (or resolved alignment) to be used. One stage decouples low-probability individual alignments to highlight higher-probability individual alignments, while another stage uses historical information and aligns edges of current individual alignments of a current lane segment with edges of permeable completed individual mappings of a group mapping.

[0036] Another stage uses similar historical data and matches lane or edge features of current individual orientations with lane features of edges of previous individual assignments of the previous road segment, in order to use only those individual orientations with matching features. Through an exemplary formula in the historical stages, each instance of sufficiently matching individual orientations and / or sufficiently matching features increments a non-ambiguity count, and the group orientation with the highest non-ambiguity count can be used as the group orientation for variance correction. Other operations that can be performed for lateral orientation and ambiguity resolution, as well as further details of these operations, are provided below.

[0037] The presented method and system applies perception-to-map association robustness for lateral deviation measurement through lateral feature alignment and map deviation correction. The disclosed method also resolves ambiguities in perception-map lane edge assignments and adds assignment robustness to lane displacements due to construction zones and lane merging scenarios, as well as the ability to assign in complex scenarios such as intersections based on longitudinal alignment logic and the ability to address perpendicular lane edges. The present method also provides association robustness to lane width mismatch and lane count mismatch between perception and map lane edges, while providing robustness to longitudinal lane edge identification switching due to noisy perceived lane edges.

[0038] With reference to Fig. 1 includes an exemplary system 101 comprising one or more vehicles 100, each performing perceptual edge map association for autonomous driving or other navigation systems. In various implementations, the perceptual map edge association system can be deployed on the vehicle 100 and can have tasks that, according to process 300 ( Fig. 3) and subprocesses and implementations thereof Fig. 4-18 according to the exemplary implementations described herein.

[0039] In particular, as described in more detail below, the vehicle 100 in various implementations has a controller 140 (or a computer system) with processor circuits comprising at least one processor 142 and a memory 144 that stores programs 150 containing perceptual map edge association system software and / or firmware that performs the perceptual map edge association tasks, as described in detail below.

[0040] By way of example, Vehicle 100 includes an automobile. Vehicle 100 can be any of a number of different types of automobiles, such as a sedan, station wagon, truck, or SUV, and can be two-wheel drive (2WD) (i.e., rear-wheel drive or front-wheel drive), four-wheel drive (4WD), or all-wheel drive (AWD), and / or various other types of vehicles in certain implementations, such as trucks with more than four wheels, and so on. In certain implementations, Vehicle 100 can also include any other motorized vehicle with cameras and / or sensors capable of sufficiently capturing or perceiving at least one environment near a vehicle and receiving map data to generate navigation data or for autonomous driving.

[0041] In some implementations, the vehicle 100 can be operated wholly or partially by a human driver, or alternatively, it can comprise an autonomous or semi-autonomous vehicle, where, for example, the vehicle control (including acceleration, deceleration, braking, and / or steering) is planned and executed wholly or partially automatically by a control system 102 of the vehicle 100. Additionally, the vehicle 100 can be operated by a human at certain times and via automated control at other times. In some forms, the vehicle may only have manual control. Thus, the vehicle 100 can include one or more functions that can be automatically controlled, for example, via the control system 102, to provide driver assistance features.

[0042] Furthermore, the exemplary vehicle 100 includes a body 104 mounted on a chassis 116. The body 104 essentially encloses other components of the vehicle 100. The body 104 and the chassis 116 can together form a frame.

[0043] The vehicle 100 also includes a plurality of wheels 112. A drive system 110 is mounted on the chassis 116 and drives the wheels 112, for example, via axles 114. The drive system 110 preferably comprises a drive system, such as an internal combustion engine and / or an electric motor / generator, any variation thereof, or any other drive system, and is coupled to a transmission thereof.

[0044] In some forms, the vehicle 100 also includes a braking system 106 and a steering system 108 with a steering wheel 109 in various implementations. In exemplary implementations, the braking system 106 controls the braking of the vehicle 100 using brake components that are controlled by inputs provided by a driver (e.g., via a brake pedal in certain implementations) and / or automatically via the steering system 102. Furthermore, in exemplary implementations, the steering system 108 controls the steering of the vehicle 100 via steering components (e.g., the steering wheel 109) that are controlled by inputs provided by a driver (e.g., via the steering wheel 109 in certain implementations) and / or automatically via the steering system 102.

[0045] Through one approach, the control system 102 is coupled with the braking system 106, the steering system 108, and the drive system 110. In various implementations, the control system 102 facilitates at least the generation and processing of observational data on camera images acquired by other sensors for the vehicle 100 and / or other vehicles. Additionally, in certain implementations where the vehicle 100 is an autonomous or semi-autonomous vehicle, the control system 102 also provides, under certain circumstances, control over automated features of the vehicle 100 (including the automated operation of the braking system 106, the steering system 108, and / or the drive system 110), including the use of one or more models trained using the observational data.

[0046] As in Fig.As shown in Figure 1, the control system 102, in various implementations, includes a sensor array 120, a display 124, a transceiver 126, and the controller 140. In one example, the sensor array 120 receives sensor data to generate the observation data. In various implementations, the sensor array 120 includes one or more cameras 130 (such as video cameras and / or still cameras). In some examples, the sensor array 120 may also include one or more other detection sensors 132 (e.g., radar, sonar, light detection and ranging (LIDAR), infrared, or the like) and / or other sensors 134 (e.g., vehicle position sensors, speed sensors, accelerometers, gyroscopes, inertial sensors, brake sensors, steering sensors, suspension sensors, and so on).

[0047] In various implementations, the vehicle cameras 130, used to obtain images of the road observation data, can include front, rear, side, and / or surround-view cameras, including wide-angle, 360-degree, and / or fisheye lens cameras, as well as monocular, stereo, infrared, time-of-flight, thermal, LiDAR cameras, and so on. These cameras 130 can capture images that are then processed by object detection algorithms. These algorithms can be used to detect and measure a roadway (hereinafter also referred to as a road or path) on which the vehicle 100 operates and are capable of detecting shape, size (e.g., dimensions), and otherwise identifying and / or labeling road surfaces, lane lines, curbs, sidewalks, driveways, and other objects near the roadway, including guardrails, barriers, traffic lights, signs, light poles, fire hydrants, and many other objects.Such imaging systems also measure distances from the vehicle to road surfaces, lane markings, signs, barriers, and the positions and movements of pedestrians, vehicles, drivers, and various other details of the roadway and activities affecting them. In various implementations, video camera images are obtained. Additionally or alternatively, still camera images can be obtained.

[0048] In various implementations, the detection sensors 132 and / or other sensors 134 receive additional information relating to the road surface and / or the operation of the vehicle 100 itself (e.g., its position, speed, deceleration and / or acceleration, etc.) for use in operating the vehicle 100, for example, according to the autonomous operation of the vehicle 100 and / or certain components thereof. This can include radar sensors, ultrasound and other types of sensors, as well as inertial measurement units (IMUs) that can detect the movement and orientation of a vehicle.

[0049] In an exemplary form, and instead of using cameras 130 alone to detect objects and perceive the environment around the vehicle, the cameras 130 are used as part of an automated driving system (ADS), an advanced driver assistance system (ADAS), and / or a similar system that uses both optics and the other detection sensors 132 and other sensors 134 to detect the roadway, lane markings, and objects near the roadway. Thus, for example, data collected from camera images, radar, LiDAR, and ultrasonic sensors can be used together to detect these objects. This can involve performing sensor fusion and machine learning or neural network models that receive input from a variety of sensors instead of image data alone, as well as other techniques.

[0050] In the present example, the vehicle 100 also includes a transceiver 126 for communicating with remote systems, servers, devices, modules, or units. Thus, one or more parts or components (or units) of the control system 102 and / or the controller 140, which perform processing for any of the operations described herein relating to perception map edge mapping, can be performed remotely as needed. In particular, the controller 140 (and in certain implementations, the control system 102 itself) is located within the body 104 of the vehicle 100 in various implementations. In one implementation, the control system 102 is mounted on the chassis 116.In certain implementations, the controller 140 and / or the control system 102 and / or one or more of its components may be located outside the body 104, for example on a remote server, in the cloud, or in another device where image processing is performed remotely. It is understood that the control system 102 and / or the controller 140 are otherwise distinct from the one in the body 104. Fig. The implementation shown in Figure 1 may differ. For example, the controller 140 may be coupled with one or more remote computer systems and / or other control systems, or may otherwise use them, for example as part of one or more of the devices and systems of the vehicle 100 identified above.

[0051] The transceiver 126 can also be used to receive map data, which is then compared with perception data generated by the vehicle's onboard sensors and other systems to add external environmental data to the onboard navigation systems, including the perception map edge association system. Thus, the transceiver 126 can function as a high-frequency transceiver for receiving real-time map data. Alternatively, the transceiver 126 can be part of a satellite communication system (SCS), such as a global positioning system (GPS) or satellite-based augmentation system (SBAS). Wi-Fi modules can be used to connect to local networks or hotspots on cellular or satellite networks.Bluetooth technology can be used to facilitate short-range data exchange with nearby vehicles or infrastructure for sharing localized map data. Mesh networks can be used to create ad-hoc mesh networks with other nearby vehicles for exchanging map data, using protocols designed for vehicle-to-vehicle (V2V) communication, for example. Such data communication networks can include 4G / 5G cellular networks, dedicated short-range communications (DSRC), and emerging vehicle communication protocols such as cellular vehicle-to-everything (C-V2X).The sources of map data can range from government agencies providing infrastructure information, private mapping companies offering dynamic road condition updates, the vehicle manufacturer, the navigation system developer and / or crowdsourced data from other vehicles on the road, to name a few examples.

[0052] Furthermore, the control system 102 can have a display on the vehicle 100 that can provide messages or show maps to the occupants of the vehicle 100, such as to show the vehicle's position and orientation on a road. The display 124 can be any type that can provide a screen for an occupant or user in the vehicle to view the images on the display 124. Such a display can be a digital display, a graphical user interface (GUI), an LED display, a plasma display, an LCD display, an organic light-emitting diode (OLED) display, a thin-film transistor (TFT) display, a head-up display (HUD), 3D displays, holographic displays, virtual or augmented displays, and so on.

[0053] In various implementations, the controller 140 is coupled with the sensor array 120, as well as with the brake system 106, the steering system 108, and the drive system 110. In various implementations, the controller 140 is also coupled with the display 124 and the transceiver 126.

[0054] In various implementations, the controller 140 comprises or is a computer system and includes the processor 142, the memory 144, an interface 146, a storage device 148, and a computer bus 149. In various implementations, the controller (or the computer system) 140 receives sensor data from the sensor array 120 and, in certain implementations, additional data via the transceiver 126. In various implementations, the controller 140 processes the observation data, including images of a road ahead on an expected path of the vehicle 100. In certain implementations, the controller 140 also uses the observation data to develop, train, and / or implement one or more autonomous driving models for the vehicle 100 (e.g., for automated control of the braking system 106, the steering system 108, and / or the drive system 110).In various implementations, the controller 140 provides these and other functions according to the steps of the processes and implementations described in . Fig. 2- Fig. 18 are shown and as described below in connection therewith.

[0055] In the implementation shown, the controller 140 (or computer system) includes the processor 142 to perform the computational and control functions of the controller 140, and may comprise circuits or circuitry forming any type of processor or multiple processors, individual integrated circuits such as a microprocessor, or any suitable number of integrated circuit devices and / or printed circuit boards that work together to achieve the functions of a processing unit. This may include a system-on-a-chip (SoC) and / or one or more processor cores.During operation, the processor 142 executes one or more programs 150, which include the perception map edge association system described herein and are contained in memory 144, and as such controls the general operation of the controller 140 and the computer system of the controller 140, generally in the execution of the processes described herein, such as the processes and implementations that are in . Fig. 2- Fig. 15 are shown and as described below in connection therewith.

[0056] Memory 144 can be any suitable type of memory. For example, memory 144 can include various types of dynamic random access memory (DRAM) such as SDRAM, the various types of static RAM (SRAM), and the various types of non-volatile memory (PROM, EPROM, and Flash). In certain examples, memory 144 is located on the same computer chip as processor 142 and / or is located on the same computer chip as processor 142. In the implementation shown, memory 144 stores the program 150 mentioned above, along with one or more databases 155 (which may relate, for example, to perceptual and / or map data and / or perceptual / map association calculations described herein) and other stored values ​​156.

[0057] Bus 149 serves to transmit programs, data, status, and other information or signals between the various components of the controller's computer system 140. Interface 146 enables communication with the controller's computer system 140, for example, from a system driver and / or another computer system, and can be implemented using any suitable method and device. In one implementation, interface 146 receives various data from the sensor array 120 and / or a navigation system with map data. Such a navigation system can be one of the programs 150 mentioned above. Interface 146 can include one or more network interfaces for communication with other systems or components.

[0058] The storage device 148 can be any suitable type of storage device, including various types of random-access memory and / or other storage devices. In an exemplary implementation, the storage device 148 comprises a program product from which the memory 144 can receive the program 150, which contains one or more implementations of the processes and implementations of Fig. 3, Fig. 4, Fig. 9, Fig. 12, Fig. 13, Fig. 15 and Fig. 17 and as described below in connection therewith. In another exemplary implementation, the program product can be stored directly in memory 144 and / or a secondary storage device (e.g. disk 157) and / or accessed in another way, such as the one mentioned below.

[0059] The bus 149 can be any suitable physical or logical means for connecting computer systems and components. This includes, but is not limited to, direct hard-wired connections, fiber optic, infrared, and wireless bus technologies. During operation, the program 150 is stored in memory 144 and executed by processor 142.

[0060] It is understood that while this exemplary implementation is described in the context of a fully functioning computer system, the mechanisms of the present disclosure can be distributed as a program product using one or more types of non-volatile, computer-readable signal-carrying media for storing the program and its instructions, and for carrying out its distribution. This includes, for example, a non-volatile, computer-readable medium carrying the program and containing computer instructions stored therein to cause a computing device, such as a computer processor (such as Processor 142), to execute the program. Such a program product can take a variety of forms, and the present disclosure applies equally regardless of the specific type of computer-readable signal-carrying media used for distribution.Examples of signal-carrying media include writable media such as floppy disks, hard drives, memory cards, and optical discs, and transmission media such as digital and analog communication links. It is understood that cloud-based storage and / or other technologies may also be used in certain implementations. Likewise, it is understood that the computer system of control 140 may also differ from the one in [the text is incomplete]. Fig. The implementation shown in Figure 1 can differ. For example, the computer system of the controller 140 may be coupled with one or more remote computer systems and / or other control systems, or may otherwise use them.

[0061] With reference to Fig. Figure 2 shows a program 150, which is a perception-map edge association (PMEA) system 200 and which is run by at least one processor 142 of the controller 140. Fig.1 is operated. The PMEA system 200 receives data from perceived road edges and objects (P) 202 for comparison with map road edges and objects (M) 204 to determine where matches exist, to remove map deviations, and to resolve ambiguities as mentioned. The PMEA 200 has a lateral alignment unit 206, a longitudinal alignment unit 208, an ambiguity resolution unit 210, a map deviation correction unit 212, and a PMEA narrower threshold unit. The output is assigned to PM road edges 216.

[0062] The lateral alignment unit 206 comprises an alternative generation unit 218, a probability value unit 220, and a ratio unit 222. The longitudinal alignment unit 208 comprises an object detection unit 224, a distance unit 226, and a group correction unit 228, while the ambiguity resolution unit 210 comprises an ambiguity test unit 230, which can be considered part of the lateral alignment unit 206 or separate from both, a decoupling unit 232, an ego lane tracking unit 234, an ego lane feature consistency unit 236, and a candidate selection unit 238. The operation of these units and subunits of the PMEA system 200 is described in detail below with processes 300, 400, 900, 1200, 1300, 1500, and 1700.

[0063] With reference to Fig.3. A process 300 is provided for performing a perception map edge mapping according to at least one of the implementations described herein. Process 300 is described by operations 302-320, which are generally numbered evenly. The systems, road facilities, processes, devices, and vehicles from one of the Fig. 1-2 and Fig. Reference may be made to 4-18 where relevant.

[0064] Process 300 generally comprises five phases for mapping perception and map lane edges, which can handle road areas with large map deviations, and includes lateral alignment, longitudinal alignment, ambiguity resolution, map deviation correction determination, and lane edge mapping. The disclosed method and system also works well even when high-resolution (HD) maps may not be available.

[0065] Process 300 can include an “input map (M)” 302, where map data 204 is a street map ( Fig. 2) can be obtained and made available for the mapping operations. The map data 204 can be obtained via various sources and communication equipment already mentioned above. Fig.The map data 204 can contain one or more road segments to be analyzed and can include any road configuration, including straight or curved segments, intersections, single-lane or multi-lane, and so on, without restriction, as long as the configurations are depicted. The map data 204 includes at least map roadway edges. The map data 204 can also include object data of any non-roadway objects near the roadway, including road and other signs, traffic lights, guardrails, construction barriers, vegetation, buildings, fences and other structures, bridge structures, hydrants and other fluid system equipment, lighting equipment such as light poles, electrical boxes, and so on. Data of such object positions can be used to determine the object positions relative to roadway positions on the map data 204.Furthermore, semantic data can also be provided, including markings for the lanes, objects or both, such as a code for a stop sign and so on.

[0066] The process 300 may include an “input perception (P)” 304, and this refers to the perception data (P) 202 ( Fig. 2) This can include data of the same roadway and objects as in the map data and for the same road segment. The perception data 202 is obtained using vehicle sensors and other components, systems, or services that provide localization and other navigation data to the vehicle control system 102 and the controller 140.

[0067] Process 300 can include a “performing a lateral edge alignment” 306 and is performed by the lateral alignment unit 206. This involves generating several alternative group alignments, each group alignment having one or more individual alignments, each with a different pair of a perception road edge that is paired with a map road edge. Features of the lines are used to generate an individual alignment probability and, in turn, a group alignment probability value. The system then compares a ratio of the probability values ​​to determine a group alignment with the best or highest probability value, which should be confirmed and used as the final group association to establish a final map deviation correction and apply the correction, for example, for navigation and autonomous driving.It is understood that the terms lateral and longitudinal here refer to a forward course of the carrier vehicle, with the control unit 140 performing the perception map assignment. Further details are provided in a lateral alignment process 400 (. Fig. 4) provided and examples are given with the Fig. Explained in sections 5-8.

[0068] Process 300 can include the query "Ambiguous alignment?" 310, which determines whether the probability ratios indicate that at least one of the group alignments, or the group alignment with the best or highest probability, has a sufficiently high probability. This can be performed by the ambiguity test unit 230.

[0069] If so, the process 300 “Determine map deviation correction” 312 may involve combining a given longitudinal map deviation, if any, and the given lateral deviation of the group orientation with the highest probability value (or highest ratio) to form a single map deviation correction. This is performed by the map deviation correction unit 212.

[0070] Otherwise, the perception map track edge alignments are too ambiguous. In other words, for example, the system cannot adequately determine whether at least one of the perception track edges aligns with a specific map track edge in a group alignment, so all resulting group alignment probability values ​​are too low. In this case, a three-stage ambiguity resolution process is performed to resolve the ambiguities.

[0071] Thus, process 300 can include an "ambiguity resolution" 314, and this is carried out by the ambiguity resolution unit 210. An overview of this process is provided in process 1200 ( Fig.12) provided. Three stages are performed, with one stage involving decoupling or removing (or ignoring) low probabilities of individual orientations. In another ambiguity stage, historical data is used to determine ambiguous individual orientations as to whether a previous perception or map lane edge of a previous road segment aligns with a current perception or map lane edge of a current road segment. In yet another historically based ambiguity stage, the system compares lane features of the current individual orientations with features of previous individual mappings.This can use a non-ambiguity counter to count the matches of individual ambiguity orientations with matching lane configurations and / or the individual ambiguity orientations with matching features. The details of the three stages are given below in processes 1300 (. Fig. 13), 1500 ( Fig. 15) and 1700 ( Fig. 17) provided. The result is the resolution of the ambiguities and a resulting most probable (or selected) group alignment, which is a resolved alignment to be used to determine and apply the lateral map deviation and final map deviation correction.

[0072] Separately, process 300 can include a “Performing a Longitudinal Object Alignment” 308 and includes a longitudinal alignment unit 208 that determines a longitudinal map deviation using the perception and map objects instead of the roadway edges. The details of the longitudinal alignment are given below in process 900 ( Fig. 9) provided.

[0073] With reference to Fig.Figure 5, as an exemplary implementation demonstrating the lateral and longitudinal deviation corrections, features an intersection or road segment 500 with a carrier vehicle 502 (or more precisely, a vehicle position) performing the analysis herein and crossing roads or streets R1, R2, R3, and R4. The perception data 501 has edges with thicker, solid lines, while the map data 503 has thinner, dashed lines, and this is maintained in all road figures herein unless otherwise noted. The perception data includes side or curb lines 506 and 514, center lines 504 and 512, stop sign 526, and traffic light 522. The map data 503 includes side or curb lines 508 and 516, center lines 510 and 518, stop sign 524, and traffic light 520. As shown, a longitudinal deviation correction b group,long (as b g,l(shown) determined that the map data should be shifted along the horizontal arrows extending from map centerlines 510 and 518 and perception centerlines 504 and 512, respectively. Similarly, horizontal arrows of the longitudinal map deviation extend from map curb lines 508 and 516 and perception curb lines 506 and 514, respectively. The same longitudinal deviation correction is shown for stop signs 524 and 526 and traffic lights 520 and 522. A lateral deviation correction b lat is determined as shown by the vertical arrows extending from map kerb lines 508 and 516 and to perception kerb lines 506 and 514, respectively.

[0074] Returning to step 312, the longitudinal map deviation is combined with the lateral map deviation to form a single final map deviation correction with lateral and longitudinal components. For clarity, the final alignments are hereby referred to as associations.

[0075] The process 300 may then include a “determination of a PM edge association” 316, performed by a PMEA narrower threshold unit 214 and by applying the final map deviation correction. This process recalculates the lateral alignment component, which minimizes the lateral distance between all edges and is calculated as in equations (1) and (2) above. However, while this process recalculates the lateral alignment, the features now include the distance between perception and map roadway edges, as well as the other roadway features mentioned above, to calculate the individual probabilities and then establish a final lateral map deviation value or final lateral map deviation correction.The use of spacing is referred to herein as the use of “narrow thresholds”, as mentioned above, and now spacing is included as a feature metric along with the road course, curvature, type and / or color.

[0076] To perform this alignment and map deviation correction, a small-radius search can be conducted around each of the perceived roadway edges to capture nearby map edges. A final feature match check for the features mentioned above, including distance, is then performed, and a probability value is calculated as described for lateral alignment. The distance feature is expected to be minimal at this point, as large errors should already have been removed, as mentioned above.

[0077] Process 300 can include an “output of a PM lane edge association” 318, whereby the associated lane edge data can now be used for other applications such as navigation and / or autonomous driving.

[0078] The process 300 may involve a “provision of a Z-1 edge association” 320, wherein the previous determinations are stored in memory, including any desired data identifying final associated lanes, features, probability values, alternative group orientations or assignments, individual orientations or assignments, etc., and are then provided to the ambiguity resolution unit 210 as the data of the previous road segment to perform the historical comparisons for the ambiguity levels if required.

[0079] With reference to Fig.4. A process 400 for performing a lateral perception map assignment according to at least one of the implementations described herein is now provided. Process 400 is described by operations 402-410, which are generally numbered evenly. The systems, road infrastructure, processes, devices, and vehicles from one of the Fig. 1-3 and Fig. Reference may be made to sections 5-18 where relevant.

[0080] Process 400 can include “Determine alternative lateral edge group orientations” 402 and is formed by the alternative generation unit 218. This generates every available group perception map orientation alternative. In this case, and by means of a form, each possible group orientation is analyzed. A group orientation is determined by maintaining or fixing the arrangement (including the orientation and position) of perception lane edges relative to each other (as in a perception image) and a global direction such as north, and by maintaining the position and arrangement of map lane edges in the map or map data used as for the perception lane edges. The perception image is then moved relative to the map to place at least one perception lane edge close to another map lane edge for each group orientation.

[0081] With reference to Fig.For example, in Figure 6-7, a road segment 600 has a pre-map deviation conversion device 602 that includes perception lane edges or lines p1, p2, p3, and p4 and map lane edges or lines m1, m2, m3, and m4, where edge m1 can be a side lane line, curb line, shoulder line, and so on. Map lanes m1, m2, m3, and m4 form lanes M100, M101, and M102, as shown. An alternative group alignment can include the following provisional pairs (p1, m1), (p2, m2), (p3, m3), and (p4, m4). Another alternative group orientation can have the following pairs: (p1, m2), (p2, m3), (p3, m4), and (p4, none (or not associated). The last alternative appears to be the closest alignment between perception and map edges, as shown, but is clearly incorrect. This is shown in Table 700. Fig.Figure 7 shows where the first column A is perception edges, the second column B is associated map edges before map deviation correction, the third column C is associated map edges after map deviation correction, and the fourth column D is ground truth associated map edges, with U standing for not associated in column B.

[0082] The formation of alternative group orientations can continue until all possible different group orientations have been produced, while maintaining the arrangement (or sequence or configuration) of the perceptual pathways. Thus, the system cannot pair p4 to m1 and p1 to m4 in a single group orientation, as this would be meaningless while maintaining the orientation of the perceptual image and the map.

[0083] With reference to Fig.Figure 8 shows, for another example of a lateral orientation, two different alternative perception group orientations (or setups) 802 and 806 relative to a single map 804. Perception group orientations 802 and 806 both show three lanes with PL1 as the center lane, and where the lanes are defined by lane lines or edges p1 to p4. The same perception picture is maintained from orientation 802 to 806. A vehicle 808 is in the right lane in group orientation 802, and a vehicle 810 is in the right lane in group orientation 806. Map 804 shows four lanes M100 to M103, defined by lane lines or edges m1 to m5.

[0084] Group orientation 802 has perceptual edges p1 to p4 aligned with map edges m1 to m4, and line m5 is not associated. Conversely, group orientation 806 has perceptual edges p1 to p4 aligned with map edges m2 to m5, and line m1 is not associated. Both of these alternatives are analyzed as follows.

[0085] An optional form allows preliminary alternatives to be discarded if they are obvious, such as for a five-lane highway where both the perceptual image and the map have edges for five lanes. Alternatives where only a single perceptual or map lane edge is aligned as a potential match can, for example, be discarded in this case.

[0086] Otherwise, the process 400 may involve “for each alternative group alignment, determining a probability value without regard to the distance” 404 and by the probability value unit 220. In particular, this process 404 may involve “using individual lane-to-lane confidences of each potential lane-to-lane match in a single alternative group alignment” 406, where a probability or probability value of a group alignment is as follows. L(i,j)=p1conf∗l(p1,m1)+p2conf∗l(p2,m2)+p3conf∗l(p3,m3)+p4conf∗l(p4,m4)p1conf+p1conf+p3conf+p4conf where (i, j) are the two perception and map edge numbers being compared, where L( ) is a group (or global) probability or probability value (or just a group probability value) of a global orientation, where each global orientation has one or more individual orientations, each with an individual orientation probability l(pi, mj) of an orientation between a single perception road edge pi and a single map road edge mj, where p iconf A confidence or weighting is applied to the individual alignment probabilities 1(pi, ji). The individual confidences p iconfis the probability of confidence that the perceived lane edge pi exists. This can be provided upstream as part of perception data and by a lane edge detector based on a neural network. To compute the individual alignment probabilities l(pi, mj), the process can involve “using multiple lane features at each individual alignment” 408. Thus, lane width matching robustness can be obtained to compensate for lane width mismatches by relaxing the feature boundary conditions by omitting the distance as a feature and as mentioned above.

[0087] In the present example, and for each alignment hypothesis, each individual alignment probability l(pi, mj) can be calculated by considering the characteristics of a roadway (or road line) under consideration, including course, curvature, roadway type, and color. The course characteristic can refer to a global course, such as north-south, or it can be a local course that is consistent across both perceptual and map data. The curvature characteristic is the linearity or radius of the roadway line, whether constant or varying through a road segment. The roadway type can include a single dashed line, a solid double line, a solid double line with a single left dashed line, a double line with a single right dashed line, a double line that is both dashed, and so on.The color feature can include white, yellow, and any other desired color. An individual alignment probability l(pi, mj) is calculated in one example by considering the probabilities of all features, but in other examples, it can be at least one of the features. Alternatively, other roadway features besides those listed here can also be used, or used instead. This calculation is repeated for each individual alignment probability l(pi, mj) in a single group alignment and repeated for each group alignment. Using the present example, the individual alignment probability can be calculated as: l(pi,mj) log(l_type(pi,mj)*l_color(pi,mj)*l_heading(pi,mj)*l_curvature(pi,mj)) where 1_[feature] is the individual feature match probability. The feature match probability can be calculated by several different probability algorithms. By way of example, the probability is a version of a weighted cost function that sums the average Mahalanobis distance, which accounts for the correlations between different attributes (e.g., course and curvature) and captures the similarity with respect to these specific properties of each feature, and which lies between all perception points in pi and corresponding nearest neighboring map points in mj. This is disclosed by U.S. Patent Publication No. 2024 / 0263965, published on August 8, 2024, which is incorporated herein in its entirety for all purposes.

[0088] Alternatively, feature matching probabilities between perceptual and map road edge features can be calculated using localization algorithms such as simultaneous localization and mapping (SLAM) and Monte Carlo localization. Alternatively, geospatial data association algorithms such as nearest neighbor search and data association filters (e.g., with common probabilistic data association filters) can be used. Another option is to use map matching techniques or algorithms that employ hidden Markov models (HMMs) or neural networks. Semantic segmentation can also be used, including deep learning models for semantic segmentation that can classify the environment into different categories (street, lane lines, signs, etc.).

[0089] Process 400 can involve “comparing a ratio of two probability values ​​with an ambiguity threshold” 410 and through the ratio unit 222. Once a probability value L(i, j) is calculated for each group alignment, a ratio is thus determined for the two highest probability values ​​to better ensure strong relative performance among multiple group alignments in addition to the isolated probabilities of each group alignment.

[0090] In this example, the ratio is then compared to an ambiguity threshold, although other criteria could be used instead. If the ratio is not above the ambiguity threshold, the alignments are considered ambiguous, and an ambiguous resolution process, here process 1200 ( Fig. 12), is done as in Operation 314 ( Fig. 3) applied.

[0091] Once the best orientation is determined, the process continues with operation 312 ( Fig. 3) continued to determine the final map deviation correction.

[0092] Returning to the example of Fig. Figure 6-7 shows a post-map deviation correction device 604, which corrects the perception and map roadway edges through individual deviation corrections. lat (numbered here b1 to b4) are moved, which result in a single group, lateral, map deviation correction b group (as b g (shown) can be combined. The "X"s indicate where it was found that the road edges do not match, such as p4 to m4 due to the road type and p1 to m2 due to the road curvature.

[0093] With reference to Fig.Section 9 now provides a process 900 for performing a longitudinal perception map assignment according to at least one of the implementations described herein. Process 900 is described by operations 902-910, which are generally numbered evenly. The systems, road infrastructure, processes, devices, and vehicles from one of the Fig. 1-8 and Fig. Reference can be made to 10-18 where relevant.

[0094] Non-roadway objects are used for the longitudinal alignment of roadway edges because aligning roadway edges longitudinally (parallel to the carrier vehicle) is particularly difficult. The error estimated from the alignment perception using corresponding map road objects can be used to correct the longitudinal deviation of longitudinal map roadway edges and to align perpendicular roadway edges, thus successfully assigning roadway edges in complex intersection scenarios.

[0095] Process 900 can include "Determining the Quantity of Longitudinal Objects" 902 and the object recognition unit 224. This process involves collecting data from identified non-roadway line objects for longitudinal alignment. This determines the position and identification of road objects such as signs, light poles, hydrants, traffic lights, guardrails, and so on, as mentioned above. A sufficient quantity of objects within a certain distance of the road segment being analyzed is used to provide a statistically significant dataset for performing longitudinal alignment. A minimum quantity of objects is determined through experimentation. If there are not enough objects, the longitudinal alignment process is omitted.

[0096] Referring again to the example from Fig. The 5 longitudinal objects are the traffic lights 522 and 520 and the stop signs 524 and 526.

[0097] With reference to Fig.As another longitudinal alignment example, Figure 10 features an intersection or road segment 1000 with roads R1, R2, R3, and R4, perceptual lane edges or an image 1002, and map lane line edges or a map 1004. A vehicle 1006 is located on road R2, which is oriented upwards, so that longitudinally it is now located up and down on the intersection 1000 and laterally to the right and left on the intersection 1000. One object for longitudinal alignment here is a perceptual traffic light 1014, which is to be aligned vertically with a map traffic light 1012. Another object for longitudinal alignment here is a perceptual stop sign 1010, which is to be aligned vertically with a map stop sign 1008. By maintaining the arrangement of the objects relative to the lane edges, the longitudinal alignment of the objects again leads to the longitudinal alignment of the perceptual and map lane edges 1002 and 1004.

[0098] Process 900 can include “Determining distances between P and M objects” 904 and by the distance unit 226, where the perception and map road objects (e.g., stop signs, traffic lights, bus stops, curbs, etc.) can be mapped based on object type, orientation, and loose position features. These mappings are used to calculate the longitudinal orientation component, which minimizes the longitudinal distance between all road objects, and the result is passed downstream. Thus, to determine distances between the perception and map objects, a longitudinal group correction b is used. group,long on the longitudinal deviation of all or several road objects, wherein b k is valued and whereby: bgroup,long*=argminbgroup,long(∑i=1Nobject‖(pi−(m^i+bgroup,long))‖22) where m̂ is the nearest map street object assigned to the perception object p i is and where equation (2) is the square of the L2 norm. Thus, process 900 can include a “calculating a longitudinal group correction” 906, which is performed by the group correction unit 228, the result being a longitudinal group correction b group,long is. The longitudinal group correction b group,long is indicated by the arrows at the intersection 1000 ( Fig. 10) shown.

[0099] With reference to Fig. 11 For yet another example, a road segment 1100 is the road segment 600 ( Fig.6) Similar and similar elements are similarly designated. However, here vehicles 1106 and 1108 are in the center lane instead of the right lane, and the longitudinal deviation applied to change from the 'pre-' fixture 1102 to the 'post-' fixture 1104 shows a much closer alignment between the perception and map stop signs 1110 and 1112 than before a longitudinal deviation b long (or only b1 as shown) is applied to the pre-arrangement 1102. This results in a much better longitudinal alignment (from right to left) of the perception lane edges p1 to p4 to the map lane edges m1 to m4, in addition to the lateral alignment shown. This is most evident when aligning the perception lane edge p1 and the map lane edge m1.

[0100] With reference to Fig.12. A process 1200 is provided for performing ambiguity resolution for a lateral perception map assignment according to at least one of the implementations described herein. Process 1200 is described by operations 1202-1212, which are generally numbered evenly. The systems, road facilities, processes, devices, and vehicles from one of the Fig. 1-11 and Fig. Reference may be made to sections 13-18 where relevant.

[0101] Returning now to the lateral alignment process, process 1200 can include the query "ambiguous alignment" 1202, which performs the same operation 310 ( Fig. 3) and Operation 410 ( Fig.4) is. In the event that the best alignment hypotheses are ambiguous, the system herein attempts to resolve the ambiguity using a multi-stage ambiguity process, with each stage employing a different ambiguity test. By one form, a three-stage robust ambiguity resolution algorithm is performed if ambiguity arises from the lateral alignment process 400 described above, and this can occur if the system detects two or more potential alignment candidate perception lane edges for a single map lane edge, or vice versa. By another form, the ambiguity is resolved by having at least one or two stages calculate and aggregate probability or individual alignment values ​​using a non-ambiguity counter. By an alternative form, all stages can use a non-ambiguity counter.In the following three-stage process, the second and third stages use the non-ambiguity counter as follows.

[0102] In particular, process 1200 may include a “decoupling operation” 1204, performed by the decoupling unit 232. This first stage involves decoupling (or filtering or ignoring) low-probability contributions and recalculating the alignment hypotheses to produce an updated group probability score or scores. Thus, this first stage limits the focus to high-probability individual alignments. This is performed by process 1300.

[0103] With reference to Fig.13. A process 1300 is provided for performing a low-probability ambiguity decoupling stage for a perception map assignment according to at least one of the implementations described herein. Process 1300 is described by operations 1302-1314, which are generally numbered evenly. The systems, road facilities, processes, devices, and vehicles from one of the Fig. 1-12 and Fig. Reference may be made to sections 14-18 where relevant.

[0104] Process 1300 may include “for each group alignment, comparing individual alignment probabilities with an individual alignment threshold” 1302. This individual alignment threshold is determined through experimentation and defines the difference between low and high (and / or non-low) individual alignment probabilities. These are the l(pi, mj) values ​​from equation (1) above.

[0105] Process 1300 may include a “removal of individual alignments below the individual alignment threshold” 1304, whereby these individual alignments with individual alignment probabilities l(pi, mj) below the threshold are now removed or ignored.

[0106] Process 1300 may include a “recalculating individual alignment probabilities and group alignment probability values” 1306. This involves performing equation (1) without the low-probability individual alignments to recalculate the group alignment L(i, j) of equation (1). This can be repeated for each group alignment or a specific subset thereof to be used.

[0107] Process 1300 may involve a “comparison of newly calculated group alignment probability values ​​with a group alignment threshold” 1308. This group alignment threshold for this ambiguity level is also determined by experimentation. It should be noted that the group alignment threshold may be used here in place of the ratio threshold, which is the same or a similar threshold to the ambiguity threshold of any process 310 ( Fig. 3), Process 410 ( Fig. 4) and process 1202 ( Fig. 12). Otherwise, both thresholds can be used, as mentioned here for this first level of ambiguity.

[0108] Process 1300 may involve “forming group alignment probability value ratios with the two highest probability values ​​of two group alignments” 1310, and the two highest probability values ​​are placed into a ratio as described above with the lateral alignment process 410. It is understood that alternatives could be used instead to form the ratios, as long as the relative performance among the group alignments can be evaluated.

[0109] As mentioned as a possible alternative, the ratios can be compared to a ratio threshold, and only those group alignments with a ratio higher than the ratio threshold are considered further. This can be in addition to, or instead of, the group alignment threshold mentioned above.

[0110] Process 1300 can involve “using the group alignment with the highest probability value of the highest ratio as the “resolved alignment” 1312 and only those that met the ratio threshold when used, as mentioned. This is done in process 1200 ( Fig. 12) is referred to as resolved 1214. When the ratio threshold is used here to approve the resolved alignment, this indicates that the resolved alignment between perception and map lanes is sufficiently aligned, as explained above.

[0111] Process 1300 can include a clause 1314 stating that "if no group alignment has a probability value above the group alignment threshold, then proceed to stage 2." Alternatively, this can refer to the ratio threshold, or both the group alignment and ratio thresholds can be met.

[0112] With reference to Fig.As an example, road segment 1400 is similar to road segment 800 and is designated similarly. However, here the "X" indicates those individual orientations that have been dropped or ignored. Thus, a small probability may exist resulting from the alignment of road edges p1 to m4 due to the road type and road edges p1 to m1 due to the road curvature. The individual orientation of p3 to m1 due to curvature has also been dropped. These individual orientations are then removed from equation (1) before a group orientation probability value is recalculated.

[0113] Returning to process 1200, which may involve "performing an ego-tracking" 1206 and is performed by an ego-tracking unit 234. This is provided by process 1500.

[0114] With reference to Fig. Section 15 provides a process 1500 for performing an ambiguity stage of ego lane tracking for a perception map mapping according to at least one of the implementations described herein. Process 1500 is described by operations 1502-1512, which are generally numbered evenly. The systems, road infrastructure, processes, devices, and vehicles from one of the Fig. 1-14 and Fig. Reference may be made to sections 16-18 where relevant.

[0115] Process 1500 can include “for each group alignment, obtaining data of the current ego lane alignments between perceptual and map lane edges of the current road segment that constitute the individual alignments” 1502. Specifically, the second stage involves tracing the lane edge association history and focuses on the previous (or previous timestamp) ego lane edges of a previous road segment and the configuration of the perceptual and / or map lane edges that constitute the previous associations. It should be noted that the timestamp may have any suitable format or any suitable numbering of maps and perceptual images of such maps and perceptual images in a sequence.Furthermore, "valuable" and "current" can refer to road segments that are relative to each other, preceding and following, and may or may not overlap, or the previous and current maps and perception images may be from the same road segment at different times (or with different timestamps). An ego lane edge refers to a right or left edge of a lane on which the carrier vehicle is traveling. In other words, the nearest lane edges to the vehicle. Alternative approaches could add more lane edges, such as the lane edges of the nearest left and right lanes to the carrier lane.

[0116] Process 1500 may involve “determining final prior individual assignments of a previous road segment that correspond to current ego lane orientations” 1504. Thus, the individual assignments refer to final determinations as opposed to the individual orientations of the current road segment, which are orientations yet to be analyzed. Alternatively, the individual assignments also refer to a correspondence (or now association) between a single perceptual lane edge and a single map lane edge, as with the individual orientations. The prior associations may also be referred to as prior timestamp ego associations.

[0117] Process 1500 may involve “comparing previous individual association patterns with the corresponding current ego-lane individual orientations, a group orientation at a time” 1506. This involves determining whether a pair of an individual previous association and an individual orientation matches the same perceptual and map lane edges. In a particular example, an ambiguous (or multiple) individual orientation is determined and has a calculated probability. The ambiguous (or multiple) individual orientation calculation is performed between a perceptual lane edge of the current road segment and a current map lane edge of the current road segment. The probability for the current orientation is then compared to the probability of the previous ego timestamp (individual) association.These comparisons are performed between alignment and association with the same identifier (for example, the same lane identifier, such as the same lane number, the same lane (edge ​​or ego) number, or a letter, etc.). It is understood that alternative processes can be used to compare the historical data instead.

[0118] With reference to Fig.16 For this example, the road facility 1600 has a prior association or road segment 1602 with a carrier vehicle 1606 on a four-lane prior road segment with map lanes M100 to M103, which are defined by prior map lane edges m1 to m5, where m2 and m3 are the map ego lanes for the vehicle 1606. Prior lane edges p1 and p2 (or only prior edges p1 and p2) of the carrier lane PL1 are the perception ego lane edges for the carrier vehicle 1606 on the prior road segment 1602.

[0119] Comparing the previous associations of perception lane edges p1 and p2 from lane PL1 with the current alignment candidates 1604 shows that respective matches occurring between perception lane edges p1 and p2, which were previously assigned to map lane edges m2 and m3, are adequate for the current alignment candidate map lane edges m2 and m3. Alternatively, when the perception lane edges p1 and p2, which were previously assigned to map lane edges m2 and m3, are compared with a different current alignment candidate using map lane edges m3 and m4, only one of the matches is adequate (p1-m3). These comparisons are shown below in Tables 1 and 2. Table 1 Previous association P1 → m2 P2 → m3 Table 2 map track candidate Number of ambiguities m2 and m3 (M101) 2 m3 and m4 (M102) 1

[0120] Process 1500 can include "for each comparison above a match threshold, increment a non-ambiguity counter by one" (1508), and this is performed for each individual ambiguity alignment in a group alignment (or, in other words, in the ambiguous group alignment formed for the ambiguity level). Thus, the comparison on the left with the carrier vehicle identified as 1608 for the comparison of m2 and m3 (M101) has two adequate probabilities and therefore receives two non-ambiguity counts, one per lane line or one per individual ambiguity alignment. The comparison on the right with the carrier vehicle identified as 1610 with m3 and m4 (M102) has only one individual alignment with an adequate probability and therefore receives only a single non-ambiguity count, as shown in Table 2.This shows that the road edge assignment tends to remain the same over time, as shown in the arrangement comparing vehicle 1608 and the previous road segment 1602.

[0121] Process 1500 can include a "if a group alignment has the highest non-ambiguity count, then use that group alignment as the resolved alignment" 1510. Thus, in this example, Fig. 16. Using the group alignment with the higher non-ambiguity count for m2 and m3 (M101), the lateral map deviation correction is determined and provided to complete the map deviation correction, as described above with process 300. The resolved alignment is used in process 1200 ( Fig. 12) is referred to as dissolved in 1216.

[0122] Otherwise, process 1500 may include a “if all group alignments have the same non-ambiguity count, then proceed to level 3” 1512.

[0123] The process 1200 can then include a “determination of ego road feature consistency” 1208 and this can be carried out by the ego road feature consistency unit 236.

[0124] With reference to Fig. 17. A process 1700 is provided for performing an ambiguity level of ego-road feature consistency for a perception map mapping according to at least one of the implementations described herein. Process 1700 is described by operations 1702-1710, which are generally numbered evenly. The systems, road facilities, processes, devices, and vehicles from one of the Fig. 1-16 and Fig. Reference may be made to paragraph 18 where relevant.

[0125] Process 1700 may involve “for each group alignment, for each individual probability in the group alignment, and for each feature probability in an individual alignment, determining a feature probability between features of a previous individual association and a current individual alignment” 1702. This may involve establishing the same ambiguous individual alignment between previous features of previous perceptual lane edges of the previous road segment and current features of current map lane edges of the current road segment. Here, the historical data being analyzed or monitored is the consistency of the edge features themselves over time. Equation (2) above can be used to determine the feature match probabilities, although other alternative algorithms could be used instead.

[0126] Process 1700 may involve “comparing feature probabilities with at least one feature threshold” 1704. The feature probabilities are then compared with feature thresholds to determine whether the feature matches are sufficiently close. Each different feature type may have its own threshold, determined through experimentation. If the previous features change significantly from a previous time (k) to the current time (k + 1), this indicates a change in orientation from the previous road segment to the current road segment that should be considered. Otherwise, if the features are consistent over time, it should be more acceptable to continue using the same orientations or mappings.

[0127] Process 1700 can include “incrementing a non-ambiguity counter by one for each individual alignment that satisfies the feature threshold for all or several features” 1706. Thus, an individual alignment is satisfactory by an exemplary form if an ambiguous individual alignment has all probabilities of all feature matches between the previous perceptual lane edges and the current map lane edges higher than their feature thresholds. In this case, where the perceptual lane edges and map lane edges are sufficiently similar or the same, the non-ambiguity count can be incremented by one for each such ambiguous individual alignment in a group alignment.

[0128] Process 1700 can involve “using the group orientation with the highest non-ambiguity count as the resolved orientation” 1708, and this can be determined by the candidate selection unit 238. Thus, the map roadway edge with the highest ambiguity count is chosen to have the most consistent roadway features. This corresponds to process 1206 and the resolved orientation designation 1218 ( Fig. 12).

[0129] With reference to Fig.As an example of feature consistency, Figure 18 shows that a road facility 1800 has a previous road segment 1802 at time k adjacent to a subsequent, current road segment 1804 at time k + 1. It is shown that perception and map lane edges intersect from one road segment 1802 to the next road segment 1804. The perception lane edges include lane lines p1, p2, and p3, which define previous perception lanes PL1 and PL2, while the map lane edges include lane lines m100, m100R / m100L, and m101R, which define current lanes M100 and M101, as indicated by the lane line names, where R is for right and L is for left. A carrier vehicle 1806 is located in the center lane PL1.As shown, the perceived road edge p1 curves inwards or downwards, indicating that there may have been a perceived reduction from two lanes to one, and a change in features from the previous to the current road segment. In contrast, the map road edges remain straight in both segments, causing a map error. The edge associations and non-ambiguity counts are shown below in Tables 3 and 4.

[0130] In this case, it is found that p3 has consistent features in the roadway M101 at time k + 1, while p1, due to the changing curvature feature, has no consistent features and a low feature match probability that does not meet the feature match thresholds. In this case, both edges of the M101 roadway can receive an incremented non-ambiguity count for the map edges m101R and m100R / m101L, although the p2 roadway line can stop and p1 actually becomes the roadway line for roadway M101. Otherwise, edge p1 does not receive an increment of the non-ambiguity count for roadway M100. Thus, the group alignments that match p1 to m100L do not have the highest non-ambiguity count. These group alignments, where p3 is paired with m101R without pairing p1 with the map edge m100L, have the higher non-ambiguity number. Table 3 Perceptual margins at k assigned map margins p1 M100L p2 M100R / M101L P3 M101R Table 4 map track candidate Number of ambiguities M100 2 M101 1->2

[0131] Process 1700 can include operation 1710, "if no feature match meets the feature thresholds, then use the group alignment with the highest probability value." This corresponds to operation 1210 of process 1200 ( Fig. 12) Therefore, if no individual alignment has all feature matches with probabilities that meet the feature thresholds, then use the group alignment with the highest probability value. This could be from the original group alignments being analyzed, updated ambiguity group alignments formed from analyzed individual ambiguity group alignments, or a combination of both.

[0132] Process 1200 then performs a "resolved assignment provision" 1212, which returns process 1200 to operation 312.

Claims

[1] Procedure (300, 400), comprising: Receiving (302) map data (204) comprising a plurality of map lane edges (m1, m2, m3, m4) of a road segment (600); Receiving (304) perception data (202) comprising a plurality of perception roadway edges (p1, p2, p3, p4) of the road segment (600); Form (402) at least one orientation which has at least one of the map roadway edges (m1, m2, m3, m4) and at least one of the perception roadway edges (p1, p2, p3, p4); Determine (404), by at least one processor (142), a probability value of each of the orientations, indicating whether at least one of the map road edges (m1, m2, m3, m4) of the orientation should be the same road line as one of the perception road edges (p1, p2, p3, p4) of the orientation; Resolving (314) an ambiguity by at least one processor (142) when no probability value of the at least one orientation satisfies at least one orientation criterion, and comprising determining a resolved orientation between the at least one perception roadway edge (p1, p2, p3, p4) and the at least one map roadway edge (m1, m2, m3, m4), comprising applying several ambiguity levels, each ambiguity level having a different ambiguity test, and comprising at least two of the ambiguity levels being arranged to determine whether a non-ambiguity count should be incremented depending on a size of probabilities with respect to the at least one orientation, and Determine, by at least one processor (142), that the resolved orientation exists if the orientation between the at least one perceptual lane edge and the at least one map lane edge has a highest non-ambiguity count, wherein the orientation is a group orientation with one or more individual orientations, each individual orientation having a perceptual lane edge (p1, p2, p3, p4) paired with a map lane edge (m1, m2, m3, m4), and wherein the multiple ambiguity levels successively comprise a first low-probability decoupling level arranged to remove low-probability individual orientations, a second ego lane tracking level taking into account previous individual associations between map and perceptual lane edges, and a third ego lane feature consistency level.The road surface characteristics of the individual orientations are compared with the road surface characteristics of the previous individual associations. [2] Method (300, 400) according to claim 1, wherein the alignment is a group alignment with one or more individual alignments, wherein each individual alignment has a perception roadway edge (p1, p2, p3, p4) paired with a map roadway edge (m1, m2, m3, m4), and wherein one of the ambiguity levels is a low-probability decoupling level that ignores individual alignments with an individual probability below an ambiguity threshold of the individual alignment to calculate the probability value of a group alignment. [3] Method (300, 400) according to claim 1, wherein the road segment (600) is a current segment and wherein one of the ambiguity levels is an ego lane tracking level comprising: generating ambiguous group orientations, each with one or more ambiguous individual orientations; and determining whether a current ego-individual orientation of a map and perception lane edge is sufficiently similar to a previous timestamp ego map and ego perception lane edge of a previous ego association and is at least partially based on an identifier that is temporally consistent in the current ego-individual orientation and the previous ego association. [4] Method (300, 400) according to claim 3, wherein the non-ambiguity count is incremented for each of the ego-individual orientations with a sufficient match between a current map and perception road edge with a previous map and perception road edge association. [5] Method (300, 400) according to claim 1, wherein the road segment (600) is a current segment and wherein one of the ambiguity levels is an ego road feature consistency level that determines probabilities of feature matches between road line features of a current map or perception road edge of the current timestamp with a corresponding previous map or perception road edge, such that each pair of current and previous map and perception road edges forms an individual ambiguity orientation, and wherein the determination of probabilities of feature matches for each individual ambiguity orientation of an ambiguity group orientation is repeated with multiple individual ambiguity orientations. [6] Method (300, 400) according to claim 5, wherein the non-ambiguity count is incremented if an individual ambiguity alignment has several features, each with a feature match probability that is above a feature match threshold. [7] Method (300, 400) according to claim 5, wherein the features include at least one of the following: road type, road color, road curvature and road course. [8] Method (300, 400) according to claim 5, wherein the distance between perception and map roadway edges is not one of the features. [9] System (101), comprising: Memory (144); and Processor circuits comprising at least one processor (142) which is communicatively coupled to the memory (144) and arranged to operate by: Receiving (302) map data (204) comprising a plurality of map lane edges (m1, m2, m3, m4) of a road segment (600), Receiving (304) perception data (202) comprising a plurality of perception roadway edges (p1, p2, p3, p4) of the road segment (600), Form (402) at least one orientation which has at least one of the map roadway edges (m1, m2, m3, m4) and at least one of the perception roadway edges (p1, p2, p3, p4); Determine (404) a probability value of each of the orientations that indicates whether at least one of the map road edges (m1, m2, m3, m4) of the orientation should be the same road line as one of the perception road edges (p1, p2, p3, p4) of the orientation, Resolving (314) an ambiguity when no probability value of the at least one orientation satisfies at least one orientation criterion, and comprising determining a resolved orientation between the at least one perception roadway edge (p1, p2, p3, p4) and the at least one map roadway edge (m1, m2, m3, m4), comprising applying several levels of ambiguity, each level of ambiguity having a different ambiguity test, and comprising at least two of the levels of ambiguity being arranged to determine whether a non-ambiguity count should be incremented depending on a magnitude of probabilities with respect to the at least one orientation, and Determine that the resolved orientation exists if the orientation between the at least one perception roadway edge (p1, p2, p3, p4) and the at least one map roadway edge (m1, m2, m3, m4) has a highest non-ambiguity count, wherein the alignment is a group alignment with one or more individual alignments, wherein each individual alignment has a perceptual lane edge (p1, p2, p3, p4) paired with a map lane edge (m1, m2, m3, m4), and wherein the multiple ambiguity levels include, in sequence, a first low-probability decoupling level arranged to remove low-probability individual alignments, a second ego lane tracking level which considers previous individual associations between map and perceptual lane edges, and a third ego lane feature consistency level which matches lane features of the individual alignments with lane features of the previous individual associations.

Images