Closed lane detection

Abstract

The technology disclosed in this application is used to detect whether a lane of a road is open or closed. Detecting that a lane is closed can include detecting an object within or near the lane, which can include determining a size, location, and / or category associated with the object, and expanding the size associated with the object. If the expanded object detection result is less than a threshold distance from another object detection result, the expanded object detection result, or a lane extent, then the lane can be indicated as closed. The technology can additionally or alternatively include determining an alternate lane shape based at least in part on one or more object detection results and / or determining that one or more lanes are closed and / or uploading the lane closure and / or the alternate lane shape to a central database for retrieval by other computing devices and / or propagation to other computing devices.

Description

[0001] This case is a divisional application of the invention patent application filed on May 20, 2020, with application number 202080051044.0 and invention title "Closed Lane Detection".

[0002] Cross-references to related applications

[0003] This PCT international application claims priority to U.S. Patent Application No. 16 / 417441, filed May 20, 2019, which is incorporated herein by reference in its entirety. Background Technology

[0004] Objects such as traffic cones, road signs, directional signs, and traffic lights (referred to as "safety objects" in this application) may be placed on or near a road to indicate that a lane is closed. For example, a lane may be closed due to construction, a vehicle collision, vehicle repair, or other reasons. In some cases, the objects placed on or near the road may not completely obstruct the road (e.g., a small sign may be placed in the middle of the lane, or one or more cones may be placed in the lane). Even if the lane is not completely obstructed, people will understand that the lane is completely closed. However, an autonomous vehicle simply programmed to avoid collisions with these objects may not recognize or notice the lane closure and may simply avoid the safety objects. For example, if the spacing between traffic cones is large enough that an autonomous vehicle can pass through them without a collision, the autonomous vehicle may enter or merge into the closed lane. Attached Figure Description

[0005] The accompanying drawings provide a detailed description. The leftmost numeral(s) on the figures identify the first figure in which that numeral appears. Identical reference numerals in different figures indicate similar or identical parts.

[0006] Figure 1 An example scenario depicting an autonomous vehicle and a closed lane is shown, in which the autonomous vehicle can detect lane closures.

[0007] Figures 2A to 2E A flowchart of an example process for detecting closed lanes is shown.

[0008] Figure 3 The image shows a bird's-eye view of a vehicle approaching a lane-changing narrowing point, which could be an example scenario where all lanes in the vehicle's direction of travel are closed.

[0009] Figure 4 A bird's-eye view shows one or more alternative lane shapes that a vehicle can determine in association with closed lane status and / or open lane status.

[0010] Figure 5A block diagram of an example system for detecting closed lanes and / or determining the shape of alternative lanes is shown. Detailed Implementation

[0011] The technical solutions discussed in this application involve detecting whether a road lane is open (e.g., available for vehicle operation) or closed and / or otherwise unavailable. The technical solutions discussed herein can improve passenger safety in autonomous vehicles employing the technologies discussed in this application. Additionally, these technical solutions can also improve the safety of construction workers, individuals assisting with vehicle maintenance, and the structural integrity of the autonomous vehicle itself. Furthermore, these technical solutions can improve the efficiency of autonomous vehicles in performing tasks such as transporting passengers and / or goods, and surveying areas. This technical solution can reduce human intervention by providing guidance to autonomous vehicles that have navigated to an area including at least one (partially or completely) closed lane. This technical solution can thereby reduce network usage for networks used to receive remote operation requests from one or more autonomous vehicles in a fleet of autonomous vehicles.

[0012] According to the technical solutions discussed in this application, autonomous vehicles can be configured with various components to detect safe objects and / or signs indicating lane closures and avoid driving in closed lanes. The technical solutions discussed in this application may include analyzing the closure of one or more lanes of a road (e.g., the current lane, one or more adjacent lanes, other lanes). Safe objects may include, for example, traffic cones, road signs, flashing traffic lights, construction vehicles / law enforcement vehicles / trailers / electronic toll collection vehicles, construction workers / law enforcement officers / trailer workers / electronic toll collection vehicle personnel, etc., and safety signs may include signs indicating construction zones, signs indicating road construction ahead, signs indicating lane merging, specific color signs indicating deviations from standard road structures (e.g., red-orange and / or orange signs in the United States), traffic signals (e.g., stop lights), flashing traffic lights, and / or similar signs. In some examples, the autonomous vehicle may receive sensor data from one or more sensors of the autonomous vehicle and determine object detection results associated with the environment surrounding the autonomous vehicle, at least in part, based on the sensor data.

[0013] For example, an autonomous vehicle may determine a region of interest (ROI) associated with a portion of an image associated with an object (e.g., bounding boxes and / or instance segmentation), a subset of depth sensors (e.g., LiDAR, radar, time-of-flight (ToF), depth cameras) of points associated with the object, and / or a 3D ROI associated with the object (e.g., a 3D shape associated with the image and / or a 3D shape that constrains the depth sensor points). In some examples, object detection results may indicate the location of the object within the environment surrounding the autonomous vehicle. In other or alternative embodiments, object detection results may indicate size (e.g., size in two or more dimensions, such as size in 3D space), shape (e.g., geometry, curve shape, boundary shape, instance segmentation of sensor data), category (e.g., "pedestrian," "vehicle," "construction worker," "construction vehicle," "safety object," "traffic cone," "traffic light," "safety sign," "road construction ahead sign," "lane merging sign"), and / or the lane associated with the object (e.g., the autonomous vehicle's current lane, the left adjacent lane, the right adjacent lane, or any other suitable indication identifying a specific lane). In some examples, lane indication can identify lanes stored in a map accessible to autonomous vehicles (e.g., this lane is stored in the memory of an autonomous vehicle that can be retrieved from a distributed computing system via a network).

[0014] This technical solution may include modifying object detection results, for example by at least partially expanding (i.e., increasing) the size indicated by the object detection results based on a scalar. In some examples, the expansion may be at least partially based on the direction associated with the road. For example, an autonomous vehicle may determine tangents associated with road curvature and / or lane curvature. The autonomous vehicle may increase the size associated with the object detection results in one or more dimensions associated with the tangent (e.g., it may bulge along the same direction as the tangent). The size of the object may be increased in any other or alternative direction (e.g., a direction orthogonal to or inclined to the tangent).

[0015] In some examples, the scalars that increase in size may be based at least in part on the width of the road or one side of it (e.g., one side of the road associated with traffic flow), lane width, number of lanes on the road or one side of it, speed at which the autonomous vehicle is moving, number of open lanes, number of closed lanes, category associated with object detection results (e.g., a scalar associated with “vehicle” may be smaller than a scalar associated with “traffic cone”) and / or closure type (e.g., road construction; vehicle repair; other construction, such as building construction; road cleaning or clearing, such as clearing snow and dirt).

[0016] In other or alternative embodiments, this technical solution may include: determining the distance between object detection results in the analyzed lane and / or the distance between an object detection result and the lane range of the analyzed lane. In some examples, this technical solution may include determining the distance between an object detection result and the nearest one of the lane range (e.g., a lane edge, such as a road edge or lane marking) or the next object detection result. In some examples, the distance between each object detection result may be calculated. In such examples, this technical solution may include determining the maximum distance among two or more distances between the object detection result and / or the lane range.

[0017] If the distance is less than a distance threshold, the technical solution may include setting a state to indicate that the analyzed lane is closed. In some examples, the aforementioned state may be a state of a state machine, a flag in a register, and / or any other persistent state. In some examples, the state may be associated with a portion of the analyzed lane and / or road, for example, by identifying a portion of a map associated with the state. In some examples, the autonomous vehicle may determine its trajectory for control based at least in part on the state. For example, if the state indicates that the current lane the vehicle is traveling in is closed ahead, the autonomous vehicle may change lanes to lanes that are identified as open (by the autonomous vehicle). In some examples, the distance threshold may be based at least in part on the autonomous vehicle's speed, lane width, road width, the category associated with one or more object detection results, and / or the proximity of pedestrians to safety signs (e.g., this might indicate a construction worker holding a construction sign).

[0018] If the distance meets or exceeds a distance threshold, the technical solution may include identifying the lane as open. However, in additional or alternative examples, the technical solution may include determining whether the current state identifies the lane as closed and / or whether sensor data indicates the absence of a safe object. In some examples, if the state identifies the lane as closed, the technical solution may include maintaining the lane as closed. In additional or alternative examples, if sensor data indicates the absence of a safe object (e.g., no object detection result matching the safe object category is generated) for a duration and / or the operating distance reaches or exceeds a threshold time period and / or threshold distance, the state may revert to an open lane indication. In other words, even if a gap is opened for the autonomous vehicle to pass through, the autonomous vehicle may maintain a closed lane state until the last time the autonomous vehicle detected a safe object and / or safety sign. In additional or alternative examples, autonomous vehicles may transition from a closed lane state to an open lane state based at least in part on positive indications that the lane has been reopened (e.g., a “road construction closed” sign, a “slow down” sign, a green light, multiple lights indicating the reopening of the lane and / or lane merging, or a reversal signal from “stop” to “slow down”).

[0019] In some examples, these techniques can fit shapes to one or more object detection results (e.g., polynomials and / or another curve, polygon) and / or expand the object detection results in additional or alternative examples, and associate states with shapes. For example, safety objects can be specified with shapes such as traffic cones (e.g., merging traffic cones, reversing traffic cones, single-lane two-way traffic cones). Autonomous vehicles can determine trajectories for controlling the autonomous vehicle based at least in part on the shape.

[0020] In some examples, autonomous vehicles can transmit an indication to a remote computing device that at least a portion of the lanes on a road are closed. The remote computing device can then send this indication to one or more vehicles in a fleet, thereby updating a stored map to indicate that the lanes associated with that portion of the road are closed. For example, such a remote computing device could send instructions to store lane closure statuses associated with a portion of a map stored by the vehicles, where that portion of the map is linked to a section of the road.

[0021] Example Scenario

[0022] Figure 1An example scenario 100 is shown, including an autonomous vehicle 102, which can be configured to detect lane closures (inoperable on) or lane openings (operable on). In some cases, the autonomous vehicle 102 may be an autonomous vehicle configured to operate according to a Level 5 category issued by the National Highway Traffic Safety Administration (NHTSA), which describes a vehicle capable of performing all safety-critical functions throughout the journey without the driver (or occupant) taking control of the autonomous vehicle at any time. However, in other examples, the autonomous vehicle 102 may be a fully autonomous vehicle or a partially autonomous vehicle with any other level or category. It is foreseeable that the techniques discussed in this specification are not limited to robot control, such as for autonomous vehicles. For example, the techniques discussed in this specification can be applied to air navigation / space navigation / water navigation / underground navigation, manufacturing, augmented reality, etc. Furthermore, even if the autonomous vehicle 102 is depicted as a land vehicle, it could also be a spacecraft, aircraft, ship, submersible, and / or the like.

[0023] According to the technical solutions discussed herein, the autonomous vehicle 102 can receive sensor data from multiple sensors 104 of the autonomous vehicle 102. For example, the multiple sensors 104 may include position sensors (e.g., global positioning system (GPS) sensors), inertial sensors (e.g., accelerometer sensors, gyroscope sensors, etc.), magnetic field sensors (e.g., compasses), orientation / velocity / accelerometer sensors (e.g., speedometers, drive system sensors), depth position sensors (e.g., lidar sensors, radar sensors, sonar sensors, time-of-flight (ToF) cameras, depth cameras), image sensors (e.g., visible spectrum cameras, depth cameras, infrared cameras), audio sensors (e.g., microphones), and / or environmental sensors (e.g., barometers, hygrometers, etc.).

[0024] Multiple sensors 104 can generate sensor data, which can be received by multiple computing devices 106 and / or stored in a memory 108 (e.g., a cache) associated with the autonomous vehicle 102. However, in other examples, some or all of the multiple sensors 104 and / or multiple computing devices 106 may be arranged separately from and / or remotely from the autonomous vehicle 102, and one or more remote computing devices may transmit data capture, processing, command, and / or control data to / from the autonomous vehicle 102 via wired and / or wireless networks.

[0025] In some examples, the computing device(s) 106 may include: a positioning unit 110, a planner 112, and / or a perception engine 114. Typically, the positioning unit 110 can determine the position and / or orientation of the autonomous vehicle 102 within a map 116, which may be stored in a memory 108; the perception engine 114 can determine what is in the environment surrounding the autonomous vehicle 102; and the planner 112 can determine how to operate the autonomous vehicle 102 based on the information about the environment received from the perception engine 114.

[0026] Map 116 may include (multiple) global maps, (multiple) drivable road surface maps, and / or (multiple) local maps. The global map may include: roads, feature points (e.g., buildings, commercial locations, parks), and feature points detectable in different sensor modalities that can be used to locate the autonomous vehicle. Drivable road surfaces may include road segments, lanes, traffic signal locations, lane restrictions (e.g., turning only, merging, yielding), etc. Local maps may include finer-grained details, and in some examples, may be at least partially based on sensor data and / or may be generated by the autonomous vehicle. For example, a local map may include features of the environment surrounding the autonomous vehicle 102, such as road slope; fire hydrants, traffic signs; traffic lights; trees, buildings, fixed seats, bus stops, and the location and / or size of any of the aforementioned objects. Determining the location of the autonomous vehicle 102 within the global map may include: determining the location of the autonomous vehicle 102 within the road (e.g., lane markings and / or orientation within one or more lanes of the road). Determining the position of the autonomous vehicle 102 within a local map may include: determining the relative distance between the autonomous vehicle 102 and various features identified in the local map, or other methods described in this application (such as...). Figure 5 (as described above). In some examples, the positioning unit 110 can output the orientation of the autonomous vehicle 102 in the map 116. In additional or alternative examples, the positioning unit 110 can determine the orientation of the autonomous vehicle 102 (e.g., yaw angle, pitch angle, roll angle), which may be related to the orientation and / or coordinates of the global map and / or local map.

[0027] Perception engine 114 can receive sensor data from sensor(s) ...)(s)(s)(s)(s)(s)(s)(s)(s)(s))(s)(s)(s)(s)(s)(s)(s)(s)(s))(s)(s)(s)(s)(s)(s)(s))(s)(s)(s)(s)(s

[0028] In the illustrated example scenario 100, as the autonomous vehicle 102 approaches a set of traffic cones 120, the autonomous vehicle 102 can receive sensor data from one or more of the multiple sensors 104. The traffic cones 120 may be an example of a safety object associated with lane closure. The perception engine 114 may include one or more ML models for detecting and / or classifying the multiple objects in the environment surrounding the autonomous vehicle 102, at least in part, based on the sensor data. For example, the autonomous vehicle 102 may receive image and / or point cloud data (e.g., data from lidar, radar, sonar), and the autonomous vehicle 102 may determine that it is associated with one or more safety objects (e.g., by determining that the object detection results are related to a safety category).

[0029] Perception engine 114 can determine the orientation, orientation, size, shape, and / or trajectory associated with detected objects (e.g., the previous orientation, current position, predicted position, velocity, and / or acceleration of the detected object). In some examples, perception engine 114 can output object detection results associated with the location of objects in identification map 116, the orientation of the objects relative to autonomous vehicle 102, and / or the volume occupied by the objects. For example, perception data map 126 can represent a combination of a portion of map 116 and data associated with the set of object detection results 128 associated with traffic cones 120 output by perception engine 114.

[0030] In some examples, if at least one of the object detection results generated by perception engine 114 indicates a category associated with a safe object (i.e., a "safe category"), perception engine 114 may trigger lane closure analysis. In additional or alternative examples, autonomous vehicle 102 may analyze at least the current lane 122; any adjacent lane(s) such as adjacent lane 124; and / or any other lane to determine whether the lane is open or closed. For example, perception engine 114 may perform the lane closure analysis described herein continuously or periodically, regardless of whether a safe object has been detected, and / or, if a safe object is detected, perception engine 114 may perform periodic lane analysis in addition to triggering lane analysis.

[0031] In some examples, perception engine 114 may store and / or maintain the state of one or more lanes of the road detected by perception engine 114 (in some examples, one or more lanes may be indicated by map 116). For example, perception engine 114 may store a state tracker 118 in memory 108, which includes the state associated with each lane analyzed by perception engine 114. In some examples, the state may identify whether the analyzed lane is closed or open and / or identify that it is located in at least a portion of map 116 associated with the state (e.g., lane identifier). In some examples, state tracker 118 may include, in additional or alternative examples, an indication of the presence of a lane adjacent to the current lane and / or whether the adjacent lanes are associated with the same or different directions of travel.

[0032] Figure 1 An example of lane state 130 is shown, which can be stored and / or output by a perception engine 114 associated with map 116 and at least in part based on the lane analysis techniques described herein. Lane state 130 may include open lane state 132 and closed lane state 134. The techniques discussed herein can receive object detection results output by perception engine 114 and can determine lane state 130 at least in part based on object detection results and the techniques discussed herein. For example, these techniques can determine to associate open lane state 132 with the current lane 122 up to the point location associated with the most recent object detection result, after which these techniques may include associating closed lane state 134 with the current lane 122. In additional or alternative examples, the techniques of this application can determine to associate adjacent lanes 124 with the same direction of travel and / or associate open lane state 132 with adjacent lanes 124. In additional or alternative examples, if the lane associated with a different direction of travel is different from the current lane 122, these technical solutions may include associating the closed lane state 134 and / or the state of the different direction of travel with that lane. Of course, as described in detail herein, such determination is not limited to predefined (or pre-mapped) lanes and can be extended to changes in drivable road surfaces.

[0033] Once the perception engine 114 generates perception data, which may include multiple states identified by the state tracker 118 and / or any other form indicating lane closure / openness, the perception engine 114 may provide the perception data to the planner 112.

[0034] Planner 112 can use perception data, including lane closure / open states discussed in this specification, to determine one or more trajectories, thereby controlling the path or route traversed by autonomous vehicle 102 and / or otherwise controlling the operation of autonomous vehicle 102, though any such operation can be performed in various other components. For example, planner 112 can determine the route of autonomous vehicle 102 from a first position to a second position; based on rolling time-domain techniques (e.g., 1 microsecond, half a second, every 10 seconds, etc.) and at least based on portions of lane state 130 (which may be associated with map 116 and / or state tracker 118) to traverse the route (e.g., to avoid any detected objects and / or avoid operation in closed lanes); and select one of the potential trajectories as trajectory 136 of autonomous vehicle 102, which can be used to generate drive control signals that can be transmitted to the drive components of autonomous vehicle 102. Figure 1 An example of such a trajectory 136 is shown, represented by arrows indicating heading, speed, and / or acceleration, although the trajectory itself may include instructions for a controller that can also actuate the drive system of the autonomous vehicle 102. In the example depicted in the figure, the planner may generate and / or select trajectory 136 based at least in part on the closed lane state 134 associated with the current lane 122. Trajectory 136 may include instructions for actuating the drive system to merge the autonomous vehicle 102 into an adjacent lane 124 (which may be associated with the same direction of travel and open lane state 132).

[0035] Example process

[0036] Figures 2A to 2E An example process 200 for analyzing lanes to determine lane status is shown. In some examples, the example process 200 may be performed by multiple components of the perception engine 114. Although the example process 200 is depicted as including multiple operations and / or an obvious operational flow, the example process 200 may include more or fewer operations, repetitive operations, and / or one or more operations may be performed serially, in parallel, and / or in a different order than shown in the figure.

[0037] Go to Figure 2AIn operation 202, example process 200 may include receiving object detection results indicating the position, shape, and / or size of objects in an environment, according to any of the techniques discussed in this application specification. In some examples, the object detection results may identify, in additional or alternative examples, the category associated with the object, the trajectory and / or lane associated with the object, or other parts of the map associated with the object. In some cases, one or more ML models may output object detection results (e.g., the category may be machine learning-based). In some examples, the object detection results may identify the two-dimensional and / or three-dimensional space associated with the object (e.g., the volume and boundaries of the object). Figure 2A A bird's-eye view of an example road 204 comprising two lanes is depicted, with one lane associated with three object detection results representing objects that may have been detected by an autonomous vehicle. For example, object detection result 206 could be one of the three object detection results.

[0038] In operation 208, example procedure 200 may include modifying the object detection result according to any techniques discussed in some examples herein. In some examples, modifying the object detection result may include increasing (“expanding”) the size associated with the object detection result for objects corresponding to a safety category. This size may be increased by a scalar. The scalar may be based at least in part on the width of the road or one side of it (e.g., the side of the road associated with traffic flow) (e.g., a larger width resulting from a larger scalar), lane width (e.g., a larger width resulting from a larger scalar), the number of lanes on the road or one side of it (e.g., an increased number result resulting from an increased scalar), the speed at which the autonomous vehicle is moving (e.g., an increased speed result resulting from an increased scalar), the number of open lanes (e.g., a larger scalar resulting from an increase in the number of available open lanes), the number of closed lanes (e.g., a reduced scaling result resulting from an increase in the number of closed lanes), the category associated with the object detection result (e.g., a scalar associated with “vehicle” may be smaller than a scalar associated with “traffic cone”), and / or the type of closure (e.g., road construction; vehicle repair; other construction such as building construction; e.g., road cleaning or clearing for snow and mud—different base scalars may be associated with any of these types). In some cases, operation 208 may include increasing the size based on one or more scalars. For example, the scalar may be associated with each dimension associated with the object. When the object detection results define the object in Euclidean space, the first scalar can be associated with the object's x-axis dimension, the second scalar with the object's y-axis dimension, and the third scalar with the object's z-axis dimension. For example, a scalar can be a vector with components in each dimension of the space defining the object.

[0039] In additional or alternative examples, the scalar may include a direction (e.g., the tangent of a curve of the lane and / or road) that determines the direction of travel of the lane and / or road. In some cases, the size of an object may be increased at least in part based on the direction / tangent. For example, an object may be compressed along this direction and at least in part based on the scalar.

[0040] In the example shown in the figure, example road 204 may have a direction parallel to the y-axis. Operation 208 may include increasing the size of the object detection result primarily along the y-axis, at least in part based on the direction of example road 204. Since operation 208 may be based on one or more scalars, in additional or alternative examples, the object detection result may be increased along the x-axis and / or a z-axis (not shown) (or any other axis in the dimension and / or coordinate space, such as a sphere or cylinder, in additional or alternative examples). For example, expanding the object detection result 210 is an example of increasing the size of object detection result 206 at least in part based on the direction associated with road 204 and / or lane.

[0041] In operation 212, according to any of the technical solutions discussed in this application specification, example process 200 may include: determining the distance between the extended object detection result and another object detection result, another extended object detection result and / or the range of the lane and / or road. Figure 2A An example of calculating distances along the x-axis is depicted, but it should be understood that distances(s) can be calculated in one or more other dimensions. For example, distance 214 represents the distance from the edge of extended object detection result 210 to the road extent. Distance 216 represents the distance from the edge of extended object detection result 210 to another extended object detection result. Distance 218 represents the distance from the edge of extended object detection result 210 to the lane extent.

[0042] In some examples, operation 212 may include, in additional or alternative examples, determining a set of distances between (extended or unextended) object detection results associated with the lane and / or determining the maximum distance within that set. For example, in the depicted example, operation 212 may include determining a first distance from the interrupted lane marker to the leftmost extended object detection result, a second distance from the leftmost extended object detection result to extended object detection result 210, a third distance from extended object detection result 210 to the rightmost extended object detection result, and a fourth distance from the rightmost extended object detection result to the lane extent.

[0043] In operation 220, example process 200 may include determining, according to any techniques discussed herein, whether the distance between an extended object detection result and the range of another object detection result, another extended object detection result, and / or the lane and / or road meets or exceeds a distance threshold. In some examples, the distance threshold may correspond to the width and / or length of the autonomous vehicle (e.g., depending on the dimension of the distance measurement—in the depicted example, the distance threshold may be at least partially based on the width of the autonomous vehicle) and / or a tolerance. Operation 220 may functionally determine whether the autonomous vehicle is fit between the extended object detection results (e.g., along the longitudinal or lateral axis of the vehicle). If the distance is less than the distance threshold, example process 200 may proceed to operation 222.

[0044] In operation 222, example procedure 200 may include determining a closure state indicating lane closure according to any of the technical solutions discussed in this application. In some examples, operation 222 may include any method of setting and / or saving the state associated with the analyzed lane, such as toggling a flag in a register or changing the state of a state machine to identify the lane as closed. For example, Figure 2A Lane state 224 is shown, which includes identifying the analyzed lane as closed. In some cases, the state may be associated with a portion of the lane, at least in part based on extended object detection results and / or the object detection results closest to the autonomous vehicle. For example, a closed lane state may be associated with the nearest edge starting from the nearest object detection result and / or the nearest edge of the expanded object corresponding to the nearest object (e.g., Figure 2A This is associated with a portion of the lane (starting from the nearest edge of the inflated object detection result 226). In some examples, example process 200 can continue. Figure 2B Operation 228 is described in the text (e.g., receiving new sensor data while the autonomous vehicle continues to operate).

[0045] Go to Figure 2B In operation 228, example process 200 may include determining whether a new object detection result has been received, according to any of the techniques discussed in this application. In some examples, operation 228 may include determining whether a new object detection result has been received from a perception engine that identifies a security category associated with a safe object and / or determining a period of time during which no such object detection result has been received / generated.

[0046] If a new object detection result is received, the example process 200 can continue with operation 230. If no new object detection result is received, the example process 200 can continue with operation 232.

[0047] Go to Figure 2CIn operation 230, example process 200 may include determining whether a new object detection result is associated with a safety category for identifying a safe object, according to any of the techniques discussed in this application. For example, as discussed in more detail above, operation 230 may determine whether the category associated with the new object detection result is a label for marking "construction vehicle," "traffic cone," "traffic light," etc. If the new object detection result is not associated with a safety category for identifying a safe object, example process 200 may return to operation 226 and / or continue to operation 232. If the object detection result is associated with a safety category for identifying a safe object, example process 200 may continue to operation 234. For example, this may be related to a situation where an autonomous vehicle is operating alongside and / or following a construction vehicle that includes traffic cones.

[0048] In operation 234, example procedure 200 may include: determining the state associated with the lane related to the new object detection result. Operation 234 may include: examining the stored state associated with the lane. If the state identifies that the lane is open, example procedure 200 may include returning to operation 202. If the state identifies that the lane is closed, example procedure 200 may include continuing to operation 236.

[0049] In operation 236, example procedure 200 may include maintaining the lane's closure by identifying safe objects, at least in part, based on the determination of new object detection results. For example, operation 236 may ensure that the lane remains identified as closed as long as there are safe objects(s).

[0050] Go to Figure 2D According to any of the techniques discussed in this application, in operation 232, example process 200 may include: determining that sensor data indicates the absence of a safe object associated with the (closed) lane for a duration that meets or exceeds a threshold duration. For example, example process 200 may include: tracking the duration for which no new object detection result is received identifying an object and / or a safe object in the closed lane. This duration can be reset when any new object detection result identifies such an object (in which case example process 200 can continue with operation 230). For example, shortly after the autonomous vehicle passes the last traffic cone, the state associated with the closed lane can thus be set to an open lane state, such as... Figure 2D As shown.

[0051] Although operation 232 is discussed here in conjunction with duration, it should be understood that, in additional or alternative examples, operation 232 may be based at least in part on the distance traveled by the autonomous vehicle. For example, in additional or alternative examples, operation 232 may include: determining that sensor data indicates the absence of a safe object associated with the (closed) lane within a distance traveled by the autonomous vehicle that meets or exceeds a threshold distance. In some examples, the threshold distance and / or threshold time may be based at least in part on the type of safe object upon which the determination of the closed lane state is based. For example, if the safe object is a “slow down” sign and flag, the duration / distance may be greater than if the safe object is a traffic cone.

[0052] Once the duration reaches or exceeds the detection threshold, example process 200 can continue with operation 238.

[0053] In operation 238, according to any of the technical solutions discussed in this application specification, example process 200 may include replacing the first state (closed lane state) with a second state indicating that the lane is open. In some examples, the open lane state may be associated with a portion of the lane corresponding to the farthest edge (along the road direction / road tangent) of the extended object detection result farthest from the autonomous vehicle.

[0054] Go to Figure 2E , Figure 2E Operations that can replace and / or be added to operations 232 and 238 are illustrated. For example, according to any of the technical solutions discussed in this application, after performing operation 232, example process 200 may additionally or alternatively include operation 240. In operation 240, example process 200 may include determining whether the perception engine has detected that the autonomous vehicle is approaching or within a road intersection (e.g., a crossroads, a T-junction, or any other intersection). For example, if the autonomous vehicle is within a threshold distance or within the threshold duration (of operation 232) and / or some of its factors, the autonomous vehicle may approach the intersection.

[0055] In operation 240, if the perception engine does not determine that the autonomous vehicle is at or near an intersection, the example process 200 can continue to operation 238. On the other hand, if in operation 240, the perception engine determines that the autonomous vehicle is at or near an intersection, the example process 200 can continue to operation 242.

[0056] In operation 242, according to any of the technical solutions discussed in this application specification, example process 200 may include maintaining a lane-associated state while the autonomous vehicle is still approaching and / or at an intersection. For example, this may include keeping the lane open or closed. This approach is useful because intersections may not contain any safety objects at all, or may not contain safety objects at the same rate as previously observed (e.g., traffic cones may be spaced further apart to accommodate traffic).

[0057] In some examples, example process 200 may additionally or alternatively include transmitting a set of states associated with portions of lane(s) and / or road(s) to a remote computing device. For example, the autonomous vehicle may send a notification indicating that a portion of a lane has been identified as closed based on the aforementioned confirmation, and / or the autonomous vehicle may periodically send “heartbeat” signals including the current state of the state machine and / or the location of the autonomous vehicle, thereby identifying the states of lane(s) in which the autonomous vehicle is located. In some examples, the remote computing device may store states received from the autonomous vehicle and may transmit updated states to one or more vehicles in a fleet. In some examples, vehicles in a fleet may be configured to determine routes based at least in part on states received from the remote computing device. In some examples, the autonomous vehicle may store lane open / closed states associated with a map to maintain the state. In some examples, such location states may be associated with an expiration time (e.g., the time from when a determined or received state will be cleared) or may persist until re-determined or updated (e.g., by receiving states associated with lanes from another vehicle and / or the remote computing device). In at least some examples, autonomous vehicles can transmit this information about partial or complete lane closures to a global map database, allowing other autonomous vehicles to retrieve the lane closure data as well. In any such example, similar steps can be implemented and / or verified / updated based on newly acquired sensor data from the attached vehicles.

[0058] In some examples, the lane analysis described above can be performed to identify lane conditions within a certain range, which can be the distance from the vehicle, at which sensor data and / or object detection results are determined to be reliable.

[0059] Example lane modification scenarios and techniques

[0060] Figure 3A bird's-eye view of example scenario 300 is shown, in which a set of safety objects designates new lanes that are not associated with the original lane markings. Despite various other lane changes (e.g., narrowing signs, narrowing shoulders, narrowing merges, narrowing single lanes, narrowing two-lane traffic flow, etc.), Figure 3 Example scenario 300 only illustrates lane narrowing. In some examples, the techniques described in this application can determine that all available lanes (e.g., lanes in the same direction of vehicle travel) are closed in scenarios similar to example scenario 300.

[0061] Example scenario 300 includes two driving directions, driving direction 302 and driving direction 304, where each driving direction has two lanes associated with it. Driving direction 304 includes two existing lanes (existing lane 306 on the left and existing lane 308 on the right), which are divided by discontinuous lane markings and defined by double (yellow) lines 310 and road boundaries 312. For example, suppose that autonomous vehicle 102 has determined, at least in part, that existing lane 308 on the right is closed based on the set of safe objects in the road (e.g., unmarked and represented by circles due to quantity) (e.g., the autonomous vehicle may have stored and / or maintained the closed state associated with existing lane 308 on the right). However, upon reaching a lane-changing narrowing point, the autonomous vehicle 102 can (e.g., due to a safe object in the lane) determine that the existing lane 306 on the left is closed and / or the lane 314 associated with the (opposite) direction of travel 302 is unavailable because the direction of travel 302 is the opposite direction of travel 304 associated with the autonomous vehicle 102 and / or because a double lane 312 is detected. In other words, by using the technical solutions discussed in conjunction with FIG2, the autonomous vehicle 102 can determine that all (existing) (multiple) lanes associated with the direction of travel are blocked (e.g., according to the technical solutions discussed in conjunction with FIG2, no existing lane has an open width sufficient for the autonomous vehicle 102 to pass). The autonomous vehicle 102 may be equipped with other or alternative technical solutions for determining alternative lane shapes and / or their associated open / closed states. Alternative lane shapes may, in some cases, avoid existing lane markings.

[0062] In addition to associating with existing lane markings, or as a replacement for methods that associate with existing lane markings, Figure 4Other or alternative technical solutions are illustrated for determining the open lane state and / or closed lane state associated with multiple sections of a road. For example, the technical solutions discussed herein may attempt to identify open lanes based on the technical solutions discussed above and existing lane markings. However, if no such lane exists, for example, in example scenario 300, the autonomous vehicle may determine the associated alternative lane shape and open / closed state(s) at least in part based on one or more object detection results and / or the state associated with the lane.

[0063] For example, the autonomous vehicle can determine that the existing lane 308 on the right is associated with a closed state, and can determine the shape of the overlapping object detection results associated with the existing lane 308 on the right. The autonomous vehicle 102 can attempt, in additional or alternative examples, to expand the shape to include objects within a threshold distance of the objects in the existing lane 308 on the right, and continue doing so until no object detection results exist within the threshold distance. For example, object detection result 400 may have a threshold distance to object detection result 402, so the autonomous vehicle can expand the shape to include object detection result 400, etc. (e.g., by modifying the boundary to include both objects by connecting the two objects with a line).

[0064] The autonomous vehicle 102 can generate a shape with continuous boundaries such that there is at least one lane with a minimum width. The minimum width can be a distance greater than a threshold distance (to avoid the generated shape including safety objects on the opposite side of the narrowing lane). In some examples, an ML model can be trained to determine such a shape (e.g., at least in part based on a category task, clustering task, etc.).

[0065] In some examples, the autonomous vehicle 102 may repeat this process for a second set of safety objects (e.g., safety objects on the left) associated with and / or at least partially based on the open lane state. In other words, the autonomous vehicle 102 may determine the shape of the open lane affirmatively based at least partially on the open lane state and one or more safety objects. In some examples, the autonomous vehicle may detect safety signs symbolizing narrowing, lane merging, and / or other traffic modifications and determine the shape of alternative lanes based at least partially on symbols on safety signs.

[0066] Figure 4Three alternative lane shapes are shown: a first alternative lane shape 404 associated with a closed lane state, a second alternative lane shape 406 associated with an open lane state, and a third alternative lane shape 406 associated with a closed lane state. In some examples, the autonomous vehicle 102 may transmit one or more of the alternative lane shapes to a remote operating system to receive confirmation of the alternative lane shapes, confirmation of crossing dual lanes 312, and / or modifications provided by a remote operator to the alternative lane shapes. In some examples, the alternative lane shapes may be stored in a global map database, which may be retrieved by one or more vehicles in a convoy or pushed to one or more vehicles in a convoy (e.g., based on generating routes for vehicles approaching the roads associated with the alternative lane shapes). In some examples, the autonomous vehicle 102 may receive modifications to the alternative lane shapes. In some examples, the autonomous vehicle 102 and / or the remote operating system may use the modifications to train an ML model configured to generate alternative lane shapes. In some cases, a first ML model may be trained to generate alternative lane shapes associated with an open lane state, and a second ML model may be trained to generate alternative lane shapes associated with a closed lane state. In additional or alternative examples, a single ML model can generate alternative lane shapes for two lane states. In any such example, such detection and updates of the drivable lane surface can be passed to a global (and / or local) map system, allowing autonomous vehicles and / or additional autonomous vehicles to rely on such remotely confirmed drivable road surfaces. In any such example, drivable road surfaces and shapes can be continuously verified, although remotely confirmed and pre-mapped data (which may be necessary in case of any changes to such surfaces) are required.

[0067] Example System

[0068] Figure 5 A block diagram of an example system implementing the techniques discussed herein is shown. In some cases, system 500 may include vehicle 502, which may represent Figure 1 The autonomous vehicle 502 is described in the document. In some cases, vehicle 502 may be an autonomous vehicle configured to operate according to a Level 5 category issued by the National Highway Traffic Safety Administration (NHTSA), which describes a vehicle capable of performing all safety-critical functions throughout the journey without the driver (or occupant) having control of the vehicle at any time. However, in other examples, vehicle 502 may be a fully or partially autonomous vehicle with any other level or category. Furthermore, in some cases, the technologies described in this application may also be used by non-autonomous vehicles.

[0069] Vehicle 502 may include: (multiple) vehicle computing devices 504, (multiple) sensors 506, (multiple) transmitters 508, (multiple) network interfaces 510 and / or (multiple) drive components 512.

[0070] In some cases, the (multiple) sensors 506 may include: lidar sensors, radar sensors, ultrasonic transducers, sonar sensors, position sensors (e.g., global positioning systems (GPS), compasses, etc.), inertial sensors (e.g., inertial measurement units (IMUs), accelerometers, magnetometers, gyroscopes, etc.), image sensors (e.g., red-green-blue (RGB) cameras, infrared (IR) cameras, intensity cameras, depth cameras, time-of-flight cameras, etc.), microphones, wheel encoders, environmental sensors (e.g., temperature sensors, humidity sensors, light sensors, pressure sensors), etc. The (multiple) sensors 506 may include various instances of each of these or other types of sensors. For example, radar sensors may include individual radar sensors located at corners, front, rear, sides, and / or top of vehicle 502. As another example, cameras may include multiple cameras positioned at different locations around the exterior and / or interior of vehicle 502. The (multiple) sensors 506 may provide input to the (multiple) vehicle computing devices 504 and / or (multiple) computing devices 514.

[0071] As described above, vehicle 502 may also include multiple transmitters 508 for emitting light and / or sound. In this example, the multiple transmitters 508 may include multiple internal audio transmitters and visual transmitters for communicating with passengers of vehicle 502. By way of example, and not limitation, the multiple internal transmitters may include: speakers, lights, signs, displays, touchscreens, multiple haptic transmitters (e.g., vibration and / or force feedback), mechanical actuators (e.g., seatbelt pretensioners, seat positioners, headrest positioners, etc.), etc. In this example, the multiple transmitters 508 may also include multiple external transmitters. By way of example, and not limitation, the multiple external transmitters in this example include: signal lights for emitting driving direction signals or other indicators of vehicle movement (e.g., indicator lights, signs, light arrays, etc.), and one or more audio transmitters (e.g., speakers, speaker arrays, horns, etc.) for auditory communication with pedestrians or other nearby vehicles, one or more of which incorporate beam steering technology.

[0072] Vehicle 502 may also include multiple network interfaces 510 that enable communication between vehicle 502 and one or more other local or remote computing devices. For example, the multiple network interfaces 510 may facilitate communication with multiple other local computing devices and / or multiple drive components 512 on vehicle 502. Furthermore, the multiple network interfaces 510 may, in additional or alternative examples, enable the vehicle to communicate with multiple other nearby computing devices (e.g., other nearby vehicles, traffic lights, etc.). Moreover, the multiple network interfaces 510 may, in additional or alternative examples, enable vehicle 502 to communicate with multiple computing devices 514. In some examples, the multiple computing devices 514 may include one or more nodes of a distributed computing system (e.g., a cloud computing architecture). The multiple computing devices 514 may be examples of the multiple remote computing devices and / or remote operating systems discussed herein.

[0073] Multiple network interfaces 510 may include physical and / or logical interfaces for connecting multiple vehicle computing devices 504 to another computing device or network, such as multiple networks 516. For example, the multiple network interfaces 510 may connect via frequencies defined by the IEEE 500.11 standard, short-range radio frequencies (e.g.,... Wi-Fi-based communication can be enabled via cellular communication (e.g., 2G, 3G, 4G, 4G LTE, 5G, etc.) or any suitable wired or wireless communication protocol that allows each computing device to interface with (multiple) other computing devices. In some cases, (multiple) vehicle computing devices 504 and / or (multiple) sensors 506 can transmit sensor data to (multiple) computing devices 514 via (multiple) networks 516 at a specific frequency in a near real-time manner after a predetermined period of time.

[0074] In some cases, vehicle 502 may include one or more drive units 512. In some cases, vehicle 502 may have a single drive unit 512. In some cases, drive unit(s) 512 may include one or more sensors to detect the conditions of drive unit(s) 512 and / or the environment surrounding vehicle 502. By way of example and not limitation, the sensors of drive unit(s) 512 may include: one or more wheel encoders (e.g., rotary encoders) to sense the rotation of the drive module's wheels, inertial sensors (e.g., inertial measurement units, accelerometers, gyroscopes, magnetometers, etc.) to measure the orientation and acceleration of the drive module, cameras or other image sensors, ultrasonic sensors to acoustically detect objects in the environment surrounding the drive module, lidar sensors, radar sensors, etc. Some sensors, such as vehicle encoders, may be unique to drive unit(s) 512. In some cases, the sensors on drive unit(s) 512 may overlap with or supplement the corresponding systems of vehicle 502 (e.g., sensors 506).

[0075] The (multiple) drive components 512 may include a number of vehicle systems, including: a high-voltage battery; an electric motor driving the vehicle; an inverter that converts direct current from the battery into alternating current for use by other vehicle systems; a steering system including a steering motor and steering bracket (which may be electric); a braking system including hydraulic or electric actuators; a suspension system including hydraulic and / or pneumatic components; a stability control system for distributing braking force to mitigate traction loss and maintain control; an HVAC system; a lighting system (e.g., headlights / taillights for illuminating the exterior environment of the vehicle); and one or more other systems (e.g., a cooling system, a safety system, an on-board charging system, and other electrical components such as DC / DC converters, high-voltage junctions, high-voltage cables, charging systems, charging ports, etc.). Additionally, the (multiple) drive components 512 may include a drive module controller that can receive and preprocess data from the (multiple) sensors and control the operation of various vehicle systems. In some cases, the drive module controller may include one or more processors and memory coupled to the one or more processors in a communicative manner. The memory can store one or more modules to perform various functions of the drive unit(s) 512. Furthermore, the drive unit(s) 512 also includes one or more communication connections that enable each drive module to communicate with another drive module or other local or remote computing device.

[0076] Multiple vehicle computing devices 504 may include multiple processors 518 and a memory 520 communicatively connected to one or more processors 518. Multiple computing devices 514 may also include multiple processors 522 and / or a memory 524. Multiple processors 518 and / or 522 may be any suitable processor capable of executing instructions to process data and perform operations as described in this specification. By way of example and not limitation, multiple processors 518 and / or 522 may include one or more central processing units (CPUs), graphics processing units (GPUs), integrated circuits (e.g., application-specific integrated circuits (ASICs)), gate arrays (e.g., field-programmable gate arrays (FPGAs)), and / or any other means or part of means for processing electronic data to convert electronic data into electronic data that can be stored in registers and / or memory.

[0077] Memory 520 and / or 524 may be examples of non-transitory computer-readable media. Memory 520 and / or 524 may store an operating system and one or more software applications, instructions, programs, and / or data to implement the methods described herein and the functions belonging to various systems. In various implementations, any suitable memory technology may be used to implement the memory, such as static random access memory (SRAM), synchronous dynamic RAM (SDRAM), non-volatile / flash memory, or any other type of memory capable of storing information. The architectures, systems, and individual elements described herein may include many other logical, program, and physical components, wherein those components shown in the accompanying drawings are merely examples relevant to the discussion herein.

[0078] In some cases, memory 520 and / or memory 524 may store perception engine 526, planner 528, state tracker 530, map(s) 532, and / or system controller(s) 534. Perception engine 526 may represent perception engine 114, planner 528 may represent planner 112, state tracker 530 may include and / or represent state tracker 118, and / or map(s) 532 may include and / or represent map 116. In some cases, perception engine 526 may include a primary perception system, a secondary perception system, a prediction system, and / or a positioning system. Memory 520 and / or 524 may additionally or alternatively store positioning components, mapping systems, planning systems, ride management systems, etc. Although the perception engine 526, planner 528, state tracker 530 and / or (multiple) maps 532 are shown as stored in memory 520, the perception engine 526, planner 528, state tracker 530 and / or (multiple) maps 532 may include processor-executable instructions, (multiple) machine learning models (e.g., neural networks) and / or hardware and / or may be stored in memory 524.

[0079] In some examples, the perception engine 526 may include one or more ML models for determining the shape of alternative lanes. Such ML models(s) may include clustering models, categorical models, and / or regression models. In some examples, the ML models(s) may include neural networks. As described in this application, the example neural network is a bio-inspired algorithm that passes input data through a series of connected layers to produce an output. Each layer in the neural network may also contain another neural network, or may contain any number of layers (whether or not convolutional). In the context of this application, it will be understood that neural networks can utilize machine learning, which can refer to a broad class of algorithms that generate outputs based on learned parameters.

[0080] Although discussed in the context of neural networks, any type of machine learning can be used according to this application specification. For example, machine learning algorithms can include, but are not limited to, regression algorithms (e.g., ordinary least squares regression (OLSR), linear regression, logarithmic regression, stepwise regression, multivariate adaptive regression splines (MARS), local estimation scatter plot smoothing (LOESS)), instance-based algorithms (e.g., ridge regression, minimum absolute shrinkage and selection operator (LASSO), elastic networks, minimum angle regression (LARS)), decision tree algorithms (e.g., category and regression tree (CART), iterative bisectioner 3 (ID3), chi-square automatic interaction detection (CHAID), single-layer decision tree, conditional decision tree), Bayesian algorithms (e.g., Naive Bayes, Gaussian Naive Bayes, multinomial Naive Bayes, average one-dependency estimator (AODE), Bayesian belief network (BNN), Bayesian network), clustering algorithms (e.g., k-means, k-median, expectation maximization (EM), hierarchical clustering), and association rule learning algorithms. (e.g., perceptron, backpropagation, Hopfield network, radial basis function network (RBFN)), deep learning algorithms (e.g., deep Boltzmann machine (DBM), deep belief network (DBN), convolutional neural network (CNN), stacked autoencoder), dimensionality reduction algorithms (e.g., principal component analysis (PCA), principal component regression (PCR), partial least squares regression (PLSR), Sammon mapping, multidimensional scaling (MDS), projection tracking, linear discriminant analysis (LDA), mixture discriminant analysis (MDA), quadratic discriminant analysis (QDA), flexible discriminant analysis (FDA)), ensemble algorithms (e.g., cueing algorithms, bootstrap aggregation (guided aggregation algorithm), AdaBoost algorithm, stacked generalization (mixture), gradient boosting machine (GBM), gradient boosting regression tree (GBRT), random forest), SVM (support vector machine), supervised learning, unsupervised learning, semi-supervised learning, etc.). Other examples of architectures include neural networks such as ResNet-50, ResNet-101, VGG, DenseNet, PointNet, etc.

[0081] In some examples, computing device 514 may generate and / or store state tracker 530 and / or maps 532. State tracker 530 may include registers, state machines, and / or any other method to continuously and / or temporarily (e.g., at least for a defined (expiration) time) identify the closed or open state of a lane. For example, state tracker may include finite state machines ("FSMs") such as deterministic finite automata ("DFAs"), data structures, registers, etc. In some examples, state tracker 530 may accumulate, store, and / or otherwise associate lane states, lane identifiers, portions of roads, portions of local maps, object detection results, extended object detection results, and / or alternative lane shapes. In other embodiments or alternative embodiments, the techniques discussed in this specification may include outputting the closed / open state associated with the current lane and / or any other lane (e.g., an adjacent lane, another lane).

[0082] In some examples, map(s) 532 may include local maps, drivable road surface maps, and / or global maps, and vehicle 502 may determine one or more of these maps based at least in part on sensor data and / or these maps may be received from computing device 514 via network 516. In some examples, map(s) 532 may include a global map database.

[0083] The memory 520 may additionally or alternatively store one or more system controllers 532, which may be configured to control the steering system, propulsion system, braking system, safety system, transmitter system, communication system, and other systems of the vehicle 502. These system controllers 532 may communicate with and / or control corresponding systems of the drive components 512 and / or other components of the vehicle 502. For example, the planner 528 may generate instructions at least in part based on perception data generated by the perception engine 526 (which may include any states and / or alternative lane shapes discussed in this specification) and transmit these instructions to the system controllers 532, which may control the operation of the vehicle 502 at least in part based on these instructions.

[0084] In some examples, the computing device 514 may include a remote operating system that can be configured to generate data representations and / or input options received from one or more vehicles in the fleet, to be presented to a remote operator, thereby providing navigation for one or more vehicles (e.g., options to select a display button for confirming a vehicle’s determined trajectory, options to modify and / or confirm the shape of alternative lanes determined by the vehicle, options to confirm crossing double solid lines).

[0085] It should be noted that, although Figure 5As illustrated in a distributed system, in an alternative example, components of vehicle 502 may be associated with computing devices 514 and / or components of computing devices 514 may be associated with vehicle 502. That is, vehicle 502 may perform one or more functions associated with computing devices 514, and vice versa.

[0086] Example Terms

[0087] A. A method comprising: receiving sensor data from one or more sensors; determining, at least in part, an object detection result based on the sensor data, the object detection result being associated with a lane of a road and indicating the position of an object in the environment and the size of the object in a first direction along the lane and in a second direction perpendicular to the first direction in a plane associated with the lane; modifying the object detection result by increasing the size indicated by the object detection result to obtain an extended object detection result; determining a distance between the extended object detection result and at least one of another object detection result, another extended object detection result, or a lane range; determining that the distance is less than or equal to a distance threshold; determining a closure state indicating lane closure based at least in part on the distance being less than or equal to the distance threshold; and controlling a vehicle based at least in part on the closure state.

[0088] B. As described in paragraph A, modifying the object detection result includes: increasing the dimension parallel to the first direction, so that the dimension parallel to the first direction is greater than the dimension parallel to the second direction.

[0089] C. The method as described in paragraph A or B, wherein: the closure state is determined and modified based at least in part on a safety category identified by determining the category associated with the object detection result; the safety category includes at least one of the following: safety object, safety personnel, safety vehicle, or safety sign or signal.

[0090] D. The method as described in any of paragraphs A through C, wherein the object detection result is a first object detection result, and the method further includes: receiving one or more second object detection results after the first object detection result; determining that at least one of the one or more second object detection results is associated with a security category; and maintaining the closed state at least in part based on determining that at least one of the one or more second object detection results is associated with a security category.

[0091] E. The method as described in any of paragraphs A through D, further comprising: receiving sensor data after receiving an object detection result; determining that the sensor data indicates that there is no safe object associated with the lane within a distance of travel that is equal to or greater than a threshold distance; and replacing the closed state with an open state indicating that the lane is open, at least in part based on the determination that the sensor data indicates that there is no travel distance.

[0092] F. The method described in any of paragraphs A through E further includes controlling the autonomous vehicle based at least in part on the open state.

[0093] G. A system comprising: one or more sensors; one or more processors; and a memory storing processor-executable instructions that, when executed by the one or more processors, cause the system to perform operations including: receiving an object detection result indicating the position of an object in an environment and multiple dimensions of the object, the dimensions including a first dimension along a lane direction associated with the object and a second dimension perpendicular to the first direction; modifying at least one of the first dimension or the second dimension based at least in part on a lane width or a lane direction to obtain an extended object detection result; determining a distance between the extended object detection result and at least one of another object detection result, another extended object detection result, or a lane width; determining that the distance is less than a distance threshold; and determining a closure state indicating lane closure based at least in part on determining that the distance is less than the distance threshold.

[0094] H. The system as described in paragraph G, wherein the system further modifies at least one of a first direction or a second direction and determines a closed state based at least in part on determining a category associated with the object detection result, including a safety category, and the safety category includes at least one of the following: a safety object, a safety person, a safety vehicle, or a safety sign or safety signal.

[0095] I. The system as described in paragraph G or H, wherein the object detection result is a first object detection result, and the operation further includes: receiving one or more second object detection results after the first object detection result; determining that at least one of the one or more second object detection results is associated with a category including a security category; and maintaining a closed state at least in part based on determining that at least one of the one or more second object detection results is associated with a security category.

[0096] J. The system as described in any of paragraphs G to I, wherein the operation further comprises: receiving sensor data after receiving an object detection result; determining a second object detection result associated with at least one of the lanes or adjacent lanes; determining that a driving direction associated with at least one of the lanes or adjacent lanes is blocked; and determining a polygon in a plane associated with the road that contains one or more object detection results identifying safety categories.

[0097] K. The system as described in any of paragraphs G to J, wherein the operation further comprises: determining the open state of a second lane adjacent to the lane based at least in part on a second object detection result or second sensor data associated with the second lane.

[0098] L. The system as described in any of paragraphs G through K, wherein the operation further comprises: receiving sensor data from one or more sensors; determining, at least in part, that an autonomous vehicle is approaching an intersection based on the sensor data; and maintaining a closed state based, at least in part, on the determination that the autonomous vehicle is approaching the intersection.

[0099] M. The system as described in any of paragraphs G through L, wherein the operation further comprises: associating a closed state with at least a portion of the road; and transmitting the closed state and an identifier of that portion to a remote computing device.

[0100] N. The system as described in any of paragraphs G through K, wherein the operation further comprises: determining, at least in part, based on sensor data from sensors on the autonomous vehicle, that the drivable road surface on which the autonomous vehicle is traveling includes at least one lane; receiving, from a remote computing system and at least in part based on the drivable road surface, a second state indicating lane closure; and determining, at least in part based on the second state and the drivable road surface, a route for controlling the autonomous vehicle.

[0101] O. A non-transitory computer-readable medium storing processor-executable instructions, which, when executed by one or more processors, cause the one or more processors to perform operations including: receiving an object detection result, the object detection result including the position and size of an object in an environment; modifying the object detection result at least in part based on a lane direction associated with the object detection result to obtain an extended object detection result; determining a distance between the extended object detection result and at least one of another object detection result associated with a lane, another extended object detection result associated with a lane, or a lane range; determining that the distance is less than a distance threshold; and determining a closure state indicating lane closure based at least in part on determining that the distance is less than the distance threshold.

[0102] P. A non-transitory computer-readable medium as described in paragraph O, wherein modifying object detection results and determining a closed state is further based, at least in part, on determining a category associated with the object detection results, including a security category, wherein the security category includes at least one of the following: a security object; a security person; a security vehicle; or a security sign or security signal.

[0103] Q. A non-transitory computer-readable medium as described in paragraph O or paragraph P, wherein the object detection result is a first object detection result, and the operation further includes: receiving one or more second object detection results after the first object detection result; determining that at least one of the one or more second object detection results is associated with a category including a safe object; and maintaining a closed state at least in part based on determining that at least one of the one or more second object detection results is associated with identifying a category of safe objects.

[0104] R. A non-transitory computer-readable medium as described in paragraph O or paragraph Q, wherein the operation further comprises: receiving sensor data after receiving an object detection result; determining a second object detection result associated with at least one of the lanes or adjacent lanes; determining that a driving direction associated with at least one of the lanes or adjacent lanes is blocked; and determining a polygon in a plane associated with the road that includes one or more object detection results identifying a safety category.

[0105] S. A non-transitory computer-readable medium as described in any of paragraphs O to R, wherein the operation further comprises: receiving sensor data from one or more sensors of the autonomous vehicle; determining, at least in part, that the autonomous vehicle is approaching an intersection based on the sensor data; and maintaining a closed state based, at least in part, on the determination that the autonomous vehicle is approaching the intersection.

[0106] T. A non-transitory computer-readable medium as described in any of paragraphs O to S, wherein the operation further comprises: associating a closed state with at least a portion of the road; and transmitting the closed state and an identifier of that portion to a remote computing device.

[0107] Although the subject matter of this application has been described using language specific to structural components and / or methodological behaviors, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described. The specific components and behaviors disclosed in this application are illustrative forms of implementing the claims of this application.

[0108] The components described herein represent instructions that can be stored in any type of computer-readable medium and can be implemented in software and / or hardware. All methods and processes described above can be fully embodied in software code components and / or computer-executable instructions executed by one or more computers or processors, hardware, or some combination thereof, and can be fully automated. Some or all of these methods may be selectively embodied in dedicated computer hardware.

[0109] Unless otherwise specified, conditional language such as “may,” “possibly,” “can,” or “perhaps” should be understood in context as indicating inclusion of a specific component, element, and / or step not included in other examples. Therefore, such conditional language generally does not imply that the aforementioned specific component, element, and / or step is absolutely necessary for one or more examples, nor does it imply that one or more examples must include a logic system to determine whether user input or prompts exist, whether the aforementioned specific component, element, and / or step is included in any particular instance, or whether the aforementioned specific component, element, and / or step is implemented in any particular instance.

[0110] Unless otherwise specified, conjunctions such as “at least one of X, Y, or Z” should be understood to mean that the items, terms, etc., can be X, Y, Z, or any combination thereof, including plural elements. Unless explicitly stated as single, “one” means both single and multiple.

[0111] Any routine descriptions, elements, or blocks depicted in the flowcharts described herein and / or in the accompanying drawings should be understood as potentially representing modules, sections, or portions of code for implementing logical functions or elements in a particular set of one or more computer-executable instruction routines. It will be understood by those skilled in the art that, in alternative embodiments within the scope of the examples described herein, elements or functions may be removed depending on the functionality involved, or the execution may be performed in a different order than that described in the figures or text, including substantially synchronous execution, reverse execution, addition of other operations, or cancellation of operations.

[0112] Various changes and modifications can be made to the above examples, and the elements therein should be understood as other acceptable examples. All such changes and modifications are intended to be included within the scope disclosed in this application and protected by the following claims.

[0113] This application includes the following sample terms:

[0114] A1. A system comprising: one or more sensors; one or more processors; and a memory storing processor-executable instructions, which, when executed by the one or more processors, cause the system to perform operations including: receiving an object detection result indicating the position of an object in an environment and multiple dimensions of the object, the multiple dimensions including: a first dimension along a lane direction associated with the object, and a second dimension perpendicular to the first direction; modifying at least one of the first dimension or the second dimension based at least in part on a lane width or at least one of the lane direction to obtain an extended object detection result; determining a distance between the extended object detection result and at least one of another object detection result, another extended object detection result, or a lane width; determining that the distance is less than a distance threshold; and determining a closure state indicating lane closure based at least in part on determining that the distance is less than the distance threshold.

[0115] A2. According to the system described in paragraph A1, modifying the object detection result includes: increasing the dimension parallel to the first direction by more than the dimension parallel to the second direction.

[0116] A3. The system according to paragraph A1 or A2, wherein modifying at least one of the first direction or the second direction and determining the closure state is further based at least in part on determining a category associated with the object detection result, including a safety category, wherein the safety category includes at least one of: a safety object; a safety person; a safety vehicle; or a safety sign or safety signal.

[0117] A4. The system according to any of paragraphs A1-A3, wherein the object detection result is a first object detection result, and the operation further includes: receiving one or more second object detection results after the first object detection result; determining that at least one of the one or more second object detection results is associated with a category including a security category; and maintaining the closed state at least in part based on determining that at least one of the one or more second object detection results is associated with a category identifying a secure object.

[0118] A5. The system according to any of paragraphs A1-A4, wherein the operation further comprises: receiving sensor data after receiving the object detection result; determining a second object detection result associated with at least one of the lane or adjacent lanes; determining that the driving direction associated with at least one of the lane or adjacent lanes is blocked; and determining a polygon in a plane associated with the road, the polygon including one or more object detection results identifying a safety category.

[0119] A6. The system according to any of paragraphs A1-A5, wherein the operation further comprises: determining the open state associated with the second lane based at least in part on a second object detection result or second sensor data associated with the second lane, wherein the second lane is adjacent to the lane.

[0120] A7. The system according to any of paragraphs A1-A6, wherein the operation further comprises: receiving sensor data from one or more sensors; determining, at least in part, that an autonomous vehicle is approaching an intersection based on the sensor data; and maintaining the closed state based, at least in part, on the determination that the autonomous vehicle is approaching the intersection.

[0121] A8. The system according to any of paragraphs A1-A7, wherein the operation further comprises: associating the closure state with at least a portion of the road; and transmitting the closure state and an identifier of the portion to a remote computing device.

[0122] A9. The system according to any of paragraphs A1-A8, wherein the operation further comprises: determining, at least in part, based on sensor data from sensors on the autonomous vehicle, that a drivable road surface on which the autonomous vehicle is traveling includes at least one lane; receiving, at least in part, a second state indicating that the lane is closed from a remote computing system based on the drivable road surface; and determining a route for controlling the autonomous vehicle based at least in part on the second state and the drivable road surface.

[0123] A10. A non-transitory computer-readable medium storing processor-executable instructions that, when executed by one or more processors, cause the one or more processors to perform the following operations: receiving an object detection result, the object detection result including the position of an object in an environment and the size of the object; modifying the object detection result at least in part based on a lane direction associated with the object detection result to obtain an extended object detection result; determining a distance between the extended object detection result and at least one of another object detection result associated with the lane, another extended object detection result associated with the lane, or the range of the lane; determining that the distance is less than a distance threshold; and determining a closure state indicating lane closure based at least in part on determining that the distance is less than the distance threshold.

[0124] A11. The non-transitory computer-readable medium according to paragraph A10, wherein modifying the object detection result and determining the closure state are further based at least in part on determining a category associated with the object detection result, including a security category, wherein the security category includes at least one of the following: a security object; a security person; a security vehicle; or a security sign or security signal.

[0125] A12. The non-transitory computer-readable medium according to paragraphs A10 or A11, wherein the object detection result is a first object detection result, and the operation further includes: receiving one or more second object detection results after the first object detection result; determining that at least one of the one or more second object detection results is associated with a category including a safe object; and maintaining the closed state at least in part based on determining that at least one of the one or more second object detection results is associated with identifying the category of the safe object.

[0126] A13. The non-transitory computer-readable medium according to any of paragraphs A10-A12, wherein the operation further comprises: receiving sensor data after receiving the object detection result; determining a second object detection result associated with at least one of the lane or adjacent lanes; determining that the driving direction associated with at least one of the lane or adjacent lanes is blocked; and determining a polygon in a plane associated with the road, the polygon including one or more object detection results identifying a safety category.

[0127] A14. The non-transitory computer-readable medium according to any of paragraphs A10-A13, wherein the operation further comprises: receiving sensor data from one or more sensors of the autonomous vehicle; determining, at least in part, that the autonomous vehicle is approaching an intersection based on the sensor data; and maintaining a closed state based, at least in part, on the determination that the autonomous vehicle is approaching the intersection.

[0128] A15. The non-transitory computer-readable medium according to any of paragraphs A10-A14, wherein the operation further comprises: associating the closure state with at least a portion of the road; and transmitting the closure state and an identifier of the portion to a remote computing device.

Claims

1. A lane analysis method, comprising: Receive sensor data from the sensor; Based at least in part on the sensor data, an object detection result is determined, the object detection result indicating the position of the object in or near a lane of the road surrounding the vehicle and the size of the object; The object detection result is modified by increasing the size of the object to obtain an expanded object detection result, wherein the modification includes increasing the size of the object according to a scalar, the scalar being determined at least in part based on one or more of the following: the width of the road, the width of the lane, the number of lanes of the road, the speed of the vehicle, the category associated with the object detection result, or the closure type associated with the road; Determine the distance between the extended object detection result and another object or drivable area restriction feature; Determine that the distance is less than or equal to the distance threshold; as well as The vehicle is controlled at least in part based on the distance being less than or equal to a distance threshold.

2. The method of claim 1, wherein the distance threshold comprises at least one of the following: The length of the vehicle; or The width of the vehicle.

3. The method of claim 1, wherein modifying the object detection result includes increasing the size in the first direction by a greater amount than increasing the size in a second direction different from the first direction; and in, Determining the distance between the extended object detection result and another object or drivable area includes determining the distance between the extended object detection result and another object, another extended object, or the lane range.

4. The method according to claim 1, wherein: The modification of the object detection result also identifies a safety category based at least in part on determining a category associated with the object detection result, the safety category including at least one of the following: Safety objects; Security personnel; Safety vehicles; or Safety signs or signals.

5. The method according to claim 1, wherein the object detection result is a first object detection result, and the method further comprises: Determine the state associated with the lane; Receive the detection result of the second object; The result of the second object detection is determined to be associated with the safety category; as well as The state is maintained at least in part based on determining that the second object detection result is associated with a safety category.

6. The method according to claim 1, further comprising: After determining the object detection result, receive data from the second sensor; The second sensor data indicates that a safety object associated with the lane is missing when the vehicle has traveled a distance that reaches or exceeds a threshold distance. as well as The vehicle is controlled at least in part based on the determination that the sensor data indicates the missing information.

7. The method according to claim 1, further comprising: Determine the category associated with the object detection result; The closing state indicating that the lane is closed is determined at least in part based on the extended object detection results and the category; as well as The control of the vehicle is also partly based on the off state.

8. A lane analysis system, comprising: One or more processors; as well as One or more non-transitory computer-readable media storing computer-executable instructions, which, when executed, cause the one or more processors to perform the following operations: Receive sensor data from the sensor; Based at least in part on the sensor data, an object detection result is determined, the object detection result indicating the position of the object in or near a lane of the road surrounding the vehicle and the size of the object; The object detection result is modified by increasing the size of the object to obtain an expanded object detection result, wherein the modification includes increasing the size of the object according to a scalar, the scalar being determined at least in part based on one or more of the following: the width of the road, the width of the lane, the number of lanes of the road, the speed of the vehicle, the category associated with the object detection result, or the closure type associated with the road; Determine the distance between the extended object detection result and another object or drivable area restriction feature; Determine that the distance is less than or equal to the distance threshold; as well as The vehicle is controlled at least in part based on the distance being less than or equal to a distance threshold.

9. The system of claim 8, wherein the distance threshold comprises at least one of the following: The length of the vehicle; or The width of the vehicle.

10. The system of claim 8, wherein modifying the object detection result includes increasing the size in a first direction by a larger size in a second direction different from the first direction.

11. The system of claim 8, wherein determining the distance between the extended object detection result and another object or drivable area includes determining the distance between the extended object detection result and another object, another extended object, or the lane range.

12. The system of claim 8, wherein modifying the object detection result is at least in part based on determining a category associated with the object detection result, including a security category, the security category including at least one of the following: Safety objects; Security personnel; Safety vehicle; or Safety signs or safety signals.

13. The system of claim 8, wherein the object detection result is a first object detection result, and the operation further includes: Receive the detection result of the second object; Determine that the second object detection result is associated with a category that includes a safety category; as well as The vehicle is controlled based at least in part on the association between the second object detection result and the safety category.

14. The system of claim 8, wherein the operation further comprises: After determining the object detection result, receive data from the second sensor; Based at least in part on the second sensor data, a second object detection result associated with at least one of the lanes or adjacent lanes of the road is determined; It is determined that at least one of the driving directions associated with the lane or adjacent lanes is blocked; as well as Determine the polygon associated with the road, the polygon including one or more object detection results that identify a safety category.

15. The system of claim 8, wherein the operation further comprises: The state associated with the lane is determined at least in part based on the object detection results or the sensor data; The vehicle is determined to be near the intersection, at least in part, based on the sensor data. as well as The state is maintained at least in part based on determining that the vehicle is close to the intersection.

16. The system of claim 8, wherein the operation further comprises: Receive the state associated with the lane from a remote computing system and at least in part based on the drivable surface; as well as The route for controlling the vehicle is determined at least in part based on the state.

17. A non-transitory computer-readable medium storing instructions executable by one or more processors, wherein the instructions, when executed, cause the one or more processors to perform the following operations: Receive sensor data from the sensor; Based at least in part on the sensor data, an object detection result is determined, which indicates the position of the object in or near a lane of the road surrounding the vehicle and the size of the object; The object detection result is modified by increasing the size of the object to obtain an expanded object detection result, wherein the modification includes increasing the size of the object according to a scalar, the scalar being determined at least in part based on one or more of the following: the width of the road, the width of the lane, the number of lanes of the road, the speed of the vehicle, the category associated with the object detection result, or the closure type associated with the road; Determine the distance between the extended object detection result and another object or drivable area restriction feature; Determine that the distance is less than or equal to the distance threshold; as well as The vehicle is controlled at least in part based on the distance being less than or equal to a distance threshold.

18. The non-transitory computer-readable medium of claim 17, wherein the distance threshold includes at least one of the following: The length of the vehicle; or The width of the vehicle.

19. The non-transitory computer-readable medium of claim 17, wherein modifying the object detection result is further based at least in part on determining a category associated with the object detection result, including a security category, wherein the security category includes at least one of the following: Safety objects; Security personnel; Safety vehicle; or Safety signs or safety signals.

20. The non-transitory computer-readable medium of claim 17, wherein the operation further comprises: After determining the object detection result, receive data from the second sensor; Based at least on the second sensor data, a second object detection result associated with at least one lane or adjacent lane of the road is determined; It is determined that the direction of travel associated with the lane or at least one of the adjacent lanes is blocked; as well as Determine the polygon associated with the road, the polygon including one or more object detection results that identify a safety category.

21. The non-transitory computer-readable medium of claim 17, wherein the operation further comprises: The state associated with the lane is determined at least in part based on the object detection results or the sensor data; The vehicle is determined to be near the intersection, at least in part, based on the sensor data. as well as The state is maintained at least in part based on determining that the vehicle is close to the intersection.

Images