Method for free space expansion for automatic driving

Abstract

A method of free space expansion for automated driving is disclosed. The vehicle includes one or more sensors arranged to detect an object adjacent to the vehicle and a device for processing sensor data. The method comprises: obtaining target object data defining one or more target object types to be detected; obtaining sensor data from one or more sensors; detecting in the sensor data one or more instances of the target object type; determining spatial location data of one or more instances of the object type, where the spatial location data includes instance location data sets that define locations of the instances, respectively; and implicitly determining the free space as a space between the vehicle and the instance location data set by determining one or more barrier-free gazes between the sensor of the vehicle and the instance location data set and generating the free space by combining the one or more barrier-free gazes.

Description

Technical Field

[0001] The disclosed technology relates to methods and systems for determining the free space adjacent to a vehicle. Specifically, but not exclusively, the disclosed technology relates to ensuring free space so that the vehicle can move forward safely. Background Technology

[0002] Automated driving systems (ADS) in passenger vehicles are rapidly improving. These systems enhance safety and comfort by supporting the driver during dynamic driving tasks. These systems can be divided into two subcategories: Automated Driving (AD) systems configured to control the vehicle without supervision, and Advanced Driver Assistance Systems (ADAS) configured to assist the driver but not necessarily provide full autonomy. Currently, a variety of ADAS / AD systems are available.

[0003] The ADS functions available in both ADAS and AD systems can be categorized as follows:

[0004] Safety features such as lane departure prevention, automatic emergency braking, emergency steering, collision warning, and blind spot detection; and

[0005] Comfort features such as lane center following (cruise steering assist), adaptive cruise control, speed control based on speed limit and curvature, and lane change assist.

[0006] These systems provide one or more sensors and computing devices for determining what is known as free space. In the field of AD (Autonomous Driving), this typically refers to the space in front of a vehicle. Ensuring that this space is free of obstacles, pedestrians, or any other objects is crucial for the safe operation of the vehicle. A common method for identifying free space today is to continuously track the portion of the road surface in front of the vehicle. This is achieved by scanning the road surface portion using onboard vehicle sensors to measure the uninterrupted continuity of road detection in front of the vehicle (thus determining the absence of objects). The road surface portion can be divided into sub-segments, allowing the free space to vary in size at different points in time. If a small free space is detected, meaning only the nearest area is free, the vehicle can be stopped or its speed can be set relatively low. Conversely, if a large free space is detected, meaning no objects have been detected at a relatively large distance from the vehicle, the vehicle can be set to operate at a relatively high speed without compromising safety.

[0007] While the freespace assessment in modern Adaptive Controllers (ADS) is safe and reliable, there is still room for improvement. For example, by being able to determine freespace more accurately and / or reliably, ADS could potentially avoid unnecessary stops or decelerations associated with incorrect freespace assessments. Furthermore, by improving freespace assessment, it is possible to expand the freespace area, thereby covering not only the immediate area in front of the vehicle but also areas further in front, such as 50 meters or more. The positive effect of expanding the freespace area is that both safety and comfort requirements can be further enhanced. Summary of the Invention

[0008] The techniques disclosed herein aim to mitigate, alleviate, or eliminate one or more of the aforementioned defects and disadvantages in the prior art to address various problems related to how to assess the road surface in front of or adjacent to a vehicle, ensuring that there are no objects in the space, thereby ensuring that the vehicle can be safely moved into that space. Current systems focus on determining the absence of objects. However, when the available sensor data does not provide sufficient information to guarantee the absence of objects, the vehicle must be stopped or at least moved forward at a reduced speed. Current systems can be used as driver-assistance systems in so-called supervised driving systems where the driver is responsible for monitoring operations and also takes action when the system fails to determine free space. Current systems can also be used as a supplement to object recognition systems. By using free space detection systems in this way, it becomes possible to safely operate a vehicle in areas where object recognition systems cannot reliably detect objects on the road.

[0009] The various aspects and embodiments of the disclosed invention are defined in the following and accompanying independent and dependent claims.

[0010] A first aspect of the disclosed technology includes a method for determining free space adjacent to a vehicle. The vehicle includes one or more sensors arranged to detect objects adjacent to the vehicle and a device for processing sensor data. The method may include: obtaining target object data defining one or more target object types to be detected; obtaining sensor data from the one or more sensors; detecting one or more instances of the target object type in the sensor data; determining spatial location data of the one or more instances of the object type, wherein the spatial location data includes an instance location dataset defining the location of each instance; and implicitly defining free space as the space between the vehicle and the instance location dataset by: determining one or more unobstructed lines of sight between the vehicle's sensors and the instance location dataset; and generating free space by combining one or more unobstructed lines of sight.

[0011] The free space adjacent to a vehicle can be defined as a region, that is, a two-dimensional or three-dimensional space that is immediately adjacent to or includes the vehicle. For example, the free space adjacent to a vehicle can be a three-dimensional space that extends from the vehicle and reaches a beam shape with a maximum threshold in the direction of the vehicle's movement. The maximum threshold can depend on the onboard sensors used, but also on external conditions such as weather conditions. For example, the maximum threshold could be 300 meters.

[0012] A second aspect of the disclosed technology includes an apparatus for determining free space adjacent to a vehicle, the apparatus including control circuitry configured to: acquire target object data defining one or more target object types to be detected; acquire sensor data from one or more sensors of the vehicle; detect one or more instances of the object type; determine spatial location data of the one or more instances of the object type, wherein the spatial location data includes an instance location dataset defining the location of each instance; and implicitly determine the free space as the space between the vehicle and the instance location dataset by: determining one or more unobstructed lines of sight between the vehicle's sensors and the instance location dataset and generating the free space by combining one or more unobstructed lines of sight.

[0013] This aspect of the disclosed technology has similar advantages and preferred features as other aspects.

[0014] As used herein, the term "non-transitory" is intended to describe a computer-readable storage medium (or "memory") that does not include propagating electromagnetic signals, but is not intended to otherwise limit the type of physical computer-readable storage device included in the term computer-readable medium or memory. For example, the terms "non-transitory computer-readable medium" or "tangible memory" are intended to cover types of storage devices that do not necessarily store information permanently, including, for example, random access memory (RAM). Program instructions and data stored in a non-transitory form on a tangible computer-accessible storage medium can be further transmitted via a transmission medium or a communication medium, such as an electrical, electromagnetic, or digital signal, that can be transmitted via a communication medium such as a network and / or a wireless link. Therefore, as used herein, the term "non-transitory" is a limitation on the medium itself (i.e., tangible, not a signal), rather than a limitation on the persistence of data storage (e.g., RAM vs. ROM).

[0015] The disclosed aspects and preferred embodiments may be suitably combined with each other in any manner that is obvious to those skilled in the art, such that one or more features or embodiments disclosed with respect to one aspect may also be regarded as embodiments disclosed with respect to another aspect or another aspect.

[0016] One advantage of some embodiments is that free space can be implicitly determined. In other words, instead of determining the absence of an object, detecting certain target object types, and once these target object types are detected and their location data is determined, makes it possible to implicitly or indirectly determine the existence of free space between the vehicle and the detected objects, since no object obstructs the sensor's line of sight. This allows for reliable free space detection even when weather conditions may be challenging. For example, implicit free space detection can be used instead of explicit free space detection, which cannot be performed reliably due to rain. For instance, during rain, implicit free space detection can identify other vehicles, and as a result of identifying these vehicles, implicitly define the space between the vehicle and other vehicles as free space. In addition to facilitating free space detection under adverse weather conditions, implicit free space detection can also have the positive effect of expanding free space. In some cases, for example, it may be possible to identify other vehicles more reliably than detecting their absence on the road surface. This could happen, for example, when the vehicle is parked at the foot of a hill. Even if only a portion of the road surface between this vehicle and other vehicles is at least partially visible to this vehicle's onboard sensors, other vehicles located in front of this vehicle and on the other side of the mountain can be used to implicitly determine the free space. In other words, by using implicit free space detection, it is possible to extend the free space beyond the road surface visible to this vehicle's onboard sensors.

[0017] Free space can be determined solely by using the implicit free space determination method described above, but it is also possible to determine free space by combining the implicit free space determination method with the explicit free space determination method (i.e., as described above, detecting objects and thus indirectly detecting free space, and also detecting the absence of objects on the road surface).

[0018] Further embodiments are defined in the dependent claims. It should be emphasized that, when used in this specification, the term "comprising" is used to specify the presence of the stated features, elements, steps, or components. It does not exclude the presence or addition of one or more other features, elements, steps, components, or groups thereof.

[0019] Referring to the embodiments described below, these and other features and advantages of the disclosed technology will be further illustrated below. Attached Figure Description

[0020] The foregoing aspects, features, and advantages of the disclosed technology will be more fully understood through the following illustrative and non-limiting detailed description of exemplary embodiments of the present disclosure, taken in conjunction with the accompanying drawings, in which:

[0021] Figure 1The example illustration shows the field of view of a sensor installed at the front of a vehicle.

[0022] Figure 2 The vehicle shown in the diagram above is provided with a device configured to provide multiple implicit free subspaces.

[0023] Figure 3 The vehicle shown in the image above is configured to provide a polygonal implicit free subspace.

[0024] Figure 4 The vehicle shown in the image above is configured to provide both implicit and explicit free space based on polygons.

[0025] Figure 5 This is a flowchart of a method for determining the free space adjacent to a vehicle.

[0026] Figure 6 The diagram shows the device arranged to define free space.

[0027] Figure 7 The overall diagram includes Figure 6 The vehicle shown in the diagram. Detailed Implementation

[0028] This disclosure will now be described in detail with reference to the accompanying drawings, in which some exemplary embodiments of the disclosed technology are illustrated. However, the disclosed technology may be implemented in other forms and should not be construed as limited to the exemplary embodiments disclosed. The exemplary embodiments disclosed are provided to fully convey the scope of the disclosed technology to those skilled in the art. Those skilled in the art will understand that the steps, services, and functions described herein can be implemented using separate hardware circuitry, software that works in conjunction with a programmed microprocessor or general-purpose computer, one or more application-specific integrated circuits (ASICs), one or more field-programmable gate arrays (FPGAs), and / or one or more digital signal processors (DSPs).

[0029] It will also be understood that when this disclosure is described in terms of method, it can also be implemented as a device including one or more processors and one or more memories coupled to the one or more processors, wherein computer code is loaded to implement the method. For example, in some embodiments, the one or more memories may store one or more computer programs that, when executed by the one or more processors, cause the device to perform the steps, services, and functions disclosed herein.

[0030] It should also be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting. It should be noted that, as used in the specification and appended claims, unless the context clearly specifies otherwise, the articles “a,” “an,” “the,” and “the” are intended to indicate the presence of one or more elements. Thus, for example, in some contexts, references to “a unit” or “the unit” may refer to more than one unit, etc. Furthermore, the words “comprising” or “including” do not exclude other elements or steps. It should be emphasized that, when used in this specification, the term “comprising” is used to specify the presence of the stated feature, integral, step, or component. It does not exclude the presence or addition of one or more other features, integrals, steps, components, or groups thereof. The term “and / or” should be interpreted as meaning “both” and that each is optional.

[0031] It will also be understood that although the terms first, second, etc., may be used herein to describe various elements or features, these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, without departing from the scope of the embodiments, a first signal may be referred to as a second signal, and similarly, a second signal may be referred to as a first signal. Both the first signal and the second signal are signals, but they are not the same signal.

[0032] Figure 1 The example illustration generally includes the area in front of the vehicle, thus providing the field of view of the sensors that provide a forward view. As shown, lane 100 (i.e., the lane in which the vehicle is located) is included in the unobstructed line of sight area 102 in this example. Since there are no objects obstructing the view, sensor data about lane 100 can be obtained up to the horizon 104, i.e., the line separating the ground from the sky.

[0033] Figure 2 The vehicle 200 is illustrated above with an example. In this example, a two-lane road is provided. The left lane has a first solid lane marking 202 on the left side of the direction of travel D, and a dashed lane marking 204 on the right side, separating the left lane from the right lane, which is this lane in this particular example. As shown, the right lane has a second solid lane marking on the right side of the direction of travel D. To the left of the first solid lane marking 202, an obstacle 208 is provided. In this particular example, the obstacle 208 is a concrete wall. Therefore, unlike the lane markings, the obstacle 208 has a height that prevents vehicles from stopping on the other side of the obstacle.

[0034] As shown, different objects exist on and beside the road. In this particular example, a lamppost 210 is provided beside the road. Next to the lamppost 210, and also beside the road, is a pedestrian 212. Another vehicle 214 is present in the left lane of the road. In addition to these objects, lane markings 202, 204, 206 and obstacles 208 can also be considered objects. Generally, these objects can be divided into static objects and dynamic objects. In the context of this patent application, static objects are fixed objects, i.e., objects that are not intended to be moved, such as the lamppost 210 and obstacle 208 in the illustrated example. On the other hand, dynamic objects are considered to be moving. For example, the other vehicle 214 and pedestrian 212 are examples of dynamic objects.

[0035] In modern Adaptive Digital Systems (ADS), objects existing in the field of view 216a and 216b of the vehicle 200 can be linked to object types for adjustments such as speed and handling. By doing so, different actions can be taken depending on whether an object is linked to a pedestrian-type object or a lamppost-type object. For example, if an object is designated as a pedestrian-type object, this may result in a reduction in speed due to the risk of a pedestrian moving; however, if the object is designated as a lamppost-type object, the speed can be maintained because there is no risk of that type of object moving.

[0036] A further effect of assigning objects to different object types, also referred to in this paper as target object types, is that this can be used to improve free space evaluation. Instead of determining free space simply by evaluating the absence of objects on the ground in front of vehicle 200, free space can also be implicitly determined by specifying the space between vehicle 200 and a dataset of locations of instances of the target object type as free space. In this context, the free space determined by scanning the ground in front of vehicle 200 to identify the absence of objects is called explicit free space. The absence of such objects can be determined by detecting the continuous smoothness of the road surface (also called the ground) or by continuously detecting asphalt. The free space determined as a result of an unobstructed line of sight between the vehicle and instances of the target object type is called implicit free space. Reliability can be improved by restricting the implicit free space to instances of objects associated with the target object type. In other words, by knowing what to look for, algorithms for target object types can be specifically adapted to different target types, and it is also possible to reuse existing algorithms developed for other purposes, such as traffic sign detection algorithms not specifically developed for free space evaluation, which can also be used to determine implicit free space. Another positive effect of using implicit free space detection is that free space can be extended in terms of distance from the vehicle. In other words, in some situations, it is difficult to reliably detect the smoothness of the road surface, and it is also difficult to continuously detect asphalt at a relatively long distance, such as 100 meters or more, from the vehicle. Therefore, by using implicit free space detection, it is possible to expand the free space, thereby making it possible to move the vehicle forward safely at, for example, a higher speed than would be possible using explicit free space detection methods.

[0037] Another positive effect is that using implicit free-space detection methods allows for improved cost efficiency. For example, to reliably detect the smoothness of the road surface at long distances (e.g., 100 meters or more) from a vehicle, advanced onboard vehicle sensors are required. On the other hand, if implicit free-space detection methods are applied, the sensor requirements can be reduced, meaning less advanced sensors may be needed. One reason for this is that rain, leaves, and other road surface imperfections can significantly affect explicit free-space detection methods, increasing the demand for sensors used in the system implementing such methods, but have little impact on the sensors used to implement implicit free-space detection methods, which can be configured to detect vehicles, pedestrians, and other object types.

[0038] exist Figure 2The diagram illustrates four distinct implicit free subspaces: a first free subspace 218, a second free subspace 220, a third free subspace 222, and a fourth free subspace 224. The first free subspace 218 includes the space between vehicle 200 and pedestrian 212. The second free subspace 220 includes the space between vehicle 200 and lamppost 210. The third free subspace 222 includes the space between vehicle 200 and obstacle 208. The fourth free subspace 224 includes the space between vehicle 200 and another vehicle 214. By linking different free subspaces to different target object types in this way, finer-grained free spaces can be provided. In other words, while explicit free spaces can provide spaces where no obstacles or objects exist on the ground, different implicit free subspaces can provide different spaces linked to different target object types. For example, for speed control purposes, it might be of interest to receive implicit free subspaces linked to dynamic target object types, such as a first free subspace 218 linked to a pedestrian object type or a fourth free subspace 224 linked to a vehicle object type.

[0039] Although the first implicit free subspace to the fourth implicit free subspace are in Figure 2 The image shows a sector-shaped portion of the field of view, but implicit free space and implicit free subspace do not necessarily need to be allocated in this way. For example... Figure 3 As shown, by defining the implicit free space as the space between the position data of the vehicle and instances of the target object type, and by making no arbitrary assumptions about the space not detected therein, the implicit free space can instead be determined as a polygon formed between the position data of one or more sensors of the vehicle and the position data of instances of the target object. Figure 3For illustrative purposes, the implicit free space formed by one of the sensors provided on vehicle 200 and the detected object is illustrated. In the illustrated example, two different target object types are arranged to be detected: vehicle-type objects and obstacle-type objects. As shown, a first polygonal implicit free subspace 300 can be obtained based on the unobstructed line of sight between the vehicle's sensor and obstacle 208. Similarly, a second polygonal implicit free subspace 302 can be obtained based on the unobstructed line of sight between the sensor and vehicle 214. In other words, while the explicit free space can cover the entire width of the field of view, the implicit subspace can only cover a portion of the width of the field of view. Furthermore, while the explicit free space can be associated with a maximum distance threshold (i.e., a distance beyond which reliable absence detection is impossible), the implicit free space is not limited by such a maximum distance threshold. Instead, this results in an implicit free space where it is possible to reliably detect objects of a predefined target object type. In other words, the evaluation of the implicit free space can be improved by improving object recognition.

[0040] Figure 4 Diagram and Figure 3 The example shown in the image is similar to the previous one, but instead of depicting the effect of only one sensor, Figure 4 The example shown in the diagram uses a sensor array, which consists of multiple sensors placed close together. The effect of using a sensor array is to expand the implicit free subspace associated with the detection of vehicle-type objects and obstacle-type objects. Figure 4 In the example illustrated, solid lane-type objects are also detected, resulting in an implicit free space existing on the right side of the vehicle when viewed from the direction of travel D. As shown, the different implicit free spaces obtained through unobstructed line of sight between sensor position data and position data of instances of the target object type can be combined into an implicit free space 400. In addition to the implicit free space 40, which can depend on static and dynamic objects identified by the vehicle's ADS or other components as described above, an explicit free space 402 can be provided. As described above, the explicit free space 402 can be the result of scanning any object in the field of view that may pose a risk of collision with the vehicle 200. As shown, by combining the implicit free space 400 and the explicit free space 402, it is possible to provide the combined free space illustrated by the thick lines. The explicit free space 402 in this combined free space is used for the purpose of identifying the absence of objects, while the implicit free space 400 is used for the purpose of identifying objects of known types (i.e., target object types) and indirectly determining objects that do not exist between the vehicle and the instances based on the identified instances of these types.

[0041] Even without illustration, one or more sensors of a vehicle can provide different detection capabilities for instances of target object types at different points in time. For example, one or more sensors might be covered in dust, hindering their reliable detection of objects. This capability can also be related to sensor defects. For example, if a subset of the sensor elements of one of the sensors is malfunctioning, this will result in a portion of the space adjacent to the vehicle not being covered by that single sensor. Any deviation or event affecting the current ability of sensors to detect objects adjacent to the vehicle can be reflected in so-called sensor health monitoring data. By accessing this data, it can be taken into account when detecting one or more instances of the target object type. In other words, this information can be taken into account and compensated for in cases where one or more sensors suffer arbitrary deviations, temporarily or not temporarily. Alternatively, sensor health monitoring data can be considered after detection has already been performed. For example, if detection has already been performed by one or more sensors that cannot reliably detect for some reason, these detections can be discarded, thus avoiding the determination of implicit free space based on, for example, faulty sensors.

[0042] Furthermore, reliability assessments of implicitly determined free spaces (i.e., implicit free spaces) can be performed when needed. In other words, reliability can be estimated by mapping a dataset of instance locations of static objects (e.g., instance coordinates) to a dataset of reference locations of reference static objects. For example, the reference... Figure 2 Assuming lamppost 210 is designated as one or more reference static objects, the location dataset determined for lamppost 210 can be mapped to the lamppost reference objects, i.e., the preset location dataset for lamppost 210. The deviation between the instance location datasets associated with lamppost 210 and the reference location dataset can serve as a measure of reliability. The lower the deviation, the better the reliability.

[0043] Back Figure 4 Furthermore, when determining the implicit free space, map data including height data can be considered to reduce the risk of incorrect object detection. For example, such as Figure 4 As shown, the surrounding environment can be shaped to obstruct the sensor's vision in certain directions. Figure 4This is exemplified by obstacle 208 on the left side as seen in the driving direction D. By obstructing the sensor's line of sight with obstacle 208, it is possible for the sensor not to detect objects behind the obstacle as seen from the vehicle. By reflecting obstacle 208 and other objects in the map data, as well as in the general topology of hills and landscapes, this data can be taken into account when performing the step of detecting one or more instances of an object type. By doing so, incorrect detection of instances of the object type can be avoided. The area in which the sensor can perform instance detection without being obstructed by objects reflected in the map data is referred to herein as the map data-based line-of-sight region 404. Figure 4 In this context, the area is formed between the vehicle, the field of view 216a, 216b, the obstacle 208, and the outer boundary.

[0044] Figure 5 This is a flowchart of a method S100 for determining the free space adjacent to a vehicle. The method may include: obtaining target object data S102 defining one or more target object types to be detected; obtaining sensor data S104 from one or more sensors; detecting one or more instances of the target object type in the sensor data S106; determining spatial location data of one or more instances of the object type S108, wherein the spatial location data includes an instance location dataset that respectively defines the location of the instance; and implicitly determining the free space S110 as the space between the vehicle and the instance location dataset by the following steps: determining one or more unobstructed lines of sight between the vehicle's sensors and the instance location dataset S112; and generating the free space S114 by combining one or more unobstructed lines of sight.

[0045] Based on the advantages mentioned above, it has been recognized that instead of explicitly determining the absence of objects to obtain free space, it is possible to use this to implicitly determine free space in cases where the type of the target object can be detected.

[0046] Method S100 may further include: obtaining sensor health monitoring data S116 from one or more sensors. The sensor health monitoring data may include information about the sensor's current ability to detect objects adjacent to the vehicle. The step of detecting one or more instances of a target object type in the sensor data can be performed while taking into account the sensor's current capability reflected in the sensor health monitoring data.

[0047] By taking current capabilities into account, more reliable free-space assessments can be made. For example, if part of a sensor is covered in dust, this can be detected and compensated for, thus reducing the risk of errors.

[0048] The objects can include static and dynamic objects. The method may further include estimating the reliability of S118 by mapping an instance location dataset of static objects to a reference location dataset of reference static objects and / or by implicitly determining the reliability of the free space through the use of multiple sensors to detect 3D objects in free space. By utilizing the fact that some objects may be associated with the reference location dataset, it is possible to measure the bias and thus obtain a reliability metric. Since the detection of 3D objects involves several sensors, this can also serve as a reliability metric.

[0049] Sensor health monitoring data can include information about sensor components being blocked, sensor components malfunctioning, and / or sensor components being blinded by light.

[0050] The method may further include: determining the explicit free space adjacent to the vehicle in S120 by detecting instances of any object regardless of object type, and combining the implicitly determined free space with the explicit free space in S122 to form a combined free space. The explicit free space can be a near free space, and the implicit free space can be a far free space. For example, the near free space can be 0m to 40m from the vehicle, and the far free space can be 40m to 100m. However, these measurements are highly dependent on the sensors used.

[0051] The method may further include transmitting free space data, including implicitly determined free space, to the Autopilot System (ADS) via S124, so that vehicle maneuvers can be performed using the free space data.

[0052] Free space data can be used in ADS to adjust the vehicle's speed.

[0053] A free space can include multiple free subspaces, and the target object data can be defined by multiple target object types, wherein each of the free subspaces is linked to one of the multiple target object types.

[0054] The advantage of linking different free subspaces to different target object types is that it allows for the provision of finer-grained data to, for example, Adaptive Detection Systems (ADS). For instance, by knowing whether the free subspace is determined based on pedestrian detection or lane sign detection, the free space can be processed differently by the ADS.

[0055] Different modules of ADS can use a combination of one or more free subspaces as input.

[0056] Method S100 may further include: obtaining map data including height data in S126, determining a line-of-sight region based on the map data in S128, wherein, while considering the line-of-sight region based on the map data, the step of detecting one or more instances of an object type is performed.

[0057] The advantage of considering map data is that static objects and their heights can be taken into account, making it possible to determine obstacle and non-obstacle areas around the vehicle. In addition to static objects, it is also possible to consider the geographic topology around the vehicle. For example, if the vehicle is located at the foot of a steep mountain, this can be reflected in the map data, and also in the line-of-sight area based on the map data.

[0058] The target object type can include size-specific objects that satisfy one or more geometric conditions. In other words, the target object type can be qualified based on geometric properties.

[0059] Sensors may include one or more lidar, one or more radars, and / or one or more cameras.

[0060] Method S100 is preferably a computer-implemented method S100 executed by a processing system of a vehicle equipped with ADS. The processing system may, for example, include one or more processors and one or more memories coupled to the one or more processors, wherein the one or more memories store one or more programs that, when executed by the one or more processors, perform the steps, services, and functions of the method S100 disclosed herein.

[0061] Executable instructions for performing these functions may optionally be included in a non-transitory computer-readable storage medium or other computer program product configured to be executed by one or more processors.

[0062] Figure 6 This is a schematic block diagram representation of a device 10 for determining free space adjacent to a vehicle. Device 10 includes control circuitry 11 (e.g., one or more processors) configured to perform functions of the method S100 disclosed herein, wherein these functions may be included in a non-transitory computer-readable storage medium 12 or other computer program product configured to be executed by the control circuitry 11. In other words, device 10 includes one or more memory storage areas 12 containing program code, the one or more memory storage areas 12 and the program code being configured to cause device 10 to perform method S100 according to any of the embodiments disclosed herein, using one or more processors 11. However, for better illustration of the embodiments disclosed herein, the control circuitry... Figure 6 The components are represented as various “modules” or blocks, each of which is linked to one or more specific functions of the control circuit.

[0063] Figure 7 It includes Figure 6 The diagram shows a schematic of a vehicle 1 equipped with an ADS for such a device 10. As used herein, "vehicle" refers to any form of motorized transport. For example, vehicle 1 can be any road vehicle, such as a car (as shown herein), a motorcycle, a (freight) truck, a bus, etc.

[0064] Device 10 includes control circuitry 11 and memory 12. Control circuitry 11 may physically comprise a single circuit device. Alternatively, control circuitry 11 may be distributed across several circuit devices. As an example, device 10 may share its control circuitry 11 with other parts of vehicle 1 (e.g., ADS 310). Furthermore, device 10 may form part of ADS 310, i.e., device 10 may be implemented as a module or feature of ADS. Control circuitry 11 may include one or more processors such as a central processing unit (CPU), microcontroller, or microprocessor. One or more processors may be configured to execute program code stored in memory 12 to implement various functions and operations of vehicle 1 in addition to the methods disclosed herein. The processor may be or include any number of hardware components for performing data or signal processing or for executing computer code stored in memory 12. Optionally, memory 12 includes high-speed random access memory such as DRAM, SRAM, DDR RAM, or other random access solid-state storage devices; and, optionally, includes non-volatile memory such as one or more disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. The memory 12 may include a database component, an object code component, a script component, or any other type of information structure for supporting various actions of this specification.

[0065] In the illustrated example, memory 12 further stores map data 308. Map data 308 can be used, for example, by the ADS 310 of vehicle 1 to perform autonomous functions of vehicle 1. Map data 308 may include high-definition (HD) map data. It is contemplated that even if memory 12 is illustrated as a separate element from ADS 310, it can be provided as an integrated element of ADS 310. In other words, according to exemplary embodiments, any distributed or local memory device can be utilized in implementations of the inventive concept. Similarly, control circuitry 11 can be distributed, for example, such that one or more processors of control circuitry 11 are provided as integrated elements of ADS 310 or any other system of vehicle 1. In other words, according to exemplary embodiments, any distributed or local control circuitry device can be utilized in implementations of the inventive concept. ADS 310 is configured to implement the functions and operations of autonomous or semi-autonomous functions of vehicle 1. ADS 310 may include multiple modules, each responsible for a different function of ADS 310.

[0066] Vehicle 1 includes several components common to autonomous or semi-autonomous vehicles. It will be understood that vehicle 1 is capable of having… Figure 7 Any combination of the various elements shown. Furthermore, vehicle 1 may include, in addition to... Figure 7 Other elements besides those shown herein. Although various elements are shown herein as being located inside vehicle 1, one or more of these elements can be located outside vehicle 1. For example, map data can be stored in a remote server and accessed by various components of vehicle 1 via communication system 326. Furthermore, as will be readily understood by those skilled in the art, even though various elements are described herein in a particular arrangement, these elements can be implemented in different arrangements. It should be further noted that various elements can be communicatively connected to each other in any suitable manner. Since the elements of vehicle 1 can be implemented in several different ways, therefore Figure 7 Vehicle 1 should be considered merely as an illustrative example.

[0067] Vehicle 1 further includes a sensor system 320. Sensor system 320 is configured to acquire sensing data about the vehicle itself or its surrounding environment. Sensor system 320 may, for example, include a Global Navigation Satellite System (GNSS) module 322 (e.g., GPS) configured to collect geographic location data of vehicle 1. Sensor system 320 may further include one or more sensors 324. Sensors 324 may be any type of onboard sensor such as a camera, lidar and radar, ultrasonic sensors, gyroscope, accelerometer, odometer, etc. It should be understood that sensor system 320 may also provide the possibility of acquiring sensing data directly or via dedicated sensor control circuitry in vehicle 1.

[0068] Vehicle 1 further includes a communication system 326. Communication system 326 is configured to communicate with external units such as other vehicles (i.e., via vehicle-to-vehicle (V2V) communication protocols), remote servers (e.g., cloud servers), databases, or other external devices (i.e., vehicle-to-infrastructure (V2I) or vehicle-to-everything (V2X) communication protocols). Communication system 318 can communicate using one or more communication technologies. Communication system 318 may include one or more antennas (not shown). Cellular communication technologies can be used for remote communication, such as to remote servers or cloud computing systems. Additionally, if the cellular communication technology used has low latency, it can also be used for V2V communication, V2I communication, or V2X communication. Examples of cellular radio technologies are GSM, GPRS, EDGE, LTE, 5G, 5G NR, etc., and future cellular solutions are also included. However, in some solutions, short-to-medium range communication technologies such as wireless local area networks (LANs) (e.g., solutions based on IEEE 802.11) can be used to communicate with other vehicles near vehicle 1 or with local infrastructure components. ETSI is developing cellular standards for vehicle communications, and 5G is considered a suitable solution due to its low latency and efficient handling of high bandwidth and communication channels.

[0069] The communication system 326 can accordingly provide the possibility for a remote location (e.g., a remote operator or control center) to send outputs and / or receive inputs from a remote location via one or more antennas. Furthermore, the communication system 326 can be further configured to allow various components of the vehicle 1 to communicate with each other. For example, the communication system can provide a local network setup such as CAN bus, I2C, Ethernet, fiber optics, etc. Local communication within the vehicle can also be a wireless type with protocols such as WiFi, LoRa, Zigbee, Bluetooth, or similar medium / short-range technologies.

[0070] Vehicle 1 further includes a control system 328. The control system 328 is configured to control the handling of vehicle 1. The control system 328 includes a steering module 330 configured to control the direction of vehicle 1. The control system 328 further includes a throttle module 332 configured to control the actuation of the throttle valve of vehicle 1. The control system 328 further includes a braking module 334 configured to control the actuation of the brakes of vehicle 1. The various modules of the control system 328 can also receive manual input from the driver of vehicle 1 (i.e., from the steering wheel, accelerator pedal, and brake pedal, respectively). However, the control system 328 can be communicatively connected to the vehicle's ADS 310 to receive instructions on how the various modules of the control system 328 should operate. Therefore, the ADS 310 can control the handling of vehicle 1, for example, via a decision and control module 318.

[0071] ADS 310 may include a positioning module 312 or a positioning block / system. The positioning module 312 is configured to determine and / or monitor the geographic location and orientation of vehicle 1, and may utilize data from sensor system 322, such as data from GNSS module 320. Alternatively or in combination, the positioning module 312 may utilize data from one or more sensors 324. Alternatively, the positioning system may be implemented as real-time dynamic (RTK) GPS to improve accuracy.

[0072] ADS 310 may further include a perception module 314 or a perception block / system 314. The perception module 314 may refer to any known module and / or function included in one or more electronic control modules and / or nodes of vehicle 1, adapted and / or configured to interpret driving-related perception data of vehicle 1 to identify, for example, obstacles, lanes, relevant signs, appropriate navigation paths, etc. Therefore, the perception module 314 may be adapted to rely on and obtain inputs from multiple data sources, such as automotive imaging, image processing, computer vision, and / or in-vehicle networking, in conjunction with sensing data from sensor system 320, for example.

[0073] The positioning module 312 and / or the sensing module 314 can be communicatively connected to the sensor system 320 to receive sensing data from the sensor system 320. The positioning module 312 and / or the sensing module 314 can further send control commands to the sensor system 320.

[0074] The present invention has been presented above with reference to specific embodiments. However, other embodiments besides those described above are also possible and are within the scope of the invention. Within the scope of the invention, method steps different from those described above, performed by hardware or software, can be provided. Therefore, according to exemplary embodiments, a (non-transitory) computer-readable storage medium is provided storing one or more programs configured to be executed by one or more processors of a vehicle control system, the programs including instructions for performing the methods according to any one of the above embodiments. Alternatively, according to another exemplary embodiment, a cloud computing system can be configured to perform any of the methods presented herein. The cloud computing system may include distributed cloud computing resources that collectively perform the methods presented herein under the control of one or more computer program products.

[0075] Generally, computer-accessible media can include any tangible or non-transitory storage medium or memory medium, such as electronic, magnetic, or optical media, for example, a disk or CD / DVD-ROM connected to a computer system via a bus. As used herein, the terms “tangible” and “non-transitory” are intended to describe computer-readable storage media (or “memory”) that do not include propagating electromagnetic signals, but are not intended to otherwise limit the types of physical computer-readable storage devices included in the term computer-readable media or memory. For example, the terms “non-transitory computer-readable medium” or “tangible memory” are intended to cover types of storage devices that do not necessarily permanently store information, including, for example, random access memory (RAM). Program instructions and data stored in a non-transitory form on a tangible computer-accessible storage medium can be further transmitted via a transmission medium or a signal, such as an electrical, electromagnetic, or digital signal, that can be transmitted via a communication medium such as a network and / or a wireless link.

[0076] The processor (associated with control circuitry 11) may be or include any number of hardware components for performing data or signal processing or for executing computer code stored in memory 12. Device 10 has an associated memory 12, and memory 12 may be one or more means for storing data and / or computer code for performing or facilitating the various methods described herein. Memory may include volatile or non-volatile memory. Memory 12 may include database components, object code components, script components, or any other type of information structure for supporting the various actions of this specification. According to exemplary embodiments, any distributed or local storage device may be used with the systems and methods of this specification. According to exemplary embodiments, memory 12 (e.g., via circuitry or any other wired, wireless, or network connection) may be communicatively connected to processor 11 and includes computer code for performing one or more of the processes described herein.

[0077] Therefore, it should be understood that multiple parts of the described solution can be implemented in vehicle 1, in a system located outside vehicle 1, or in a combination of inside and outside the vehicle; for example, in a server communicating with the vehicle, i.e., a so-called cloud solution. For example, sensor data can be sent to an external system, and this system performs steps for comparing the sensor data (motion of other vehicles) with a predetermined behavior model. Different features and steps of the embodiments can be combined in combinations other than those described.

[0078] It should be noted that no reference numerals in the drawings limit the scope of the claims. The invention can be implemented, at least in part, by both hardware and software, and multiple “devices” or “units” can be represented by the same hardware item.

[0079] Although the accompanying drawings may show a specific order of method steps, the order of steps may differ from the depicted order. Furthermore, two or more steps may be performed simultaneously or partially concurrently. This variation will depend on the chosen software and hardware system and the designer's choices. All these variations are within the scope of this invention. Similarly, standard programming techniques with rule-based logic and other logic can be used to accomplish the various connection steps, processing steps, comparison steps, and decision steps. The embodiments mentioned and described above are given by way of example only and should not be construed as limiting the invention. Other solutions, uses, objectives, and functions within the scope of the invention claimed in the patent claims described below will be apparent to those skilled in the art.

[0080] AD characteristics / functions

[0081] ADS includes, preferably, Level 3 or higher ADS features (also referred to as ADS functions) of SAE J3016 level 3 or higher for driving automation of road vehicles. ADS features can be, for example, traffic jam navigation, highway navigation, or any other SAE J3016 Level 3 or higher ADS features.

[0082] Geographical location

[0083] In this context, the vehicle's geographical location is interpreted as its map location (also known as its location within the map). In other words, geographical location or map location can be understood as a set (two or more) of coordinates in a global coordinate system.

[0084] Surrounding environment

[0085] The surrounding environment of this vehicle can be understood as the general area around this vehicle where objects (e.g., other vehicles, landmarks, obstacles, etc.) can be detected and identified by vehicle sensors (radar, lidar, cameras, etc.) (i.e., within the sensor range of this vehicle).

[0086] The term "if"

[0087] As used herein, the term “if” may be interpreted, depending on the context, as meaning “when”, “based on”, “in response to determination”, or “in response to detection”. Similarly, the phrases “if determined”, “when determined”, or “in the instance of” may be interpreted, depending on the context, as meaning “when determined”, “in response to determination”, “when the occurrence of an event is detected and identified”, or “in response to the occurrence of an event being detected”.

[0088] get

[0089] The term "acquire" is to be interpreted broadly herein and includes receiving, retrieving, collecting, acquiring, etc., directly and / or indirectly between two entities configured to communicate with each other or further with other external entities. However, in some embodiments, the term "acquire" will be interpreted as determining, deriving, forming, calculating, etc. In other words, acquiring the attitude of a vehicle may include determining or calculating the vehicle's attitude based on, for example, GNSS data and / or perception data and map data. Therefore, as used herein, "acquire" may indicate receiving parameters at a first entity / unit from a second entity / unit, or determining parameters at a first entity / unit, for example, based on data received from another entity / unit.

Claims

1. A method (S100) for determining a free space (218, 220, 222, 224, 300, 302, 400) adjacent to a vehicle (200, 1), wherein, The vehicle comprises one or more sensors (324) arranged to detect objects adjacent to the vehicle and a device (10) for processing sensor data, the method comprising: obtaining (S102) target object data defining one or more target object types to be detected; obtaining (S104) the sensor data from the one or more sensors; detecting (S106) one or more instances of the target object types in the sensor data; determining (S108) spatial position data of the one or more instances of the object types, wherein the spatial position data comprises a set of instance position data respectively defining a position of the instances; and implicitly determining (S110) the free space as a space between the vehicle and the set of instance position data by determining (S112) one or more unobstructed lines of sight between the sensors of the vehicle and the set of instance position data and by generating (S114) the free space by combining the one or more unobstructed lines of sight.

2. The method according to claim 1, further comprising: obtaining (S116) sensor health monitoring data from the one or more sensors, wherein the sensor health monitoring data comprises information about a current capability of the sensors to detect the objects adjacent to the vehicle, wherein the step of detecting the one or more instances of the target object types in the sensor data is performed while taking into account the current capability of the sensors reflected in the sensor health monitoring data.

3. The method of any one of claims 1-2, wherein, The objects comprise static objects and dynamic objects, the method further comprising: estimating (S118) a reliability of the free space by mapping the set of instance position data of the static objects to a set of reference position data of reference static objects and / or by implicitly determining the free space by detecting three-dimensional objects in the free space by using multiple sensors.

4. The method of claim 2, wherein, The sensor health monitoring data comprises information about elements of the sensors being obstructed, elements of the sensors not functioning and / or elements of the sensors being blinded by light.

5. The method according to claim 1, further comprising: determining (S120) explicit free space (402) adjacent to the vehicle by detecting instances of arbitrary objects independent of the object types; and combining (S122) the implicitly determined free space with the explicit free space into combined free space.

6. The method according to claim 1, further comprising: transmitting (S124) free space data comprising the implicitly determined free space to an autonomous driving system, ADS, such that a maneuver of the vehicle can be performed using the free space data.

7. The method of claim 6, wherein, The free space data is used in the ADS for adjusting a speed of the vehicle.

8. The method of claim 1, wherein, The free space comprises a plurality of free subspaces (218, 220, 222, 224) and the target object data defines a plurality of target object types, wherein each of the free subspaces is linked to one of the plurality of target object types.

9. The method of claim 8, wherein, Different modules of the ADS use a combination of one or more of the free subspaces as input.

10. The method according to claim 1, further comprising: obtaining (S126) map data comprising height data; determining (S128) a map data based line of sight region (404) based on the map data, wherein the step of detecting one or more instances of the object type is performed while taking the map data based line of sight region into account.

11. The method of claim 1, wherein, The target object type comprises size specific objects that satisfy one or more geometric conditions.

12. The method of claim 1, wherein, The sensor comprises one or more lidars, one or more radars, and / or one or more cameras.

13. A computer program product comprising instructions which, when executed by a computing device, cause the computing device to implement the method according to claim 1.

14. A device (10) for determining a free space (218, 220, 222, 224, 300, 302, 400) adjacent to a vehicle, the device comprising a control circuit (11) configured to: obtain target object data defining one or more target object types to be detected; obtain sensor data from one or more sensors of the vehicle; detect one or more instances of the object type; determine spatial position data of the one or more instances of the object type, wherein the spatial position data comprises a set of instance position data respectively defining a position of the instances; and implicitly determine the free space as a space between the vehicle and the set of instance position data by determining one or more unobstructed lines of sight between the sensors of the vehicle and the set of instance position data and generating the free space by combining the one or more unobstructed lines of sight.

15. A vehicle (1) comprising: one or more sensors arranged to detect objects in front of the vehicle, and a device (10) according to claim 14.

Images