Method for detecting and locating a non-line-of-sight object
The non-line-of-sight obstacle detection system using a Doppler radar with MIMO array radar processes multipath reflections to identify and locate objects beyond the line of sight, addressing limitations of existing systems and improving safety in automated vehicles.
Patent Information
- Application Number
- DE102017106931
- Authority / Receiving Office
- DE · DE
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2016-05-23
- Filing Date
- 2017-03-30
- Publication Date
- 2024-11-28
- Estimated Expiration
- 2037-03-30
AI Technical Summary
Current obstacle detection systems rely on line-of-sight methods, limiting their effectiveness in scenarios where objects are not within the direct line of sight, such as in vehicle collision avoidance systems.
A non-line-of-sight obstacle detection and localization system using a Doppler radar system with a MIMO array radar, combining direct and multipath reflections to identify and locate objects beyond the line of sight by processing reflections with a transmitter, receiver, and processing system, incorporating landscape information and motion parameters.
Enables the detection and localization of objects not in the direct line of sight, enhancing safety in automated systems by providing advanced collision avoidance capabilities.
Smart Images

Figure 00000000_0000_ABST
Abstract
Description
AREA OF INVENTION
[0001] The present invention relates to obstacle detection and in particular to methods for detecting and locating a non-line-of-sight object according to the preamble of claim 1, as is known essentially from DE 10 2015 210 069 A1.
[0002] Further state of the art can also be found in the publications DE 10 2004 058 844 A1, DE 103 11 959 B4, DE 10 2014 002 435 A1 and DE 10 2011 119 767 A1. BACKGROUND
[0003] Obstacle detection in its various forms is an integral part of numerous systems. In automated manufacturing facilities, for example, machines that transport components and equipment to different areas of the plant must detect and avoid obstacles. Another example is that automated vacuum cleaners must detect and avoid obstacles such as stairs. Obstacle detection is also a crucial task for increasingly automated vehicles. Currently, obstacle detection focuses on identifying obstacles within the line of sight. Therefore, providing non-line-of-sight obstacle detection and localization capabilities is desirable. SUMMARY OF THE INVENTION
[0004] According to the invention, a method for detecting and locating a non-line-of-sight object is presented, characterized by the features of claim 1.
[0005] Furthermore, a non-line-of-sight obstacle detection and localization system is described.The system is arranged on a movable platform and comprises a transmitter section configured to transmit radio frequency signals from a variety of transmission elements; a receiver section configured to receive reflections from a variety of receiving antenna elements, including direct and multilateral reflections; and a processing system configured to identify reflections associated with static targets, retain reflections associated with moving targets, distinguish between line-of-sight and non-line-of-sight objects among the moving targets, locate the non-line-of-sight objects relative to the platform, and approach non-line-of-sight objects among the non-line-of-sight objects, with approaching non-line-of-sight objects moving towards the platform along a path that intersects the platform.
[0006] The aforementioned features and advantages, as well as further features and advantages of the invention, are readily apparent from the following detailed description of the invention in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0007] Other features, advantages and details appear, only by way of example, in the following detailed description of the embodiments and the detailed description which refers to the following drawings: Fig. 1 is a representation of the non-line-of-sight obstacle detection according to embodiments; Fig. Figure 2 is a block diagram of the detection system according to embodiments; and Fig. 3 is a process flow of a method for performing non-line-of-sight obstacle detection according to embodiments. DESCRIPTION OF THE EXECUTION FORMS
[0008] As mentioned above, obstacle detection is part of the operation of many systems that incorporate automated path control. Depending on the location and distance, different types of obstacle detection can be used. For example, automated vacuum cleaners that need to detect obstacles on the order of inches away can use infrared transmissions and reflections. In other applications, such as vehicle and aircraft applications, where obstacle detection over longer distances is of interest, radio-based detection and range (radar) are generally used. In general, radar systems transmit radio waves and determine the range, angle (azimuth and elevation), and speed of an object based on the reflection of the radio waves from the object. As such, typical radar detection relies on a line of sight to the object (target) being detected.However, in vehicle collision avoidance systems, for example, there is an interest in detecting objects that are not yet within the line of sight of the vehicle sensors. In accordance with exemplary embodiments of the invention, non-line-of-sight obstacle detection and localization is facilitated. As described in more detail below, radar data is combined with a predefined or learned model of the current topology to derive information about objects that lie outside the line of sight. While the exemplary case of vehicle-based radar systems is described for illustrative purposes, the embodiments described herein are not limited to use in a vehicle system. Other vehicles (e.g., construction machinery, agricultural equipment) and other types of platforms are also considered.Additionally, while a Doppler radar system is discussed herein as an exemplary embodiment of a non-line-of-sight detection system, any sensor system that provides range, azimuth, height, and velocity information may be used in accordance with the detailed embodiments.
[0009] Fig. Figure 1 is a representation of the non-line-of-sight obstacle detection according to embodiments. An exemplary intersection is shown, and the exemplary platform 10 for the non-line-of-sight obstacle detection system 110 is a vehicle. A host vehicle 100, which includes the detection system 110 (2) according to the embodiments described below, is shown at the exemplary intersection. The host vehicle 100 may include other sensors 105 (e.g., camera, lidar system). Other vehicles 120a, 120b (which may also include the detection system 110) and other objects 130 (which are buildings in the exemplary representation) are also shown in Figure 1. Fig. Figure 1 shows that the other objects 130a and one of the other vehicles 120a are within the line of sight of the detection system 110 of the host vehicle 100. One of the other objects 130b and the other vehicle 120b are not within the line of sight of the detection system 110 of the host vehicle 100. That is, the transmissions 215 from the detection system 110 of the host vehicle 100 cannot directly detect vehicle 120b or object 130b based on the relative positions of the host vehicle 100 and vehicle 120b or the object 130b shown in Figure 1. Fig. 1. Object 130b shown. As such, reflections 225 directly from vehicle 120b or object 130b cannot be obtained with the detection system 110 of the host vehicle 100.
[0010] However, how do the dashed lines in Fig. As indicated by the signal 1, transmissions 215 from the detection system 110 of the host vehicle 100 can bounce off the other vehicle 120a and other objects 130a and reach the vehicle 120b or the object 130b that are outside the line of sight of the detection system 110 of the host vehicle 100. Reflections 225 from the vehicle 120b or the object 130b can also bounce off the other vehicle 120a and other objects 130 to reach the host vehicle 100. These bounced signals are called multipath signals because, based on a transmission 215 by the detection system 110, reflections 225 can result from multiple paths. To be clear, only one of the direct transmissions 215x and the resulting reflections 225x within the line of sight of the host vehicle 100 (to and from the vehicle 120a and the other objects 130) is shown, in order to obscure the multipath impact signals (215 / 225) of interest according to the embodiments.For example, a transmission 215x can be reflected by the other object 130a and leads to a reflection 225x directly back to the detection system 110, and additionally, a transmission 215 can bounce off the other object 130a and lead to a reflection 225 that actually originates from the non-line-of-sight object of the vehicle 120b.
[0011] In a typical radar system, these bounced transmissions 215 and resulting reflected reflections 225, or multipath signals, are an undesirable effect, while the direct transmissions 215 and reflections 225 from targets within the line of sight (vehicle 120, objects 130) are of interest. However, according to embodiments of the invention, these multipath reflections 225 are isolated and processed to perform non-line-of-sight obstacle detection and localization. The detection system 110 is described with reference to Fig. 2 executed.
[0012] Fig. Figure 2 is a block diagram of the detection system 110 according to the embodiments. The detection system 110, which is a Doppler radar system, is an exemplary embodiment, but any sensor system that provides similar information (e.g., speed, range, azimuth, elevation) can be used in alternative embodiments. Vehicle detection systems 110 used in vehicle platforms 10, such as the host vehicle 100, can generally operate in continuous linear wave frequency modulation (CW-LFM) and can operate in frequency ranges from 21 to 27 gigahertz or 77 to 81 gigahertz. Power is transferred and received over a number of cycles to perform the non-line-of-sight obstacle detection and localization according to the embodiments discussed herein. Information obtained from the detection system 110 can be supplied to the vehicle control systems 270 and the display systems 280.The detection system 110 can also communicate with other sensor systems 105 of the host vehicle 100. The vehicle control systems 270 can include, for example, automatic braking or steering control systems. The display systems 280 can show the driver a non-line-of-sight object (e.g., another vehicle 120b). Fig. 1) Display. Other sensor systems 105 of the host vehicle 100 may include imaging systems (e.g., a global positioning system (GPS)), visual systems (e.g., mono or stereo camera systems), and distance systems (e.g., LiDAR). These other sensor systems 105 can facilitate the determination of the location of the host vehicle 100 and the creation of a model of the topology at the location.
[0013] The detection system 110 comprises a transmitter section 210, a receiver section 220, and a processing system 230. The detection system 110 can be a multiple-input multiple-output (MIMO) array radar, as shown. Thus, the transmitter section 210 can include multiple antenna elements 214 that transmit multiple signals 215, and the receiver section 220 can include multiple antenna elements 224 to receive the reflections 225. Such reflections are obtained over an azimuth and altitude range by the arrays of elements 214 and 224. The detection system 110 can use known techniques such as beamforming at the antenna elements 224 of the receiver section 220 and specifically use the Doppler effect to determine the velocity of the detected objects. Range and power (intensity) are also obtained from the reflections 225.Thus, the array of antenna elements 214, 224 facilitates the acquisition of an image in which each pixel is assumed to be assigned an azimuth, altitude, range, and velocity value, as well as an intensity. Additionally, the detection system 110 can use a topology model (which specifies the targets in the line of sight) to simplify the identification of non-line-of-sight objects.
[0014] The transmitter section 210 and the receiver section 220 are known and are not described in detail here. As shown in the extended view in Fig. As shown in Figure 2, the transmitter section 210 generally includes an oscillator 211, a buffer 212, and a power amplifier 213, and the receiver section 220 generally includes a preamplifier 221, a mixer 222, and an analog-to-digital converter (A / D converter) 223. The processing system 230 includes one or more storage devices 240 and one or more processors 250 to perform the non-line-of-sight obstacle detection. While the processing system 230 is shown as part of the detection system 110 and is separate from other vehicle control systems 270, for example, the processing system 230, which processes the reflections 225 to perform non-line-of-sight obstacle detection, can be shared among one or more systems in the host vehicle 100.Communication between the various systems (110, 270, 280, 105) can be based on hardwired or wireless communication, or on a combination of known communication schemes, including, for example, via a shared bus. The processing of reflections 225, performed by the processing system 230 to identify and locate non-line-of-sight objects approaching the host vehicle 100, is described with reference to... Fig. 3 described.
[0015] Fig. Figure 3 describes a process flow for performing non-line-of-sight obstacle detection and localization according to the embodiments. The exemplary embodiments described below relate to the reflections 225 received by the detection system 110 for illustrative purposes. As mentioned above, information (range, velocity, azimuth, height) used to perform non-line-of-sight detection can instead be obtained from other known sensor systems or other configurations of radar systems. In Block 310, receiving reflections 225 includes receiving both direct and multipath reflections 225 based on the existing objects. Receiving reflections 225 also includes performing multiple transmissions 215 from each transmitting antenna element 214 and receiving multiple reflections 225 at each receiving antenna element 224.The processing of reflections 225 at block 320 involves several processes 330, 340, 350, as shown. In addition to the reflections 225 obtained in block 310, other information (obtained at blocks 325 and 335) is used to identify non-line-of-sight objects (e.g., another vehicle 120b). Fig. 1) to capture and locate. Obtaining other information at Block 325 refers to obtaining information from other sensors 105 or other processing systems of the platform 10 (e.g., host vehicle 100). The other information may include landscape information about the topology of the current location of the platform 10 (host vehicle 100). The landscape information may, for example, include the location of objects 130 such as buildings. This information may be provided as input for the host vehicle 100 or may be learned by the host vehicle 100 from previous visits to the same location according to a known dynamic learning algorithm. Monitoring host vehicle 100 movement parameters at Block 335 may involve monitoring other sensors 105, such as a GPS receiver of the host vehicle 100, to determine position and movement. How Fig. 3 is displayed, the location information obtained by monitoring the host vehicle 100 (at block 335) is needed to obtain other information, such as landscape information (at block 325).
[0016] The identification of reflections 225 from static environments, at block 330, refers to the identification of pixels with zero velocity (zero Doppler). These pixels can then be assigned to non-moving objects (e.g., objects 130a, 130b, Fig. 1) be assigned. Identification at block 330 can be supported if landscape information is available (from block 325 onwards). The landscape information helps to distinguish between the stationary objects 130 in the scene. That is, if the vehicle 120a in Fig. When block 1 is stopped, it may appear as a non-moving object. However, the landscape information (from block 325) can be used to distinguish vehicle 120a (which may be stationary but not moving at the time of processing) from objects 130 (which are both stationary and moving at the time of processing). The identification of reflections 225 from static environments at block 330 can first be performed as a kind of filtering of the reflections 225 to isolate those associated with moving (or moving) objects (both within and outside the line of sight of the host vehicle 100 of the monitoring system 110). This filtering of static objects (at block 330) involves distinguishing between line of sight (e.g., 120a) and non-line of sight (e.g., another vehicle 120b). Fig. 1) the objects in Block 340 move. As part of the processing at Block 340, pixels associated with a movement can first be used to identify objects based on a known clustering algorithm. The remainder of the processing at Block 340 then involves categorizing the objects as being inside or outside the line of sight of Platform 10.
[0017] The differentiation of line-of-sight moving objects from non-line-of-sight moving objects, as described in block 340, can be performed according to various embodiments. According to one embodiment, other sensors 105 can be used based on the information obtained in block 325. For example, a camera (105) mounted on the host vehicle 100 can be used, and a known motion object detection procedure can be performed within the camera's field of view. The azimuth and elevation of pixels associated with any moving objects in the camera's field of view can be translated into an azimuth and elevation corresponding to the field of view of the detection system 110.If the translated azimuth and elevation values match azimuth and elevation values of moving objects or reflections 225 that are not filtered out as static (at block 330), then these objects or reflections 225 are associated with a line of sight. According to another embodiment, a known statistical modeling approach is used on reflections 225 that are associated with moving objects. Once the non-line-of-sight moving objects are identified from line-of-sight moving objects (at block 340), the non-line-of-sight moving objects (e.g., 120b) are located at block 350, obtaining other information (at block 325), as in . Fig. 3 is shown. The other information can, for example, include a diagram of the current position of host vehicle 100. Specifying an approaching non-line-of-sight moving object in block 360 involves determining, based on the localization (at block 350), whether the non-line-of-sight moving object of the line-of-sight moving object intersects with host vehicle 100. For example, in the case of the Fig.In the scenario shown, the non-line-of-sight object intersects with the host vehicle 100 if both vehicles 100 are traveling on the current path. However, in another exemplary scenario, the path on which the other vehicle 120b is shown may pass under the path on which the host vehicle 100 is shown. In such a scenario, the other vehicle 120b and the host vehicle 100 would not intersect. As such, the localization (at block 350), which uses a map, can prevent the other vehicle 120b from being displayed as an approaching non-line-of-sight object at block 360. The display provided at block 360 can be visible on a map (via a display system 280), which is visible to the driver of the host vehicle 100, as a warning of the approaching object (e.g., another vehicle 120b, 1). In additional or alternative embodiments, the display can be connected to a vehicle control system 270 (e.g.Collision avoidance or automated steering system) to facilitate decisions regarding the control of the host vehicle 100 based on the position and movement of the non-line-of-sight object(s).
Claims
[1] A method for detecting and locating a non-line-of-sight object (120b), the method comprising: Receiving reflections at a detection system (110) of a movable platform (10), the reflections including direct and multi-part reflections; Obtaining information from sensors (105) other than the sensing system (110), obtaining a location of the platform (10) and a map of the location; Identify the reflections associated with static targets in order to retain the reflections associated with moving targets; Identify the static targets on the map; Distinguishing between line-of-sight objects (120a) and non-line-of-sight objects (120b) among the moving targets; Localizing the non-line-of-sight objects (120b) relative to the platform (10); and Displaying approaching non-line-of-sight objects (120b) among the non-line-of-sight objects (120b), wherein the approaching non-line-of-sight objects (120b) move toward the platform (10) on a path that intersects the platform (10); characterized by , that the non-line-of-sight objects (120b) are located by determining the location of the non-line-of-sight objects (120b) on the map; and that determining on the basis of the map whether the non-line-of-sight objects (120b) are approaching non-line-of-sight objects (120b); wherein obtaining information comprises receiving information about moving objects from a camera, wherein distinguishing between the line-of-sight objects (120a) and the non-line-of-sight objects (120b) comprises translating heading and elevation angles of pixels associated with moving objects in the field of view of the camera into heading and elevation angles associated with the field of view of the detection system (110).
Citation Information
Patent Citations
Traffic condition detecting method, involves detecting virtual reflection points by looped around radar radiations, where reflection points are included in computation for detection of traffic condition
DE102004058844A1
APPEARANCE-BASED UNION OF CAMERA AND DISTANCE DATA FOR MULTI-OBJECTS
DE102011119767A1
Method for representation of environment information of vehicle, involves evaluating radar echo signals as radar data, where radar echo signals are transferred from direct signal path between radar device and reflected objects
DE102014002435A1
DRIVING ASSISTANCE DEVICE
DE102015210069A1
Object detection device, radar device and method for detecting an object
DE10311959B4
