Perceptual sensor for determining precise spatial coordinates of various targets in visible area using spatially separated receive / transmit antenna groups

Abstract

The present disclosure discloses a perceptual sensor that uses spatially separated receive / transmit antenna groups to determine precise spatial coordinates of various targets in a visible area. Systems, devices, methods, and instructions for determining spatial coordinates of a target in a visible area using a radar device having one or more spatially separated transmit and receive antenna groups, including: receiving at least one data set from at least one radar sensor, including a target response and coordinates of the at least one transmit and receive antenna group; calculating one or more address trajectories of the spatial distribution points forming the grid with the relative coordinates; and comparing the frames sequentially or synchronously received from the one or more radar sensors to determine coordinates of the target.

Description

Technical Field

[0001] Embodiments of the present invention generally relate to radar, and more specifically to systems, apparatus, methods, and instructions for determining the location of one or more observed targets in a visible area (e.g., line of sight) using one or more spatially separated receiving / transmitting antenna arrays. Background Technology

[0002] Current radar systems offer radar rangefinders mounted on rotating support equipment and radar devices based on multiple-input multiple-output (“MIMO”) technology. MIMO technology, applied to environmental sensors (e.g., MIMO sensors), transmits signals and receives echo signals. Radar sensor devices independently transmit signals through multiple transmitting antennas and independently receive echo signals through multiple receiving antennas.

[0003] The signals received by each receiving antenna from its respective transmitting antenna form a pair. Data from each pair is processed independently. Within each pair, the geometric arrangement of the receiving and transmitting antennas is unique, allowing each pair to be considered an independent “virtual antenna.” These virtual antennas collectively form a virtual antenna array (i.e., a phased array antenna or “PAA”).

[0004] MIMO-based devices are widely used in environmental control systems, for example, when solving the problem of detecting obstacles around a mobile platform (e.g., a carrier) on which the device is mounted. For instance, MIMO-based devices can be used in the autonomous navigation systems of driverless vehicles. In another example, MIMO-based devices can be used in vehicles of various levels of automation, including automobiles.

[0005] Several problems exist associated with the current use of radar sensors. A major drawback of radar sensor-based solutions is the high noise level caused by multipath interference. This problem is widely known and is relevant to all radar systems, including communication systems and radars, as well as those using MIMO technology. Multipath interference causes the device to record both the distance to the actual obstacle and the signal generated by the random superposition of secondary echo signals in each individual measurement cycle.

[0006] Eliminating interference noise is a current research area, and the specific methods for doing so may differ for each individual device. A common approach in well-known techniques is to aggregate several measurements at different time points or at different sensor locations. However, a general solution to this problem remains lacking.

[0007] Another issue with MIMO-based sensors and other similar radar sensors is determining the location of an obstacle within a measurement cycle by analyzing and comparing signals received from different directions relative to the device (i.e., echo responses). The accuracy of determining the angular coordinates of the detected target is determined by the accuracy of the method or means used to calculate that angle.

[0008] The method based on the physical rotation of the sensor in the direction of the target is used in many devices. This method is used in optical detection and ranging (“LIDAR”) systems, which utilize mechanically movable mirrors in the sensor and radar that utilizes movable antennas mounted on mechanically rotating devices.

[0009] MIMO methods used to form virtual antennas are limited by the physical size of their sensors (i.e., aperture) and have limited accuracy in determining direction toward obstacles. Synthetic Aperture Radar (“SAR”) methods—which synthesize larger virtual antennas to achieve any desired angular resolution—are used in radar. SAR methods enable decimeter-level spatial resolution from virtually any distance on the ground. It is this technology used in Earth remote sensing radars mounted on spacecraft, ensuring their operation at distances of approximately 1000 kilometers from targets.

[0010] However, the application of SAR technology in autonomous driving is limited because it requires long observation times and has an inherent delay before data is received. SAR technology is not currently used in commercially available (i.e., publicly accessible) real-time monitoring equipment, including vehicles equipped with driver assistance technologies. Summary of the Invention

[0011] Therefore, the present invention relates to systems, apparatus, methods, and instructions for determining the location of one or more observed targets, which substantially eliminate one or more problems caused by the limitations and disadvantages of related technologies.

[0012] Embodiments of the present invention provide systems, apparatus, methods, and instructions for determining the location of one or more observed targets within a visible area (e.g., line of sight) using one or more spatially separated groups of receiving / transmitting antennas. In some embodiments, this is achieved by comparing two or more datasets in a coordinate system, the datasets being obtained from one or more groups of receiving / transmitting antenna devices spaced apart at known locations.

[0013] Additional features and advantages of the invention will be set forth in the following description, and will be apparent from the description or may be learned by practice of the invention. The objects and other advantages of the invention will be realized and obtained by means of the structures particularly pointed out in the written description and its claims, as well as the accompanying drawings.

[0014] To achieve these and other advantages, and in accordance with the purposes of the invention, as embodied and broadly described, systems, apparatuses, methods, and instructions for determining the location of one or more observed targets include systems, apparatuses, methods, and instructions for determining the spatial coordinates of targets in a visible area using radar apparatus having one or more spatially separated groups of transmitting and receiving antennas, comprising: receiving at least one dataset from at least one radar sensor, including target responses and coordinates of at least one group of transmitting and receiving antennas; calculating one or more address trajectories of spatially distributed points forming a grid with relative coordinates; and comparing frames received sequentially or synchronously from one or more radar sensors to determine the coordinates of the targets.

[0015] In another aspect, additionally or in combination with other aspects, the implementation is performed in real time and / or continuously and / or periodically. In another aspect, additionally or in combination with other aspects, data from at least one radar sensor is stored in memory and subsequently retrieved. In another aspect, additionally or in combination with other aspects, data from at least one radar sensor is received in the form of a set of recorded analog-to-digital conversion (“ADC”) data. In another aspect, additionally or in combination with other aspects, the data from the sensors contains not only information about coordinates but also information about the moving speed of the target and / or platform. In another aspect, additionally or in combination with other aspects, the radar sensors are capable of moving relative to each other during system operation. In another aspect, additionally or in combination with other aspects, address trajectories are obtained in the form of formulas. In another aspect, additionally or in combination with other aspects, a probability map of target detection is stored in memory instead of an obstacle map. In another aspect, when no other information source is available in the system, sensor data is used to obtain odometry data. In another aspect, additionally or in combination with other aspects, a thermal map is used instead of discrete prediction frames, on which, instead of points, the probability of detecting a target at each point is determined. On the other hand, additionally or in combination with other aspects, a standalone MIMO-based radar system is used as a radar sensor. Alternatively, additionally or in combination with other aspects, instead of a radar sensor, one or more sensors may be used that receive the same information but use other physical principles, such as LiDAR, sonar, camera equipment, thermal imaging cameras, and / or various types of stereo pairs.

[0016] It should be understood that both the foregoing general description and the following detailed description are exemplary and illustrative, and are intended to provide further explanation of the claimed invention. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and, together with the specification, serve to explain the principles of the invention.

[0018] Figure 1 A block diagram of the sensing sensor is shown.

[0019] Figure 2 An example method is shown for determining the precise spatial coordinates of various targets in the visible area using a radar device with one or more spatially separated transmit and receive antenna groups.

[0020] Figure 3 An example of a perception sensor with two radar sensors installed on a vehicle is shown.

[0021] Figure 4 A schematic diagram is shown of a coordinate system called a map and different arrangements of grid nodes on it. Specifically, Figure 4 a) in the diagram illustrates the use of a uniform grid; and Figure 4 b) in the figure illustrates the use of a non-uniform grid in polar coordinates.

[0022] Figure 5 The relationship between the frames generated during the operation, the predicted frames, and the address trajectories is shown.

[0023] Figure 6 The diagram shows sensor data obtained from a sensor and the results of forming frames based on the sensor data.

[0024] Figure 7 An example of combining frame data and grid data during the generation of a global map is shown. Detailed Implementation

[0025] The embodiments of the present invention will now be described in detail, examples of which are shown in the accompanying drawings.

[0026] Embodiments of the present invention can provide systems, apparatus, methods, and instructions for determining the location of one or more observed targets within a visible area (e.g., a line of sight) using one or more spatially separated groups of receiving / transmitting antennas. In some embodiments, this can be achieved by comparing two or more datasets in a coordinate system, said two or more datasets being obtained from one or more groups of receiving / transmitting antenna devices spaced apart at known locations.

[0027] An example use case of the implementation will now be described. First, a terrain coordinate system can be generated in the device memory, within which the position of the target can be calculated. Such a coordinate system is called a "map". A "grid" is applied to the map, with a set of abstract points located on the map at a specific step size. The grid step size determines the accuracy of determining the target coordinates as well as the computational cost.

[0028] In the iterative method, each iteration can be assigned a sequence number. At the beginning of each iteration, each radar device can collect data about the surrounding space (e.g., within line of sight). The data can generally include information about one coordinate (i.e., distance to the target) or several coordinates (i.e., distance to one or more targets and their angular coordinates) for each detected obstacle.

[0029] If the device generates only one coordinate information (e.g., distance to the target), the operation can utilize accurate knowledge of the spatial arrangement of at least two devices on the platform. If the device generates two or more coordinate information (e.g., distance to the target, angle, etc.), at least one radar sensor is sufficient to enable the invention to operate.

[0030] For each set of data obtained in the previous steps, a "frame" can be formed. Here, using data about the platform's movement, the coordinates of each device on the map can be recorded. For each set of data on the grid, a subset of points (i.e., grid nodes) can be selected, where obstacles can be located based on the device data. The collection of these subsets, along with information about the precise coordinates of the devices, can constitute a "frame".

[0031] For the entire grid and for each device, "predictive frames" can be generated based on data about the movement of the carrier at each step. These "predictive frames" contain a model of the frames received by the radar sensors in the case of an obstacle at each individual grid node. The set of predicted frames for each grid node can be represented as a discrete or continuous function of the coordinates of each obstacle, independent of time (or the number of iterations). Here, this function is referred to as the "address trajectory".

[0032] By using correlation functions, the obtained "frames" and the obtained "predicted frames" can be compared, and based on the data from previous iterations, a statistical probability of finding the target at these coordinates can be assigned to each grid node.

[0033] After at least two iterations and when the probability of detecting a target exceeds a reliability threshold (e.g., P = 0.95), the corresponding grid node can be considered a detected obstacle, and the corresponding address trajectory can be stored as an address trajectory containing the obstacle. In subsequent frames, it is also expected that signals will be received from this address trajectory in frames from the radar equipment. To reduce computational complexity, address trajectories containing obstacles can be excluded from further checks. The result of each iteration can be a grid on which the detected target is marked.

[0034] Figure 1 A block diagram of the sensing sensor 100 is shown. (As shown) Figure 1 As shown, the perception sensor (100) may include one or more transmit / receive antenna groups (110 and 120) connected to one or more radar sensors (130). The number of antennas and radars used can be determined by the desired size and shape of the device's field of view (FoV). For example, for a car, using two radars with a common antenna array consisting of two or three transmit antennas and two or four receive antennas may be sufficient. The perception sensor (100) may include one or more devices configured as digital signal processors (140) from their respective radar sensors (130). Digital signal processing data output by the digital signal processor (140) can be sent to a processor (150), where the spatial coordinates of each obstacle in the device's visible area can be calculated. In some cases using a virtual PAA, it may be preferred to use dedicated digital signal processing (i.e., a dedicated "DSP") to calculate the coordinates of obstacles. Although described separately, the functions of the digital signal processor (140) and the processor (150) can be combined in a single device or chip.

[0035] In some implementations, the sensing sensor (100) may additionally include a navigation system (170). The navigation system (170) can be used to obtain data about the movement of the vehicle (e.g., in...). Figure 2 In step 220), the precise spatial coordinates are determined.

[0036] In some implementations, the sensing sensor (100) may additionally include a storage device (160) in the device memory. The storage device (160) may store data received from the radar sensor (130) and the results of its processing.

[0037] The system diagram of the sensing sensor (100) shown is an example, and various modifications can be made within the spirit of the invention. For example, the implementation can be implemented as a separate program that replicates the function of the processor (150) as part of a virtual device that processes data recorded on a storage medium. Additionally or alternatively, the implementation can be implemented as a separate program that includes the use of a dedicated processor, such as a graphics card and / or other computing module of various architectures. In another example, the implementation can be implemented as embedded software, including embedded software for programmable logic (FPGA). In yet another example, the implementation can be implemented as a single device or one of several environmental sensors in a more complex system. In yet another example, the implementation can be implemented modularly, such that data is received from one or more different systems or individual sensors during the information reception process and after system operation is complete.

[0038] The implementation method can be readily applied to obtain data on platform movement using various information sources, including but not limited to: satellite navigation systems, inertial navigation, integrated motion sensors based on Doppler radar, data from photographic and video equipment, thermal imaging cameras, wheel position sensors, airspeed sensors, etc. Alternatively or additionally, the implementation method may use analytical functions describing the expected movement trajectory instead of data on platform movement. Platform movement data is collectively referred to as "odometer data."

[0039] Although not shown, the sensing sensor (100) may include a bus and / or other communication mechanisms configured to transfer information between various components of the system diagram, such as processors, memory, database memory, and other storage devices.

[0040] In addition, communication devices can enable connections between processors and other devices by encoding data sent from a processor to another device via a network and decoding data received from another system of the processor via a network.

[0041] The processor (150) may include one or more general-purpose or special-purpose processors to perform computational and control functions of the sensing sensor (100). The processor (150) may include a single integrated circuit, such as a microprocessor device, or may include multiple integrated circuit devices and / or circuit boards that work together to implement the functions of the processor (150). For example, an ARM-based processing unit may be used with Nvidia digital signal processing technology.

[0042] The system may include memory for storing information and instructions executed by the processor. The memory may contain various components for retrieving, presenting, modifying, and storing data. For example, the memory may store software modules that provide functionality when executed by the processor. The software modules may include an operating system that provides operating system functionality to the system. The software modules may also include modules for implementing the functions of the sensing sensor (100).

[0043] The memory may include a variety of computer-readable media that can be accessed by the processor (150). For example, the memory may include any combination of random access memory (“RAM”), dynamic RAM (“DRAM”), static RAM (“SRAM”), read-only memory (“ROM”), flash memory, cache memory and / or any other type of non-transitory or transient computer-readable media.

[0044] Although shown as a single system, the functionality of the sensing sensor (100) can be implemented as a distributed system. Furthermore, the functionality disclosed herein can be implemented on independent devices that can be communicatively coupled together. Additionally, one or more components of the sensing sensor (100) may be omitted.

[0045] Figure 2 An example method 200 is shown to determine the precise spatial coordinates of various targets in a visible area (e.g., the line of sight of the radar device) using a radar device having one or more spatially separated transmit and receive antenna groups. Method 200 comprises successive steps, which are described below.

[0046] In the following discussion, a map can describe a topographic coordinate system in which each point is described by a set of values, such as (x, y) coordinates in a planar Cartesian coordinate system; and an M-map can be characterized by continuous coordinate values. A grid can describe discrete partitions of a map, which are represented by a set of countable coordinate pairs (x, y). i y j ) and value Ψ(x) i y j This characterizes and determines the probability that an obstacle exists at a given coordinate.

[0047] At the initial time point, the probability map can be zero-filled. Under certain conditions, it is a uniform grid defined by the following relationship:

[0048] x i =i·hx,i=-N,...,-1,0,1,...,N

[0049] y j =j·hy,j=-M,...,-1,0,1,...,M (1) Where: i is the number of grid nodes along coordinate x; j is the number of grid nodes along coordinate y; hx is the step size of the uniform grid along coordinate x; hy is the step size of the uniform grid along coordinate y; and N and M are the boundaries of the grid region (e.g., Figure 4 The map shown and its grid.

[0050] Sensor data is a set of values ​​describing the response and the distance to the response. In general, this sensor data is described by a countable, finite array of triplets (r, α, p), where r is the distance to the target, α is the angle from the target normal, and p is the complex value of the response (its magnitude and phase, specified as an ordered pair). An example implementation uses the following definition of sensor data:

[0051]

[0052] (r i α i p i ), i = 1, 2, ..., P (2)

[0053] Where k1 is the number of iterations; k2 is the sensor number within the number of iterations; (x k1，k2 y k1，k2 ) represents the coordinates of sensor k2 within iteration k1 at the grid node; i represents the number of targets detected within sensor data k2 within iteration k1; P represents the number of targets detected by sensor k2 within iteration k1; r i α is the distance to the i-th target specified by accuracy dr, in meters; α∈[-π,+π] is the direction angle relative to the OX axis of the map to target i, specified by accuracy dα; p i =pa i +j·pb i , where pa i For the real part of the response, pb i The imaginary part of the response is j, and j is the imaginary unit.

[0054] A frame can be a set or subset of grid nodes, where the presence of obstacles does not contradict the sensor data in Equation (2). Frames may intentionally contain too many nodes, and one purpose is to avoid errors, i.e., missing the target. The specific method for determining the set of grid points based on the data in Equation (2) can depend on the implementation of various algorithms and can vary accordingly. Typically, a frame is defined by coordinate pairs and power data:

[0055] {p i，k1，k2 x i，k1，k2 y i，k1，k2} (3)

[0056] The prediction frame is a set or subset of grid nodes, where the location of obstacles is calculated. Unlike the frames in Equation (3), the calculation can be performed once for each iteration of the loop, taking into account all data from all sensors. Therefore, the calculation is independent of the number of sensors.

[0057] {p i，k1 x i，k1 y i，k1} (4)

[0058] An address trajectory is a set of grid nodes, but unlike the frames in Equations (3) and (4), an address trajectory can be limited to a series of interconnected points for each target over a period of time (i.e., a certain number of iterations k1).

[0059] {x i，k1 y i，k1} (5)

[0060] The frames, prediction frames, and address trajectories can be defined based on the following considerations: Frames describe obstacles visible to the sensor, address trajectories show how obstacles change over time, and prediction frames describe assumptions about the location of a target that correspond to the address trajectories and do not contradict the sensor data.

[0061] Odometry data can be a collection of different types of data, making it possible to determine the position of each sensor at each iteration of the loop. In an example implementation, data from an inertial navigation system can be used, which directly determines the coordinates of the sensors in any space.

[0062] (x k1，k2 y k1，k2 (6)

[0063] Here, data formula (6) can be used to generate data formula (2).

[0064] In formulas (2) through (6), the same exponents i, k1, and k2 can be used. These three variables determine the following: the coordinates of the target in formula (2); the target being recorded in formula (6) based on the sensor coordinates; the corresponding coordinates of the target on the frame in formula (3); whether the target belongs to any address trajectory in formula (5); and the conclusion regarding the presence of obstacles on the global map at the current calculation stage, as expressed in the prediction frame formula (4).

[0065] Consider the implementation of an example embodiment of a radar sensor with an accuracy of dr = 0.1 meters and dα = 0.1°, which are typical values ​​for a continuous wave radar sensor with signal linear frequency modulation (“FMCW”) and MIMO technology.

[0066] Step 210: Generate the map. In this stage, the program settings determine the boundaries N and M, and the grid step size hx,hy. Both the boundary and grid step size can be directly calculated based on the sensor range and map resolution. For example, r max =100 meters and resolution dx=0.1 meters are typical values ​​for a radar sensor with FMCW. In the example implementation, this is used immediately after the device is powered on. Figure 1 The processor (150) is used to create the map. The map and any other data acquired during the execution of the steps are stored... Figure 1 In the storage device (160).

[0067]

[0068] Step 220: Determine the loop parameters. At this stage, the loop frequency can be determined based on the polling frequency of the radar sensor, as determined by the sensor manufacturer, and the accuracy of the odometer data. The proposed method is implemented using a sensor that can directly provide information about the coordinates in Equation (6), and therefore the loop can be determined by numbering the data packets using index k2. In the example implementation, a measurement frequency of at least 10 measurements per unit map resolution can be provided, for example, a measurement frequency of 1000 Hz and a vehicle speed of less than 100 km / h for a 10 cm map.

[0069] Step 230: Collect sensor data. This step can be performed by sequentially polling the sensors and storing the data in the format of formula (2). Sensors based on FMCW radar are able to provide data in the already generated packets corresponding to (2). Therefore, when using such a device, no further processing is required. Otherwise, processing can be performed according to the instructions of a specific type of sensor. In the example implementation, two radars (e.g., Figure 3 Each of the radar sensors 320 can transmit signals and receive echo signals, calculating the area to be visible in the public field of view (e.g., Figure 3 The distance and direction of each obstacle in the intersecting field of view 310.

[0070] Here, taking into account the amplitude and phase of the received signal, MIMO technology can analyze the echo signals received by different combinations of receiving and transmitting antennas, thus forming a virtual directional antenna.

[0071] Step 240: Frame Construction. In this step, information in the format of formula (2) can be converted into the format of formula (3). The difference between steps 230 and 240 may be that in step 230, data is collected in analog form, while in step 240, frames are constructed, and the collected data is converted by formulas (2) and (3) and input as discrete values ​​of frame coordinates. In an example of implementation, for example, continuous data may be data in which orientation and coordinates are stored in the memory of the processor (150) in the form of floating-point numbers, and the map may be a coordinate grid with a predetermined grid step size, such as 10 cm. Therefore, before marking targets on the map, the processor (150) can quantize the coordinates according to the formula specified below.

[0072] To convert the data into the format of formula (3), the rounding principle can be used. Considering the need to select the grid step size in a way that meets accuracy requirements, the fastest way to convert the coordinates into a discrete form is to quantize them and round them to the nearest integer.

[0073]

[0074] The operator round(*) means rounding to the nearest integer.

[0075] Here, the target response can be averaged relative to different target types. At this stage, the phase can be determined by the distance to the target. Moving from sensor data to frame data, the following formula suffices to retain only the power information.

[0076]

[0077] Note that in formulas (9) and (10), the rounding result depends on the general random error caused by the radar sensor (which has an accuracy of dr = 0.1 meters and dα = 0.1°). Therefore, the implementation utilizes expressions (9) and (10) even if the number of points included in the frame is increased by a factor of 10 or 100 (e.g., for systems with low angle determination accuracy).

[0078] Step 250: Calculate the address trajectory and generate the prediction frame for the current iteration. At step 240, using formulas (9) and (10), several points (e.g., from 2 to 1000 with selected device characteristics) can be obtained (and marked) for each target on the frame. Assuming that obstacles may exist at each of the selected grid points, additional points must be eliminated. For this purpose, a prediction frame can be used, which is obtained using the dataset acquired at step k1-1 (i.e., at the previous step).

[0079] The intermediate frame set can be generated according to the following formula.

[0080] dx k1,k2 =(x k1,k2 -x k-1,k2 (12)

[0081] dy k1,k2 =(y k1,k2 -y k-1,k2 (13)

[0082]

[0083] Operators This indicates taking the integer part.

[0084] In the absence of system error, equations (14) and (15) and equations (9) and (10) should give the same results. However, in practice, due to random errors in equations (9) and (10), the frames may differ from one another.

[0085] By eliminating intermediate frames The coordinates of at least one unresponsive intermediate frame can be used to further improve the calculation of formulas (14) and (15). This can be easily achieved by multiplying by the corresponding power.

[0086]

[0087] Operators This indicates taking the integer part.

[0088] Points with zero power can be discarded and the resulting set is the prediction frame {p}. i,k1 ,x i,k1 ,y i,k1}

[0089] if

[0090] if

[0091] if

[0092] Note that for intermediate frames, the odometer data is assumed to be known and its accuracy exceeds that of the radar sensor. In systems where this is not the case, equations (12)-(16) can take different forms.

[0093] It is worth noting that, in order to improve accuracy, it is reasonable to use a series of prediction frames calculated for steps k1-2, k1-3, etc., similar to formulas (12)-(16).

[0094] The set of predicted frames for all values ​​of k1, k1-1, k1-2, k1-3, ... can form an address trajectory. In some implementations of this method, to simplify the calculations in formulas (12)-(19), continuous values ​​are used instead of discrete coordinate values, and the address trajectory can then be interpolated by a smooth curve, which should theoretically increase the accuracy of the method.

[0095] Step 260: Compare the “frame” and the “predicted frame”. Assign statistical probabilities and overlay the data on the map and grid.

[0096] In the current step, based on information about the obstacle from the sensors {p i，k1，k2 x i，k1，k2 y i，k1，k2} and predicted frame data {p i，k1 x i，k1 y i，k1 By identifying a set of grid points, we can draw conclusions about the probability of the presence of obstacles within that set of grid points.

[0097] This conclusion is derived from the reasonableness of the following reasoning: despite the presence of errors in sensor data and the difficulty in distinguishing obstacles from one another, the probability of missing a target in a series of steps can be a significantly low reliability threshold (e.g., p = 0.05), and therefore, it stems from the consideration that it is reasonable to assume that there are no obstacles at grid nodes where there is no signal.

[0098] In the proposed implementation of this technology, the concept embedded in the method can be expressed by formulating the following estimation. First, points where the signal strength differs significantly from the responses received on other frames can be excluded. This exclusion can be performed based on a threshold, which depends on the sensor's sensitivity and the noise level of the received signal. A typical implementation may use a threshold p = 0.05.

[0099]

[0100] After using formula (23), points where the power variability between sensors exceeds a certain threshold can be disregarded.

[0101] Considering the complex structure of the target's radiation pattern, formula (23) is meaningful in the following cases.

[0102]

[0103] Next, a weighted estimate can be made of the proportion of frames with reliable signals present in the current iteration along the grid coordinates. The result can be input into the current prediction frame. This formula can be normalized to values ​​between 0 and 1. Weighting coefficients can be selected that are proportional to the ratio of the current power to the maximum signal power recorded at a given point.

[0104]

[0105] Where p max =max k2 (p i,k1,k2 (26)

[0106] Finally, based on the last K predicted frames (i.e., K points based on the address trajectory), a weighted evaluation of the presence of obstacles at a given point can be performed. The results can be immediately plotted on a grid. The proposed design uses geometric weighting, resulting in a smaller contribution from older signals.

[0107]

[0108] Step 270: Filtering grid nodes. This can be done by resetting all values ​​not exceeding a threshold p, for example, p = 0.95, to zero.

[0109] Ψ(x i，k1 y i，k1 If Ψ(x) = 0, then... i，k1 y i，k1 )≤p (28)

[0110] Step 280: Filter address traces. Address traces are filtered to reduce computational complexity. Address traces with a current grid point value of 0 can be excluded from the set of address traces.

[0111] Step 290: The result is generated. Proceed to step 230.

[0112] To generate the results, for a given iteration k1, the grid Ψ(x) can be... i y i Convert it into a binary mask.

[0113]

[0114] The iteration number k1 can be increased by 1, and the operation can be repeated starting from step 230.

[0115] In some implementations, computational complexity can be reduced. For example, to reduce computational complexity, the response of a target whose coordinates have been obtained with the required accuracy can be excluded (i.e., subtracted) from the radar signal. In another example, to reduce computational complexity, when processing frame sequences, spatial coordinates that have been confirmed to not contain any targets can be deliberately omitted.

[0116] In some embodiments of the invention, separate frames can be used to detect the coordinates of a moving target that moves relative to other observed targets at a constant speed and direction or at a variable speed and direction.

[0117] In some implementations, MIMO-based sensors can be used as radar sensors. In some implementations, continuous-action radar, pulse-action radar, or devices with different actions can be used simultaneously. Some implementations can utilize on-chip radar formats. In some implementations, a continuous coordinate system can be used instead of grids and grid nodes. In some implementations, when calculating address trajectories, the inherent errors of the radar sensor can be considered, and a probability distribution function can be used instead of the precise coordinates of the target.

[0118] Figure 3 An example of the mounting diagram of a perception sensor (330) with two radar sensors (320) on a vehicle (340) is shown. Figure 3 The diagram shows the antenna arrangement and the sensing sensor (330, Figure 1 An example of the placement of element 100) on a vehicle. In this example configuration, the sensing sensor (330) itself may include two radar sensors (320) having intersecting fields of view (FoV) (310). In some configurations, the radar sensors may be movable relative to each other. Various types of mobile platforms, both manually driven and unmanned, can be envisioned replacing the car (340). These implementations can be performed within the spatial region described by the intersecting portion (310) of the fields of view of the transmitting and receiving antennas. Here, Figure 3 The location of the operating area relative to its installation position is shown.

[0119] Figure 4 A schematic diagram is shown of a coordinate system known as a map and different arrangements of its grid nodes. Specifically, Figure 4 a) in the diagram illustrates the use of a uniform grid; and Figure 4 b) illustrates the use of a non-uniform grid in a polar coordinate system recommended for use with sensors that detect more than one coordinate.

[0120] As shown in the figure Figure 4The embodiment illustrates the spatial localization of points capable of obstacle detection. The grid step size h can be selected based on the physical dimensions of the obstacle. x and h y For example, the grid step size can use the resolution provided by the radar sensor along the distance coordinates, i.e., the accuracy with which the sensor determines the distance to the obstacle.

[0121] Figure 5 The relationship between the frames generated during operation, the predicted frames, and the address trajectories is shown.

[0122] The frames (510) generated based on sensor data show information about the obstacle (540) and its distance obtained from each of the radars used. Implementations are capable of identifying signals on different frames as belonging to the same physical target. This association can be represented by a set of coordinates collectively referred to as address trajectories (530). In some configurations, implementations are capable of tracking echo signals obtained from a single obstacle by different sensors. These spatial trajectories can be stored in memory as a set of coordinates and are referred to as "address trajectories".

[0123] By using frame data and address trajectories, and by analyzing the current frame (550), a prediction of future expected radar data is determined. This prediction is represented as a set of target coordinates at each time point, collectively forming a set of prediction frames (520). Based on a set of address trajectories and frames obtained at a given time point, the implementation can calculate a set of expected echo signals that are likely to be received by the radar sensor in the next few seconds. These “most likely sets” can be referred to as prediction frames (520).

[0124] By comparing the predicted frames (520) stored in the address trajectory memory (530) with data continuously received from the radar, a complete list of obstacles and their coordinates within the field of view (560) of the present invention is generated. Furthermore, by comparing a set of predicted frames with actual radar data, a map of the surrounding area can be generated.

[0125] Figure 6 The diagram shows sensor data obtained from a sensor and the results of forming frames based on the sensor data.

[0126] In this sensor (610), the signal from the target can be represented as a set of distances to the obstacle in multiple (at least two) spatial directions, determined by the configuration of the device's transmitting and receiving antennas. The echo signal obtained by a typical MIMO radar sensor can be represented by a digital array of detected obstacles and measured distances to said obstacles. The number of arrays can correspond to the number of beams formed by the MIMO radar (i.e., at least two).

[0127] In operation, sensor data can be represented as points located at grid nodes (630). For this purpose, each obstacle detected by the sensor (620) can be marked on multiple grid nodes along a direction and distance determined by radar. Using these arrays and spatial positioning of the beams provided by the sensor manufacturer, the actual coordinates in the Cartesian system can be calculated. Nodes in the grid (630) that may potentially contain obstacles can be marked.

[0128] Figure 7 An example of combining frame data and grid data during the generation of a global map is shown. The illustration shows obstacles (720) within the current field of view (730). Using data of global coordinates and the coordinates of previously detected obstacles (710), all obstacles can be mapped onto the global map (740).

[0129] Various implementations can be used to create maps within the radar field of view, as well as to build global maps in the process of solving the Simultaneous Localization and Mapping (SLAM) problem. To build a global map, once the coordinates of obstacles are obtained, they can be accessed via a navigation module (i.e., Figure 1 The navigation system 170) transforms them into a global coordinate system. These implementations can be integrated with real-time SLAM systems, enabling enhanced navigation capabilities that can process radar signals in real time, which is crucial for autonomous driving and robotic applications that require rapid decision-making.

[0130] The embodiments described above can be used in radar devices (regardless of modulation type and radiation wavelength) designed to determine the coordinates of a target relative to the radar device within the field of view of the transceiver antenna. Therefore, the embodiments can be implemented, wholly or partially, in devices where a unique modulation type is used in each virtual antenna. These embodiments can be used in two types of systems: systems that determine only one target coordinate (e.g., distance to the target), and systems that determine several coordinates (e.g., range and angular coordinates or target coordinates in a universal terrain coordinate system).

[0131] As described in this article, the general principles of forming a virtual PAA and the device for implementing such a virtual PAA to calculate the spatial coordinates of an obstacle can be introduced. The introduction of a MIMO-based radar sensor enables the acquisition of both the distance to the observed target and the angular coordinates of the observed target relative to the sensor itself. The angular coordinates can be calculated by comparing signals received from different spatially separated receiving and transmitting antenna arrays (virtual PAAs).

[0132] Therefore, the embodiments of the present invention can solve two main problems. First, these embodiments are less susceptible to noise, such as interference noise present in the signal (so-called "speckle noise"). Thus, signal suppression caused by multipath interference effects can be achieved. Multi-beam interferometry suppression characteristics can be achieved by setting a limit on the minimum distance traveled to confirm the target's coordinates. This distance can be at least twice the radar correlation radius, determined by wavelength, platform speed, and other factors. Second, the embodiments can be more accurate in determining the coordinates of obstacles even at greater distances. The accuracy of the obtained coordinates is independent of the distance to the observed target.

[0133] By implementing the above embodiment, an additional result is the possibility of generating a system that uses a radar device to determine the spatial coordinates of a target by measuring only one coordinate (i.e., the distance to the target). This result can be achieved when using several radar sensors (at least two radar sensors) located on a platform and spatially spaced by a known amount. Furthermore, this result can be achieved due to the double accumulation of signals. Here, the embodiment can utilize the movement parameters of the platform on which all the radar sensors are mounted.

[0134] Therefore, implementations can provide a radar vision sensing system that can be integrated with automotive radar, advanced driver assistance systems (“ADAS”), and autonomous vehicles. These implementations can combine radar technology with data processing methods to improve radar accuracy and performance, particularly for automotive and robotic applications (e.g., robotic transportation systems, robotic passenger taxis, robotic freight and last-mile delivery systems, container terminals, etc.). Additionally, implementations can be integrated with various sensors such as lidar, camera equipment, and global positioning systems (“GPS”).

[0135] By implementing the embodiments, target detection can be enhanced and false alarms reduced. For example, compared to conventional techniques, the embodiments can improve accuracy by approximately 210% for long-range applications and approximately 133% for short-range applications. These improvements can be achieved even without additional segmentation or filtering, exhibiting robust performance in detecting anticipated obstacles. Furthermore, the embodiments can effectively manage the noise and target separation problems prevalent in conventional MIMO systems. Therefore, by implementing the embodiments of the present invention, radar accuracy can be improved, and expensive lidar systems can be replaced with more cost-effective radar solutions.

[0136] It will be apparent to those skilled in the art that various modifications and variations can be made to the system, apparatus, method, and instructions for determining the position of one or more observed targets without departing from the spirit or scope of the invention. Therefore, this invention is intended to cover such modifications and variations as long as they fall within the scope of the appended claims and their equivalents.

Claims

1. A method for determining the spatial coordinates of a target in a visible area using a radar device having one or more spatially separated transmitting and receiving antenna groups, comprising: Receive at least one dataset from at least one radar sensor, including the target response and the coordinates of at least one transmit and receive antenna array; Calculate the address trajectory of one or more spatially distributed points forming a grid with relative coordinates; as well as The coordinates of the target are determined by comparing frames received sequentially or synchronously from one or more radar sensors.

2. The method according to claim 1, wherein, The target is detected in real time.

3. The method according to claim 1, wherein, The dataset is received from the at least one radar sensor in the form of a set of recorded analog-to-digital converter (ADC) data.

4. The method according to claim 1, wherein, The dataset received from the at least one radar sensor includes information about the target's speed of movement.

5. The method according to claim 1, wherein, The dataset received from the at least one radar sensor includes information about the platform's movement speed.

6. The method according to claim 1, wherein, The probability map of the target is stored in a non-transitory memory.

7. The method according to claim 1, wherein, The at least one radar sensor is capable of moving relative to the other radar sensor during operation.

8. The method according to claim 1, wherein, The dataset was used to obtain odometer data.

9. An apparatus for determining the spatial coordinates of a target in a visible area using a radar device having one or more spatially separated transmitting and receiving antenna groups, comprising: Non-transitory memory; as well as The processor performs the following operations: Receive at least one dataset from at least one radar sensor, including the target response and the coordinates of at least one transmit and receive antenna array; Calculate the address trajectory of one or more spatially distributed points forming a grid with relative coordinates; as well as The coordinates of the target are determined by comparing frames received sequentially or synchronously from one or more radar sensors.

10. The device according to claim 9, wherein, The target is detected in real time.

11. The device according to claim 9, wherein, The dataset is received from the at least one radar sensor in the form of a set of recorded analog-to-digital converter (ADC) data.

12. The device according to claim 9, wherein, The dataset received from the at least one radar sensor includes information about the target's speed of movement.

13. The device according to claim 9, wherein, The dataset received from the at least one radar sensor includes information about the platform's movement speed.

14. The device according to claim 9, wherein, The probability map of the target is stored in the non-transient memory.

15. The device according to claim 9, wherein, The at least one radar sensor is capable of moving relative to the other radar sensor during operation.

16. The device according to claim 9, wherein, The dataset was used to obtain odometer data.

17. A radar vision sensing system, comprising: At least one radar sensor, the at least one radar sensor being configured to detect targets within a predetermined range; A processing unit, operatively connected to the at least one radar sensor, is configured to perform a method comprising: Receive at least one dataset from the at least one radar sensor, including the target response and the coordinates of at least one transmitting and receiving antenna group; Calculate the address trajectory of one or more spatially distributed points forming a grid with relative coordinates; and The coordinates of the target are determined by comparing frames received sequentially or synchronously from one or more radar sensors.

Images