Computer-implemented method for performing a joint radon transformation association and system

Abstract

Computer-implemented method (200) for performing a joint radon transformation association, wherein the method (200) comprises: Detection (202) by a processing device of a target object to be tracked relative to a vehicle (100); Performing (204) the common radon transformation association on the target object by means of the processing device to generate association candidates; Tracking (206) of the target object in relation to the vehicle (100) by the processing device using the association candidates; and The control (208) of the vehicle (100) is based at least partially on the tracking of the target object by the processing device.

Description

[0001] The present description refers to the implementation of a joint radon transformation association.

[0002] Modern vehicles (for example, a car, motorcycle, boat, or any other type of motorized vehicle) can be equipped with a vehicle communication system that enables various types of communication between the vehicle and other entities. For instance, a vehicle communication system can facilitate vehicle-to-infrastructure (V2I), vehicle-to-vehicle (V2V), vehicle-to-pedestrian (V2P), and / or vehicle-to-grid (V2G) communication. Collectively, these can be referred to as vehicle-to-everything (V2X) communication, which enables the communication of information from the vehicle to any other suitable entity. Various applications (for example, V2X applications) can utilize V2X communication to send and / or receive safety alerts, maintenance messages, vehicle status messages, and similar information.

[0003] Modern vehicles can also include one or more cameras that serve as backup aids, capture images of the driver to determine driver fatigue or attentiveness, provide images of the road while driving for collision avoidance, enable structure recognition such as road signs, and so on. For example, a vehicle can be equipped with multiple cameras, and the images from several cameras (referred to as "surround view cameras") can be used to create a "surround" or "bird's-eye view" of the vehicle. Some of the cameras (referred to as "long-range cameras") can be used to capture images over a long distance (for example, for object recognition for collision avoidance, structure recognition, etc.).

[0004] Such vehicles may also be equipped with one or more radar systems, LiDAR devices, and / or similar equipment for target tracking. Target tracking involves identifying a target object and tracking it over time as it moves relative to the vehicle observing it. Images from one or more of the vehicle's cameras may also be used for target tracking.

[0005] US 2018 / 0322642A1 describes a system for predicting the movements of multiple agents. A Radon Cumulative Distribution Transform (Radon-CDT) is applied to pairs of signature formations representing agent movements. The components of canonical correlation analysis (CCA) are identified for the pairs of signature formations. Subsequently, a relationship between the pairs of signature formations is learned using the CCA components. Based on the learned relationship and a new signature formation, a countersignature formation is predicted for a new dataset. The control parameters of a device can be adjusted based on the predicted countersignature formation.

[0006] US 10 032 077 B1 describes various technologies for identifying vehicle tracks in coherent change detection image data from synthetic aperture radar. The coherent change detection images are analyzed in a parameter space using Radon transforms. The peaks of the Radon transforms correspond to features of interest, including vehicle tracks, which are identified and classified. New coherent change detection images are then generated using inverse Radon transforms, in which the features of interest and their properties are labeled.

[0007] According to the invention, a computer-implemented method for performing a joint radon transformation association is provided. The method comprises detecting a target object to be tracked relative to a vehicle by a processing device. The method further comprises performing the joint radon transformation association on the target object by the processing device to generate association candidates. The method further comprises tracking the target object with respect to the vehicle by the processing device using the association candidates. The method further comprises controlling the vehicle by the processing device, at least partially based on the tracking of the target object.

[0008] According to one embodiment, the common radon transformation association is based at least partially on an energy value.

[0009] According to another embodiment, performing the common radon transformation association includes organizing a list of traces and detections into association pairs and, for each pair, constructing a common signal defined by the following equation: s[n,m]=[s1[n,m],s2[n,m]e2πj2D1λT] where s is an energy value of the association pair, s1 is a signal of the track, s2 is a signal of the detection, λ is a signal wavelength, T is a time gap, n is a sample index, m is a chirp index, j is the complex coefficient for -1 and D1 is a trace doubler.

[0010] According to another embodiment, performing the common radon transformation association includes using a common radon transformation defined by the following equation: S=∑k∈K|∑m=1M∑n=1Ns[n,m]e−2πjR(n,m,D1)e−2πjlmM| where e -2πjR(n,m,D1)determines a sloping radon integration curve with slope D1: R(n,m,D1)=(mPRI2SlcD1)(nfs) where M is a number of chirps in a frame, N is a number of samples in each chirp, k is a track range container, l is a track Doppler container, PRI is a pulse repetition interval, Sl is a chirp slope, c is the speed of light, and f s a sampling frequency.

[0011] According to another embodiment, it includes performing a first association on the target object before carrying out the joint Radon transformation association in order to generate second association candidates.

[0012] According to another embodiment, the method includes performing a statistical association on the target object to generate third association candidates before carrying out the joint Radon transformation association.

[0013] According to another embodiment, the joint radon transformation association is carried out in response to the finding that the third association candidates are below a certainty threshold.

[0014] According to the invention, a system comprises a memory containing computer-readable instructions and a processing device for executing the computer-readable instructions to perform a method for carrying out a joint radon transformation association. The method comprises the detection of a target object to be tracked relative to a vehicle by a processing device. The method further comprises the processing device performing the joint radon transformation association on the target object to generate association candidates. The method further comprises the processing device tracking the target object relative to the vehicle using the association candidates. The method further comprises the processing device controlling the vehicle, at least partially based on the tracking of the target object.

[0015] According to one embodiment, the common radon transformation association is based at least partially on an energy value.

[0016] According to another embodiment, performing the common radon transformation association includes organizing a list of traces and detections into association pairs and, for each pair, constructing a common signal defined by the following equation: s[n,m]=[s1[n,m],s2[n,m]e2πj2D1λT] where s is an energy value of the association pair, s1 is a signal of the track, s2 is a signal of the detection, λ is a signal wavelength, T is a time gap, n is a sample index, m is a chirp index, j is the complex coefficient for -1 and D1 is a trace doubler.

[0017] According to another embodiment, a common radon transformation is used in the preformation of the common radon transformation association, which is defined by the following equation: S=∑k∈K|∑m=1M∑n=1Ns[n,m]e−2πjR(n,m,D1)e−2πjknNe−2πjlmM| where e -2πjR(n,m,D1) determines a sloping radon integration curve with slope D1: R(n,m,D1)=(mPRI2SlcD1)(nfs) where M is a number of chirps in a frame, N is a number of samples in each chirp, k is a track range container, l is a track Doppler container, PRI is a pulse repetition interval, Sl is a chirp slope, c is the speed of light, and f sa sampling frequency. In further examples, the procedure also includes performing a first association on the target object before carrying out the joint radon transformation association in order to generate second association candidates. In further examples, the procedure includes performing a statistical association on the target object before carrying out the joint radon transformation association in order to generate third association candidates. In further examples, the joint radon transformation association is carried out in response to the finding that the third association candidates are below a certainty threshold.

[0018] In one application, a computer program product comprises a computer-readable storage medium containing program instructions embodied therein, wherein the computer-readable storage medium is not a transitory signal in itself, but the program instructions are executable by a processing device to cause the processing device to execute the method according to the invention and its embodiments, which is implemented on the computer program product.

[0019] The features and advantages mentioned above, as well as further features and advantages described below, are easily apparent from the following detailed description when considered in conjunction with the accompanying figures.

[0020] Further features, advantages and details appear only as examples in the following detailed description, which refers to the figures in which they are included: Fig. Figure 1 shows a vehicle with sensors and a processing system for performing a joint radon transformation association; Fig. Figure 2 shows a flowchart of a procedure for performing a joint radon transformation association to track a target object relative to a vehicle; Fig. Figure 3 shows a flowchart illustrating a procedure for carrying out associations; Fig. Figure 4 shows a flowchart illustrating a procedure for carrying out associations; and Fig. Figure 5 shows a block diagram of a processing system for implementing the techniques described here.

[0021] It should be understood that in the figures, corresponding reference numerals indicate identical or corresponding parts and features. The term "module" as used here refers to processing circuits that may include an application-specific integrated circuit (ASIC), an electronic circuit, a processor (shared, dedicated, or grouped), memory executing one or more software or firmware programs, a combinational logic circuit, and / or other suitable components that provide the described functionality.

[0022] The technical solutions described here employ shared radon transformation associations for object tracking. Radar-based object tracking uses target mapping to link a detected object with motion predictions (tracks) associated with that object. For example, Cartesian distances can be used to determine which candidate pairs match. Such implementations are designed for sparse, point-like targets and do not perform as well in urban environments containing dense, extended objects.

[0023] In existing association processes, the traces of objects are predicted with respect to time. A Cartesian distance between the predictions and discoveries is calculated, and the Cartesian distances are then used as a pair-candidate score in a many-to-many procedure.

[0024] Current techniques replace distance-calculation-based associations with joint radon transformation associations. Joint radon transformation associations use an energy evaluation between candidate pairs, and the many-to-many process uses the energy evaluation (instead of distance calculations) as the association criterion. Accordingly, the presented techniques improve target tracking technologies by performing a joint radon transformation association on a target object to generate association candidates. The target object is then tracked relative to the vehicle using the association candidates generated by the joint radon transformation association, and the vehicle is steerable based on the tracking of the target object.

[0025] Fig. Figure 1 shows a vehicle 100 with sensors and a processing system 110 for performing a common radon transformation association according to one or more embodiments described herein. In the example of Fig. The vehicle 100 comprises the processing system 110, cameras 120, 121, 122, 123, cameras 130, 131, 132, 133, a radar sensor 140 and a LiDAR sensor 141. The vehicle 100 can be a car, a truck, a van, a bus, a motorcycle, a boat, an aircraft or any other suitable vehicle 100.

[0026] Cameras 120-123 are surround-view cameras that capture images outside and near the vehicle 100. The images captured by cameras 120-123 together form a 360-degree view (sometimes referred to as a "top view" or "bird's-eye view") of the vehicle 100. These images can be useful for operating the vehicle (for example, when parking, reversing, etc.). Cameras 130-133 are long-range cameras that capture images outside the vehicle and at a greater distance than cameras 120-123. These images can be useful, for example, for object detection and avoidance. It is estimated that although eight cameras 120-123 and 130-133 are shown, more or fewer cameras in various configurations could be used.

[0027] The captured images can be displayed on a screen (not shown) to provide the driver / operator with external views of the vehicle. The captured images can be displayed as live images, still images, or a combination thereof. In some examples, the images can be combined into a composite view, such as a surround view.

[0028] The radar sensor 140 measures the distance to a target object by emitting electromagnetic waves and measuring the reflected waves with a sensor. This information is useful for determining the distance / location of a target object relative to the vehicle 100.

[0029] The LiDAR sensor 141 (Light detection and ranging) measures the distance to a target object by illuminating the target with pulsed laser light and measuring the reflected pulses with a sensor. This information is useful for determining the distance / location of a target object relative to the vehicle 100.

[0030] The data generated by cameras 120-123, 130-133, radar sensor 140 and / or LiDAR sensor 141 can be used to track a target object relative to vehicle 100. Examples of target objects include other vehicles, pedestrians, bicycles, animals, and the like.

[0031] The processing system 110 comprises a recognition engine 112, an association engine 114, and a vehicle control engine 116. Although not shown, the processing system 110 may include other components, engines, modules, etc., such as a processor (e.g., a central processing unit, a graphics processing unit, a microprocessor, etc.), memory (e.g., random access memory, read-only memory, etc.), data storage (e.g., a solid-state drive, a hard disk drive, etc.), and the like. The features and functionality of the components of the processing system 110 are described in more detail below. The processing system 110 of the vehicle 100 performs a common Radon transformation association to track a target object relative to a vehicle. This process is described with reference to Fig. 2 further described.

[0032] In particular Fig. Figure 2 shows a flowchart of a method 200 for performing a common radon transformation association for tracking a target object relative to a vehicle according to one or more of the embodiments described herein. The method 200 can be performed by any suitable system or device, such as the processing system 110 from [reference missing]. Fig. 1, the processing system 500 from Fig. 5 or any other suitable processing system and / or processing device (for example, a processor).

[0033] In block 202, the detection engine 112 identifies a target object to be tracked relative to a vehicle. Specifically, the detection engine 112 generates a detection for the target object. This detection indicates the location where the target object is recognized as moving.

[0034] In block 204, the association and tracking engine 114 performs a joint Radon transformation association on the target object to generate association candidates. The association and tracking engine 114 receives a list of detections from the detection engine 112 and a list of traces from the previous cycle for various target objects. The association and tracking engine 114 organizes the traces and detections into pairs and constructs a joint signal for each pair from its individual signal, defined by the following equation: s[n,m]=[s1[n,m],s2[n,m]e2πj2D1λT] where s is an energy value of the association pair, s1 is a signal of the track, s2 is a signal of the detection, λ is a signal wavelength, T is a time gap, n is a sample index, m is a chirp index, j is the complex coefficient for -1 D1 is a trace doubler. The exponent that multiplies s2 is a phrase correction component used due to the time difference between the previous and current frames.

[0035] The joint radon transformation is defined by the following equation: S=∑k∈K|∑m=1M∑n=1Ns[n,m]e−2πjR(n,m,D1)e−2πjknNe−2πjlmM| where e -2πjR(n,m,D1) determines a sloping radon integration curve with slope D1: R(n,m,D1)=(mPRI2SlcD1)(nfs) where M is a number of chirps in a frame, N is a number of samples in each chirp, k is a track range container, l is a track Doppler container, PRI is a pulse repetition interval, Sl is a chirp slope, c is the speed of light, and f s a sampling frequency.

[0036] In block 206, the Association and Tracking Engine 114 uses association candidates to track the target object relative to the vehicle. That is, over time, the Association and Tracking Engine 114 continues to determine the target object's position relative to the vehicle.

[0037] In block 208, the control motor 116 controls the vehicle, at least partially, based on tracking the target object. Controlling the vehicle 100 can include increasing / decreasing speed, changing direction, and similar actions. For example, if the target object's position relative to the vehicle 100 would cause a collision, the control motor 116 can steer the vehicle 100 to avoid the target object. This is possible by tracking the target object using the common Radon transformation association. Accordingly, the vehicle technology is improved by controlling the vehicle based on such positional data.

[0038] Additional processes can also be included, and it should be understood that the in Fig. The process shown is for illustrative purposes only and it is understood that other processes may be added or existing processes removed, modified or rearranged without departing from the scope and spirit of the present description.

[0039] Fig. Figure 3 shows a flowchart of a method 300 for performing associations according to one or more of the embodiments described herein. The method 300 can be performed with any suitable system or device, such as the processing system 110 from [reference missing]. Fig. 1, the processing system 500 from Fig. 5 or any other suitable processing system and / or processing equipment (for example, a processor).

[0040] In block 302, the processing system 110 receives a signal from the radar sensor 140 and performs an analog-to-digital conversion to transform the signal from an analog signal into a digital signal as sample values.

[0041] In block 304, the processing system 110 receives the sample values ​​and performs a range-fast Fourier transformation on the sample values ​​to generate a range-chirp channel mapping.

[0042] In block 306, the processing system 110 performs a fast Doppler-Fourier transform on the range chirp channel map to generate a range Doppler channel map.

[0043] In block 308, the processing system 110 performs digital beam shaping on the range Doppler channel map to generate a range Doppler beam map.

[0044] In block 310, the processing system 110 generates discoveries in the form of detections from the range-Doppler ray map.

[0045] In block 312, the processing system performs 110 associations, such as the use of the association and tracking engine 114, which is based at least partially on the data stored in a track database 314 and on the recognitions. The track database stores information about the tracks. The associations can include an initial association, a statistical association, and / or a common Radon transformation association. These associations are in Fig. 4 described in more detail.

[0046] With continued reference to Fig. 3. At block 316, the processing system then tracks the target object using the association and tracking engine 114, based on the results of the association(s) performed at block 312. The tracking results can be stored as tracks in the tracks database 314. Additional processes can also be involved, and it should be understood that the Fig. The process shown in section 3 is for illustrative purposes only and it is understood that other processes may be added or existing processes removed, modified or rearranged without departing from the scope and spirit of the present description.

[0047] Fig. Figure 4 shows a flowchart of a method 400 for performing associations according to one or more of the embodiments described herein. The method 200 can be performed by any suitable system or device, such as the processing system 110 from [reference to relevant document]. Fig. 1, the processing system 500 from Fig. 5 or any other suitable processing system and / or processing equipment (for example, a processor).

[0048] Performing a joint radon transformation association is computationally intensive and complex. Therefore, it is advantageous to use joint radon transformation associations in combination with other types of associations. For example, an efficient implementation of joint radon transformation associations is their integration into other association programs. For instance, a candidate for an association undergoes an initial association and a statistical association. Then, targets with low statistical association confidence are subjected to a joint radon transformation association.This approach reduces the number of association candidates that undergo a common radon transformation association, thereby reducing the high computational load associated with performing a common radon transformation association, while simultaneously allowing the common radon transformation association to be performed on suitable candidates. Fig. Figure 4 shows an example of the implementation of such associations.

[0049] In block 402, the association and tracking engine receives 114 association candidates for performing an association as part of a target tracking process.

[0050] In block 404, the association and tracking engine 114 performs an initial association. For each target (i.e., association candidate), an association window is created based on the target's location, kinematics, and a time difference between a past target update time and a current target detection time.

[0051] In block 406, the association and tracking engine 114 performs a statistical association. A target state is predicted up to the detection time. The statistical distance is calculated between the predicted target state and a detected target state. If a trace has a statistical distance below a confidence threshold with more than one detection, these association candidates are considered low confidence. That is, if the trace has only one or zero candidates with a statistical distance below a confidence threshold in block 408, the association in block 412 is considered valid.

[0052] If the association candidates in block 408 have low confidence, the procedure proceeds to block 410, where the association and tracking engines 114 perform a joint radon transformation association. If a detection is associated with more than one trace and the difference between the statistical distances of the different candidates is below the confidence threshold in block 408, the joint radon transformation association is performed. The results of the joint radon association in block 410 are considered a valid association in block 412.

[0053] Additional processes can also be included, and it should be understood that the in Fig. The process described is for illustrative purposes only, and other processes may be added, or existing processes may be removed, modified, or rearranged without departing from the scope and spirit of the present description.

[0054] It is assumed that the present description can be implemented in conjunction with any other type of computer environment known today or developed in the future. Fig.Figure 5, for example, shows a block diagram of a processing system 500 for implementing the techniques described here. In examples, the processing system 500 has one or more central processing units (processors) 521a, 521b, 521c, etc. (collectively or generally referred to as processor(s) 521 and / or processing device(s)). With regard to the aspects of this description, each processor 521 may contain a reduced instruction set (RISC) microprocessor. The processors 521 are coupled to the system memory (for example, RAM 524) and various other components via a system bus 533. The read-only memory (ROM) 522 is coupled to the system bus 533 and may contain a basic input / output system (BIOS) that controls certain basic functions of the processing system 500.

[0055] Also shown are an input / output (I / O) adapter 527 and a network adapter 526, which are connected to the system bus 533. The I / O adapter 527 can be a small SCSI (Computer System Interface) adapter that communicates with a hard disk 523 and / or a storage device 525 or another similar component. The I / O adapter 527, hard disk 523, and storage device 525 are collectively referred to here as mass storage 534. The operating system 540 for execution on the processing system 500 can be stored in the mass storage 534. The network adapter 526 connects the system bus 533 to an external network 536, enabling the processing system 500 to communicate with other such systems.

[0056] A display (for example, a display monitor) 535 is connected to the system bus 533 via the display adapter 532. The system bus 533 may contain a graphics adapter to improve the performance of graphics-intensive applications and a video controller. In one aspect of this description, the adapters 526, 527, and / or 532 can be connected to one or more I / O buses that are connected to the system bus 533 via an intermediate bus bridge (not shown). Suitable I / O buses for connecting peripheral devices such as hard disk controllers, network adapters, and graphics adapters typically include common protocols, such as Peripheral Component Interconnect (PCI). Additional input / output devices are shown as being connected to the system bus 533 via the user interface adapter 528 and the display adapter 532.A keyboard 529, a mouse 530, and a speaker 531 can be connected to the system bus 533 via the user interface adapter 528, which may, for example, contain a super I / O chip that integrates multiple device adapters into a single integrated circuit. One or more of the cameras 120-123, 130-133 are also connected to the system bus 533.

[0057] In some aspects of this description, the Processing System 500 includes a Graphics Processing Unit 537. The Graphics Processing Unit 537 is a specialized electronic circuit for manipulating and modifying memory to accelerate the generation of images in a frame buffer intended for output on a screen. In general, the Graphics Processing Unit 537 is very efficient at manipulating computer graphics and image processing and has a highly parallel structure, which makes it more effective than general-purpose CPUs for algorithms where the processing of large blocks of data is performed in parallel.

[0058] The processing system 500 configured here thus comprises processing capability in the form of processors 521, storage capability including system memory (for example, RAM 524) and mass storage 534, input devices such as a keyboard 529 and mouse 530, and output capability including speakers 531 and a screen 535. In some aspects of this description, a portion of the system memory (for example, RAM 524) and the mass storage 534 together store the operating system 540 in order to coordinate the functions of the various components shown in the processing system 500.

Claims

[1] A computer-implemented method (200) for performing a joint Radon transform association, the method (200) comprising: Detecting (202), by a processing device, a target object to be tracked relative to a vehicle (100); performing (204), by the processing device, the joint Radon transform association on the target object to generate association candidates; Tracking (206), by the processing device, the target object with respect to the vehicle (100) using the association candidates; and Controlling (208), by the processing device, of the vehicle (100) is based at least in part on tracking the target object. [2] The computer-implemented method (200) of claim 1, wherein the joint Radon transform association is based at least in part on an energy assessment. [3] The computer-implemented method (200) of claim 1, wherein performing the joint Radon transform association comprises organizing a list of traces and detections into association pairs and, for each pair, constructing a joint signal defined by the following equation: s[n,m]=[s1[n,m],s2[n,m]e2πj2D1λT] where s is an energy value of the association pair, s1 is a signal of the trace, s2 is a signal of the acquisition, λ is a signal wavelength, T is a time gap, n is a sample index, m is a chirp index, j is the complex coefficient for −1, and D1 is a trace Doppler. [4] The computer-implemented method (200) of claim 3, wherein performing the joint Radon transform association uses a joint Radon transform defined by the following equation: S=∑k∈K|∑m=1M∑n=1Ns[n,m]e−2πjR(n,m,D1)e−2πjknNe−2πjlmM| where e -2πjR(n,m,D1)an inclined Radon integration curve with slope D1 is determined: R(n,m,D1)=(mPRI2SlcD1)(nfs) where M is a number of chirps in a frame, N is a number of samples in each chirp, k is a track range bin, l is a track Doppler bin, PRI is a pulse repetition interval, Sl is a chirp slope, c is the speed of light, and f s is a sampling frequency. [5] The computer-implemented method (200) of claim 1, further comprising, prior to performing the joint Radon transform association, performing a first association on the target object to generate second association candidates. [6] The computer-implemented method (200) of claim 1, further comprising performing a statistical association on the target object to generate third association candidates prior to performing the joint Radon transform association. [7] The computer-implemented method (200) of claim 6, wherein the joint Radon transform association is performed in response to determining that the candidates for the third association are below a certainty threshold. [8] System (500), comprising: a memory (525) with computer-readable instructions; and a processing device for executing the computer-readable instructions for performing a method (200) for performing a joint Radon transform association, wherein the method (200) comprises: Detecting (202), by the processing device, a target object to be tracked relative to a vehicle (100); performing (204), by the processing device, the joint Radon transform association on the target object by the processing device to generate association candidates; Tracking (206), by the processing device, the target object with respect to the vehicle (100) using the association candidates; and Controlling (208), by the processing device, the vehicle (100) based at least in part on tracking the target object. [9] The system (500) of claim 8, wherein the common Radon transform association is based at least in part on an energy assessment. [10] The system (500) of claim 8, wherein performing the joint Radon transform association comprises organizing a list of traces and detections into association pairs and, for each pair, constructing a joint signal defined by the following equation: s[n,m]=[s1[n,m],s2[n,m]e2πj2D1λT] where s is an energy value of the association pair, s1 is a signal of the trace, s2 is a signal of the acquisition, λ is a signal wavelength, T is a time gap, n is a sample index, m is a chirp index, j is the complex coefficient for −1, and D1 is a trace Doppler.

Images