Method for estimating moving direction of front vehicle using radar and apparatus thereof

Abstract

Disclosed are a method for estimating the moving direction of a front vehicle using radar and an apparatus thereof. The method for estimating the moving direction of a front vehicle using radar comprises the steps of: (a) acquiring a radar detection result and then signal-processing the radar detection result to acquire front vehicle information; (b) converting the front vehicle information into point cloud data in a 2D space; (c) converting the point cloud data into an image by preprocessing the point cloud data; and (d) outputting progress direction and angle information of a front vehicle by applying the converted image to a trained deep learning model.

Description

Method for estimating the direction of movement of a forward vehicle using radar and device therefor

[0001] The present invention relates to a method and device for estimating the direction of travel of a front vehicle using only radar reception signals.

[0002]

[0003] Recently, there has been a growing demand for autonomous vehicles capable of safely navigating complex and dynamic road environments. Autonomous vehicles utilize various types of sensors, such as cameras, lidar, and radar, to perceive their surroundings. While these sensors must operate accurately in all conditions to ensure driver safety, camera performance suffers in low-light conditions, including night and fog. Lidar sensors also face the same challenges as cameras in adverse weather conditions like snow, rain, and fog.

[0004] Additionally, rear-end collisions can occur due to problems such as a vehicle cutting in from the front while driving, lack of night lighting or difficulty securing visibility due to fog, or driver carelessness.

[0005] Radar has the advantage of being able to operate stably regardless of weather or lighting conditions, but vehicle-mounted radar is limited to obstacle detection and distance estimation research.

[0006]

[0007] The present invention provides a method and device for estimating a vehicle movement direction using radar.

[0008] In addition, the present invention provides a method and device for estimating the direction of movement of a vehicle using radar, which can estimate the direction of movement and the exact angle of a front vehicle using only radar reception signals.

[0009]

[0010] According to one aspect of the present invention, a method for estimating a vehicle movement direction using radar is provided.

[0011] According to one embodiment of the present invention, a method for estimating a vehicle movement direction using radar may be provided, including the steps of (a) obtaining radar detection results and then performing signal processing to obtain information on a front vehicle; (b) converting the front vehicle information into point cloud data in a 2D space; (c) preprocessing the point cloud data to convert it into an image; and (d) applying the converted image to a trained deep learning model to output information on the direction of travel and angle of the front vehicle.

[0012] The above step (b) can accumulate the results for 10 frames of the above front vehicle information and convert them into one XY coordinate system to convert them into point cloud data.

[0013] The above step (c) can convert the point cloud data into an image by resizing it to a size of 100 x 100.

[0014] The above front vehicle information is distance and angle information, and is converted into point cloud data in the 2D space using the following mathematical formula.

[0015]

[0016] Here, is the distance to the kth target, represents the azimuth to the kth target, represents the elevation angle to the kth target, where k is a natural number.

[0017]

[0018] According to another aspect of the present invention, a device for estimating a vehicle movement direction using radar is provided.

[0019] According to one embodiment of the present invention, a device for estimating a vehicle's movement direction using radar may be provided, including: a radar sensor for acquiring a radar detection result; a signal processing unit for signal processing the radar detection result to acquire information on a front vehicle; a conversion unit for converting the front vehicle information into point cloud data in a 2D space; a preprocessing unit for preprocessing the point cloud data to convert it into an image; and an estimation unit for applying the converted image to a trained deep learning model to output information on the direction of travel and angle of the front vehicle.

[0020] The above conversion unit can accumulate the results for 10 frames of the above front vehicle information and convert them into one XY coordinate system to convert them into point cloud data.

[0021] The above preprocessing unit can convert the point cloud data into an image by resizing it to a size of 100 x 100.

[0022]

[0023] By providing a method and device for estimating a vehicle movement direction using radar according to one embodiment of the present invention, it is possible to estimate the direction and accurate angle of movement of a front vehicle using only radar reception signals.

[0024]

[0025] Figure 1 is a flowchart illustrating a method for estimating a vehicle movement direction using radar according to one embodiment of the present invention.

[0026] FIG. 2 is a schematic diagram illustrating a radar system according to one embodiment of the present invention.

[0027] FIG. 3 is a diagram illustrating an FMCW radar transmission signal according to one embodiment of the present invention.

[0028] FIG. 4 is a diagram illustrating distance and speed estimation according to one embodiment of the present invention.

[0029] FIG. 5 is a diagram illustrating a data cube according to one embodiment of the present invention.

[0030] FIG. 6 is a diagram illustrating channel selection for azimuth estimation in a data cube according to one embodiment of the present invention.

[0031] FIG. 7 is a diagram illustrating preprocessing of point cloud data according to one embodiment of the present invention.

[0032] FIG. 8 is a diagram illustrating the structure of a deep learning model according to one embodiment of the present invention.

[0033] FIG. 9 is a diagram illustrating learning of a deep learning model according to one embodiment of the present invention.

[0034] Fig. 10 is a block diagram schematically illustrating the internal configuration of a movement direction estimation device according to one embodiment of the present invention.

[0035] FIG. 11 is a diagram illustrating radar specifications according to one embodiment of the present invention.

[0036] FIG. 12 is a diagram illustrating the result of orthogonally projecting a 3D point cloud onto the XY plane according to one embodiment of the present invention.

[0037]

[0038] As used herein, singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as "consist of" or "include" should not be construed to necessarily include all components or steps described in the specification, and should be construed to mean that some of the components or steps may not be included, or that additional components or steps may be included. In addition, terms such as "part" and "module" described in the specification mean a unit that processes at least one function or operation, which may be implemented by hardware or software, or by a combination of hardware and software.

[0039] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

[0040]

[0041] FIG. 1 is a flowchart illustrating a method for estimating a vehicle movement direction using radar according to an embodiment of the present invention, FIG. 2 is a diagram schematically illustrating a radar system according to an embodiment of the present invention, FIG. 3 is a diagram illustrating an FMCW radar transmission signal according to an embodiment of the present invention, FIG. 4 is a diagram illustrating distance and speed estimation according to an embodiment of the present invention, FIG. 5 is a diagram illustrating a data cube according to an embodiment of the present invention, FIG. 6 is a diagram illustrating channel selection for azimuth estimation in a data cube according to an embodiment of the present invention, FIG. 7 is a diagram illustrating preprocessing of point cloud data according to an embodiment of the present invention, FIG. 8 is a diagram illustrating a structure of a deep learning model according to an embodiment of the present invention, and FIG. 9 is a diagram illustrating learning of a deep learning model according to an embodiment of the present invention.

[0042] In step 110, the movement direction estimation device (100) obtains radar detection results.

[0043] In step 115, the movement direction estimation device (100) processes the radar detection result to obtain information on the vehicle ahead.

[0044] Radar detection results are signal-processed to obtain information on the vehicle ahead. Here, the information on the vehicle ahead may include the distance, speed, azimuth, and elevation of the vehicle ahead.

[0045] It is assumed that the forward vehicle classification device (100) described below is equipped with an FMCW radar sensor as illustrated in Fig. 2. The specifications for the radar sensor are as illustrated in Fig. 11.

[0046] The FMCW radar sensor can simultaneously track the distance and speed to the vehicle ahead. As shown in Figure 3, the FMCW radar sensor has a frequency that increases linearly over time. Chirps can be transmitted sequentially. In Fig. 3, represent the center frequency, operating bandwidth, and duration of each chirp, respectively. In addition, represents the entire transmission cycle, which is defined as one frame.

[0047] The transmitted FMCW radar signal at Assuming that the radar signal is reflected from the kth target (more specifically, a point target) moving at a relative speed of , the radar reception signal will have a distance between the radar and the kth target. Time delay and velocity between the radar and the kth target Includes Doppler shift due to

[0048] Also, the Doppler shift is , where c represents the speed of light. The FMCW radar reception signal is down-converted to a baseband signal through a frequency mixer and a low pass filter (LPF), and the baseband signal sampled by the analog-to-digital converter (ADC) can be expressed as in mathematical equation 1.

[0049]

[0050] Here, represents the total number of points recognized as targets, represents the amplitude of the baseband signal for the kth target recognized as a target, and represents the distance and speed between the kth point recognized as a target and the radar, respectively. In mathematical expression 1, represents the index for the time sample of each layer, represents the index of each concubine. Also, represents the sampling frequency, is the sampling period.

[0051] The FMCW radar sensor, as shown in Fig. 2, consists of a waveform generator, a voltage-controlled oscillator (VCO), an amplifier, a transmit antenna element (Tx) and a receive antenna element (Rx), a frequency mixer, a phase shifter, an LPF, an ADC, and a digital signal processor.

[0052] The time-sampled baseband signal of mathematical expression 1 can be expressed in the form of a 2D matrix as shown in (a) of Fig. 4. By applying Fourier transforms in the sampling axis direction (i.e., p-axis) and the chirp axis direction (i.e., q-axis), the distance and speed to the preceding vehicle can be estimated.

[0053] That is, by applying the 2D Fourier transform to Equation 1, the distance and speed of the preceding vehicle can be extracted simultaneously. The result of the 2D Fourier transform for Equation 1 can be expressed as Equation 2.

[0054]

[0055] A MIMO antenna system consisting of multiple transmitting and receiving antenna elements can be used to estimate the angle of the front vehicle. The azimuth and elevation angles between the antenna center and the kth target are and Assuming this, mathematical expression 1 can be expanded as in mathematical expression 3.

[0056]

[0057] Here, and represents the distance between the transmitting antenna elements arranged in the azimuth and elevation directions, respectively. In addition, and represents the distance between receiving antenna elements arranged in the azimuth and elevation directions. In addition, represents the index of the transmitting antenna element. For example, if the number of transmitting antenna elements is , and the number of receiving antenna elements is In a MIMO antenna system, the number of receiving channels is virtually maximum. can increase up to . Since each resolution is proportional to the number of receiving channels, MIMO antenna systems can achieve high angular resolution with limited device size. When a receiving channel is created, the total The Fourier transform result of the mathematical equation 2 is generated. That is, A data cube is created and can be represented as shown in Figure 5.

[0058] To estimate the azimuth of the vehicle ahead, select the receiving channel arranged in the azimuth direction from all virtual receiving channels. For example, the number of channels in the azimuth direction is Assuming that, as in (a) of Fig. 5 A data cube of size can be generated. The corresponding index of the kth target in the 2D Fourier transform result of each channel is and Then, as in (b) of Figure 6, the total You can obtain the sampling value. Therefore, A signal vector composed of values ​​sampled from a channel can be expressed as in mathematical expression 4.

[0059]

[0060] An existing beamformer can be used for angle estimation, one of the digital beamforming techniques, based on the signal vector of Equation 4. To this end, a correlation matrix can be generated using the extracted signal vector of Equation 4, which can be expressed as Equation 5.

[0061]

[0062] In mathematical equation 5 Since represents a Hermitian operator, Is It becomes a square matrix of size .

[0063] The normalized pseudospectrum of the existing beamformer can be expressed as in mathematical equation 6.

[0064]

[0065] Here, represents a steering vector that takes into account the distance between the receiving channels in the azimuth direction, and the azimuth of the front vehicle is the one that maximizes the value of the normalized pseudospectrum. can be determined by

[0066] elevation The process of estimating the azimuth It is similar to the process of estimating azimuth. The difference from azimuth estimation is in the altitude direction. It is to select the receiving channel and the steering vector The distance between the receiving channels considered in is changed.

[0067] In step 120, the movement direction estimation device (100) converts the extracted information on each front vehicle into 2D point cloud data.

[0068] For example, if the detection result using the radar reception signal is displayed in the form of a point cloud in a three-dimensional coordinate system, it is as shown in Fig. 12. In Fig. 12, the value that can be estimated using the radar is , which represent the distance to the kth target, the azimuth of the kth target, and the elevation of the kth target, respectively. Here, k can be a natural number. Using this, the coordinates in the 3D space for the kth target (e.g., the front vehicle) ) can be obtained, and if expressed as a mathematical formula, it is as shown in mathematical formula 7.

[0069]

[0070]

[0071] Using this, if the kth point is orthogonally projected onto the XY plane, the two-dimensional coordinates ( ) can be obtained.

[0072] Accordingly, the movement direction estimation device (100) can convert the forward vehicle information (i.e., distance and angle information) into point cloud data by displaying it in an XY coordinate system using a relational expression such as mathematical expression 7.

[0073] In one embodiment of the present invention, the results for 10 frames (1 second) are accumulated and converted into an XY coordinate system by projecting them onto the XY plane.

[0074] In step 125, the movement direction estimation device (100) preprocesses the point cloud data converted into an XY coordinate system and converts it into an image.

[0075] For example, as shown in Fig. 7, point cloud data converted to an XY coordinate system can be resized to a size of 100 x 100 suitable for deep learning and converted into an image. Of course, when converting point cloud data into an image, the Density-based spatial clustering of applications with noise (DBSCAN) clustering algorithm can be applied to the point cloud data to remove unnecessary parts such as noise around the object being observed, and then converted into an image with a single color (e.g., black or white).

[0076] In addition, the movement direction estimation device (100) may be preceded by a preprocessing process in which the center of the data is found to convert the two-dimensional data converted to the XY coordinate system into an image, and then points other than the target are removed by observing the same range in the upper, lower, left, and right directions based on the center.

[0077] Next, in step 130, the movement direction estimation device (100) applies the converted image to a multi-output deep learning model to generate a forward vehicle direction estimation result. Here, the forward vehicle direction estimation result includes the direction and angle of movement.

[0078] At this time, the detailed structure of the multi-output deep learning model is as shown in Fig. 8.

[0079] A multi-output deep learning model can be trained to estimate the direction of travel of a front vehicle, including angle information, as illustrated in Fig. 9. Here, the direction of travel can be left or right.

[0080] For example, in the case of Fig. 9, the angle information may be 60 degrees, and since the angle information of the vehicle is 60 degrees, it means a vehicle approaching from the left side in front. In other words, it can be estimated as a vehicle proceeding to the right.

[0081]

[0082] Fig. 10 is a block diagram schematically illustrating the internal configuration of a movement direction estimation device according to one embodiment of the present invention.

[0083] Referring to FIG. 10, a movement direction estimation device (100) according to one embodiment of the present invention is configured to include a radar sensor (1010), a signal processing unit (1020), a conversion unit (1030), a preprocessing unit (1040), an estimation unit (1050), a memory (1060), and a processor (1070).

[0084] The radar sensor (1010) is a means for transmitting a radar transmission signal and then obtaining the radar detection result reflected by the target (front vehicle).

[0085] The signal processing unit (1020) is a means for processing radar detection results to obtain information on the vehicle ahead. Here, the information on the vehicle ahead may include distance and angle.

[0086] The conversion unit (1030) is a means for converting front vehicle information into x,y coordinates in 2D space and converting it into point cloud data for each front vehicle.

[0087] The preprocessing unit (1040) is a means for preprocessing point cloud data and converting it into an image. For example, the preprocessing unit (1040) can resize point cloud data to a size of 100 x 100 and convert it into an image suitable for training a deep learning model.

[0088] The estimation unit (1050) is a means for estimating the direction and angle of travel of the preceding vehicle by applying the converted image to a deep learning model. Here, it is assumed that the deep learning model has been pre-trained to estimate the direction and angle of travel using training data.

[0089] The learning unit (1055) is a means for training a deep learning model to estimate the direction and angle of travel of the front vehicle using a training data set. The training data set includes correct labels. Accordingly, the learning unit (1055) can update and train the deep learning model's learning parameters so that the difference between the correct labels and the estimated direction and angle of travel is minimized.

[0090] The memory (1060) stores a program code for performing a method for estimating a vehicle movement direction using radar according to one embodiment of the present invention.

[0091] The processor (1070) is a means for controlling internal components (e.g., radar sensor (1010), signal processing unit (1020), conversion unit (1030), preprocessing unit (1040), estimation unit (1050), memory (1060), etc.) of a vehicle movement direction estimation device (100) using radar according to one embodiment of the present invention.

[0092]

[0093] The device and method according to the embodiment of the present invention may be implemented in the form of program commands that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program commands, data files, data structures, etc., alone or in combination. The program commands recorded on the computer-readable medium may be those specially designed and configured for the present invention or may be those known and usable by those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program commands such as ROMs, RAMs, and flash memories. Examples of program commands include not only machine language codes generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc.

[0094] The hardware device described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

[0095] The present invention has been described above, focusing on specific embodiments thereof. Those skilled in the art will appreciate that the present invention can be implemented in modified forms without departing from its essential characteristics. Therefore, the disclosed embodiments should be considered illustrative rather than limiting. The scope of the present invention is set forth in the claims, not the foregoing description, and all differences within the scope equivalent thereto should be construed as being encompassed by the present invention.

Claims

1. (a) A step of obtaining radar detection results and then processing the signals to obtain information on each vehicle ahead; (b) a step of converting the above front vehicle information into point cloud data in 2D space; (c) a step of preprocessing the above point cloud data and converting it into an image; and (d) A method for estimating the direction of movement of a vehicle using radar, including a step of applying the converted image to a learned deep learning model to output information on the direction and angle of movement of the vehicle ahead.

2. In paragraph 1, Step (b) above, A method for estimating the direction of movement of a vehicle using radar, characterized in that the results for 10 frames of the above-mentioned front vehicle information are accumulated and then converted into one XY coordinate system and converted into point cloud data.

3. In paragraph 1, Step (c) above, A method for estimating a vehicle movement direction using radar, characterized in that the above point cloud data is resized to a size of 100 x 100 and converted into an image.

4. In paragraph 1, A method for estimating a vehicle movement direction using radar, characterized in that the above-mentioned front vehicle information is distance and angle information, and is converted into point cloud data in the 2D space using the following mathematical formula. Here, is the distance to the kth target, represents the azimuth to the kth target, represents the elevation angle to the kth target, where k is a natural number.

5. A computer-readable recording medium recording a program code for performing a method according to any one of clauses 1 to 4.

6. Radar sensor that obtains radar detection results; A signal processing unit that processes the radar detection results to obtain information on each vehicle ahead; A conversion unit that converts the above front vehicle information into point cloud data in 2D space; A preprocessing unit that preprocesses the above point cloud data and converts it into an image; and A device for estimating the direction of movement of a vehicle using radar, including an estimation unit that applies the above-mentioned converted image to a learned deep learning model and outputs information on the direction and angle of movement of the vehicle ahead.

7. In paragraph 6, The above conversion part, A device for estimating the direction of movement of a vehicle using radar, characterized in that the results for 10 frames of the above-mentioned front vehicle information are accumulated and then converted into one XY coordinate system and converted into point cloud data.

8. In paragraph 6, The above preprocessing unit, A device for estimating a vehicle movement direction using radar, characterized in that the size of the above point cloud data is resized to 100 x 100 and converted into an image.

Images