Foreign matter detection method and system based on millimeter wave radar three-dimensional point cloud imaging

Abstract

The invention discloses a foreign matter detection method and system based on millimeter wave radar three-dimensional point cloud imaging, and the method comprises the steps: carrying out the Fourier transform of a distance dimension and a speed dimension, achieving the coherent accumulation, and effectively improving the signal energy; incoherent accumulation is achieved by means of a multi-channel technology, target information is extracted by applying a two-dimensional CFAR algorithm, so that the anti-interference capability in the recognition process is enhanced, the problem that clutter and noise interference in the environment is extremely serious is effectively solved, the detection precision is improved, meanwhile, a clustering algorithm and multi-frame point cloud data fusion are combined to solve the point cloud data sparsification problem, and the detection accuracy is improved. An overfitting phenomenon occurring when a classification algorithm model is directly used is prevented, the efficiency of target feature extraction is improved, and remote detection is broken through.

Description

Technical Field

[0001] The present invention belongs to the technical field of radar signal processing and safety detection, and relates to a foreign object detection method and system based on three-dimensional point cloud imaging of millimeter-wave radar. Background Art

[0002] In the field of modern safety monitoring, from transportation to industrial production, foreign object identification is a key link to ensure the stable operation of the system. Traditional foreign object detection technologies mainly rely on manual inspections or simple sensor devices, which have problems such as low detection efficiency, high false negative rate, and inability to adapt to complex environments. Although lidar and vision sensors can provide high-resolution data, their performance drops significantly in extreme weather conditions such as rain, snow, and haze, and the cost is relatively high. For camera recognition technology, it mainly relies on visible light or infrared light to capture images, which makes its recognition performance significantly affected in complex environments such as rain, fog, strong light, and darkness. Especially in extreme weather conditions, such as thick fog or heavy rain, the "line of sight" of the camera will be severely blocked, resulting in a significant decline in its recognition ability and inability to accurately capture abnormal situations. In addition, although infrared cameras perform well at night or in low-light environments, the intensity change of infrared radiation under strong sunlight will also affect its imaging quality and thus the monitoring effect.

[0003] Regarding lidar technology, it measures distances and identifies objects by emitting laser beams and receiving the reflected light signals. However, the laser beams of lidar are easily scattered or absorbed by particulate matter in the atmosphere, such as raindrops, snowflakes, and dust, which is particularly obvious in bad weather such as rain, snow, and sandstorms. This scattering effect not only reduces the detection distance and accuracy of lidar, but also may cause false alarms or missed detections, especially when there are multiple high-reflectivity objects in the environment. These problems limit the application scope of lidar in various scenarios.

[0004] In contrast, millimeter-wave radar demonstrates its superior performance in bad weather. Millimeter-wave radar uses electromagnetic waves in the millimeter-wave band for detection. This type of electromagnetic wave with a longer wavelength is not easily affected by water vapor and particulate matter in the atmosphere, so it can maintain a relatively high recognition performance in bad weather such as rain, snow, and haze. More importantly, millimeter-wave radar is not affected by lighting conditions and has all-weather and all-time anti-interference capabilities, making it suitable for safety monitoring in various environments. In addition, compared with lidar and cameras, the price of millimeter-wave radar is more affordable, which provides economic feasibility for its large-scale deployment. However, the existing millimeter-wave radar technology has the following limitations. First, the long-distance resolution is insufficient. As the detection distance increases, the point cloud data becomes sparse, resulting in difficulties in extracting target features, and directly using classification algorithm models is prone to overfitting. Second, the anti-interference ability is limited. In complex environments, clutter and noise interference are severe, affecting the detection accuracy. Summary of the Invention

[0005] The object of the present invention is to solve the problems in the prior art that when detecting foreign objects, the resolution at a long distance is insufficient, resulting in difficult extraction of target features. In addition, the anti-interference ability is limited, affecting the detection accuracy, and to provide a foreign object detection method and system based on millimeter-wave radar three-dimensional point cloud imaging.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] A foreign object detection method based on millimeter-wave radar three-dimensional point cloud imaging includes the following steps:

[0008] Obtain the original data matrix for foreign object detection;

[0009] Generate the range-Doppler spectrum of the target information based on the original data matrix, extract the effective targets from the range-Doppler spectrum, and calculate the range information of the effective targets and the azimuth and elevation angles of the effective targets; [[ID=XX]] [[ID=YY]]

[0010] Convert the azimuth, elevation angle, and range information into three-dimensional Cartesian coordinate system point cloud data, and obtain the original grayscale image data based on the point cloud data; [[ID=XX]] [[ID=YY]]

[0011] Based on the original grayscale image data, perform oversampling processing on the samples to generate new grayscale image data, obtain the grayscale image data set according to the original grayscale image data and the new grayscale image data, perform dimensionality reduction processing on the data in the grayscale image data set, and perform foreign object recognition on the dimensionally reduced data to obtain the foreign object detection result.

[0012] A further improvement of the present invention lies in:

[0013] The obtaining of the original data matrix for foreign object detection includes:

[0014] Collect the reflected signals of the target through a millimeter-wave radar array, convert the reflected signals into intermediate-frequency signals, and construct the original data matrix according to the intermediate-frequency signals.

[0015] The extraction of the effective targets from the range-Doppler spectrum includes:

[0016] Put the range-Doppler spectrum F(u, v) into a two-dimensional CFAR detector to remove clutter and noise:

[0017] Let the number of training units be N train , the false alarm rate be P fa , the CFAR window function be w(u, v), and the calculation processes of the clutter noise power Z(u, v) and the threshold factor α are as follows:

[0018]

[0019] Reference threshold level V Note: There seem to be some incomplete or unclear parts in the original text, especially around the formulas in steps related to CFAR detector parameters. The translation has been done as accurately as possible based on the available text.TH (u, v) = αZ(u, v) is used to determine the valid target. If F(u, v) > V TH (u, v), then the current target is determined as a potential target.

[0020] The calculation of the distance information r of the valid target includes:

[0021]

[0022] where c represents the speed of light and B represents the radar frequency modulation bandwidth.

[0023] The calculation of the azimuth angle and elevation angle of the valid target includes:

[0024] Organize each valid target information into a 12-dimensional vector X, calculate the covariance matrix R of the vector X, and perform eigenvalue decomposition. The expression form is as follows:

[0025] R = XX H = U∑U H

[0026] Decompose the covariance matrix into the following form:

[0027]

[0028] where U s represents the signal subspace U s , which is the eigenvector corresponding to the largest eigenvalue; U N represents the noise subspace, which is the remaining eigenvectors;

[0029] Let the coordinates of 12 array elements be (x n , y n ), the first array element is the reference array element located at the coordinate origin, the operating wavelength is λ, and the steering vector is expressed as:

[0030]

[0031] Through the orthogonality of the noise subspace and the steering vector, construct the two-dimensional spatial spectrum function P MUSIC :

[0032]

[0033] Search for the peak position of P MUSIC in the two-dimensional parameter space, and the estimated values of the azimuth angle and elevation angle of the corresponding target are:

[0034]

[0035] Converting the azimuth angle, elevation angle, and distance information into 3D Cartesian coordinate point cloud data includes:

[0036] Based on the target distance r, azimuth angle θ, and elevation angle Construct a 3D rectangular coordinate system, obtain the point coordinates (x, y, z), filter out the outliers and noise points through the DBSCAN clustering method, and merge the point cloud matrices of n f frames to achieve multi-frame fusion of point cloud data.

[0037] Obtaining the original grayscale image data based on the point cloud data includes:

[0038] Normalize the coordinate data;

[0039] Map the normalized coordinates to the M g ×N g discrete grid,

[0040] Calculate the number of point clouds K within each grid (p, q) pq , and calculate the height feature pq of the point cloud data according to the number of point clouds K and the density feature d pq ;

[0041] Linearly fuse the height and density features to generate the grayscale value I pq , and obtain the grayscale image matrix I according to the grayscale value I pq , and convert the grayscale image matrix I into an M g ×N g -dimensional vector I'.

[0042] A foreign object detection method based on millimeter-wave radar 3D point cloud imaging includes the following steps:

[0043] A data acquisition module for acquiring the original data matrix for foreign object detection;

[0044] An effective target information extraction module for generating a range-Doppler spectrum of target information based on the original data matrix, extracting effective targets from the range-Doppler spectrum, and calculating the range information of the effective targets and the azimuth angle and elevation angle of the effective targets;

[0045] An initial grayscale image data generation module for converting the azimuth angle, elevation angle, and distance information into 3D Cartesian coordinate point cloud data and obtaining the original grayscale image data based on the point cloud data;

[0046] A foreign object detection module, which is used to perform oversampling processing on a sample based on the original grayscale image data to generate new grayscale image data, obtain a grayscale image dataset according to the original grayscale image data and the new grayscale image data, perform dimensionality reduction processing on the data in the grayscale image dataset, and perform foreign object recognition on the dimensionally reduced data to obtain a foreign object detection result.

[0047] A terminal device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of any method described in the present invention are implemented.

[0048] A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the steps of any method described in the present invention are implemented.

[0049] Compared with the prior art, the present invention has the following beneficial effects:

[0050] The present invention discloses a foreign object detection method based on millimeter-wave radar three-dimensional point cloud imaging. By performing Fourier transform on the range dimension and velocity dimension, coherent accumulation is achieved, effectively improving the signal energy; non-coherent accumulation is achieved by means of multi-channel technology, and the two-dimensional CFAR algorithm is used to extract target information, thereby enhancing the anti-interference ability in the recognition process, effectively solving the problem of extremely serious clutter and noise interference in complex environments, improving the detection accuracy. At the same time, combined with the clustering algorithm and multi-frame point cloud data fusion, the problem of sparse point cloud data is resolved, preventing overfitting when directly using the classification algorithm model, improving the efficiency of target feature extraction, and breaking through long-distance detection. Description of the Drawings

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the present invention, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0052] Figure 1 It is the range-Doppler spectrogram of the 152nd frame in the embodiment of the present invention;

[0053] Figure 2 It is the CFAR threshold level diagram of the 152nd frame in the embodiment of the present invention;

[0054] Figure 3 It is the two-dimensional MUSIC spatial spectrogram of the 36th target point in the 152nd frame in the embodiment of the present invention;

[0055] Figure 4This is the comparison diagram of the 152nd frame of 3D point cloud image processing before and after in the embodiments of the present invention (where a is the schematic diagram before 3D point cloud image processing, and b is the schematic diagram after 3D point cloud image processing);

[0056] Figure 5 This is the 152nd frame grayscale image in the embodiments of the present invention. Detailed implementation manners

[0057] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some but not all of the embodiments of the present invention. Usually, the components of the embodiments of the present invention described and illustrated in the accompanying drawings here can be arranged and designed in various different configurations.

[0058] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed present invention, but merely represents selected embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention.

[0059] It should be noted that: similar reference numerals and letters denote similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0060] In the description of the embodiments of the present invention, it should be noted that if terms such as "upper", "lower", "horizontal", "inner", etc. are used to indicate the orientation or positional relationship, it is based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship in which the product of the invention is usually placed during use. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be construed as a limitation of the present invention. In addition, terms such as "first", "second", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0061] In addition, if the term "horizontal" appears, it does not mean that the component is required to be absolutely horizontal, but it can be slightly inclined. For example, "horizontal" only means that its direction is more horizontal relative to "vertical", and does not mean that the structure must be completely horizontal, but it can be slightly inclined.

[0062] In the description of the embodiments of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "disposed," "installed," "connected," and "connected" should be understood in a broad sense. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to internal connections between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0063] The present invention is described in further detail below with reference to the accompanying drawings:

[0064] This embodiment discloses a foreign object detection method based on millimeter-wave radar 3D point cloud imaging. Compared with cameras and lidar, it has stronger anti-interference capabilities, can work in all weather conditions, and is low-cost. With the help of point cloud generation and classification algorithms, millimeter-wave radar can overcome the limitations of long-range detection. The method includes the following steps:

[0065] Step 1: Use a multiple-input multiple-output (MIMO) millimeter-wave radar array to transmit a linear frequency modulated continuous wave (FMCW) signal, receive the target reflection signal, obtain the intermediate frequency signal through mixing, filtering, and analog-to-digital conversion, and construct the original data matrix.

[0066] The specific steps include:

[0067] Data was collected using the TI-IWR6843 in TDM-MIMO mode. The TI-IWR6843 has three transmit antennas and four receive antennas, for a total of 12 channels. Each receive antenna is used for I / Q channel acquisition.

[0068] Assume that there are N packets f frames, one frame has N c Chirp signals, each chirp signal has N s Sampling points, each sampling point is 16 bits. Combine the I / Q signals and organize the data into N f N c Row 12N s The two-dimensional matrix Y of the column, for each frame of the two-dimensional matrix Y i Perform differential operation with the two-dimensional matrix Y1 of the first frame to filter out the DC component in the environment. i Decomposed into 12 N s Row N c The two-dimensional matrix f1(j, k) to f 12 (j, k), where j and k represent the column index and row index of the matrix respectively.

[0069] Step 2: Perform fast Fourier transforms (FFTs) on the original data in the range dimension and the Doppler dimension respectively to generate the range-Doppler spectrum.

[0070] Specifically, it includes the following steps:

[0071] For each channel f i (j, k), perform an N s -point FFT on the rows and an N c -point FFT on the columns to obtain the range-Doppler spectra F1(u, v) to F 12 (u, v). Incoherently accumulate F1(u, v) to F 12 (u, v) to obtain F(u, v), where u and v represent the column index and row index of the matrix respectively, and ⊙ represents the Hadamard product.

[0072]

[0073] Step 3: Use the Constant False Alarm Rate (CFAR) detection technique to filter out noise, extract valid targets, and calculate the range information of the valid targets;

[0074] Specifically, it includes the following steps:

[0075] To detect target information from clutter and noise, put F(u, v) into a two-dimensional CFAR detector. Let the number of training cells be N train , the false alarm rate be P fa , the CFAR window function be w(u, v), and the calculation formulas for the clutter noise power Z(u, v) and the threshold factor α are as follows:

[0076]

[0077] The reference threshold level V TH (u, v) = αZ(u, v). If F(u, v) > V TH (u, v), then it is determined as a potential target, extract its corresponding row index ν, and calculate the range information r of the target through the following formula, where c represents the speed of light and B represents the radar frequency modulation bandwidth:

[0078]

[0079] Step 4: Estimate the target azimuth angle and elevation angle through the Multiple Signal Classification (MUSIC) algorithm, and convert them into point cloud data in a three-dimensional Cartesian coordinate system in combination with the range information.

[0080] Specifically, it includes the following steps:

[0081] Organize each target information into a 12-dimensional vector X, and use the two-dimensional MUSIC algorithm to calculate its azimuth angle θ and elevation angle simultaneously

[0082] Calculate its covariance matrix R and perform eigenvalue decomposition, and the expression form is as follows:

[0083] R = XX H = U∑U H (6)

[0084] Sort the eigenvalues from large to small, and according to the sorting result, take the eigenvector corresponding to the largest eigenvalue as the signal subspace U S , and form the remaining eigenvectors into the noise subspace U N , and decompose the covariance matrix into the following form:

[0085]

[0086] Let the coordinates of 12 array elements be (x n , y n ), the first array element is the reference array element located at the coordinate origin, the working wavelength is λ, and the steering vector is expressed as:

[0087]

[0088] Construct a two-dimensional spatial spectrum function P MUSIC through the orthogonality of the noise subspace and the steering vector:

[0089]

[0090] Search for the peak position of P MUSIC in the two-dimensional parameter space, corresponding to the estimated values of the azimuth angle and elevation angle of the target:

[0091]

[0092] Step 5: Fuse multi-frame point cloud data, and use the density clustering algorithm (Density-Based Spatial Clustering of Applications with Noise, DBSCAN) to remove outliers and optimize the point cloud quality.

[0093] Specifically, it includes the following steps:

[0094] At the known target distance r, azimuth angle θ and elevation angle After that, a three-dimensional rectangular coordinate system is constructed, the point coordinates (x, y, z) are obtained, and the DBSCAN clustering algorithm is used to filter out the miscellaneous points and noise points, and the point cloud matrices of n f frames are merged to achieve multi-frame fusion and improve the density of the point cloud.

[0095]

[0096] Step 6: Map the point cloud to a regular grid, fuse the height and density features to generate a grayscale image, and enhance the foreign object characterization ability.

[0097] Specifically, it includes the following steps:

[0098] To generate a grayscale image, the coordinates are normalized and physically range-mapped, where conflicts with the point cloud-free area (value 0) are avoided:

[0099]

[0100] The normalized coordinates are mapped to the discrete grid of M g ×N g , where [·] represents the rounding operation, and (p i , q i ) are the pixel coordinates corresponding to the point cloud:

[0101]

[0102] Calculate the number of point clouds K within each grid (p, q) pq , and calculate the height feature using the average height of the grid Calculate the density feature d using logarithmic compression density pq , and the calculation method is as follows:

[0103] where Q5 and Q 95 represent the 5th percentile and the 95th percentile of ln(1 + K pq ) in the non-zero density region, respectively;

[0104]

[0105]

[0106] Linearly fuse the height and density features to generate the grayscale value I pq :

[0107]

[0108] Finally, convert the grayscale image matrix I into a vector I' of dimension M g ×N g .

[0109] Step 7: Use the synthetic minority oversampling technique (SMOTE) to analyze the minority samples and artificially synthesize new samples based on the minority samples and add them to the dataset.

[0110] The specific steps include:

[0111] SMOTE oversampling is used to reduce the bias caused by class imbalance.

[0112] Set the oversampling factor β, which represents the number of synthetic samples generated by each minority class sample. i ∈χ min , calculate its min The k nearest neighbors in The distance metric is usually Euclidean distance. i and each of its neighbors Randomly interpolate along the line connecting the two to generate a new sample x syn :

[0113] x syn =x i +δ·(x j -x i ) (17)

[0114] Among them, δ~U(0,1) is a uniformly distributed random number that controls the interpolation position.

[0115] Step 8: Use the principal component analysis (PCA) method to reduce the dimension of the grayscale image data and apply the reduced dimension data to the K-nearest neighbors (KNN) classification algorithm to achieve the task of identifying foreign objects.

[0116] The specific steps include:

[0117] PCA is used to reduce the high-dimensional grayscale image vector I' to d dimensions (d<<M g ×M g ):

[0118] Suppose the grayscale image dataset X (containing N samples, M g ×N g Features), the grayscale image dataset X is centered and standardized according to the features to obtain the standardized dataset X std Calculate the covariance matrix and eigenvalue decomposition, select the first d principal components to form the projection matrix W, and map the original data to the low-dimensional space X std W.

[0119] Using KNN classification, for each sample after dimensionality reduction, calculate its Euclidean distance from the training set, select the nearest k samples, and determine the category according to the majority voting rule.

[0120] The present invention realizes coherent accumulation by performing Fourier transform on the distance dimension and the velocity dimension, effectively improving the signal energy; achieves non-coherent accumulation by means of multi-channel technology, and uses a two-dimensional CFAR algorithm to extract target information, thereby enhancing the anti-interference ability in the recognition process and effectively solving the problem of extremely severe clutter and noise interference in complex environments. At the same time, combining the clustering algorithm with multi-frame point cloud data fusion to solve the problem of sparse point cloud data and prevent overfitting when directly applying the classification algorithm model.

[0121] The present invention is realized through the above process. To prove the effectiveness of the method of the present invention, next, a foreign object detection scenario is simulated to collect data, and radar signal analysis and foreign object classification detection are carried out with the aid of a computer, specifically including:

[0122] First, a frequency-modulated continuous wave signal is transmitted through a MIMO radar array, multi-channel reflection data is collected, environmental interference is eliminated through differential operation, and a standardized data matrix is constructed.

[0123] Secondly, targets are extracted through range-Doppler spectrum analysis combined with CFAR detection, the azimuth angle and the elevation angle are obtained by using a spatial spectrum estimation algorithm, and three-dimensional point clouds are generated by fusing range information.

[0124] Furthermore, multi-frame fusion and density clustering are adopted to optimize the point cloud quality, which is mapped into a grayscale image integrating height and density features, and the classification robustness is enhanced through data balancing and dimensionality reduction techniques, significantly improving the anti-interference ability and foreign object detection accuracy of the millimeter-wave radar in complex environments, and having the advantages of all-weather and low cost.

[0125] Experiment 1: Moving target data is collected using the millimeter-wave radar TI-IWR6843. The signal emitted by the radar system is a linear frequency-modulated signal, the frequency modulation bandwidth is 767.5 MHZ, the pulse width is 25.6 us, the pulse repetition period is 160 us, the sampling frequency is 10 MHz, and the operating wavelength is 5 mm. There are 3 receiving antennas and 4 transmitting antennas. The data packet has 200 frames, one frame has 64 chirp signals, and each chirp signal has 256 sampling points.

[0126] Through Figure 1 the results, it can be seen that the moving target returns to the origin at a speed of 1.5 m / s at a position of about 6 m, but due to the presence of clutter in the environment, there are also other bright spots in the image.

[0127] Through Figure 2 the detection threshold, the position of the target information can be effectively extracted.

[0128] The two-dimensional MUSIC algorithm is used to plot Figure 3 , and the generated spatial spectrum is very stable in most cases. The peaks of the spatial spectrum can be clearly seen. At this time, the azimuth angle of the 36th target point is -26 degrees, and the elevation angle is -3 degrees.

[0129] Figure 4 Figure 152 shows the comparison of 3D point cloud images before and after processing. The directly generated point cloud image has many miscellaneous points and noise, and the point cloud distribution is sparse. After processing by the clustering algorithm and multi-frame fusion, the contour of the target is clearer and the point cloud distribution is denser.

[0130] Figure 5 This is the result of converting the processed point cloud image into a grayscale image.

[0131] A foreign object detection method based on millimeter-wave radar 3D point cloud imaging includes the following steps:

[0132] A data acquisition module for acquiring the original data matrix for foreign object detection;

[0133] An effective target information extraction module for generating a range-Doppler spectrum of target information based on the original data matrix, extracting effective targets from the range-Doppler spectrum, and calculating the range information of the effective targets and the azimuth angle and elevation angle of the effective targets;

[0134] An initial grayscale image data generation module for converting the azimuth angle, elevation angle, and range information into 3D Cartesian coordinate system point cloud data, and acquiring the original grayscale image data based on the point cloud data;

[0135] A foreign object detection module for oversampling the sample based on the original grayscale image data to generate new grayscale image data, obtaining a grayscale image data set according to the original grayscale image data and the new grayscale image data, performing dimensionality reduction processing on the data in the grayscale image data set, and performing foreign object recognition on the dimensionally reduced data to obtain a foreign object detection result.

[0136] The schematic diagram of the terminal device provided by an embodiment of the present invention. The terminal device of this embodiment includes: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps in the above-mentioned method embodiments are implemented. Alternatively, when the processor executes the computer program, the functions of each module / unit in the above-mentioned device embodiments are implemented.

[0137] The computer program can be divided into one or more modules / units, and the one or more modules / units are stored in the memory and executed by the processor to complete the present invention.

[0138] The terminal device may be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. The terminal device may include, but is not limited to, a processor and a memory.

[0139] The processor may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0140] The memory may be used to store the computer program and / or module. The processor realizes various functions of the terminal device by running or executing the computer program and / or module stored in the memory, and by calling the data stored in the memory.

[0141] If the modules / units integrated in the terminal device are implemented in the form of software functional units and sold or used as independent products, they may be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the above-mentioned embodiment methods of the present invention, it may also be completed by a computer program instructing relevant hardware. The computer program may be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments may be implemented. Among them, the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc. It should be noted that the content included in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0142] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, various modifications and variations can be made to the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A foreign object detection method based on millimeter-wave radar three-dimensional point cloud imaging, characterized in that, Including the following steps: Obtain the original data matrix for foreign object detection; Generate the range-Doppler spectrum of the target information based on the original data matrix, extract the effective targets from the range-Doppler spectrum, and calculate the range information of the effective targets and the azimuth and elevation angles of the effective targets; Convert the azimuth, elevation angle, and range information into 3D Cartesian coordinate system point cloud data, and obtain the original grayscale image data based on the point cloud data; Based on the original grayscale image data, perform oversampling processing on the samples to generate new grayscale image data, obtain the grayscale image data set according to the original grayscale image data and the new grayscale image data, perform dimensionality reduction processing on the data in the grayscale image data set, and perform foreign object recognition on the dimensionality-reduced data to obtain the foreign object detection result.

2. The foreign object detection method based on millimeter-wave radar three-dimensional point cloud imaging according to claim 1, wherein, The obtaining of the original data matrix for foreign object detection includes: Collect the reflected signals of the target through the millimeter-wave radar array, convert the reflected signals into intermediate-frequency signals, and construct the original data matrix according to the intermediate-frequency signals.

3. A foreign object detection method based on millimeter-wave radar three-dimensional point cloud imaging according to claim 1, characterized in that, The extracting of the effective targets from the range-Doppler spectrum includes: Put the range-Doppler spectrum F(u, v) into a two-dimensional CFAR detector to remove clutter and noise: Let the number of training units be N train , and the false alarm rate be P fa , the CFAR window function is w(u, v), and the calculation processes of the clutter noise power Z(u, v) and the threshold factor α are as follows: Reference threshold level V TH (u, v) = αZ(u, v) is used to determine valid targets. If F(u, v) > V TH (u, v), then the current target is determined to be a potential target.

4. A foreign object detection method based on three-dimensional point cloud imaging of millimeter-wave radar according to claim 1, characterized in that The calculating of the range information r of the effective targets includes: Where c represents the speed of light and B represents the radar frequency modulation bandwidth.

5. A foreign object detection method based on millimeter-wave radar three-dimensional point cloud imaging according to claim 1, characterized in that, The calculating of the azimuth and elevation angles of the effective targets includes: Organize the information of each effective target into a 12-dimensional vector X, calculate the covariance matrix R of the vector X, and perform eigen-decomposition. The expression form is as follows: R = XX H = UΣU H Decompose the covariance matrix into the following form: Among them, U S represents the signal subspace U S , which is the eigenvector corresponding to the largest eigenvalue; U N represents the noise subspace and is the remaining eigenvectors; Let the coordinates of 12 array elements be (x n , y n ). The first array element is the reference array element located at the coordinate origin, the operating wavelength is λ, and the steering vector is expressed as: Construct the two-dimensional spatial spectrum function P through the orthogonality between the noise subspace and the steering vector MUSIC : Search for the peak position of P in the two-dimensional parameter space, and the estimated values of the azimuth angle and elevation angle corresponding to the target are as follows: MUSIC ​ 6. The foreign object detection method based on millimeter-wave radar three-dimensional point cloud imaging according to claim 1, wherein, The converting of the azimuth, elevation angle, and range information into 3D Cartesian coordinate system point cloud data includes: According to the target distance r, azimuth angle θ and elevation angle Construct a three-dimensional rectangular coordinate system, obtain the point coordinates (x, y, z), filter out the miscellaneous points and noise points through the DBSCAN clustering method, and merge the point cloud matrices of n f frames to achieve multi-frame fusion of point cloud data.

7. A foreign object detection method based on three-dimensional point cloud imaging of millimeter-wave radar according to claim 1, characterized in that, The obtaining of the original grayscale image data based on the point cloud data includes: Perform normalization processing on the coordinate data; Map the normalized coordinates to a discrete grid of M g × N g and Calculate the number of point clouds K within each grid (p, q) pq , according to the number of point clouds K pq Calculate the height feature of the point cloud data and the density feature d pq ; Linearly fuse the height and density features to generate the grayscale value I pq , according to the grayscale value I pq Obtain the grayscale image matrix I, and convert the grayscale image matrix I into a vector I' of M g ×N g dimensions.

8. A foreign object detection method based on millimeter-wave radar three-dimensional point cloud imaging, characterized in that, Including the following steps: A data acquisition module for obtaining the original data matrix for foreign object detection; An effective target information extraction module for generating the range-Doppler spectrum of the target information based on the original data matrix, extracting the effective targets from the range-Doppler spectrum, and calculating the range information of the effective targets and the azimuth and elevation angles of the effective targets; An original grayscale image data generation module for converting the azimuth, elevation angle, and range information into 3D Cartesian coordinate system point cloud data and obtaining the original grayscale image data based on the point cloud data; A foreign object detection module for performing oversampling processing on the samples based on the original grayscale image data to generate new grayscale image data, obtaining the grayscale image data set according to the original grayscale image data and the new grayscale image data, performing dimensionality reduction processing on the data in the grayscale image data set, and performing foreign object recognition on the dimensionality-reduced data to obtain the foreign object detection result.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1-7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the method according to any one of claims 1-7.

Images