A method for fusion of material features of moving objects based on millimeter-wave radar

Abstract

The present invention discloses a method for fusion of material features of moving objects based on millimeter wave radar: Step 1: Obtain a one-dimensional complex sequence S received by the radar receiver raw ; Step 2: The one-dimensional complex sequence S received by the radar receiver raw The method involves splitting subsequences, each representing a signal sequence s(n; k, c) received by a single radar channel. These subsequences are then preprocessed to obtain the precise radar range-amplitude spectrum corresponding to the single channel. Step three: p(d; k, c) is converted into a compressed sequence representing target characteristics and concatenated to obtain a one-dimensional sequence representing the target characteristics. Step four: The one-dimensional sequence is filtered according to M×N transceiver channels to obtain a two-dimensional spatiotemporal feature matrix. This method improves real-time recognition and is applicable not only to object material recognition but also to target perception using millimeter-wave signals, enriching target feature information and compressing data.

Description

Technical Field

[0001] The present invention belongs to the field of feature processing and relates to a method for fusing material features of moving objects based on millimeter-wave radar. Background Art

[0002] Material recognition is key to furthering the interaction between many smart devices and scenarios. For example, when grasping objects made of different materials, robots need to skillfully control the force based on the material to avoid damage or failure due to insufficient force. Similarly, smart vehicles can avoid falling into water by identifying the material of the road while driving.

[0003] Generally speaking, material recognition scenarios can be categorized into two main types: the first involves scenarios where the recognition device and the target object are relatively stationary; the second involves scenarios where the device and the target object are in relative motion. Because the target's material characteristics are dependent on the relative position of the device and the recognition device, in mobile scenarios, it is often necessary to retain the target's position information for each recognition session. This approach eliminates interference caused by relative motion between the device and the target object, ensuring high recognition rates.

[0004] However, this method adds additional time-varying position information of the target object when identifying the material type of the target object. The existing method of retaining the target object's position information is to retain the entire received signal sequence, that is, to retain the range amplitude spectrum of the entire radar field of view to determine the relative position of the target object and the radar. Therefore, in order to retain the target's position information, a large amount of redundant data is introduced, which increases the burden on subsequent processing and makes it difficult to ensure the real-time recognition. Therefore, the application of such methods in practical environments will be severely limited. At the same time, existing recognition methods often adopt a single-link signal transmission and reception mode, which puts forward requirements for the robustness of the link, and the recognition equipment often has multiple transmitting and receiving antennas. Only a single link is used to identify the target material, and the potential spatial characteristics brought by multiple antennas and multiple links are not utilized. Summary of the Invention

[0005] In response to the problems of poor real-time performance in the above-mentioned existing technologies and the use of a single link to identify target materials without utilizing the potential spatial features brought by multi-antenna multi-links, the purpose of the present invention is to provide a method for fusion of material features of moving objects based on millimeter-wave radar.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] A method for fusing material features of moving objects based on millimeter-wave radar includes the following steps:

[0008] Step 1: Limit the target object's moving range to the radar's field of view, and the radar transmits a signal to detect the target; the number of the radar's transmitting antennas and receiving antennas are and For each receiving and transmitting channel, the radar receiver performs analog-to-digital conversion on the captured signal and performs discrete sampling on the received signal to obtain the signal. complex number of received signal samples ,in ; In sending and receiving After the signal is received, the one-dimensional complex sequence received by the radar receiver is obtained ;

[0009] Step 2: Convert the one-dimensional complex sequence received by the radar receiver Split into subsequences, each subsequence has a length of , each subsequence is the signal sequence received by the radar in a single channel ,in , ; then Perform preprocessing to obtain the accurate radar range amplitude spectrum corresponding to a single channel;

[0010] Step 3: The precise range-amplitude spectrum obtained in step 2 Convert it into a compressed sequence representing the target features and concatenate them to obtain a one-dimensional sequence representing the target features;

[0011] Step 4: The one-dimensional sequence obtained in step 3 according to The two-dimensional spatiotemporal feature matrix is ​​obtained by filtering the receiving and transmitting channels. .

[0012] Further, in step 2, the Preprocessing is performed to obtain the accurate radar range amplitude spectrum corresponding to a single channel, including the following sub-steps:

[0013] Step 21, the signal sequence Perform fast Fourier transform and take the modulus to obtain its frequency domain signal sequence. The ordinal number of the converted frequency domain signal sequence is and distance Linear correlation to obtain the original distance amplitude spectrum :

[0014]

[0015] in, represents the total number of signal components received by the radar, Represents the The path loss of each signal component, represents the radar's transmit power, Indicates the Half of the propagation distance of the signal component, that is, the distance between the target object and the radar;

[0016] Original range-amplitude spectrum middle, , , indicating the When receiving for the first time, The original range amplitude spectrum of the signal received by each channel;

[0017] Step 22: the original distance amplitude spectrum The amplitude dimension is spline interpolated to obtain the radar range amplitude spectrum with high range resolution ;

[0018] Step 23, initialization phase, point the device at the air and measure the reference signal , obtain the spectrum peak data of the pseudo target , after multiple measurements, obtain the peak data The average value is used as the reference value for compensating amplitude distortion ; In the measurement phase, a scaling factor is introduced Used to compensate for the amplitude distortion caused by the automatic gain control unit; calculate the amplitude of the pseudo target obtained in each measurement ,make , get the signal after amplitude compensation :

[0019]

[0020] Step 24: First, within the range of the radar's detection limit, Measure the distance points and calculate the true distance of each measuring point and the measured distance The residual value between , ;

[0021] Then, use a curve To fit , where the independent variable is the measured distance value, For curve The polynomial parameter matrix of the curve is solved by the least squares method. The parameter matrix The optimal value of :

[0022]

[0023] Finally, using the residual compensation curve Compensate the distance measured, that is, during the actual measurement, the distance value measured by the radar is , using the residual compensation curve right Perform compensation and obtain the compensated distance value :

[0024]

[0025] Finally, the accurate range amplitude spectrum corresponding to a single channel is obtained :

[0026] .

[0027] Furthermore, step three specifically includes the following sub-steps:

[0028] Step 31, the precise range amplitude spectrum obtained in step 2 is a one-dimensional sequence, which can be expressed as ,in = , express The corresponding ordinal number; in the sequence Find the maximum value in and its corresponding ordinal number ;The maximum point of the radar range amplitude spectrum It is about the feature points of the target object;

[0029] Step 32: Set the amplitude threshold, select half power as the amplitude threshold, and set the maximum value point nearby The range is greater than The points are regarded as reliable target feature points, and then these points are formed into subsequences ;

[0030] Step 33, subsequence Perform compression processing to obtain a single-shot single-channel target feature sequence after range amplitude compression :

[0031] ;

[0032] Step 34: convert the single-channel target feature sequence Splice into a one-dimensional sequence according to the order of sending and receiving , , then the one-dimensional sequence Characterize the target features of a single channel.

[0033] Compared with the prior art, the present invention has the following advantages:

[0034] 1. By analyzing the object's material characteristics, a subsequence of the target object is found within the entire radar range-amplitude spectrum. The material-related distance information in the subsequence is fused with the material-related signal amplitude information, achieving data compression. The compressed subsequences obtained from multiple measurements are then concatenated into a one-dimensional sequence, reflecting the temporal movement of the target features. The resulting feature sequence not only contains the temporal variations of the target's distance and amplitude, but also significantly reduces the length of the fused sequence compared to the original sequence, reducing the burden of subsequent processing and improving real-time recognition.

[0035] 2. By utilizing multi-antenna technology, the target features measured under multiple links are fused. Due to the different spatial characteristics of each link, the signals received under each link reflect the spatial feature information of the target object.

[0036] 3. According to the temporal and spatial characteristics of the target object, a two-dimensional space-time matrix reflecting the target characteristics is formed. This matrix contains the temporal and spatial information of the moving target and compresses the data volume of the target characteristics, thereby improving the recognition accuracy and real-time performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 is a flow chart of the method of the present invention;

[0038] Figure 2 This is a confusion matrix diagram of 11 different material recognition rates in an embodiment of the present invention;

[0039] Figure 3 This is a confusion matrix diagram of the recognition rates of 20 daily objects in an embodiment of the present invention.

[0040] The present invention will be further explained and illustrated in detail below with reference to the accompanying drawings and specific embodiments. DETAILED DESCRIPTION

[0041] Follow the above technical solution, see Figure 1 The method for fusion of material features of moving objects based on millimeter wave radar provided by the present invention specifically includes the following steps:

[0042] Step 1: Limit the target object's moving range to the radar's field of view, and the radar transmits a signal to detect the target; the number of the radar's transmitting antennas and receiving antennas are and , that is, each time the signal is transmitted Each transmitting antenna sends a signal. The receiving antennas receive simultaneously; After the signal is received, the one-dimensional complex sequence received by the radar receiver is obtained .

[0043] The signal received by the radar receiver is the reflected signal from the radar transmitter after it is reflected on the surface of the target object and then captured by the radar receiving antenna. Secondary signal, and the transmitting and receiving antennas can be composed For each receiving and transmitting channel, the radar receiver performs analog-to-digital conversion on the captured signal and performs discrete sampling on the received signal to obtain the signal. complex number of received signal samples ,in ;right Send and receive operations and After the same processing is performed on the transmitting and receiving antenna pairs, the final one-dimensional complex sequence is obtained , sequence length for:

[0044] (1)

[0045] Step 2: Convert the one-dimensional complex sequence received by the radar receiver Split into subsequences, each subsequence has a length of , each subsequence is the signal sequence received by the radar in a single channel ,in , . Then Preprocessing is performed to obtain the accurate radar range amplitude spectrum corresponding to a single channel. This radar range amplitude spectrum can represent the radar field of view information.

[0046] The preprocessing refers to: Perform fast Fourier transform to realize the time-frequency domain conversion of the signal and obtain the original distance amplitude spectrum; then perform interpolation processing, distortion compensation and offset correction on the original example amplitude spectrum based on the propagation distance of the signal to obtain the signal sequence The corresponding accurate distance amplitude spectrum. Specifically includes the following sub-steps:

[0047] Step 21, the reflected signal components from different target objects together constitute the received signal captured by the radar receiver. These signal components have different propagation distances. In order to distinguish these signal components according to the signal propagation distance and obtain the original range amplitude spectrum, the signal sequence is usually Perform fast Fourier transform and take the modulus to obtain its frequency domain signal sequence. The ordinal number of the converted frequency domain signal sequence is and distance Linearly related, so the original distance amplitude spectrum can be obtained :

[0048] (2)

[0049] in, represents the total number of signal components received by the radar, Represents the The path loss of each signal component, represents the radar's transmit power (which can be considered a known parameter), Indicates the The distance between the target object and the radar is half of the propagation distance of the signal component.

[0050] Original range-amplitude spectrum middle, , , indicating the When receiving for the first time, The original range amplitude spectrum of the signal received by each channel.

[0051] Step 22, get the original distance amplitude spectrum Finally, in order to obtain accurate radar range amplitude spectrum, it is necessary to first improve the original radar range amplitude spectrum The distance resolution can be improved by interpolation. Specifically, the original range amplitude spectrum The amplitude dimension is spline interpolated to obtain the radar range amplitude spectrum with high range resolution The amplitude is sub-sampling effectively, and the range-amplitude relationship of each signal component with high range resolution is obtained.

[0052] Step 23: The radar range amplitude spectrum is distorted. In step 22, a high range resolution range amplitude spectrum is obtained. After that, distortion compensation is required. Since there is an automatic gain control unit at the radar receiving end to compensate for the power of the received signal, the amplitude of the received signal may be distorted, which may interfere with material recognition. The present invention uses the direct leakage of the equipment to compensate for the distortion caused by the automatic gain control unit. Millimeter wave equipment has direct leakage from the transmitter to the receiver, and generates a false target on the range amplitude spectrum. Since the relative positions of the transmitter and the receiver are fixed, the position where the false target appears on the range amplitude spectrum is is fixed, the amplitude of the pseudo target is , and the amplitude of the pseudo target Affected by the transmit power and the automatic gain control unit. Since the transmit power of the device is a known parameter, The change of is only related to the automatic gain control unit, so it can be used Compensate for the effects of the automatic gain control unit. Since pseudo targets always appear at the same position on the range-amplitude spectrum, the present invention dynamically compensates for the amplitude distortion caused by the automatic gain control unit by comparing pseudo targets obtained through multiple measurements. The specific operation is as follows: In the initialization phase, point the device at the air and measure the reference signal. , obtain the spectrum peak data of the pseudo target , after multiple measurements, obtain the peak data The average value is used as the reference value for compensating amplitude distortion During the measurement phase, a scaling factor is introduced Used to compensate for the amplitude distortion caused by the automatic gain control unit. Calculate the amplitude of the pseudo target obtained in each measurement ,make . Then we can get the signal after amplitude compensation :

[0053] (3)

[0054] Since the reference value of direct leakage is universal, it only needs to be calculated once during the initialization phase before the actual measurement and can be applied to the entire measurement phase.

[0055] Step 24: In order to obtain an accurate radar range amplitude spectrum, after obtaining a high range resolution and a distortion-compensated signal After that, it is still necessary to compensate for the inherent error caused by the equipment. Since there are always manufacturing errors in the production of equipment, there is an inherent error between the measured frequency and the true frequency of the sampling point when sampling the signal. This error is reflected in the distance amplitude, that is, there is an error between the nominal value and the true value of each distance point. Because correctly estimating the distance of the target reflector relative to the radar is important for material characteristics, this error needs to be corrected. The correction method is to use the least squares method to fit the error curve of the inherent error as the distance changes, and then use this error curve to compensate for the inherent error of each sampling point. The specific operation is:

[0056] First, within the range of the radar's detection limit, Measure the distance points and calculate the true distance of each measuring point and the measured distance The residual value between , ;

[0057] Then, to predict the residual value of all distance points, a curve is used To fit , where the independent variable is the measured distance value, For curve The polynomial parameter matrix of . The fitting curve It can be regarded as the distance value When the distance residual value measured by the radar is , is the independent variable Any distance value within the value range. And use the least squares method to solve the curve The parameter matrix The optimal value of :

[0058] (4)

[0059] Finally, using the residual compensation curve Compensate the distance measured. That is, during the actual measurement, the distance value measured by the radar is , using the residual compensation curve right Perform compensation and obtain the compensated distance value :

[0060] (5)

[0061] This correction is designed to eliminate the differences caused by the manufacturing process and device components. For the same device, it is usually only necessary to compensate for the inherent error once. After compensating for the inherent error, the accurate distance amplitude spectrum corresponding to a single channel is finally obtained. :

[0062] (6)

[0063] Step 3: The precise range-amplitude spectrum obtained in step 2 Convert it into a compressed sequence that represents the target features and concatenate them to obtain a one-dimensional sequence that represents the target features. This includes the following sub-steps:

[0064] Step 31, the precise range-amplitude spectrum obtained in step 2 is a one-dimensional sequence, which can be expressed as ,in = , express The corresponding ordinal number. Find the maximum value in and its corresponding ordinal number The present invention assumes that, in the material recognition scenario, the target object is the main object that appears in the radar field of view, and the strongest spectrum peak on the radar's range amplitude spectrum is formed by the reflection of the target signal. Therefore, the found That is, it is formed by the reflection of the target object, and Corresponding ordinal number Related to the position of the target object. That is, the maximum point of the radar range amplitude spectrum It is the feature point of the target object.

[0065] Step 32, although the maximum point It is the characteristic point of the target object, but the characteristic point of the target object is not limited to the maximum point A. Due to the size and shape of the target object and the angle of incidence of the signal, the reflection forms a series of points, and the point with the largest amplitude among these points is So, at the maximum point There are points nearby that are formed by the reflection of the target object, and these points can still be regarded as the characteristic points of the target object. Based on this idea, the present invention sets the amplitude threshold and Filter other feature points nearby. Specifically, select half power as the amplitude threshold and take the maximum value point as nearby The range is greater than The points are regarded as reliable target feature points, and then these points are formed into subsequences , then the subsequence It is a feature sequence that reliably represents the target.

[0066] Step 33, for the subsequence obtained in step 32 , calculate the characteristic compression sequence The specific operations are:

[0067] Under the irradiation of radar, the radar cross-sectional area of ​​an object can characterize the material type of the object. According to the radar propagation formula, the radar cross-sectional area of ​​an object is for:

[0068] (7)

[0069] in , , , , They represent the transmitted signal power, received signal power, transmitter gain, receiver gain, and wavelength of the signal in free space. Once the radar equipment is determined, these parameters are constants. . Represents the distance between the target and the radar. In the mobile recognition scenario, as the relative position of the radar and the target changes, The value of is also changing. It represents the signal power received by the radar, which changes with the material type of the target object and the distance between the target object and the radar. Therefore, the radar cross-section that characterizes the target characteristics can be expressed as:

[0070] (8)

[0071] Among them, the power of the received signal , the power of the received signal can be Converted to the amplitude of the received signal According to step 31, the one-dimensional sequence formed by the radar range amplitude spectrum middle, The value of is the amplitude value of the received signal, The value of is linearly related to the distance value. According to formula (7), it can be inferred that:

[0072] (9)

[0073] For the subsequence obtained in step 32 Perform compression processing to obtain the characteristic compression sequence , Evolved from formula 8:

[0074] (10)

[0075] It is a single-shot single-channel target feature sequence after range amplitude compression.

[0076] Step 34: The single-pass target feature sequence obtained in step 33 is Splice into a one-dimensional sequence according to the order of sending and receiving , , then the one-dimensional sequence Characterize the target features of a single channel.

[0077] Step 4: The one-dimensional sequence obtained in step 3 according to The two-dimensional spatiotemporal feature matrix is ​​obtained by filtering the receiving and transmitting channels. The specific operations are as follows:

[0078] By using multi-antenna technology, the target features measured under multiple links are fused. Due to the different spatial characteristics of each link, the signals received under each link reflect the spatial feature information of the target object. The one-dimensional sequence obtained in step 3.4 Represents the target feature under a transceiver channel. Such a feature sequence has a total of , will The feature sequences constitute the final two-dimensional spatiotemporal feature matrix .

[0079] In order to demonstrate the feasibility and effectiveness of the method of the present invention, the present invention conducted the following experiments.

[0080] Microbenchmark experiments were conducted using 11 common materials to verify the effectiveness of the moving object material feature fusion method. These materials include copper, brass, steel, aluminum, ceramic, glass, acrylic, ABS resin, elm, poplar, and pine, as listed in Table 1. These test objects varied in size and thickness. In this experiment, a handheld millimeter-wave radar was used for mobile measurement to identify stationary objects. The present invention used the IWR1843BOOST commercial millimeter-wave radar board produced by Texas Instruments (TI) for data acquisition. The IWR1843 chip operates in the 77GHz millimeter-wave frequency band (77-81GHz). It integrates seven on-board antennas (3 transmit and 4 receive antennas). In the experiment, all three transmit antennas operated at a 4GHz bandwidth, transmitting signals sequentially and time-sharingly, while the four receive antennas received reflected signals. The program was implemented in Matlab, and data was processed in real time on a Windows desktop. The program was configured on an AMD Ryzen 5 2600 processor with 16GB of RAM. The entire data collection process was completed between June 10, 2022, and August 15, 2022. The experiment involved different time periods within a day and was conducted at multiple experimental locations. The simulation experimental results are described as follows:

[0081] The test results for 11 different materials are as follows Figure 2 As shown in the figure, the overall average recognition rate is 96.9%. This method can effectively distinguish different material categories. For metal materials with strong electromagnetic wave reflection ability, the average recognition rate is 99.05%. This method has better recognition performance for metal materials. Figure 2 As can be seen from the figure, this method can fully identify objects of very different materials, such as copper and glass. It can also effectively distinguish objects of similar materials, with only some errors. For example, in the case of copper and brass, 1.6% of brass samples were misidentified as copper. This is because the radar cross-section feature essentially distinguishes different materials based on their ability to reflect electromagnetic waves. Therefore, materials with similar electromagnetic reflection abilities will result in some errors.

[0082] Table 111 different materials

[0083]

[0084] Table 220 everyday objects

[0085]

[0086] This experiment also identified 20 common daily objects as shown in Table 2, and the results are as follows: Figure 3 As shown, the overall recognition rate for the 20 different objects was 81.1%. The results show that our method can still fully distinguish objects with widely varying material categories, such as copper-clad laminates and plastic cup lids. Experiments also demonstrate that our method can tolerate objects with varying surface textures, shapes, sizes, and hardness. The surfaces of the test objects can be rough (such as wooden lids) or fine (such as tin cans), soft (such as plastic buckets) or hard (such as ceramic plates). Irregularly shaped objects (such as glass bottles) were also considered. Furthermore, they can be composed of a single material (such as copper-clad laminates) or a mixture of materials (such as routers). However, these factors do not appear to have a significant impact on recognition.

Claims

1. A method for fusion of material features of moving objects based on millimeter wave radar, characterized in that: The specific steps include: Step 1: Limit the target object's moving range to the radar's field of view, and the radar transmits a signal to detect the target; the number of the radar's transmitting antennas and receiving antennas are and For each receiving and transmitting channel, the radar receiver performs analog-to-digital conversion on the captured signal and performs discrete sampling on the received signal to obtain the signal. complex number of received signal samples ,in ; In sending and receiving After the signal is received, the one-dimensional complex sequence received by the radar receiver is obtained ; Step 2: Convert the one-dimensional complex sequence received by the radar receiver Split into subsequences, each subsequence has a length of , each subsequence is the signal sequence received by the radar in a single channel ,in , ; then Perform preprocessing to obtain the accurate radar range amplitude spectrum corresponding to a single channel; Step 3: The precise range-amplitude spectrum obtained in step 2 Converting into a compressed sequence representing the target features and concatenating them to obtain a one-dimensional sequence representing the target features; specifically, the steps include: Step 31, the precise range amplitude spectrum obtained in step 2 is a one-dimensional sequence, which can be expressed as ,in = , express The corresponding ordinal number; in the sequence Find the maximum value in and its corresponding ordinal number ; The maximum point of the radar range amplitude spectrum It is about the feature points of the target object; Step 32: Set the amplitude threshold, select half power as the amplitude threshold, and set the maximum value point nearby The range is greater than The points are regarded as reliable target feature points, and then these points are formed into subsequences ; Step 33, subsequence Perform compression processing to obtain a single-shot single-channel target feature sequence after range amplitude compression : ； Step 34: convert the single-channel target feature sequence Splice into a one-dimensional sequence according to the order of sending and receiving , , then the one-dimensional sequence Characterizes the target features of a single channel; Step 4: The one-dimensional sequence obtained in step 3 according to The two-dimensional spatiotemporal feature matrix is ​​obtained by filtering the receiving and transmitting channels. .

2. The method for fusion of material features of moving objects based on millimeter wave radar according to claim 1, characterized in that: In step 2, the Preprocessing is performed to obtain the accurate radar range amplitude spectrum corresponding to a single channel, including the following sub-steps: Step 21, the signal sequence Perform fast Fourier transform and take the modulus to obtain its frequency domain signal sequence. The ordinal number of the converted frequency domain signal sequence is and distance Linear correlation to obtain the original distance amplitude spectrum : in, represents the total number of signal components received by the radar, Represents the The path loss of each signal component, represents the radar's transmit power, Indicates the Half of the propagation distance of the signal component, that is, the distance between the target object and the radar; Original range-amplitude spectrum middle, , , indicating the When receiving for the first time, The original range amplitude spectrum of the signal received by each channel; Step 22: the original distance amplitude spectrum The amplitude dimension is spline interpolated to obtain the radar range amplitude spectrum with high range resolution ; Step 23, initialization phase, point the device at the air and measure the reference signal , obtain the spectrum peak data of the pseudo target , after multiple measurements, obtain the peak data The average value is used as the reference value for compensating amplitude distortion ; In the measurement phase, a scaling factor is introduced Used to compensate for the amplitude distortion caused by the automatic gain control unit; calculate the amplitude of the pseudo target obtained in each measurement ,make , get the signal after amplitude compensation : Step 24: First, within the range of the radar's detection limit, Measure the distance points and calculate the true distance of each measuring point and the measured distance The residual value between , ; Then, use a curve To fit , where the independent variable is the measured distance value, For curve The polynomial parameter matrix of the curve is solved by the least squares method. The parameter matrix The optimal value of : Finally, using the residual compensation curve Compensate the distance measured, that is, during the actual measurement, the distance value measured by the radar is , using the residual compensation curve right Perform compensation and obtain the compensated distance value : Finally, the accurate range amplitude spectrum corresponding to a single channel is obtained : 。

Images