Multi-feature fusion traffic target recognition method, system, computer equipment and application

Abstract

The present invention belongs to the fields of computer, embedded system, radar detection technology, and deep learning technology, and discloses a multi-feature fusion traffic target recognition method, system, computer equipment, and application; uses FMCW radar to collect beat frequency signals of different traffic targets, and obtains the motion parameter information of the target through radar signal processing algorithm; couples the motion parameter information with time to obtain the distance-time diagram and Doppler-time diagram of the target, and constructs a data set; after pre-processing, the constructed data set is fused in a specific way and sent to a convolutional neural network for training, and the trained network is used as a traffic target classifier; the fused target features are sent to the classifier to obtain different traffic target classifications. The present invention innovatively proposes a feature fusion method that uses multi-dimensional features to characterize the same traffic target, and compared with the radar target recognition method based on single-dimensional features, the accuracy rate is improved to a certain extent.

Description

Technical Field

[0001] The present invention belongs to the fields of computer, embedded system, radar detection technology and deep learning technology, and in particular to a multi-feature fusion traffic target recognition method, system, computer equipment and application. Background Art

[0002] In recent years, the identification and supervision of traffic targets has become an important part of the smart transportation construction system. In the past, it often required a large amount of manpower to maintain the supervision of urban roads, which would lead to an excessively large workload, a long supervision cycle, and low efficiency. With the advancement of science and technology, the traffic target recognition method based on visual sensors has become the mainstream recognition method. Although this method can improve the efficiency of traffic target recognition, the application conditions are easily affected by extreme conditions such as sandstorms, fog, and strong light, resulting in a decrease in recognition rate. In addition, based on the imaging mechanism of the camera, it is difficult for the video to accurately identify targets that are far away from the data acquisition device.

[0003] Through the above analysis, the problems and defects of the existing technology are: the current traffic target recognition method based on visual sensors is difficult to identify distant targets, and cannot be applied to extreme environmental conditions such as sandstorms, heavy fog, strong light, and darkness.

[0004] The difficulty of solving the above problems and defects is:

[0005] (1) False alarms may occur when using radar for target detection.

[0006] (2) Rainy and snowy weather conditions may cause dynamic disturbances to the radar, increasing the detection background noise and making it difficult to identify the target.

[0007] (3) When using convolutional neural networks for target recognition training, a large amount of traffic target data samples need to be collected.

[0008] The significance of solving the above problems and defects is as follows: the present invention is a multi-feature fusion traffic target recognition method of FMCW radar, which collects instantaneous, time series and other different dimensional features of the same traffic target based on FMCW radar, and fuses the multi-dimensional features, and finally uses the fused features to characterize the target, and finally sends it to the classification network for target recognition. The method of the present invention has many application scenarios and is not limited to extreme weather conditions such as sandstorms and heavy fog. Compared with the video method, it has a large monitoring range, good safety, and flexible working mode. It can realize the ability to monitor urban road traffic all day and all weather, and greatly solves the problem of low efficiency of traditional manual operation and video monitoring. In order to assist in the management of urban traffic conditions, a stable and accurate traffic target detection and recognition method is provided, which has certain engineering application value. Summary of the invention

[0009] In view of the problems existing in the prior art, the present invention provides a multi-feature fusion traffic target recognition method, system, computer equipment and application.

[0010] The present invention is implemented as follows: a multi-feature fusion traffic target recognition method, = the multi-feature fusion traffic target recognition method comprises:

[0011] FMCW radar is used to collect the beat frequency signals of different traffic targets, and the range-Doppler map and radial distance and velocity motion parameter information of the target are obtained through radar signal processing algorithm;

[0012] The radar raw data is traversed and detected by windowed CFAR to obtain the range-Doppler map that represents the instantaneous characteristics of the target. The target motion parameter information is coupled with time to obtain the target's range-time map and Doppler-time map, and a data set is constructed.

[0013] After preprocessing, the constructed data set is fused in a specific way and sent to the convolutional neural network for training, and the trained network is used as a traffic target classifier;

[0014] The fused target features are sent to the classifier to obtain different traffic target classifications.

[0015] Furthermore, the multi-feature fusion traffic target recognition method further includes:

[0016] Continuous wave radar obtains the distance information of the target by comparing the difference between the frequency of the echo signal at any time and the frequency of the transmitted signal at this time;

[0017] The distance is proportional to the frequency difference between the two. The radial velocity and distance of the target can be obtained by processing the measured frequency difference between the two to detect the position and speed information of the traffic target.

[0018] Furthermore, the multi-feature fusion traffic target recognition method analyzes the target's motion parameters by processing the beat frequency signal after the transmission signal and the echo signal are mixed. In a sawtooth wave linear frequency modulation pulse width period, the radar transmission signal frequency increases linearly with time. The beat frequency signal frequency generated by mixing the echo signal reflected by the target at different distances with the local oscillator signal is different. Through the formula Where d is the distance between the radar and the target, C is the speed of light, and f IF represents the beat frequency after mixing, T represents the pulse width of the sawtooth linear frequency modulation signal, and B represents the bandwidth of the linear frequency modulation signal, and the distance information between the target and the radar is obtained.

[0019] Furthermore, if there is a radial velocity between the target and the radar, due to the existence of the Doppler effect between two adjacent pulses, the beat frequency of the echo signal received by different radar cycles is different, and this frequency implies the velocity information of the target, which is calculated by the formula f d represents the Doppler frequency, f0 represents the center frequency of the carrier, v represents the radial velocity of the target, c represents the electromagnetic wave velocity, and the radial velocity of the target is obtained. Combined with the above measurement method, the position and velocity information of the traffic target can be obtained.

[0020] Furthermore, the multi-feature fusion traffic target recognition method obtains the distance and speed information of the target, uses a sliding window constant false alarm algorithm to traverse and detect each frame of the range-Doppler matrix and performs heat map visualization according to the signal amplitude to obtain a range-Doppler map characterizing the target characteristics. The range-Doppler map of the target is regarded as a micro-Doppler map characterizing the instantaneous characteristics of the target.

[0021] Furthermore, the multi-feature fusion traffic target recognition method mines the motion feature information expressed by the target in a continuous period of time on the distance dimension, and couples the distance parameter and time of the target; firstly, the two columns of antenna channel signals on the same frame distance spectrum are subjected to dot product operation, and incoherent accumulation is performed in the slow time domain within the same frame, and finally the distance information of the target is accumulated within a frame length of 50 frames to obtain the distance-time diagram of the target, which is regarded as a micro-Doppler diagram representing the change of the distance of the target over time in a continuous period of time;

[0022] The multi-feature fusion traffic target recognition method is to mine the motion feature information expressed by the target in the speed dimension in a continuous period of time, and couple the speed parameter and time of the target. Based on the above-mentioned target parameter measurement method, the speed information of the target in each frame is extracted, and its main lobe and side lobe signals are retained. Finally, the Doppler-time diagram of the target can be obtained, which is regarded as a micro-Doppler diagram that characterizes the change of the target's speed over time in a continuous period of time;

[0023] The multi-feature fusion traffic target recognition method uses a minimum value normalization method to process data frame by frame;

[0024] The multi-feature fusion traffic target recognition method expresses the motion features of the RDM, RTM, and DTM images of the same target object from the perspectives of distance, speed, instantaneousness, and timing. In order to use the convolutional neural network to simultaneously train the three-dimensional motion features of the same target, the three-dimensional feature images of the same target are fused along the stepping direction after the convolution kernel, and the feature textures of all dimensions of the target are extracted dimension by dimension.

[0025] The multi-feature fusion traffic target recognition method adopts a traffic target classifier based on a convolutional neural network to realize the function of traffic target recognition, including 6 cascaded convolution layers, each layer uses 32 3×3 convolution kernels, 2 Maxpooling pooling layers, and 1 fully connected layer with a Softmax activation function as the output layer;

[0026] The processed data is input into the equipped network for training, and the trained network can be used to complete the traffic target recognition task.

[0027] Another object of the present invention is to provide a computer device, the computer device comprising a memory and a processor, the memory storing a computer program, and when the computer program is executed by the processor, the processor performs the following steps:

[0028] FMCW radar is used to collect the beat frequency signals of different traffic targets, and the range-Doppler map and radial distance and velocity motion parameter information of the target are obtained through radar signal processing algorithm;

[0029] The motion parameter information is coupled with time to obtain the target's range-time map and Doppler-time map, and a data set is constructed;

[0030] After preprocessing, the constructed data set is fused in a specific way and sent to the convolutional neural network for training, and the trained network is used as a traffic target classifier;

[0031] The fused target features are sent to the classifier to obtain different traffic target classifications.

[0032] Another object of the present invention is to provide a feature fusion traffic target recognition system for implementing the feature fusion traffic target recognition method, the feature fusion traffic target recognition system comprising:

[0033] The motion parameter information acquisition module is used to collect the beat frequency signals of different traffic targets using the FMCW radar, and obtain the target's range-Doppler map and radial distance and velocity motion parameter information through the radar signal processing algorithm;

[0034] The data set construction module uses the windowed CFAR to perform traversal detection on the radar raw data to obtain the range-Doppler map that characterizes the instantaneous characteristics of the target. It is used to couple the motion parameter information with time to obtain the range-time map and Doppler-time map of the target, and construct the data set.

[0035] The data set training module is used to send the constructed data set into the convolutional neural network for training after preprocessing and fusing it in a specific way, and use the trained network as a traffic target classifier;

[0036] The traffic target classification acquisition module is used to send the fused target features into the classifier to obtain different traffic target classifications.

[0037] Another object of the present invention is to provide an application of the feature fusion traffic target recognition method in traffic target recognition.

[0038] Another object of the present invention is to provide an application of the feature fusion traffic target recognition method in traffic target supervision.

[0039] In combination with all the above technical solutions, the advantages and positive effects of the present invention are as follows: based on the signal processing technology of FMCW radar, the present invention studies the motion parameter extraction method of traffic targets, and proposes to combine the instantaneous motion characteristics of the target with the temporal motion characteristics to characterize the motion state of the target, so that the motion characteristics of the traffic target are more comprehensively expressed; combined with the characteristics of the characteristic graph of the radar echo signal of the traffic target, which has distinct features and less clutter, a preprocessing method such as data normalization for the traffic target data set is given, and a feature fusion method for image fusion along the stepping direction after the convolution kernel is proposed, which retains the three-dimensional motion characteristics of the target while simplifying the network structure; an end-to-end convolutional neural network is built for the recognition of traffic targets, and the fused feature image is directly used for training, eliminating the inconvenience and feature loss caused by the traditional method of manually extracting specific target features.

[0040] The present invention comprehensively utilizes radar detection technology, deep learning and other technologies to provide a method for end-to-end traffic target identification. At the same time, with the help of FMCW radar, traffic targets on urban roads can be monitored around the clock, and the application scenarios are not limited to extreme environments such as sandstorms, heavy fog, strong light, and darkness.

[0041] The present invention uses FMCW radar to collect beat frequency signals of different traffic targets, obtains the range-Doppler map and radial distance and speed motion parameter information of the target through radar signal processing algorithm; after traversing and detecting the radar raw data with CFAR, a range-Doppler map representing the instantaneous characteristics of the target can be obtained, and the motion parameter information is coupled with time to obtain the range-time map and Doppler-time map of the target, and construct a data set; after pre-processing, the constructed data set is fused along the stepping direction after the convolution kernel and then sent to the convolution neural network for training, and the trained network is used as a traffic target classifier; the fused target features are sent to the classifier to obtain different traffic target classifications;

[0042] The present invention obtains features of different dimensions representing the same traffic target, and describes the same traffic target from three perspectives: instantaneous features of the target, time series features of distance changing over time, and time series features of speed changing over time. Compared with a single feature, the number of feature expressions of the target is increased. The data set is normalized, data enhanced, and other processes are performed to optimize the overall quality of the data set. At the same time, the fusion method of fusing the features of three different dimensions in sequence along the stepping direction after the convolution kernel also ensures that the network can extract the feature textures of different dimensions of the same target one by one, which improves the overall recognition accuracy compared to using a single feature for traffic target recognition.

[0043] Compared with the traffic target detection and recognition method based on traditional visual sensors, FMCW radar has the characteristics of high resolution and sensitivity to speed, so the accuracy of measuring traffic target parameters by FMCW radar is higher than that of video method. At the same time, due to the use of FMCW radar hardware, its working principle does not depend on external lighting conditions, so this method can weaken the influence of extreme environmental conditions such as darkness, strong light, and sandstorms in traditional video methods. The traditional traffic target parameter measurement method based on visual sensors has the error of pinhole imaging, which makes the measured motion parameters of traffic targets not accurate enough. However, the use of FMCW radar to detect the motion parameters of traffic targets has the characteristics of high resolution and sensitivity to Doppler frequency shift, which makes the measured motion parameters of traffic targets more accurate. Unlike other pulse wave radars, the linear frequency modulated continuous wave (FMCW) radar measures the motion parameters of targets based on the analysis of the beat frequency signal of the target, which further explains the working principle of FMCW radar and better explains the principle of accurate parameter measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a flow chart of the multi-feature fusion traffic target recognition method provided by an embodiment of the present invention.

[0045] Figure 2 It is a structural schematic diagram of a multi-feature fusion traffic target recognition system provided by an embodiment of the present invention;

[0046] Figure 2 In: 1. Motion parameter information acquisition module; 2. Data set construction module; 3. Data set training module; 4. Traffic target classification acquisition module.

[0047] Figure 3 It is a flow chart for implementing the multi-feature fusion traffic target recognition method provided by an embodiment of the present invention.

[0048] Figure 4 This is an image of a dual-target radar signal after CFAR provided by an embodiment of the present invention.

[0049] Figure 5 The RDM diagram provided by the embodiment of the present invention shows pedestrians, electric vehicles and cars from left to right.

[0050] Figure 6 This is an imaging diagram of the original echo signal of the dual-channel radar provided by an embodiment of the present invention.

[0051] Figure 7 The RTM diagrams provided by the embodiment of the present invention respectively show pedestrians, electric vehicles, and cars from left to right.

[0052] Figure 8 The DTM diagrams provided by the embodiment of the present invention respectively show pedestrians, electric vehicles, and cars from left to right.

[0053] Fig. 9 This is an imaging diagram after data normalization provided by an embodiment of the present invention that solves the problem of feature masking.

[0054] Fig.10 It is a feature map after the three-dimensional feature map provided by the embodiment of the present invention is fused.

[0055] Fig.11 It is a network structure diagram of a traffic target classifier provided in an embodiment of the present invention.

[0056] Fig.12 This is an actual scene test diagram provided by an embodiment of the present invention.

[0057] Fig.13 It is a structural diagram of A24BF provided in an embodiment of the present invention.

[0058] Fig.14 It is a schematic diagram of the test accuracy change curve of different two-dimensional feature fusion data sets provided by an embodiment of the present invention.

[0059] Fig.15 It is a schematic diagram of a test accuracy change curve of a three-dimensional feature fusion data set provided in an embodiment of the present invention.

[0060] Fig.16 It is a comparison chart of the accuracy before and after data standardization provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0062] In view of the problems existing in the prior art, the present invention provides a multi-feature fusion traffic target recognition method, system, computer device and application. The present invention is described in detail below with reference to the accompanying drawings.

[0063] like Figure 1 As shown, the multi-feature fusion traffic target recognition method provided by the present invention comprises the following steps:

[0064] S101: Use FMCW radar to collect beat frequency signals of different traffic targets, and obtain the target's range-Doppler map and radial distance, speed and other motion parameter information through radar signal processing algorithm;

[0065] S102: coupling the motion parameter information with time to obtain a range-time map and a Doppler-time map of the target, and constructing a data set based on the range-time map and the Doppler-time map;

[0066] S103: After preprocessing, the constructed data set is fused in a specific way and then sent to a convolutional neural network for training, and the trained network is used as a traffic target classifier;

[0067] S104: Send the fused target features to the classifier to obtain different traffic target classifications.

[0068] Ordinary technicians in the industry can also use other steps to implement the multi-feature fusion traffic target recognition method provided by the present invention. Figure 1 The multi-feature fusion traffic target recognition method provided by the present invention is only a specific embodiment.

[0069] like Figure 2 As shown, the multi-feature fusion traffic target recognition system provided by the present invention includes:

[0070] The motion parameter information acquisition module 1 is used to collect the beat frequency signals of different traffic targets using the FMCW radar, and obtain the target's range-Doppler map and radial distance, speed and other motion parameter information through the radar signal processing algorithm;

[0071] The data set construction module 2 is used to couple the motion parameter information with time to obtain the distance-time map and Doppler-time map of the target, and construct the data set based on the distance-time map and Doppler-time map;

[0072] The data set training module 3 is used to send the constructed data set into the convolutional neural network for training after preprocessing and fusing it in a specific way, and use the trained network as a traffic target classifier;

[0073] The traffic target classification acquisition module 4 is used to send the fused target features to the classifier to obtain different traffic target classifications.

[0074] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0075] The multi-feature fusion traffic target recognition method based on FMCW (Frequency Modulated Continuous Wave) radar provided in an embodiment of the present invention includes:

[0076] Continuous wave radar obtains the distance information of the target by comparing the difference between the frequency of the echo signal at any time and the frequency of the transmitted signal at this time;

[0077] The distance is proportional to the frequency difference between the two. The radial velocity and distance of the target can be obtained by processing the measured frequency difference between the two to detect the position and speed information of the traffic target.

[0078] The FMCW radar of the present invention refers to a continuous wave radar with a linearly modulated transmission frequency. The motion parameters of the target are analyzed by processing the beat frequency signal after the transmission signal is mixed with the echo signal. In a sawtooth wave linear frequency modulation pulse width period, the radar transmission signal frequency increases linearly with time. The beat frequency signal frequency generated by mixing the echo signal reflected by the target at different distances with the local oscillator signal is different. The formula (where d is the distance between the radar and the target, C is the speed of light, and f IF represents the beat frequency after mixing, T represents the pulse width of the sawtooth linear frequency modulation signal, and B represents the bandwidth of the linear frequency modulation signal) can obtain the distance information between the target and the radar.

[0079] If there is radial velocity between the target and the radar, due to the existence of the Doppler effect between two adjacent pulses, the beat frequency of the echo signal received by different radar cycles is different, and this frequency implies the velocity information of the target, which is calculated by the formula (f d represents the Doppler frequency, f0 represents the center frequency of the carrier, v represents the radial velocity of the target, and c represents the electromagnetic wave velocity) can be used to obtain the radial velocity of the target. Combined with the above measurement method, the position and velocity information of the traffic target can be obtained.

[0080] In the present invention, the distance and speed information of the target are obtained, and then a sliding window constant false alarm algorithm is used to traverse and detect each frame of the range-Doppler matrix and perform heat map visualization according to the signal amplitude, so that a range-Doppler map representing the target characteristics can be obtained. Because the time represented by one frame of data is about 0.033 seconds, which is much shorter than the continuous time when the human eye observes the traffic target under normal circumstances, the range-Doppler map (RDM map) of the target can be regarded as a micro-Doppler map representing the instantaneous characteristics of the target.

[0081] In order to mine the motion characteristic information expressed by the target object in the distance dimension within a continuous period of time, the present method couples the distance parameter and time of the target object. First, the dot product operation is performed on the two columns of antenna channel signals on the distance spectrum of the same frame, and incoherent accumulation is performed in the slow time domain within the same frame. Finally, the distance information of the target is accumulated within a frame length of 50 frames (about 1.65s), and the distance-time diagram (RTM diagram) of the target can be obtained. The RTM diagram can be regarded as a micro-Doppler diagram that characterizes the change of the distance of the target object over time within a continuous period of time.

[0082] In the present invention, the speed characteristics of different traffic targets are also different. In order to mine the motion characteristic information expressed by the target in the speed dimension in a continuous period of time, this method couples the speed parameters and time of the target. Based on the above-mentioned parameter measurement method of the target, the speed information of the target in each frame is extracted, and its main lobe and side lobe signals are retained. Finally, the Doppler-time map (DTM map) of the target can be obtained by accumulating within a frame length of 50 frames (about 1.65s). The DTM map can be regarded as a micro-Doppler map that characterizes the change of the speed of the target over time in a continuous period of time.

[0083] Since the energy spectrum of radar frame data collected at different times fluctuates, the spectrum features of frame data with low peak energy may be masked by frame data with high peak energy, resulting in unclear expression of the motion feature map of the same target. To solve the problem of feature masking, the present invention uses the minimum value normalization method to process the data frame by frame.

[0084] In the present invention, the RDM, RTM, and DTM images of the same target object express the motion characteristics from the perspectives of distance, speed, instantaneousness, and timing. In order to use the convolutional neural network to simultaneously train the three-dimensional motion characteristics of the same target, the present invention fuses the three-dimensional feature images of the same target along the stepping direction after the convolution kernel, and can extract the feature texture of all dimensions of the target dimension by dimension.

[0085] In order to realize the function of traffic target recognition in the present invention, a traffic target classifier based on convolutional neural network is designed in combination with the characteristics of distinct features and less clutter of traffic target data measured by radar. The network structure includes 6 cascaded convolutional layers, each layer uses 32 3×3 convolution kernels, 2 Maxpooling pooling layers, and 1 fully connected layer with Softmax activation function as the output layer.

[0086] In the present invention, the processed data is input into a mounted network for training, and the trained network can be used to complete the traffic target recognition task.

[0087] The technical effects of the present invention are described in detail below in conjunction with experiments.

[0088] The FMCW radar platform used in this experiment is the 24GHz radar development board A24BF produced by Analog Devices. A24BF is an independent radar system designed with two transmit channels, four receive channels and a DSP module. The antennas are arranged to support a unified virtual array with a spacing of half the wavelength. Fig.13 shown.

[0089] The specific parameters of the FMCW radar development board used in the experiment of the present invention are as follows: the center carrier frequency is 24GHz, the maximum bandwidth of the linear frequency modulation sawtooth wave is 300MHz, the sampling frequency is 1MHz, and the number of sweeps per frame is 128 sawtooth wave cycles. According to the above radar configuration parameter analysis, the development board has a range resolution d res is 0.5m. Under normal traffic background, the actual size and interval distance of traffic targets are greater than 0.5m. Therefore, in theory, the development board meets the requirements for traffic target detection in terms of spatial resolution. According to the above radar configuration parameter analysis, the velocity resolution v of the development board is res It is 0.19m / s. Under normal traffic background, the actual movement speed and movement speed difference of traffic targets are both greater than 0.19m / s. Therefore, theoretically, the development board meets the requirements for traffic target detection in terms of speed resolution.

[0090] Using the single-dimensional motion feature map dataset of different traffic targets, a three-classification test of pedestrians, electric vehicles, and cars was conducted from the different motion feature dimensions of the targets. The confusion matrices of pedestrians, electric vehicles, and cars based on the single-dimensional RDM (range-Doppler map), RTM (range-time map), and DTM (Doppler-time map) datasets are shown in Table 1.

[0091] Table 1 Confusion matrix of pedestrian, electric vehicle and car classification based on RDM dataset

[0092]

[0093] As shown in Table 1, the average accuracy of the three-classification task based on the traffic target RDM feature atlas is 87.33%. From Table 2, it can be found that the probability of electric vehicles and cars being confused with each other is slightly higher. The main consideration is that when the distance from the radar is far, the target echo energy is reduced, resulting in the loss of some target micro-Doppler characteristics.

[0094] Table 2 Confusion matrix of pedestrian, electric vehicle and car classification based on RTM dataset

[0095]

[0096]

[0097] As shown in Table 2, the average accuracy of the three-classification task based on the traffic target RTM feature atlas is 88%. From Table 3, it can be found that from the RTM perspective, the recognition errors of pedestrians and electric vehicles are relatively large. This is considered to be due to the fact that the relationship between the radial distance between pedestrians and electric vehicles and the radar changes over time within the observation frame length has a certain similarity.

[0098] Table 3 Confusion matrix of pedestrian, electric vehicle and car classification based on DTM dataset

[0099]

[0100] According to Table 3, the average accuracy of the three-classification task of traffic targets based on the DTM dataset is 90%, among which the recognition rate of pedestrian targets is relatively high. This is because the micro-Doppler characteristics formed by the swinging arms of pedestrians when walking will be more obvious on the DTM spectrum.

[0101] The above gives the results of the three-classification test of traffic targets using a single-dimensional feature map data set of traffic targets. The present invention uses a fused multi-dimensional feature image data set to perform a three-classification test of traffic targets. First, the RDM, RTM, and DTM feature images of the same traffic target are fused in pairs to obtain three two-dimensional feature fusion data sets, namely, RDM and RTM fused feature data sets, RDM and DTM fused feature data sets, and RTM and DTM fused feature data sets. Tests are then performed on the three two-dimensional feature fusion data sets every 1,000 iterations to obtain accuracy change curves. Finally, the optimal model is used to test the enhanced data set, and a confusion matrix is ​​plotted based on the test results. Fig.14 shown.

[0102] The traffic target confusion matrix based on the fusion of different two-dimensional features is given below.

[0103] Table 4 Confusion matrix of three-classification of traffic targets based on RDM and RTM two-dimensional feature fusion datasets

[0104]

[0105]

[0106] As shown in Table 4, the average accuracy of the three-classification task of traffic targets based on the RDM and RTM two-dimensional feature fusion dataset is 88.67%. Compared with the RDM dataset alone, the average accuracy obtained by using the RDM and RTM two-dimensional feature fusion dataset has improved to a certain extent, but compared with the RTM dataset alone, the improvement in the average accuracy is not obvious. After analysis, this phenomenon may be due to the fact that after the feature image is fused, the background of the RDM image accounts for a large proportion, resulting in a slightly insufficient feature expression ability.

[0107] Table 5 shows the confusion matrix of three-classification of traffic targets based on RDM and DTM two-dimensional feature fusion dataset.

[0108] Table 5 Confusion matrix of three-classification of traffic targets based on RDM and DTM two-dimensional feature fusion dataset

[0109]

[0110] It can be seen from Table 5 that the average accuracy of the three-classification task of traffic targets based on the RDM and DTM two-dimensional feature fusion data set is 90.33%, among which the recognition of pedestrian targets is relatively accurate. Analysis shows that the unique micro-Doppler features formed by the natural swing of pedestrians' arms when walking have a greater degree of distinction than other traffic targets.

[0111] Table 6 shows the confusion matrix of three-classification of traffic targets based on RTM and DTM two-dimensional feature fusion datasets.

[0112] Table 6 Confusion matrix of three-classification of traffic targets based on RTM and DTM two-dimensional feature fusion datasets

[0113]

[0114] As shown in Table 6, the average accuracy of the three-classification task of traffic targets based on the RTM and DTM two-dimensional feature fusion dataset is 91.33%. It can be seen that the recognition accuracy of electric vehicles is significantly improved by using the image after the fusion of RTM and DTM two-dimensional features as a dataset. After analysis, it is found that the feature performance of electric vehicles is easily confused with the feature performance of people and cars on the RTM and DTM single-dimensional feature image datasets. After fusing the two-dimensional feature images, the expression of easily confused features in the image is suppressed to a certain extent.

[0115] In order to observe the classification performance of the three-dimensional feature fusion data set, the present invention fully fuses the RDM, RTM, and DTM three-dimensional feature images, and uses the RDM, RTM, and DTM three-dimensional feature fusion data sets for training. The test verification is performed every 1000 iterations, and the accuracy change curve is obtained as shown in the figure. Fig.15 shown.

[0116] The trained network model is used to test on the enhanced dataset, and the confusion matrix drawn according to the test results is shown in Table 7.

[0117] Table 7 Confusion matrix of pedestrian, electric vehicle and car classification based on 3D feature fusion dataset

[0118]

[0119] As shown in Table 7, the average accuracy of the three-classification task based on the three-dimensional motion feature dataset is 92%. Compared with the single-dimensional feature dataset, the average accuracy is improved when the three-dimensional feature dataset is used. Considering the randomness of interference during data acquisition, the energy of the RDM spectrum representing the instantaneous motion characteristics and the RTM and DTM spectrum representing the time series characteristics fluctuate, which may weaken the micro-Doppler characteristics of the target in a certain frame or a continuous frame. The use of the feature fused dataset can correct the micro-Doppler characteristics of one dimension fluctuation to a certain extent.

[0120] To further explore the effectiveness of the dataset preprocessing method, the present invention conducted a comparative verification on a three-dimensional motion feature dataset that has not been standardized. The test verification was performed every 1000 iterations. The comparison accuracy curve is shown in the figure below. Fig.16 shown.

[0121] Fig.16 In the data set after normalization, the classification performance of the network is significantly improved. This is because data normalization can weaken the feature masking phenomenon on the target RTM and DTM feature spectra, thereby strengthening the micro-Doppler characteristics of different traffic targets, thereby improving the overall quality of the data set. Fig.16 It also fully proves that high-quality data sets are helpful in improving network performance.

[0122] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. It can be understood by a person of ordinary skill in the art that the above-mentioned devices and methods can be implemented using computer executable instructions and / or contained in a processor control code, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. Such code is provided on the carrier medium. The device and its modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, and can also be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.

[0123] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principle of the present invention should be covered by the protection scope of the present invention.

Claims

1. A multi-feature fusion traffic target recognition method, characterized in that: The multi-feature fusion traffic target recognition method comprises: FMCW radar is used to collect the beat frequency signals of different traffic targets, and the range-Doppler map and radial distance and velocity motion parameter information of the target are obtained through radar signal processing algorithm; The radar raw data is traversed and detected by CFAR to obtain the range-Doppler map that represents the instantaneous characteristics of the target. The motion parameter information is coupled with the time parameter to obtain the target's range-time map and Doppler-time map, and a data set is constructed. After preprocessing, the constructed data set is fused along the stepping direction of the convolution kernel, and then sent to the convolutional neural network for training, and the trained network is used as the traffic target classifier; The fused target features are sent to the classifier to obtain different traffic target classifications; The multi-feature fusion traffic target recognition method obtains the distance and speed information of the target, uses a sliding window constant false alarm algorithm to traverse and detect each frame of the range-Doppler matrix, and performs heat map visualization according to the signal amplitude to obtain a range-Doppler map representing the target characteristics. The range-Doppler map of the target is regarded as a micro-Doppler map representing the instantaneous characteristics of the target; The multi-feature fusion traffic target recognition method is to mine the motion feature information expressed by the target in a continuous period of time on the distance dimension, and couple the distance parameter and time of the target; firstly, the two columns of antenna channel signals on the same frame distance spectrum are subjected to dot product operation, and incoherent accumulation is performed in the slow time domain within the same frame, and finally the distance information of the target is accumulated within a frame length of 50 frames to obtain the distance-time diagram of the target, which is regarded as a micro-Doppler diagram representing the change of the distance of the target over time in a continuous period of time; The multi-feature fusion traffic target recognition method is to mine the motion feature information expressed by the target in the speed dimension in a continuous period of time, and couple the speed parameter and time of the target; based on the parameter measurement method of the target, the speed information of the target in each frame is extracted, and its main lobe and side lobe signals are retained, and finally accumulated within a frame length of 50 frames to obtain the target. The Doppler-time diagram is regarded as a micro-Doppler diagram that characterizes the change of the speed of the target over time in a continuous period of time; The multi-feature fusion traffic target recognition method uses a minimum value normalization method to process data frame by frame; The multi-feature fusion traffic target recognition method expresses the motion features of the RDM, RTM, and DTM images of the same target object from the perspectives of distance, speed, instantaneousness, and timing. In order to use the convolutional neural network to simultaneously train the three-dimensional motion features of the same target, the three-dimensional feature images of the same target are fused along the stepping direction after the convolution kernel, and the feature textures of all dimensions of the target are extracted dimension by dimension. The multi-feature fusion traffic target recognition method adopts a traffic target classifier based on a convolutional neural network to realize the function of traffic target recognition, including 6 cascaded convolution layers, each layer uses 32 3×3 convolution kernels, 2 Maxpooling pooling layers, and 1 fully connected layer with a Softmax activation function as the output layer; The processed data is input into the equipped network for training, and the trained network can be used to complete the traffic target recognition task.

2. The multi-feature fusion traffic target recognition method according to claim 1, characterized in that: The multi-feature fusion traffic target recognition method also includes: Continuous wave radar obtains the distance information of the target by comparing the difference between the frequency of the echo signal at any time and the frequency of the transmitted signal at this time; The distance is proportional to the frequency difference between the two. The radial velocity and distance of the target are obtained by processing the measured frequency difference between the two to detect the position and speed information of the traffic target.

3. The multi-feature fusion traffic target recognition method according to claim 1, characterized in that: The multi-feature fusion traffic target recognition method analyzes the target's motion parameters by processing the beat frequency signal after the transmission signal and the echo signal are mixed. In a sawtooth wave linear frequency modulation pulse width period, the radar transmission signal frequency increases linearly with time. The beat frequency signal frequency generated by mixing the echo signal reflected by the target at different distances with the local oscillator signal is different. Through the formula Where d is the distance between the radar and the target, C is the speed of light, and f IF represents the beat frequency after mixing, T represents the pulse width of the sawtooth linear frequency modulation signal, and B represents the bandwidth of the linear frequency modulation signal, and the distance information between the target and the radar is obtained.

4. The multi-feature fusion traffic target recognition method according to claim 3, characterized in that: If there is radial velocity between the target and the radar, due to the existence of the Doppler effect between two adjacent pulses, the beat frequency of the echo signal received by different radar cycles is different, and this frequency implies the velocity information of the target, which is calculated by the formula f d represents the Doppler frequency, f0 represents the center frequency of the carrier, v represents the radial velocity of the target, c represents the electromagnetic wave velocity, and the radial velocity of the target is obtained.

5. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the multi-feature fusion traffic target recognition method according to any one of claims 1 to 4, including the following steps: FMCW radar is used to collect the beat frequency signals of different traffic targets, and the range-Doppler map and radial distance and velocity motion parameter information of the target are obtained through radar signal processing algorithm; The motion parameter information is coupled with the time parameter to obtain the distance-time map and Doppler-time map of the target, and a data set is constructed; After preprocessing, the constructed data set is fused along the stepping direction of the convolution kernel and then sent to the convolutional neural network for training, and the trained network is used as a traffic target classifier; The fused target features are sent to the classifier to obtain different traffic target classifications.

6. A feature fusion traffic target recognition system implementing the feature fusion traffic target recognition method according to any one of claims 1 to 4, characterized in that: The feature fusion traffic target recognition system comprises: The motion parameter information acquisition module is used to collect the beat frequency signals of different traffic targets using the FMCW radar, and obtain the target's range-Doppler map and radial distance and velocity motion parameter information through the radar signal processing algorithm; A data set construction module is used to couple motion parameter information with time to obtain a range-time map and a Doppler-time map of the target, and to construct a data set; The data set training module is used to send the constructed data set after preprocessing to the convolutional neural network for training after fusion along the stepping direction of the convolution kernel, and use the trained network as a traffic target classifier; The traffic target classification acquisition module is used to send the fused target features into the classifier to obtain different traffic target classifications.

7. A feature fusion traffic target recognition method as described in any one of claims 1 to 4 is applied to traffic target recognition.

8. A feature fusion traffic target recognition method as described in any one of claims 1 to 4 applied to traffic target supervision.

Images