Real-time adaptive multi-modal hybrid feature vehicle-mounted radar signal recognition method and system

Abstract

The application discloses a kind of real-time adaptive multimodal hybrid feature vehicle-mounted radar signal identification method and system, method includes: calibration is carried out between vehicle sensor, establishes energy efficiency utilization maximization model, and joins real-time constraint, establishes target model;Vehicle radar and sensor data are acquired to carry out data preprocessing;By modeling target feature to mixed Gaussian model and carrying out model training, the state vector of initialization Kalman filter is carried out state prediction, the target recognition result of mixed Gaussian model is fused with the target tracking result of Kalman filter, and target recognition and tracking result are obtained.The application can automatically adjust the adaptability to different environment and interference, provide more accurate, reliable target recognition and perception ability, can optimize system performance, real-time adaptive algorithm can adjust the parameter of identification algorithm according to the characteristics of current target, to better adapt to various targets.

Description

Technical Field

[0001] This invention relates to the field of wireless communication network technology, and in particular to a real-time adaptive multimodal hybrid feature vehicle radar signal recognition method and system. Background Technology

[0002] With the rapid development of modern transportation facilities and urban construction, the vehicle environment, including urban roads, highways, and intersections, is becoming increasingly complex. Vehicle radar signal recognition systems with better performance are a key component for realizing intelligent transportation systems. Improving vehicle safety, preventing collisions and accidents, and enhancing emergency response capabilities in special circumstances have become new challenges.

[0003] By introducing multimodal fusion features, the system can acquire rich information from different sensors, improving comprehensive target perception and enhancing the system's target detection, tracking, and recognition performance. Different types of sensors may exhibit different advantages in different environments. Multimodal systems can select appropriate sensors or combinations based on the characteristics of the current environment, improving system adaptability. Furthermore, in extreme environments, multimodal fusion ensures robustness even when a sensor fails or performance degrades under specific conditions, enhancing reliability in various practical application scenarios. The widespread application of adaptive algorithms has proven that they can dynamically adjust system parameters based on real-time changing environmental and signal conditions, thereby improving system performance. In vehicle-mounted radar signal recognition systems, this can help suppress interference in radar signals, such as multipath effects and signal interference from other radar devices. By adjusting filter parameters in real time, the system can more effectively filter out interfering signals, improving target recognition accuracy. Summary of the Invention

[0004] The purpose of this section is to outline some aspects of embodiments of the present invention and to briefly describe some preferred embodiments. Simplifications or omissions may be made in this section, as well as in the abstract and title of this application, to avoid obscuring the purpose of these documents; however, such simplifications or omissions should not be construed as limiting the scope of the invention.

[0005] In view of the aforementioned existing problems, this invention is proposed. Therefore, this invention provides a real-time adaptive multimodal hybrid feature vehicle radar signal recognition method to address the problems of how to improve vehicle safety, prevent collisions and accidents, and enhance emergency response capabilities in special circumstances.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] In a first aspect, the present invention provides a real-time adaptive multimodal hybrid feature vehicle radar signal recognition method, comprising:

[0008] Calibrate among vehicle sensors to determine the selection and deployment of vehicle sensors. With the goal of optimizing vehicle energy efficiency, establish an energy efficiency maximization model and add real-time constraints to establish a target model.

[0009] Vehicle radar and sensor data are acquired based on the target model, parameters are adjusted, and data preprocessing is performed.

[0010] Based on preprocessed vehicle data, target features are modeled and trained using a Gaussian mixture model. The state vector of the Kalman filter is initialized for state prediction. The target recognition results of the Gaussian mixture model are fused with the target tracking results of the Kalman filter to obtain the target recognition and tracking results.

[0011] As a preferred embodiment of the real-time adaptive multimodal hybrid feature vehicle radar signal recognition method described in this invention, the establishment of an energy efficiency maximization model includes:

[0012] To determine the optimal vehicle energy efficiency through the selection and deployment of vehicle sensors, an energy efficiency maximization model is established, expressed as:

[0013]

[0014] V min ≤V(t)≤V MAX

[0015] T sensor,i ≤t≤T sensor,i +Δt sensor,i

[0016] E sensor,i ≤E max,i

[0017] E total,i =E drive,i +E sensors,i

[0018] E total i, ≤E supply,i

[0019] The objective function is set to minimize the combined cost of total energy consumption and driving time, where α and β represent coefficients set according to actual needs to balance energy consumption and driving time. total,i E represents the total energy consumed during driving time. sensor,i T represents the total energy consumption of all sensors within a given time period, where t represents the total driving time of the vehicle. sensor,i Δt represents the time when the sensor starts working. sensor,i E represents the time the sensor operates. max,iE represents the upper limit of energy consumed during the sensor's operating time. drive,i E represents the total energy consumed by a vehicle during its driving time. supply,i It represents the total energy available to the vehicle over a period of time, provided by the vehicle's energy system.

[0020] As a preferred embodiment of the real-time adaptive multimodal hybrid feature vehicle radar signal recognition method of the present invention, the target model is represented as:

[0021]

[0022] R j =T sensor,j -T start,j +Δt sensor,j

[0023] Among them, R j T represents the response time of sensor j. sensor,j T represents the time when the sensor starts working. start,j Δt represents the time when the system starts recording data. sensor,j γ represents the sensor's operating time, and γ represents the real-time coefficient set according to actual needs.

[0024] As a preferred embodiment of the real-time adaptive multimodal hybrid feature vehicle radar signal recognition method of the present invention, the data preprocessing includes:

[0025] The different types of data collected by the sensor are preprocessed, Gaussian filter is used to remove noise, and zero mean unit variance is normalized, which is represented as y[n];

[0026] The t-distributed nearest neighbor embedding algorithm is used to reduce the dimensionality of high-dimensional sensor data, with the objective of minimizing the Kullback-Leibler divergence, as shown below:

[0027]

[0028]

[0029]

[0030] Given a high-dimensional dataset X, where each sample is represented as x i And define a similarity matrix P to represent the similarity between samples in a high-dimensional space, q ij y represents the similarity distribution in a low-dimensional space. i y j p represents different samples in a low-dimensional space. ijLet P(P||Q) represent the similarity distribution in a high-dimensional space, and let KL(P||Q) represent the Kullback-Leibler divergence.

[0031] As a preferred embodiment of the real-time adaptive multimodal hybrid feature vehicle radar signal recognition method described in this invention, the filter parameters are dynamically adjusted according to the real-time properties of the input data using an adaptive filtering algorithm. The filter weights are continuously adjusted using the LMS algorithm to adapt to changes in the input signal and minimize the mean square value of the estimation error. At each time step, the filter weights are updated based on the current input and error signals, as expressed below:

[0032] ω(n+1)=ω(n)+μe(n)x(n)

[0033] Where ω(n) represents the value of the weight vector at time, μ represents the learning rate, e(n) represents the error signal at the current time, and x(n) is the input signal at the current time;

[0034] Based on the multimodal features collected by sensors, a Bayesian feature fusion algorithm is used to perform multimodal feature fusion after preprocessing, which is represented as:

[0035] x→ω j ,

[0036] ifF(ω j ) = maxP(w k |x)

[0037]

[0038] Z→w j

[0039]

[0040] P(ω k |x k )=P(ω k (1+δ) ki )

[0041] Where ω represents the classification label, x represents the classifier output, and F(ω) j P(ω) represents the function that selects the class label with the highest posterior probability, M represents the number of input features, and P(ω) represents the number of input features. k ) represents category ω k The posterior probability, P(w) k |x) represents the category ω given the input x. k The posterior probability, P(ω) k |x k ) indicates category ω k Below, we observe the input x.k The probability, δ ki This represents a correction factor that adjusts the posterior probability when the prior and posterior probabilities are equal.

[0042] As a preferred embodiment of the real-time adaptive multimodal hybrid feature vehicle radar signal recognition method of the present invention, the target recognition and tracking results include:

[0043] Determine the number of components K, divide the samples using the k-means algorithm, and calculate the mean and covariance of the samples corresponding to each component as μ. K The initial values ​​of ∑K and α K Take 1 / K, and the weights of each component are equal;

[0044] Based on the current α K μ K And ∑K calculates the posterior probability γ(Z) nk ), represented as:

[0045]

[0046] The features of the target are modeled using a Gaussian mixture model, which can be represented as follows for the multidimensional case:

[0047]

[0048] According to the posterior probability γ(Z) nk Recalculate α K ,μ K And ∑K, expressed as:

[0049]

[0050]

[0051]

[0052]

[0053] Where, μ K This represents the mean of the corresponding sample.

[0054] Calculate the maximum likelihood function, determine if the maximum likelihood function converges, and if it converges, output the current α. K μ K And ∑K, otherwise return to recalculate based on the current α. K μ K And ∑K calculates the posterior probability γ(Z) nk ).

[0055] As a preferred embodiment of the real-time adaptive multimodal hybrid feature vehicle radar signal recognition method of the present invention, wherein: the state vector of the Kalman filter is initialized based on the result of target recognition, including the position and velocity of the target;

[0056] State prediction is performed using a Kalman filter to estimate the target's state at the next time step. For a one-dimensional state, it can be represented as:

[0057]

[0058] Based on the target recognition results of the Gaussian mixture model or actual measurements, the Kalman filter is updated to correct the state estimate, expressed as:

[0059]

[0060] P k|k =(IK k H k )P k|k-1

[0061] in, H represents a state update (estimation). k1 Indicates the Kalman gain, z k H represents the measurement vector. k2 Let P represent the observation matrix. k|k-1 Represents the state covariance matrix;

[0062] Iterative state prediction and measurement updates are performed in time series data to achieve continuous tracking of the target.

[0063] Secondly, this invention provides a real-time adaptive multimodal hybrid feature vehicle radar signal recognition system, comprising:

[0064] The first construction module is used to calibrate between vehicle sensors, determine the selection and deployment of vehicle sensors, establish an energy efficiency maximization model with the goal of optimizing vehicle energy efficiency, and add real-time constraints to establish a target model.

[0065] The processing module is used to acquire vehicle radar and sensor data based on the target model, adjust parameters according to actual application requirements, and perform data preprocessing.

[0066] The output module is used to model target features and train the model based on preprocessed vehicle data using a Gaussian mixture model, initialize the state vector of the Kalman filter for state prediction, and fuse the target recognition results of the Gaussian mixture model with the target tracking results of the Kalman filter to obtain the target recognition and tracking results.

[0067] Thirdly, the present invention provides an electronic device, comprising:

[0068] Memory and processor;

[0069] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, they implement the steps of the real-time adaptive multimodal hybrid feature vehicle radar signal recognition method.

[0070] Fourthly, the present invention provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the real-time adaptive multimodal hybrid feature vehicle radar signal recognition method.

[0071] Compared with existing technologies, the beneficial effects of this invention are as follows: This invention aims to better adapt to complex vehicle environments. By introducing a real-time adaptive algorithm, it can automatically adjust its adaptability to different environments and interferences, providing more accurate and reliable target recognition and perception capabilities. This includes adjusting the feature extraction algorithm and the target recognition algorithm, which can optimize system performance. The radar signal characteristics of different targets may vary greatly. Introducing multimodal features can better adapt to different vehicle environments. The real-time adaptive algorithm can adjust the parameters of the recognition algorithm according to the characteristics of the current target to better adapt to various targets. Attached Figure Description

[0072] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0073] Figure 1 This is a schematic diagram of the overall process of the real-time adaptive multimodal hybrid feature vehicle radar signal recognition method according to an embodiment of the present invention;

[0074] Figure 2 This is a schematic diagram of a model of the real-time adaptive multimodal hybrid feature vehicle radar signal recognition method according to an embodiment of the present invention; Figure 3 This is a schematic diagram illustrating target recognition and tracking in a real-time adaptive multimodal hybrid feature vehicle radar signal recognition method according to an embodiment of the present invention. Detailed Implementation

[0075] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0076] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0077] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0078] This invention is described in detail with reference to the schematic diagrams. When detailing the embodiments of this invention, for ease of explanation, the cross-sectional views illustrating the device structure may be partially enlarged, not adhering to the usual scale. Furthermore, the schematic diagrams are merely examples and should not be construed as limiting the scope of protection of this invention. In actual fabrication, the three-dimensional spatial dimensions of length, width, and depth should be included.

[0079] Furthermore, in the description of this invention, it should be noted that the terms "upper," "lower," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These terms are used solely for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. In addition, the terms "first," "second," or "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0080] Unless otherwise explicitly specified and limited, the terms "installation," "connection," and "joining" in this invention should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; similarly, they can refer to mechanical connections, electrical connections, or direct connections, or indirect connections through an intermediate medium, or internal connections between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0081] Example 1

[0082] Reference Figures 1-2As an embodiment of the present invention, a real-time adaptive multimodal hybrid feature vehicle radar signal recognition method is provided, comprising:

[0083] S1. Calibrate among vehicle sensors to determine the selection and deployment of vehicle sensors. With the goal of optimizing vehicle energy efficiency, establish an energy efficiency maximization model and add real-time constraints to establish the target model.

[0084] Specifically, the model in this embodiment is as follows: Figure 1 As shown, sensor calibration includes,

[0085] The sensors include an onboard radar sensor, K onboard cameras, and K onboard infrared sensors. Calibration is performed between these sensors to ensure effective data fusion from different sensors within the same coordinate system, represented as:

[0086]

[0087] Where R represents the rotation matrix from the radar to the camera, and T represents the translation vector for each point i in the radar point cloud. Represents the coordinates in the radar coordinate system. This represents the coordinates in the camera coordinate system.

[0088] It should be noted that the vehicle equipped with sensors and receivers moves on random road segments, with L representing the current position. The radar sensor is used to collect radar signal data of the vehicle's surrounding environment, the camera is used to collect image data and provide visual information, and the infrared sensor further enhances the system's perception capabilities.

[0089] Preferably, appropriate parameters are set for each sensor, such as the scanning angle of the radar, the exposure time of the camera, and the resolution of the lidar. The parameters are adjusted according to the actual application requirements to optimize performance. Different sensors collect different information on the test road, such as image data and visual information, to complete the collection of vehicle environment information.

[0090] Vehicles equipped with vehicle-mounted radar systems move along designated road sections in units of vehicle length. Furthermore, the real-time requirements in the vehicle environment are higher than in other scenarios. However, different sensors share channel transmission resources, and the differences in signal transmission can lead to real-time issues.

[0091] Radar sensors typically measure distance (l) and velocity (v) against multiple targets in free space according to a normal distribution. Since the angles of the targets may be uniformly distributed within the measurement range, angle measurements usually satisfy a uniform distribution. The brightness information (σ) received by camera sensors can usually follow a normal distribution, but other distributions may be used in special cases. Color information (c) depends on the scene and may use a multidimensional distribution. The distance (d') and intensity (s) measurements of laser sensors also typically follow a normal distribution.

[0092] Furthermore, the selection and configuration of sensors need to comprehensively consider the optimization of vehicle energy efficiency, and the establishment of an energy efficiency maximization model includes,

[0093] To determine the optimal vehicle energy efficiency through the selection and deployment of vehicle sensors, an energy efficiency maximization model is established, expressed as:

[0094]

[0095] V min ≤V(t)≤V MAX

[0096] T sensor,i ≤t≤T sensor,i +Δt sensor,i

[0097] E sensor,i ≤E max,i

[0098] E total,i =E drive,i +E sensors,i

[0099] E total i, ≤E supply,i

[0100] The objective function is set to minimize the combined cost of total energy consumption and driving time, where α and β represent coefficients set according to actual needs to balance energy consumption and driving time. total,i E represents the total energy consumed during driving time. sensor,i T represents the total energy consumption of all sensors within a given time period, where t represents the total driving time of the vehicle. sensor,i Δt represents the time when the sensor starts working. sensor,i F represents the time the sensor operates. max,i E represents the upper limit of energy consumed during the sensor's operating time. drive,i E represents the total energy consumed by a vehicle during its driving time. supply,iIt represents the total energy available to the vehicle over a period of time, provided by the vehicle's energy devices. Based on the set optimization problem, it can solve the problem of equipping and setting up vehicle sensors with the goal of optimizing vehicle energy efficiency.

[0101] Furthermore, the use of sensors introduces real-time challenges, necessitating the discovery of optimal sensor operating strategies to balance energy efficiency, driving time, and real-time performance across different scenarios. This is addressed by minimizing the objective function. Therefore, the optimization problem is extended by incorporating real-time constraints, resulting in the following objective model:

[0102]

[0103] R j =T sensor,j -T start,j +Δt sensor,j

[0104] Among them, R j T represents the response time of sensor j. sensor,j T represents the time when the sensor starts working. start,j Δt represents the time when the system starts recording data. sensor,j γ represents the sensor's operating time, and γ represents the real-time coefficient set according to actual needs.

[0105] S2 acquires vehicle radar and sensor data based on the target model, adjusts parameters, and performs data preprocessing.

[0106] Specifically, data preprocessing includes,

[0107] The different types of data collected by the sensor are preprocessed, using a Gaussian filter for noise reduction and zero-mean, unit-variance normalization, as shown below:

[0108]

[0109]

[0110] Where μ represents the mean of the data, σ represents the standard deviation of the data, and x n Let x[i] represent normalized data, and x[i] represent a sample of the signal containing noise taken around time n.

[0111] The t-distributed nearest neighbor embedding algorithm is used to reduce the dimensionality of high-dimensional sensor data, with the objective of minimizing the Kullback-Leibler divergence, as shown below:

[0112]

[0113]

[0114]

[0115] Given a high-dimensional dataset X, where each sample is represented as x i And define a similarity matrix P to represent the similarity between samples in a high-dimensional space, q ij y represents the similarity distribution in a low-dimensional space. i y j p represents different samples in a low-dimensional space. ij KL(P||Q) represents the similarity distribution in a high-dimensional space. KL(P||Q) represents the Kullback-Leibler divergence. By using optimization methods such as gradient descent, the Kullback-Leibler divergence can be minimized, thereby preserving the high-dimensional similarity structure in a low-dimensional space.

[0116] Furthermore, an adaptive filtering algorithm is introduced. This algorithm dynamically adjusts the filter parameters based on the real-time properties of the input data. The LMS algorithm continuously adjusts the filter weights to adapt to changes in the input signal and minimize the mean square value of the estimation error. At each time step, the filter weights are updated based on the current input and error signals, as expressed below:

[0117] ω(n+1)=ω(n)+μe(n)x(n)

[0118] Where ω(n) represents the value of the weight vector at time, μ represents the learning rate, which is used to control the step size of weight adjustment, e(n) represents the error signal at the current time, and x(n) is the input signal at the current time.

[0119] Based on the multimodal features collected by sensors, a Bayesian feature fusion algorithm is used to perform multimodal feature fusion after preprocessing, which is represented as:

[0120] x→ω j ,

[0121] ifF(ω j ) = maxP(w k |x)

[0122]

[0123] Z→w j

[0124]

[0125] P(ω k |x k )=P(ω k (1+δ) ki )

[0126] Where ω represents the classification label, x represents the classifier output, and F(ω) j P(ω) represents the function that selects the class label with the highest posterior probability, M represents the number of input features, and P(ω) represents the number of input features. k ) represents category ω k The posterior probability, P(w) k |x) represents the category ω given the input x. k The posterior probability, P(ω) k |x k ) indicates category ω k Below, we observe the input x. k The probability, δ ki This represents a correction factor that adjusts the posterior probability when the prior and posterior probabilities are equal.

[0127] S3, based on preprocessed vehicle data, models the target features using a Gaussian mixture model and trains the model, initializes the state vector of the Kalman filter for state prediction, and fuses the target recognition results of the Gaussian mixture model with the target tracking results of the Kalman filter to obtain the target recognition and tracking results.

[0128] It should be noted that, as Figure 2 As shown, this paper demonstrates a target recognition and tracking method for a multimodal hybrid feature vehicle radar signal recognition system that incorporates a real-time adaptive algorithm. For preprocessed radar and sensor data, a combination of a Gaussian mixture model (Gaussian model) and a Kalman filter is used for target recognition and tracking. The Gaussian mixture model is used to model the target features, with each Gaussian component corresponding to a target. Its parameters include the mean μ. K Covariance matrix ∑K and weights α K .

[0129] Each Gaussian component corresponds to a target, with parameters including mean, covariance matrix, and weights. The Gaussian mixture model is trained, and the number, position, and motion state of the targets are estimated based on the model parameters. Using the target recognition results of the Gaussian mixture model, the state vector of the Kalman filter is initialized, including position and velocity. The Kalman filter is used to predict the state and estimate the target's state at the next time step. Using the target recognition results of the Gaussian mixture model or actual measurements, the Kalman filter is updated to correct the state estimation. The target recognition results of the Gaussian mixture model are fused with the target tracking results of the Kalman filter to obtain more accurate and robust target recognition and tracking results.

[0130] Specifically, the number of components K is determined, the samples are divided using the k-means algorithm, and the mean and covariance of the samples corresponding to each component are calculated as μ. K The initial values ​​of ∑K and αK Take 1 / K, and the weights of each component are equal;

[0131] Based on the current α K μ K And ∑K calculates the posterior probability γ(Z) nk ), represented as:

[0132]

[0133] The features of the target are modeled using a Gaussian mixture model, which can be represented as follows for the multidimensional case:

[0134]

[0135] Based on the posterior probability γ(Z) nK Recalculate α K ,μ K And ∑K, expressed as:

[0136]

[0137]

[0138]

[0139]

[0140] Where, μ k This represents the mean of the corresponding sample.

[0141] Calculate the maximum likelihood function, determine if the maximum likelihood function converges, and if it converges, output the current α. K μ K And ∑K, otherwise return to recalculate based on the current α. K μ K And ∑K calculates the posterior probability γ(Z) nk );

[0142] The state vector of the Kalman filter is initialized based on the target recognition results, including the target's position and velocity.

[0143] State prediction is performed using a Kalman filter to estimate the target's state at the next time step. For a one-dimensional state, it can be represented as:

[0144]

[0145] Based on the target recognition results of the Gaussian mixture model or actual measurements, the Kalman filter is updated to correct the state estimate, expressed as:

[0146]

[0147] P k|k =(IH k H k )P k|k-1

[0148] in, H represents a state update (estimation). k1 Indicates the Kalman gain, z k H represents the measurement vector. k2 Let P represent the observation matrix. k|k-1 Represents the state covariance matrix;

[0149] Iterative state prediction and measurement updates are performed in time series data to achieve continuous tracking of the target.

[0150] Furthermore, an adaptive sliding window algorithm is introduced. This algorithm uses an initial sliding window size to detect the target. It monitors the target's motion and environmental conditions in real time. When the target's scale changes or its speed increases, the algorithm adaptively adjusts the sliding window size to ensure the target remains within the window, thus maintaining accurate target tracking. This is represented as:

[0151] W t+1 =w t +αx*f

[0152] H t+1 =h t +βx*f

[0153] Among them, W t+1 and H t+1 The width and height are at time t+1, and α and β are parameters used to control the adjustment speed of the sliding window.

[0154] It should be noted that by introducing real-time adaptive algorithms, including adjusting the feature extraction algorithm and the target recognition algorithm, the system performance can be optimized. Furthermore, the radar signal characteristics of different targets may vary greatly, and the characteristics of the targets may change over time. The real-time adaptive algorithm can adjust the parameters of the recognition algorithm according to the characteristics of the current target in order to better adapt to various targets.

[0155] This invention aims to better adapt to complex in-vehicle environments. By introducing a real-time adaptive algorithm, it can automatically adjust its adaptability to different environments and interferences, providing more accurate and reliable target recognition and perception capabilities. This includes adjusting the feature extraction algorithm and the target recognition algorithm, which can optimize system performance. The radar signal characteristics of different targets may vary greatly. Introducing multimodal features can better adapt to different in-vehicle environments. The real-time adaptive algorithm can adjust the parameters of the recognition algorithm according to the characteristics of the current target to better adapt to various targets.

[0156] The above is an illustrative scheme of a real-time adaptive multimodal hybrid feature vehicle radar signal recognition method according to this embodiment. It should be noted that the technical solution of this real-time adaptive multimodal hybrid feature vehicle radar signal recognition system and the technical solution of the aforementioned real-time adaptive multimodal hybrid feature vehicle radar signal recognition method belong to the same concept. Details not described in detail in the technical solution of the real-time adaptive multimodal hybrid feature vehicle radar signal recognition system in this embodiment can be found in the description of the technical solution of the aforementioned real-time adaptive multimodal hybrid feature vehicle radar signal recognition method.

[0157] The real-time adaptive multimodal hybrid feature vehicle radar signal recognition system in this embodiment includes:

[0158] The first construction module is used to calibrate between vehicle sensors, determine the selection and deployment of vehicle sensors, establish an energy efficiency maximization model with the goal of optimizing vehicle energy efficiency, and add real-time constraints to establish a target model.

[0159] The processing module is used to acquire vehicle radar and sensor data based on the target model, adjust parameters according to actual application requirements, and perform data preprocessing.

[0160] The output module is used to model target features and train the model based on preprocessed vehicle data using a Gaussian mixture model, initialize the state vector of the Kalman filter for state prediction, and fuse the target recognition results of the Gaussian mixture model with the target tracking results of the Kalman filter to obtain the target recognition and tracking results.

[0161] It also includes corresponding communication modules and a high-performance centralized computing platform.

[0162] This embodiment also provides an electronic device suitable for real-time adaptive multimodal hybrid feature vehicle radar signal recognition, including:

[0163] The system includes a memory and a processor. The memory stores computer-executable instructions, and the processor executes these instructions to implement the real-time adaptive multimodal hybrid feature vehicle radar signal recognition method proposed in the above embodiments.

[0164] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, it implements the real-time adaptive multimodal hybrid feature vehicle radar signal recognition method proposed in the above embodiments.

[0165] The storage medium proposed in this embodiment belongs to the same inventive concept as the method for real-time adaptive multimodal hybrid feature vehicle radar signal recognition proposed in the above embodiments. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0166] Based on the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0167] Example 2

[0168] Referring to Table 1, an embodiment of the present invention provides a real-time adaptive multimodal hybrid feature vehicle radar signal recognition method. To verify its beneficial effects, a comparison of two schemes is provided.

[0169] This embodiment aims to better adapt to complex in-vehicle environments through the method of the present invention. By introducing a real-time adaptive algorithm, the system can automatically adjust its adaptability to different environments and interferences, providing more accurate and reliable target recognition and perception capabilities.

[0170] Test dataset: The open-source DAIR-V2X dataset for multimodal and multi-view vehicle-road cooperative systems. Running environment: Matplotlib 3.3.1, Numpy 1.19.2, Pandas 1.1.3, Pillow 9.1.0, SciPy 1.5.2, TensorboardX 2.5, Torch 1.5.1.

[0171] This embodiment introduces a real-time adaptive algorithm to optimize system performance. Different targets may have significantly different radar signal characteristics; introducing multimodal features allows for better adaptation to different vehicle environments. The real-time adaptive algorithm can adjust the parameters of the recognition algorithm based on the characteristics of the current target to better suit various targets. For performance comparison, it is compared with CDMC, RODNet-HG, and RODNet-CDC.

[0172] Table 1 Performance comparison of different methods

[0173] AP 30.55％ 71.84％ 71.46％ 81.23％ AR 54.79％ 76.03％ 78.15％ 84.25％

[0174] As shown in Table 1, the method of this invention achieved performance of 81.23% and 84.25% on AP and AR respectively when tested on the DAIR-V2X dataset. Both of these performances are higher than the control methods CDMC, RODNet-HG, and RODNet-CDC. The method can automatically adjust its adaptability to different environments and interferences, providing more accurate and reliable target recognition and perception capabilities.

[0175] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A real-time adaptive multimodal hybrid feature vehicle radar signal recognition method, characterized in that, include, Calibrate among vehicle sensors to determine the selection and deployment of vehicle sensors. With the goal of optimizing vehicle energy efficiency, establish an energy efficiency maximization model and add real-time constraints to establish a target model. Vehicle radar and sensor data are acquired based on the target model, parameters are adjusted, and data preprocessing is performed. Based on preprocessed vehicle data, target features are modeled and trained using a Gaussian mixture model. The state vector of the Kalman filter is initialized for state prediction. The target recognition results of the Gaussian mixture model are fused with the target tracking results of the Kalman filter. The measurement of the Kalman filter is then updated to correct the state estimation. The state prediction and measurement update are iteratively performed in the time series to achieve continuous target tracking. The state vector of the Kalman filter is initialized based on the target recognition results, including the target's position and velocity. State prediction is performed using a Kalman filter to estimate the target's state at the next time step. For a one-dimensional state, it can be represented as: Based on the target recognition results of the Gaussian mixture model or actual measurements, the Kalman filter is updated to correct the state estimate, expressed as: in, Indicates a status update. Indicates Kalman gain, Represents the measurement vector. Represents the observation matrix. Represents the state covariance matrix; Iterative state prediction and measurement updates are performed in the time series to achieve continuous tracking of the target; An adaptive sliding window algorithm is introduced, which uses an initial sliding window size to detect targets. The algorithm monitors the target's motion state and environmental conditions in real time. When the target's scale changes or the target's speed is faster than the preset value, the size of the sliding window will be adjusted adaptively to ensure that the target is still within the sliding window, thereby maintaining accurate tracking of the target and obtaining target recognition and tracking results.

2. The real-time adaptive multimodal hybrid feature vehicle radar signal recognition method as described in claim 1, characterized in that, Establishing an energy efficiency maximization model involves determining the optimal vehicle sensor selection and deployment to maximize vehicle energy efficiency, and then establishing the energy efficiency maximization model, expressed as: ≤ = ≤ The objective function is set to minimize the combined cost of total energy consumption and driving time. and This represents a coefficient set based on actual needs to balance energy consumption and driving time. This represents the total energy consumed during driving time. This represents the total energy consumption of all sensors within a given time period. This indicates the total travel time of the vehicle. Indicates the time when the sensor starts working. Indicates the time the sensor operates. This indicates the upper limit of energy consumed during the sensor's operating time. This represents the total energy consumed by a vehicle during its driving time. It represents the total energy available to the vehicle over a period of time, provided by the vehicle's energy system.

3. The real-time adaptive multimodal hybrid feature vehicle radar signal recognition method as described in claim 2, characterized in that, The target model is represented as follows: in, Indicates sensor j Response time Indicates the time when the sensor starts working. Indicates the time when the system started recording data. Indicates the sensor's operating time. This indicates the real-time coefficient set according to actual needs.

4. The real-time adaptive multimodal hybrid feature vehicle radar signal recognition method as described in claim 1 or 3, characterized in that, The data preprocessing includes preprocessing different types of data collected by the sensors, using a Gaussian filter for noise reduction, and performing zero-mean, unit-variance normalization, represented as... ; use - The distributed nearest neighbor embedding algorithm reduces the dimensionality of high-dimensional sensor data to minimize The divergence is the objective, expressed as: Given a high-dimensional dataset Each sample is represented as And set a similarity matrix This represents the similarity between samples in a high-dimensional space. Represents the similarity distribution in low-dimensional space. Represents different samples in a low-dimensional space. Represents the similarity distribution in high-dimensional space. This represents the Kullback-Leibler divergence.

5. The real-time adaptive multimodal hybrid feature vehicle radar signal recognition method as described in claim 4, characterized in that, The adaptive filtering algorithm dynamically adjusts the filter parameters based on the real-time properties of the input data. The LMS algorithm continuously adjusts the filter weights to adapt to changes in the input signal and minimize the mean square value of the estimation error. At each time step, the filter weights are updated based on the current input and error signals, as expressed below: in, This represents the value of the weight vector over time. Indicates the learning rate. This represents the error signal at the current moment. It is the input signal at the current moment; Based on the multimodal features collected by sensors, a Bayesian feature fusion algorithm is used to perform multimodal feature fusion after preprocessing, which is represented as: in, Indicates category labels, This represents the output of the classifier. This represents a function that selects the class label with the highest posterior probability. Indicates the number of input features. Indicates category The posterior probability, Indicates the given input In the case of category The posterior probability, Indicates in category Below, we observe the input. The probability, This represents a correction factor that adjusts the posterior probability when the prior and posterior probabilities are equal.

6. The real-time adaptive multimodal hybrid feature vehicle radar signal recognition method as described in claim 5, characterized in that, Obtaining target recognition and tracking results includes determining the number of components K, dividing the samples using the k-means algorithm, and calculating the mean and covariance of the samples corresponding to each component. and initial value, Take 1 / K, and the weights of each component are equal; According to the current 、 as well as Calculate the posterior probability , is represented as: The features of the target are modeled using a Gaussian mixture model, which can be represented as follows for the multidimensional case: According to the posterior probability Recalculate , and , is represented as: in, ; Calculate the maximum likelihood function, determine if the maximum likelihood function converges, and if it converges, output the current value. , as well as Otherwise, return and recalculate based on the current... , as well as Calculate the posterior probability .

7. A real-time adaptive multimodal hybrid feature vehicle radar signal recognition system, applied to the method described in any one of claims 1-6, characterized in that, include: The first construction module is used to calibrate between vehicle sensors, determine the selection and deployment of vehicle sensors, establish an energy efficiency maximization model with the goal of optimizing vehicle energy efficiency, and add real-time constraints to establish a target model. The processing module is used to acquire vehicle radar and sensor data based on the target model, adjust parameters according to actual application requirements, and perform data preprocessing. The output module is used to model target features and train the model based on preprocessed vehicle data using a Gaussian mixture model, initialize the state vector of the Kalman filter for state prediction, and fuse the target recognition results of the Gaussian mixture model with the target tracking results of the Kalman filter to obtain the target recognition and tracking results.

8. An electronic device, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, they implement the steps of the real-time adaptive multimodal hybrid feature vehicle radar signal recognition method according to any one of claims 1 to 6.

9. A computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the real-time adaptive multimodal hybrid feature vehicle radar signal recognition method according to any one of claims 1 to 6.

Images