A microwave radar and graphic vision data fusion method based on machine learning

Abstract

The application discloses a microwave radar and graphic visual data fusion method based on machine learning, which comprises the following steps: monitoring a foundation pit support structure by means of a microwave radar and computer vision simultaneously, acquiring time series data, wherein the time series data comprises a beat frequency signal measured by the microwave radar and displacement data measured by the computer vision; fusing the beat frequency signal and the displacement data; performing time series analysis and feature extraction on the fused data; constructing a model based on a machine learning method, and predicting and analyzing the data; and the application can realize high-precision, real-time and non-contact monitoring, can greatly improve the monitoring precision and reduce errors by fusing and processing the microwave radar and computer visual data, can provide timely and accurate data support for engineering management by real-time monitoring and timely feedback of engineering deformation, and can protect the integrity of the engineering structure by using a non-contact monitoring method without destructive detection of the engineering.

Description

Technical Field

[0001] This invention belongs to the field of foundation pit support monitoring technology, and in particular relates to a method for fusing microwave radar and graphic visual data based on machine learning. Background Technology

[0002] In foundation pit support, monitoring the displacement and deformation of structures is crucial for ensuring construction safety and project quality. Currently, microwave radar and computer vision technologies are widely used in foundation pit support monitoring. Microwave radar can measure deformation information of structures buried deep underground, but its monitoring accuracy is affected by the underground medium and engineering interference; while computer vision can achieve high-precision measurement of surface displacement of structures, but its monitoring range is limited by factors such as field of view and lighting. Furthermore, single monitoring methods suffer from insufficient accuracy and complex data processing, thus necessitating the integration of the two technologies.

[0003] Therefore, a method for fusing microwave radar and graphic vision data based on machine learning was developed to solve the above problems. Summary of the Invention

[0004] The purpose of this invention is to design a method for fusing microwave radar and graphic vision data based on machine learning in order to solve the above problems.

[0005] The present invention achieves the above objectives through the following technical solutions:

[0006] A machine learning-based method for fusing microwave radar and graphic vision data includes the following steps:

[0007] S1. Simultaneously monitor the foundation pit support structure using microwave radar and computer vision to acquire time-series data, which includes the difference frequency signal measured by microwave radar and the displacement data measured by computer vision.

[0008] S2. Perform data fusion of the difference frequency signal and displacement data;

[0009] S3. Perform time series analysis and feature extraction on the fused data;

[0010] S4. Build models based on machine learning methods to predict and analyze data.

[0011] The beneficial effects of this invention are as follows:

[0012] High-precision monitoring: By fusing microwave radar and computer vision data, the accuracy of monitoring can be greatly improved and errors reduced.

[0013] Real-time monitoring: Enables real-time monitoring and timely feedback on engineering deformation, providing timely and accurate data support for engineering management.

[0014] Automated processing: By using computer vision technology, the processing and analysis of large amounts of data can be automated, reducing labor costs.

[0015] Non-contact monitoring: Both microwave radar and computer vision technology are non-contact monitoring methods that do not require destructive testing of the project and can protect the structural integrity of the project. Attached Figure Description

[0016] Figure 1 This is a flowchart of the method of the present invention;

[0017] Figure 2 This is a test signal diagram for microwave radar;

[0018] Figure 3 This is a graph of microwave radar test data;

[0019] Figure 4 This is a graph of machine vision test data. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0021] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0022] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0023] In the description of this invention, it should be understood that the terms "upper," "lower," "inner," "outer," "left," "right," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product of this invention is in use, or the orientation or positional relationship commonly understood by those skilled in the art. They are only used to facilitate the description of this invention and to simplify the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0024] Furthermore, the terms "first," "second," etc., are used only to distinguish descriptions and should not be interpreted as indicating or implying relative importance.

[0025] In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, terms such as "set" and "connection" should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be a connection within two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0026] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0027] like Figure 1 As shown, this invention provides a method for fusing microwave radar and graphic vision data based on machine learning, including the following steps:

[0028] S1. Microwave radar and computer vision are used simultaneously to monitor the foundation pit support structure and acquire time-series data. The time-series data includes the difference frequency signal measured by microwave radar and the displacement data measured by computer vision; such as Figure 2 As shown, the microwave radar continuously transmits signals, and the received echo signals are sampled and preprocessed to obtain the original microwave radar signals.

[0029] In this step, microwave radar and computer vision technologies simultaneously monitor the foundation pit support structure to acquire time-series data. Microwave radar technology enables high-precision monitoring of structural deformation, while computer vision technology acquires displacement information. The collaborative data acquisition efforts of both technologies yield more comprehensive and accurate information on support deformation, providing fundamental data for subsequent data fusion and analysis.

[0030] S2. Perform data fusion of the difference frequency signal and displacement data;

[0031] In this step, the difference frequency signal obtained from microwave radar measurement and the displacement data obtained from computer vision measurement are fused to obtain more comprehensive and accurate information on support deformation. The microwave radar difference frequency signal reflects the deformation of the structure, while the computer vision displacement data provides relative displacement information. The fusion of these two data points provides a more comprehensive picture of the deformation of the support structure.

[0032] S3. Perform time series analysis and feature extraction on the fused data;

[0033] In this step, time series analysis and feature extraction are performed on the fused data to reveal the deformation patterns and characteristics of the support structure. Time series analysis can reveal the periodicity and trends of the data, while feature extraction can transform complex support deformation information into quantifiable feature vectors, providing feature data for subsequent machine learning models.

[0034] S4. Build models based on machine learning methods to predict and analyze data.

[0035] In this step, a model is built based on machine learning methods to predict and analyze the data. By training on existing support deformation data, a support deformation prediction model is established, which can predict the future deformation of the support structure. Simultaneously, analyzing existing support deformation data can identify abnormal deformations in the support structure and allow for timely intervention, ensuring the safety of the project.

[0036] In some embodiments, in step S1, the microwave radar continuously transmits signals, samples and processes the echo signals to obtain a difference frequency signal Sd(t) representing the target motion information, such as... Figure 3 As shown; computer vision acquires and analyzes images of the inner side of the foundation pit at times t1 and t2 to obtain displacement data Da(t1) and Da(t2); computer vision acquires and analyzes images of the outer side of the foundation pit at times t1 and t2 to obtain displacement data Db(t1) and Db(t2). Figure 4 As shown, computer vision processing can obtain the displacement monitoring results of the inner and outer walls of the foundation pit at different times, namely displacement data Da(t) and Db(t).

[0037] The differential frequency signal and displacement data fusion technology provided in this patent achieves data fusion from two monitoring methods by mapping microwave radar and computer vision data. Microwave radar can acquire underground deformation information in real time, while computer vision can acquire surface deformation information. By fusing the data from these two monitoring methods, more comprehensive and accurate deformation information can be obtained.

[0038] The fusion technology of difference frequency signals and displacement data is a signal processing technique that can map microwave radar and computer vision data to achieve data fusion from the two monitoring methods. The core of this technology lies in establishing the correspondence between difference frequency signals and displacement data. This patent employs the differential method, which calculates the difference in displacement data and then maps it to the difference frequency signal obtained from microwave radar measurements.

[0039] The differential method is a commonly used signal processing method that can convert time-domain signals into frequency-domain signals, thereby enabling signal processing and analysis. In this patent, the main function of the differential method is to calculate the displacement difference between the inside and outside of the foundation pit, thereby establishing the correspondence between the difference frequency signal and the displacement data.

[0040] Specifically, in step S2, data fusion includes:

[0041] S21. Calculate the displacement difference between the inner and outer sides of the foundation pit at the same time:

[0042] ΔDa(t)=Da(t2)-Da(t1);

[0043] ΔDb(t)=Db(t2)-Db(t1);

[0044] S22. Calculate the difference between the inner and outer displacements, and use it as the displacement characteristic after fusion:

[0045] ΔD(t)=ΔDa(t)-ΔDb(t)=[Da(t2)-Da(t1)]-[Db(t2)-Db(t1)];

[0046] S23. Establish the linear correspondence between Sd(t) and ΔD(t):

[0047] Sd(t) = k1 * ΔD(t) + k2;

[0048] Where k1 and k2 are the linear correlation coefficients to be determined;

[0049] A mathematical correspondence model between the microwave radar signal Sd(t) and the visual displacement difference ΔD(t) is established through parameter fitting.

[0050] It is important to note that the differential method is only a commonly used signal processing method. In practical applications, various interference factors, such as signal noise and signal drift, must be considered. Therefore, in specific implementations, the differential method needs to be appropriately adjusted and improved based on the actual situation to enhance the fusion effect of the difference frequency signal and displacement data.

[0051] In some embodiments, step S3 specifically includes:

[0052] S31. Based on wavelet transform, analyze the time-frequency characteristics of the fused signal:

[0053] The fused data sequence X(t) is decomposed into coefficient components at different scales by wavelet transform:

[0054]

[0055] Among them, c j，k—The k-th scaling coefficient of the j-th wavelet transform, d j，k —The k-th detail coefficient of the j-th wavelet transform. —The k-th scaling function of the j-th layer, ψ j，k —The k-th wavelet function of the j-th layer;

[0056] S32. Calculate the wavelet coefficients of each layer after multi-layer wavelet transform:

[0057] Iterate through the different scales j of the wavelet transform, and calculate the k-th scale coefficient c of the j-th level wavelet transform of each level. j，k and the k-th detail coefficient d of the j-th level wavelet transform j，k ;

[0058] S33. Extract singularity information from the signal using wavelet coefficients to obtain abrupt change features:

[0059] Analyze the k-th detail coefficient d of the j-th level wavelet transform j，k The sign of the signal changes, and when the sign changes, the position k is determined to be a singular point of the signal.

[0060] Sk={k|sign(d j，k )≠sign(d j，k-1 )};

[0061] Where Sk represents the set of singular points in the j-th layer;

[0062] The intensity of singularities is represented by the magnitude of the detail coefficients:

[0063] Ik=|d j，k |, when k∈Sk;

[0064] S34. Calculate the energy characteristics of wavelet coefficients at each level and analyze the signal energy distribution:

[0065] Calculate the energy of detail coefficients dj and k at each scale:

[0066] Ej=∑k(dj,k)2;

[0067] Its statistical properties, energy entropy, kurtosis, and variance are defined as follows:

[0068] Energy entropy: Hj = -∑(pj, i*logpj, i); Energy entropy is a commonly used energy characteristic in wavelet transform, which can describe the distribution of a signal. The larger the energy entropy, the more uniform the signal distribution, that is, the more even the energy distribution of the signal; the smaller the energy entropy, the more uneven the signal distribution, that is, the more concentrated the energy distribution of the signal.

[0069] Energy kurtosis: Kj = (1 / N*∑(pj, i-μ)⁴ / σ⁴) - 3; Energy kurtosis is an indicator of the sharpness of a signal, reflecting the degree of local variation. A larger energy kurtosis indicates more drastic changes in certain regions of the signal, resulting in sharper edges; a smaller energy kurtosis indicates smoother changes in the signal. Therefore, energy kurtosis can be used to describe the sharpness and texture characteristics of a signal.

[0070] Energy variance: Vj = 1 / N * ∑(pj, i-μ)²; Energy variance is an indicator that measures the distribution of signal energy, describing the degree of concentration of the signal. A larger energy variance indicates a more dispersed energy distribution, while a smaller energy variance indicates a more concentrated energy distribution. Therefore, energy variance can be used to describe the stability and overall distribution of a signal.

[0071] Where pj,i represents the energy percentage of subband i, and μ and σ represent the mean and standard deviation;

[0072] In addition to the local extremum detection method, this application also employs an energy feature extraction method based on wavelet transform. This method can calculate the energy value of each wavelet coefficient sub-band, and then extract the energy features of each sub-band.

[0073] S35. The final eigenvector is:

[0074] F=[S1, I1, ..., SN, IN, H1, ... HN, K1, ... KN, V1, ... VN];

[0075] S36. The above features comprehensively reflect the time-frequency distribution and statistical characteristics of the signal, and serve as model inputs for representing complex changes.

[0076] This application employs wavelet analysis to analyze and extract features from time series data. Wavelet analysis is a signal processing method based on wavelet transform, which can decompose a signal into different frequency components, thereby enabling signal analysis and processing.

[0077] Specifically, the fused difference-frequency signal and displacement data are decomposed using wavelet decomposition to obtain wavelet coefficients for different frequency components. Then, based on the energy distribution of the wavelet coefficients, effective feature information is extracted. This application employs a local extremum detection method based on wavelet transform, which can extract the location and amplitude information of local extrema from the wavelet coefficients, thereby achieving feature extraction of the time series.

[0078] Time series analysis is a data analysis method based on time-varying changes, which can be used to monitor and predict system changes. In this application, by fusing difference frequency signals and displacement data, comprehensive information on the deformation of the foundation pit support structure can be obtained. Next, wavelet analysis is used to analyze the time series and extract features to further improve the accuracy and reliability of the deformation data.

[0079] Wavelet analysis is a time-frequency analysis method that can decompose a signal into different frequency components and analyze each frequency component in different time periods.

[0080] In some embodiments, step S4 includes constructing a support vector regression model for structural deformation prediction, as detailed below:

[0081] (1) Establish the prediction function for the support vector regression model:

[0082]

[0083] Where wi and wi* are the parameters to be solved, i.e., the weights, b is the bias, K(x, xi) is the kernel function, and n is the number of support vectors;

[0084] (2) Prepare training data:

[0085] The fused feature vector obtained after data fusion in step S2 and time series analysis and feature extraction in step S3 is used as input xi, and the corresponding structural deformation is used as output yi, forming the training set data pair {xi, yi}.

[0086] (3) Selecting the Gaussian kernel function for nonlinear mapping:

[0087] (4) Establish the loss function and regularization term:

[0088] L(w,ξ)=1 / 2||w||2+C∑(ξi+ξi*);

[0089] Where w is the weight vector, ξi and ξi* are slack variables, and C is the regularization parameter; the regularization term is used to control the model complexity and avoid overfitting.

[0090] (5) Add constraints to limit the error between the predicted value and the actual value:

[0091] |yi-f(xi)|≤ε+ξi;

[0092] ε is the precision parameter, and ξi is the slack variable;

[0093] (6) Solve the constrained quadratic programming problem to determine the optimal parameters w and b;

[0094] Model training: Learn parameters w and b through iterative optimization; select appropriate γ and C; determine the optimal parameters that minimize the loss function;

[0095] (8) Validation: K-fold cross-validation was used; multiple training and validations were performed to obtain the optimal parameters with the minimum generalization error.

[0096] (9) The final support vector regression model is obtained.

[0097] This method predicts structural deformation based on fused features by constructing and training an SVR (Support Vector Regression) model. The selection of relevant parameters affects the prediction performance. This method fully utilizes the nonlinear fitting capability of SVR and improves the stability and generalization ability of the prediction through regularization and cross-validation.

[0098] In some embodiments, the specific steps for solving the support vector regression model using the quadratic programming method are as follows:

[0099] (1) Construct the loss function and regularization term as the optimization objective;

[0100] (2) Add prediction error constraints;

[0101] (3) Transform into a constrained quadratic programming problem:

[0102] min 1 / 2||w||2+C∑(ξi+ξi*);

[0103] st|yi-f(xi)|≤ε+ξi;

[0104] (4) Iteratively optimize and solve the quadratic objective function;

[0105] (5) Update the parameters in each iteration and gradually decrease the value of the objective function;

[0106] (6) Stop iteration when the convergence condition is met, set to the number of iterations or the change of the objective function reaching a threshold;

[0107] (7) Finally, the objective function value with the global minimum is obtained, and the optimal parameters w and b are solved.

[0108] (8) Obtain the trained support vector regression prediction model.

[0109] Quadratic programming can effectively solve for the parameters of SVR. By iteratively optimizing the objective function and considering the constraints of prediction error, the minimum objective function value, corresponding to the optimal parameters, will eventually be obtained, thus achieving the best prediction performance.

[0110] In some embodiments, the specific values ​​of the Gaussian kernel function are:

[0111] K(x, y) = exp(-γ||xy||2);

[0112] Where x and y are the feature vectors of two samples; ||xy||2 is the square of the Euclidean distance between the two vectors; γ is the kernel function parameter used to control the model complexity; the Gaussian kernel function can effectively handle nonlinear relationships;

[0113] A larger γ value results in faster convergence of the kernel function, but it can easily lead to overfitting; a smaller γ value reduces complexity, tends towards linearity, but is prone to underfitting. Therefore, the choice of γ value affects both complexity and generalization ability. Cross-validation is typically used to select the optimal γ. The training set is divided into groups, with each group serving as the validation set in turn, and the remainder as the training set. This process is repeated to determine the γ that minimizes the squared loss.

[0114] Furthermore, kernel tricks can map low-dimensional inputs to high-dimensional feature spaces, effectively handling nonlinear relationships. Combining the Gaussian kernel function with SVR can yield excellent prediction results.

[0115] In some embodiments, the regularization parameter C is determined as follows:

[0116] C is the penalty parameter, which takes a positive value. The larger the value of C, the greater the degree of penalty for slack variables.

[0117] When C is too small, underfitting is likely; when C is too large, overfitting is likely. Cross-validation can be used to select the optimal value of C.

[0118] Divide the training set into n groups; use n-1 groups for training and 1 group for validation each time, and calculate the training error and validation error; iterate through different C values ​​and select the C with the smallest validation error.

[0119] Alternatively, set an optional range for C, with an exponential increase based on 2 or 10, and gradually test different values ​​of C to find the optimal value.

[0120] Determining the regularization parameter C is crucial to preventing the model from being too complex or too simple, and it is key to achieving generalization ability.

[0121] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for fusing microwave radar and graphic vision data based on machine learning, characterized in that, Includes the following steps: S1. Simultaneously, microwave radar and computer vision monitor the foundation pit support structure to acquire time-series data. The time-series data includes the difference frequency signal measured by microwave radar and the displacement data measured by computer vision. In step S1, microwave radar continuously transmits signals, samples and processes the echo signals to obtain the difference frequency signal Sd(t) representing the target motion information. Computer vision acquires images of the inner side of the foundation pit at times t1 and t2 and analyzes them to obtain displacement data Da(t1) and Da(t2). Computer vision acquires images of the outer side of the foundation pit at times t1 and t2 and analyzes them to obtain displacement data Db(t1) and Db(t2). S2. Perform data fusion of the difference frequency signal and displacement data; In step S2, data fusion specifically includes: S21. Calculate the displacement difference between the inner and outer sides of the foundation pit at the same time: ΔDa(t) = Da(t2) - Da(t1); ΔDb(t) = Db(t2) - Db(t1); S22. Calculate the difference between the inner and outer displacements, and use it as the displacement characteristic after fusion: ΔD(t) = ΔDa(t) - ΔDb(t)= [Da(t2) - Da(t1)] - [Db(t2) - Db(t1)]; S23. Establish the linear correspondence between Sd(t) and ΔD(t): Sd(t) = k1*ΔD(t) + k2; Where k1 and k2 are the linear correlation coefficients to be determined; A mathematical correspondence model between microwave radar signal Sd(t) and visual displacement difference ΔD(t) is established by parameter fitting; S3. Perform time series analysis and feature extraction on the fused data; S4. Build models based on machine learning methods to predict and analyze data.

2. The method for fusing microwave radar and graphic vision data based on machine learning according to claim 1, characterized in that, Step S3 specifically includes: S31. Based on wavelet transform, analyze the time-frequency characteristics of the fused signal: Wavelet transform is performed on the fused data sequence X(t) to decompose it into coefficient components at different scales; S32. Calculate the wavelet coefficients of each layer after multi-layer wavelet transform: Iterate through the different scales j of the wavelet transform and calculate the k-th scale coefficient of the j-th level wavelet transform for each level. and the k-th detail coefficient of the j-th level wavelet transform ; S33. Extract singularity information from the signal using wavelet coefficients to obtain abrupt change features: Analyze the k-th detail coefficient of the j-th level wavelet transform The sign of the signal changes, and when the sign changes, the position k is determined to be a singular point of the signal. Sk = {k | sign( ) ≠ sign( )}; Where Sk represents the set of singular points in the j-th layer; The intensity of singularities is represented using the magnitude of the detail factor: Ik = | |, when k ∈ Sk; S34. Calculate the energy characteristics of wavelet coefficients at each level and analyze the signal energy distribution; specifically, this includes calculating the detail coefficients at each scale. Energy: ； Analyze the statistical properties of this energy, including energy entropy Hj, energy kurtosis Kj, and energy variance Vj; S35. The final eigenvector is: F = [S1, I1, ..., SN, IN, H1, ... HN, K1, ... KN, V1, ... VN]; S36. The above features comprehensively reflect the time-frequency distribution and statistical characteristics of the signal, and serve as model inputs for representing complex changes.

3. The method for fusing microwave radar and graphic vision data based on machine learning according to claim 2, characterized in that, Step S4 involves constructing a support vector regression model for structural deformation prediction, and the specific steps are as follows: (1) Establish the prediction function for the support vector regression model: f(x)= ; Where wi and wi* are the parameters to be solved, i.e., the weights, b is the bias, K(x, xi) is the kernel function, and n is the number of support vectors; (2) Prepare training data: The fused feature vector obtained after data fusion in step S2 and time series analysis and feature extraction in step S3 is used as input xi, and the corresponding structural deformation is used as output yi, forming the training set data pair {xi, yi}. (3) Select the Gaussian kernel function for nonlinear mapping: (4) Establish the loss function and regularization term: ； Where w is the weight vector. and C is the slack variable, and C is the regularization parameter; (5) Add constraints to limit the error between the predicted value and the actual value: |yi-f(xi)|≤ε+ξi; ε is the precision parameter, and ξi is the slack variable; (6) Solve the constrained quadratic programming problem to determine the optimal parameters w and b; (7) Model training: Learn parameters w and b through iterative optimization; select appropriate γ and C; determine the optimal parameters that minimize the loss function; (8) Validation: K-fold cross-validation was used; multiple training and validations were performed to obtain the optimal parameters with the minimum generalization error; (9) The final support vector regression model is obtained.

4. The method for fusing microwave radar and graphic vision data based on machine learning according to claim 3, characterized in that, The specific values ​​for the Gaussian kernel function are: K(x,y)=exp(-γ||x-y|| 2 ); Where x and y are the feature vectors of the two samples; ||xy|| 2 γ is the square of the Euclidean distance between the two vectors; γ is the kernel function parameter used to control the model complexity.

5. The method for fusing microwave radar and graphic vision data based on machine learning according to claim 3, characterized in that, The regularization parameter C is determined as follows: Divide the training set into n groups; use n-1 groups for training and 1 group for validation each time, and calculate the training error and validation error; iterate through different C values ​​and select the C with the smallest validation error. Alternatively, set an optional range for C, with an exponential increase based on 2 or 10, and gradually test different values ​​of C to find the optimal value.

Images