Information data management system for vehicle calibration

Abstract

The invention relates to the technical field of vehicle calibration, in particular to an information data management system for vehicle calibration. A measurement noise covariance matrix is optimized by fusing data acquired by a visual sensor and a millimeter wave radar and introducing the size, the volume and the dispersity of an obstacle as well as the illumination condition and the shielding condition in image data, so that the position of the obstacle is determined, and the sensor calibration deviation is judged in a real-time environment; when calibration is needed, angular points in the image data are extracted through an angular point detection algorithm, extraction of feature points in the image data is accurately optimized by introducing the angular points, and calibration of the visual sensor and the millimeter-wave radar is achieved by combining distance and angle information measured by the millimeter-wave radar.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle calibration, and in particular to an information data management system for vehicle calibration. Background Art

[0002] The visual sensor (such as camera) and millimeter wave radar on the vehicle are one of the important sensors. The visual sensor provides image information of obstacles, and the millimeter wave radar provides distance and speed information of obstacles. However, these sensors may produce errors during installation and calibration, resulting in inaccurate data fusion in actual driving scenarios.

[0003] Among them, actual environmental factors (such as lighting conditions, occlusion, etc.) will affect the data accuracy of the visual sensor, while factors such as the shape, volume, and dispersion of obstacles will affect the stability of the measurement data of the millimeter-wave radar. Therefore, it is currently impossible to improve the accuracy of data fusion and ensure the reliability of the fusion results in various complex environments. In other words, accurate and stable processing between information data cannot be achieved, which affects the accurate judgment of whether the visual sensor and millimeter-wave radar on the vehicle are calibrated normally;

[0004] In addition, after determining whether the calibration is normal, how to accurately calibrate the vehicle's visual sensors and millimeter-wave radars, and how to perform calibration analysis after calibration and manage data security during the calibration process are also issues that need to be addressed. Summary of the invention

[0005] In view of the above-mentioned shortcomings of the prior art, the present invention provides a vehicle calibration information data management system, which can effectively solve the problem in the prior art that it is difficult to judge the calibration accuracy of the sensor due to actual environmental factors and the structure of obstacles, and affects the subsequent calibration.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0007] The present invention provides a vehicle calibration information data management system, which at least comprises:

[0008] The data processing module obtains the image data of the obstacle collected by the visual sensor, extracts the location, size and shape information of the obstacle, obtains the distance, speed and angle data of the obstacle measured by the millimeter-wave radar, introduces the Kalman filter algorithm to fuse the measurement data of the visual sensor and the millimeter-wave radar to update the location of the obstacle. In the process of fusing the measurement data of the visual sensor and the millimeter-wave radar, the size, volume, dispersion, lighting conditions and occlusion of the obstacle are introduced to optimize the measurement noise covariance matrix R to determine the location of the obstacle;

[0009] Among them, when determining the dispersion, the dispersion is clarified by combining the shape and geometric characteristics of the obstacles, the density distribution of the point cloud and the influence of environmental factors;

[0010] as well as,

[0011] The sensor calibration module obtains the image data captured by the visual sensor, determines the corner points in the image data based on the corner detection algorithm, and introduces the coordinates of the feature points extracted by the corner points to accurately determine the distance and angle information measured by the millimeter-wave radar, establishes the coordinate relationship between the visual sensor and the millimeter-wave radar, and determines the rotation matrix R vr and the translation vector T vr , to achieve calibration of visual sensors and millimeter-wave radars.

[0012] Furthermore, the method steps of updating the obstacle position by the Kalman filter algorithm are as follows:

[0013] Execution status prediction,

[0014]

[0015] in, represents the state vector prediction at the kth time step, represents the state vector of the previous time step, A represents the state transfer matrix, B represents the control matrix, and u k represents the control input vector;

[0016] Error covariance prediction:

[0017]

[0018] in, represents the predicted error covariance matrix, A T represents the transpose of the state transfer matrix A, Q represents the process noise covariance matrix;

[0019] Define the measurement matrix H,

[0020] Calculate the Kalman gain matrix K k ,

[0021]

[0022] Among them, H T represents the transpose of the measurement matrix H, R represents the measurement noise covariance matrix, represents the inverse matrix of the measurement prediction error covariance;

[0023] Execution status update:

[0024]

[0025] in, represents the updated state vector, z k represents the observed data at the kth time step, Indicates measurement innovation item;

[0026] Perform error covariance update,

[0027]

[0028] Among them, P k represents the updated error covariance matrix, and I represents the identity matrix.

[0029] Furthermore, the method steps for optimizing the measurement noise covariance matrix R are as follows:

[0030] Define the measurement noise covariance matrix R as a diagonal matrix,

[0031]

[0032] in, represents the basic measurement error variance in the x direction, represents the basic measurement error variance in the y direction, represents the basic measurement error variance in the z direction;

[0033] The measurement noise influence coefficient F is obtained based on the size S, volume V, dispersion D, illumination factor L, and occlusion factor O of the obstacle:

[0034] According to the measurement noise influence coefficient F Optimization is performed to regain the optimized measurement noise covariance matrix R.

[0035] Furthermore, the algorithm expression of the measurement noise influence coefficient F is:

[0036] F=(1+α s S+β v V+γ d ·D+δ l ·(1-L)+δ o ·O)

[0037] Among them, α s , β v , γ d , δ l and δ o are the corresponding adjustment coefficients respectively.

[0038] 5. The vehicle calibration information data management system according to claim 1, characterized in that the dispersion is determined by:

[0039] Assume that the obstacle consists of n points {(x i， y i , z i )|i=1,2,···,n}, n represents the total number of points in the point cloud, (x i ,y i , z i ) represents the three-dimensional coordinates of the i-th point;

[0040] Calculate the centroid of the point cloud;

[0041] The centroid is used to calculate the degree of deviation of each point from the centroid, and the initial dispersion Dσ is defined as the variance or standard deviation of the Euclidean distance of all points from the centroid;

[0042] The shape, geometric features, density distribution of point clouds, illumination and occlusion of obstacles are introduced to jointly optimize the initial dispersion Dσ and obtain the dispersion D.

[0043] Furthermore, the centroid of the point cloud is calculated according to the following relationship:

[0044]

[0045] in, Represents the centroid coordinates of the point cloud.

[0046] Furthermore, the initial dispersion Dσ is calculated according to the following relationship,

[0047]

[0048] Furthermore, the dispersion D is calculated according to the following relationship:

[0049]

[0050] Among them, aspect represents the ratio of the major axis to the minor axis of the obstacle, ρ represents the density of the point cloud, and α, β, and υ represent the weight factors adjusted according to actual needs. The weight reflecting the light intensity, The weight reflecting the degree of occlusion, The weight that reflects the density of the point cloud, and n represents the total number of points in the point cloud cluster.

[0051] Furthermore, the corner point detection algorithm determines the corner points in the image data, including:

[0052] Gradient calculation;

[0053] Construct the autocorrelation matrix M;

[0054] Calculate the corner point response value R;

[0055] R = det(M) - k(trace(M))2

[0056] in, is the determinant of the matrix M, I x and I y Represent the gradient of the image in the x and y directions respectively, is the trace of the matrix M, k represents an empirical constant;

[0057] When the corner point response value R is greater than the threshold, it is judged as a corner point.

[0058] Furthermore, the method for refining the corner points is:

[0059] If the detected corner point coordinates are integer pixels (x i ,y i ), and assume that the actual position of the corner point is (x i +Δx,y i +Δy), construct the error function E of the local gray value for optimization,

[0060]

[0061] Among them, E(Δx, Δy) represents the optimization error function, ω(u, v) represents the window function, I(x, y) represents the image grayscale value, and (u, v) represents the offset in the window W. By minimizing the error function E, the sub-pixel displacement Δx and Δy are solved, and the sub-pixel feature point coordinates (u i +Δx,v i +Δy), thereby regenerating the feature point coordinates (u i ,v i ).

[0062] Compared with the known prior art, the technical solution provided by the present invention has the following beneficial effects:

[0063] By fusing the data collected by the visual sensor and millimeter-wave radar, and introducing the size, volume, dispersion of obstacles, as well as the lighting conditions and occlusion in the image data, the measurement noise covariance matrix is ​​optimized to determine the location of obstacles and judge the sensor calibration deviation in a real-time environment;

[0064] And when calibration is needed, the corner points in the image data are extracted through the corner detection algorithm, and the corner point precision is introduced to optimize the extraction of feature points in the image data, so as to combine the distance and angle information measured by the millimeter-wave radar to realize the calibration of the visual sensor and millimeter-wave radar. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0066] Figure 1 It is the overall module block diagram of the present invention. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0068] In modern automobiles’ advanced driver assistance systems (ADAS), visual sensor and millimeter-wave radar calibration is an indispensable part.

[0069] In order to ensure the accuracy and reliability of calibration data, it is necessary to build an information data management solution for vehicle calibration to effectively collect, store, analyze and manage information during the calibration process and promote multi-sensor fusion calibration.

[0070] The present invention will be further described below in conjunction with the embodiments.

[0071] Example 1 (see Figure 1 ): A vehicle calibration information data management system, comprising at least:

[0072] The data processing module obtains the image data of obstacles collected by the visual sensor after the vehicle visual sensor and millimeter-wave radar are calibrated, and extracts the location, size and shape information of the obstacles through the target detection algorithm (such as YOLO, SSD, etc.); obtains the distance, speed, angle and other data of the obstacles measured by the millimeter-wave radar;

[0073] The Kalman filter algorithm is introduced to combine the measurement data of the visual sensor and millimeter wave radar to update the position estimate of the obstacle, including:

[0074] Initial steps:

[0075] Set the initial state vector x0, y0, z0 represent the initial position coordinates of the obstacle (obtained from the initial measurements of the visual sensor and millimeter-wave radar), x′0, y′0, z′0 represent the initial velocity of the obstacle (obtained from the millimeter-wave radar data), and T represents the transposed sign;

[0076] Initialize the error covariance matrix P0=I, which represents the uncertainty of the initial state, and I represents the unit matrix.

[0077] Prediction Steps:

[0078] Execution status prediction,

[0079]

[0080] in, represents the state vector prediction at the kth time step, represents the state vector of the previous time step, A represents the state transfer matrix, which is used to describe the change of the state over time, B represents the control matrix, which is used to describe the influence of the control input on the state, and u k represents the control input vector, which represents the control signal at the kth time step, such as acceleration;

[0081] Error covariance prediction:

[0082]

[0083] in, Represents the predicted error covariance matrix, which is used to describe the error prediction at the kth time step, A T represents the transpose of the state transfer matrix A, Q represents the process noise covariance matrix, which is used to describe the uncertainty in the state model and reflects the noise caused by modeling errors, external environmental influences and other factors.

[0084] The measurement update step adjusts the predicted state based on the data from the camera and mmWave radar:

[0085] Define the measurement matrix H to map the state vector to the observation space, assuming that the observation only provides the location information of the obstacle:

[0086]

[0087] Among them, H extracts the x, y, and z positions of the obstacles;

[0088] Calculate the Kalman gain matrix K k , which is used to weight prediction and observation, balance the uncertainty of prediction and observation data, and improve fusion accuracy:

[0089]

[0090] Among them, H T represents the transpose of the measurement matrix H, R represents the measurement noise covariance matrix, which is used to describe the degree of uncertainty and noise introduced in the sensor measurement process. represents the inverse matrix of the measurement prediction error covariance;

[0091] Perform state update (update the state of the obstacle using Kalman gain and measured innovation):

[0092]

[0093] in, represents the updated state vector, which is used to describe the fusion result at the kth time step (i.e., the estimation of obstacle position, speed, etc. after fusion), z k represents the observation data at the kth time step, which is the fusion observation value from the camera and millimeter-wave radar. represents the measurement innovation term, which is used to describe the difference between the observed value and the predicted value;

[0094] The error covariance is updated to adjust the error estimate to ensure a higher accuracy for the next prediction:

[0095]

[0096] Among them, P k Represents the updated error covariance matrix, and I represents the unit matrix, which is used to maintain the consistency of the matrix dimension. Therefore, the final position and speed of the obstacle can be determined based on the updated state, and the position information measured by the visual sensor and the millimeter-wave radar can be obtained for comparison to determine whether there is any calibration anomaly between the visual sensor and the millimeter-wave radar. If so, the visual sensor and the millimeter-wave radar are recalibrated.

[0097] In the above scheme, in order to more accurately fuse the data of the camera (i.e., visual sensor) and millimeter-wave radar, the size and volume of the obstacle are introduced (the size and volume of the obstacle determine the visibility and feature prominence of the target in the sensor, and directly affect the measurement accuracy of the visual sensor and millimeter-wave radar. For millimeter-wave radar, large obstacles have higher reflected signal intensity, smaller distance and speed measurement errors, and reduced measurement noise variance, while small obstacles increase it), and dispersion (the dispersion of the obstacle refers to whether the shape of the obstacle is regular and uniform. For irregular, complex or multi-dispersed targets, the measurement error is usually larger). For millimeter-wave radars, there are scattered obstacles (such as trees and rubble), signal reflections are cluttered, and distance measurement errors and noise increase), lighting conditions (for visual sensors, in low light or backlight conditions, image noise increases, feature extraction is difficult, and measurement errors increase), and occlusion conditions (for visual sensors, occlusions can cause some feature points to be lost or misdetected, increasing visual measurement errors). The measurement noise covariance matrix R is optimized to improve the robustness of the algorithm in different scenarios, so as to process the data as described above. In this embodiment, the steps for optimizing the measurement noise covariance matrix R are as follows:

[0098] The basic measurement noise covariance matrix R is defined as a diagonal matrix, where each diagonal element represents the noise variance in different measurement dimensions:

[0099]

[0100] in, represents the basic measurement error variance in the x direction, represents the basic measurement error variance in the y direction, represents the basic measurement error variance in the z direction;

[0101] The measurement noise influence coefficient F is obtained based on the size S, volume V, dispersion D, illumination factor L, and occlusion factor O of the obstacle:

[0102] F=(1+α s S+β v V+γ d ·D+δl·(1-L)+δ o ·O)

[0103] Among them, α s , β v , γ d , δ l and δ o They are the corresponding adjustment coefficients, so according to the measurement noise influence coefficient F For optimization, we have

[0104]

[0105] in, represents the optimized measurement noise variance, so we get The optimized measurement noise covariance matrix R is obtained. In the above, the obstacle size, volume and dispersion are introduced to directly optimize the measurement accuracy of the visual sensor and millimeter wave radar. This is because obstacles with large size and high dispersion often lead to increased measurement uncertainty. By dynamically adjusting R, the measurement errors caused by these characteristics can be compensated, thereby improving the accuracy of obstacle position estimation.

[0106] In addition, lighting conditions and occlusion are important factors that affect the performance of visual sensors. In low-light or strong occlusion conditions, the measurement noise of visual sensors, that is, cameras, will be greater. By optimizing R, the error can be reduced in these complex environments and the stability of the Kalman filter algorithm can be improved.

[0107] Furthermore, the method for determining the dispersion degree D in the above scheme is as follows:

[0108] Assume that the obstacle consists of n points {(x i， y i， z i )|i=1,2,···,n}, n represents the total number of points in the point cloud, (x i ,y i , z i ) represents the three-dimensional coordinates of the i-th point, which are usually collected by sensors such as visual sensors (cameras) or millimeter-wave radars;

[0109] Calculate the centroid of the point cloud,

[0110]

[0111] in, Represents the centroid coordinates of the point cloud;

[0112] Use the centroid to calculate the degree of deviation of each point from the centroid, and define the initial dispersion Dσ as the variance or standard deviation of the Euclidean distance between all points and the centroid, then,

[0113]

[0114] Among them, the larger the initial dispersion Dσ is, the more dispersed the obstacle points are and the more complex their shapes are; the smaller the initial dispersion Dσ is, the more concentrated the point distribution is.

[0115] In the above, considering that the initial dispersion Dσ of the obstacle obtained by the above point cloud-based variance calculation is also affected by the shape of the obstacle (obstacles of different shapes, such as spheres, cubes, cuboids, etc.), geometric features (the shape of the obstacle is an ellipsoid or irregular) and the density distribution of the point cloud (the dense part and the sparse part of the obstacle may correspond to different dispersions respectively), therefore, in this embodiment, the initial dispersion is further optimized and determined by introducing the shape of the obstacle, the geometric features, and the density distribution of the point cloud, and the method is as follows:

[0116] Obtain point cloud data based on visual sensors and millimeter wave radars, perform filtering, denoising and segmentation to ensure that the point cloud information in the data set is as accurate as possible;

[0117] Filter out irrelevant point cloud data, such as background stray points or irrelevant object point clouds, based on preset features of obstacles;

[0118] According to the spatial distribution of the point cloud, it is divided into multiple small areas or clusters, and the k-means clustering algorithm or DBSCAN algorithm is used to cluster the point cloud;

[0119] Calculate the density ρ of each cluster point i For example, the density of the area is obtained by the number of points in each cluster and the volume of the cluster. The density calculation formula is:

[0120]

[0121] Among them, N i represents the number of points in cluster i, V i represents the volume in cluster i.

[0122] Perform shape fitting on the point cloud data within each cluster to determine the geometric shape of the obstacle. Common methods include fitting the minimum bounding sphere or ellipsoid. For irregularly shaped obstacles, use principal component analysis (PCA) to calculate the main axis directions.

[0123] Assume that the point cloud data P is an N×3 matrix, where each row represents a three-dimensional space point. Use PCA to calculate the principal component of the point cloud (i.e., the direction of the principal axis) and the aspect ratio of the major and minor axes.

[0124]

[0125] Among them, λ1 and λ2 are the eigenvalues ​​of the point cloud in the two main axis directions, reflecting the expansion degree of the point cloud in each direction.

[0126] Introducing the influence of illumination and occlusion (environment), adjusting the measurement noise weight of each point. In areas with poor illumination or more occlusion, the dispersion may increase. Therefore, the noise weight ω of each point is calculated.i , introduce the ambient light intensity I light and occlusion degree O occl , so that the noise weights are adjusted as the lighting changes,

[0127]

[0128] Among them, ε1 and ε2 are adjustment factors;

[0129] Thus, we can get the above-mentioned final dispersion D again.

[0130]

[0131] Among them, ρ represents the density of the point cloud, which reflects the data distribution of the obstacle collected by the sensor in space, and can indirectly describe the complexity of the obstacle or the measurement stability. The higher the point cloud density, the more detailed and accurate the sensor's measurement of the obstacle is. A low-density point cloud may indicate sparse or inaccurate measurement data. α, β, and υ represent weight factors adjusted according to actual needs. The weight reflecting the light intensity, The weight reflecting the degree of occlusion, The weight that reflects the density of the point cloud, and n represents the total number of points in the point cloud cluster.

[0132] In the above, a method based on the Kalman filter algorithm is given to determine whether the calibration of the visual sensor and the millimeter-wave radar is normal. Therefore, if there is an abnormal calibration, the sensor calibration module recalibrates the visual sensor and the millimeter-wave radar. The method includes:

[0133] Use a specific calibration scene, including calibration objects of known position and size (reflective balls, chessboards, or obstacles of known size) to ensure that they can be captured by both the visual sensor and the millimeter-wave radar at the same time (this is currently a well-known method and will not be described here);

[0134] Get the data collected by the visual sensor and millimeter wave radar:

[0135] In the calibration scene, the image data of the calibration object captured by the visual sensor and the distance and angle information of the calibration object collected by the millimeter wave radar are collected, where:

[0136] Based on the image data captured by the visual sensor, the coordinates of the feature points in the image data (u i ,v i );

[0137] Based on the distance and angle information measured by the millimeter wave radar, the three-dimensional coordinates (x r ,y r ,z r) (can be directly measured by millimeter wave radar, so it will not be described in detail in this embodiment);

[0138] The method for extracting feature point coordinates from the image data captured by the visual sensor is:

[0139] The visual sensor captures image data of multiple calibration objects of known size from different angles;

[0140] In each image data, extract the feature points on the calibration object (expressed as (u i ,v i ), is the pixel coordinate in the image), that is, the corner point of the chessboard, wherein the method for determining the feature point on the calibration plate includes:

[0141] By using the corner detection algorithm to determine the corner points in the image data, we have:

[0142] Gradient calculation (gradient of the image in the x and y directions):

[0143]

[0144] Among them, I x and I y Represent the gradient of the image in the x and y directions respectively, and Respectively represent the partial derivatives of the grayscale image;

[0145] Construct the autocorrelation matrix M:

[0146]

[0147] in, and Represents the sum of squared gradients in the x and y directions, ΣI x I y Represents the product of the gradients in the x and y directions;

[0148] Calculate the corner point response value R:

[0149] R = det(M) - k(trace(M)) 2

[0150] in, is the determinant of the matrix M, is the trace of the matrix M, k represents an empirical constant;

[0151] Therefore, when the corner point response value R is greater than the threshold, it is judged as a corner point.

[0152] Considering that the detection results given by the corner detection algorithm are in integer pixel coordinates, however, in practice, the corner position is likely to fall between pixels, so it is necessary to be refined at the sub-pixel level, then,

[0153] If the detected corner point coordinates are integer pixels (x i ,y i ), and assume that the actual position of the corner point is (x i +Δx,y i +Δy), construct the error function E of the local gray value for optimization,

[0154]

[0155] Among them, E(Δx, Δy) represents the optimization error function, ω(u, v) represents the window function used to smooth the local area, I(x, y) represents the image grayscale value, and (u, v) represents the offset in the window W. Therefore, by minimizing the error function E, the sub-pixel displacement Δx and Δy are solved to obtain a more accurate corner point position, that is, the sub-pixel feature point coordinates (u i +Δx,v i +Δy), reducing the accumulation of system errors caused by detection errors. Therefore, according to the current (u i +Δx,v i +Δy) regenerates (u i ,v i ), and then, the feature points of the image data are accurately extracted.

[0156] Establish the coordinate relationship between the visual sensor and the millimeter wave radar:

[0157] Including defining the coordinate systems of the visual sensor and millimeter wave radar as C v , C r , C v , C r Represent the image coordinate system and the sensor coordinate system respectively, then:

[0158] Point in the visual coordinate system, Pv=(x v ,y v ,z v );

[0159] Point in radar coordinate system, P r =(x r ,y r ,z r );

[0160] By rotating the matrix R vr and the translation vector T vr Describe the relationship between the two.

[0161] P v =R vr ·P r +T vr

[0162] Among them, R vr Represents the rotation matrix from the radar coordinate system to the visual coordinate system, T vr Represents the translation vector from the radar coordinate system to the visual coordinate system;

[0163] Obtain the matching point set of the visual sensor and the millimeter-wave radar, and obtain multiple groups of corresponding point pairs (P vi ,P ri ), P vi ,P ri Represent the visual sensor coordinates and millimeter wave radar coordinates respectively;

[0164] Solve the rotation matrix R according to the least squares method vr (Rotation matrix from visual sensor to radar coordinate system) and translation vector T vr (translation vector from the visual sensor to the radar coordinate system) (because it is a well-known conventional technology to solve it by the least square method, that is, it is not repeated in this embodiment), so that the error between the feature point pairs of the visual sensor and the millimeter wave radar is minimized, and the corresponding optimization formula is:

[0165]

[0166] Among them, ||·|| represents the Euclidean norm, that is, the length of the vector, thus, the optimized rotation matrix R vr and the translation vector T vr The data of the visual sensor and millimeter-wave radar are converted and aligned, and then, it is checked whether the measurement results of the visual sensor and millimeter-wave radar on the same calibration object are consistent in the unified coordinate system after conversion, thereby realizing the calibration of the visual sensor and millimeter-wave radar.

[0167] Also includes:

[0168] The calibration data storage module is used to store the calibration information data generated by the above-mentioned vehicle during the calibration process (collection time, location, environmental conditions, equipment used, parameter settings (such as resolution and focal length of the visual sensor, frequency band and angle range of the millimeter wave radar, etc.) in a file database (where an encryption key is preset during storage), and encrypt the file database using the encryption key.

[0169] The encryption key is set as follows:

[0170] By obtaining local influence information during vehicle calibration, including size S, volume V, dispersion D, illumination factor L, and occlusion factor O;

[0171] The above-mentioned influence information is sorted from large to small (according to the size S (the sum of length, width and height), volume V, dispersion D, illumination factor L, and occlusion factor O), and the influence information sequence group is obtained again;

[0172] Therefore, based on the corresponding local impact information in the impact information sequence group, the encryption key is collected one by one and composed. For example, the impact information sequence group is volume V (30), illumination factor L (29), occlusion factor O (25), size S (23) and dispersion D (10), which constitute the encryption key 3029252310. The encryption key is defined by the impact information generated during the calibration process so that the calibrator can remember it.

[0173] Monitor the decryption key input to the file database, and segment the input decryption key to obtain segment decryption key data corresponding to the local impact information volume V, illumination factor L, occlusion factor O, size S and dispersion D in the impact information sequence group, and determine whether they are the same. If they are the same, release the security restrictions of the file database;

[0174] On the contrary, if they are not the same, the segment decryption key data corresponding to the data in the impact information sequence group is obtained, and a judgment condition is set (taking into account the calibration personnel forgetting). If the correct local impact information (such as volume, illumination factor and size, which is 3) corresponding to the segment decryption key data in the input decryption key accounts for a larger proportion than the wrong local impact information (such as occlusion factor and dispersion, which is 2), a key guidance instruction is generated (including generating a pop-up window indicating that the encryption key of the current file database has been switched, and regenerating key data based on the wrong local impact information and inputting it into the encryption key to obtain a new encryption key, and replacing the original encryption key with the new encryption key), wherein the method for regenerating key data based on the wrong local impact information is:

[0175] Obtain the start time (e.g., 08:00 on March 15) and end time (e.g., 11:00 on March 15) of the vehicle's visual sensor and millimeter-wave radar calibration;

[0176] The start time and the end time are aligned and superimposed to generate fused time data (063019);

[0177] The positions corresponding to the local impact information of the error in the impact information sequence group are obtained respectively, and the data are extracted in the fused time data respectively and input into the positions corresponding to the initial encryption key (i.e., equivalent to the impact information sequence group) to form a new encryption key, where:

[0178] The method of extracting data from fused time data is:

[0179] Obtain the position mark corresponding to the erroneous local impact information in the impact information sequence group, determine the extraction starting point based on the position mark, and determine the extraction quantity based on the number of correct local impact information corresponding to the erroneous local impact information, thereby extracting in sequence in the fused time data based on the starting point and the extraction quantity (the order can be from left to right, or from right to left, if the starting point is 2 and the extraction quantity is 3, then 630 is extracted from 063019), and fill in the corresponding position in the initial encryption key after extraction, therefore, extract data from the fused time data for other (remaining) erroneous local impact information and input it into the initial encryption key (impact information sequence group) The method is similar to that of the embodiment, and further, not only can the security protection of the file database be achieved, but also the encryption key pre-constructed by local impact information can be convenient for the calibration personnel to remember. At the same time, considering the situation that the calibration personnel forget the encryption key and it is inconvenient to log in to the file database to obtain data, the encryption key is redefined by executing the start and time of the calibration of the visual sensor and millimeter-wave radar of the vehicle, so that the calibration personnel can remember and fill in the corresponding correct decryption key according to the past data. At the same time, when the original decryption key initially input is incorrect, a risk emergency protection for the file database is formed by switching the encryption key (equal to the decryption key).

[0180] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

1. A vehicle calibration information data management system, comprising: The data processing module obtains the image data of the obstacle collected by the visual sensor, extracts the position, size and shape information of the obstacle, obtains the distance, speed and angle data of the obstacle measured by the millimeter wave radar, and introduces the Kalman filter algorithm to fuse the measurement data of the visual sensor and the millimeter wave radar to update the position of the obstacle. It is characterized by: In the process of fusing the measurement data of the visual sensor and the millimeter-wave radar, the size, volume, dispersion of the obstacle, as well as the lighting conditions and occlusion in the image data are introduced to optimize the measurement noise covariance matrix R to determine the location of the obstacle; Among them, when determining the dispersion, the shape and geometric characteristics of the obstacles, the density distribution of the point cloud and the influence of environmental factors are combined to clarify the dispersion; as well as, The sensor calibration module obtains the image data captured by the visual sensor, determines the corner points in the image data based on the corner detection algorithm, and introduces the coordinates of the feature points extracted by the corner points to accurately determine the distance and angle information measured by the millimeter-wave radar, establishes the coordinate relationship between the visual sensor and the millimeter-wave radar, and determines the rotation matrix R vr and the translation vector T vr , to achieve calibration of visual sensors and millimeter-wave radars.

2. The vehicle calibration information data management system according to claim 1, characterized in that: The method steps of updating the obstacle position by the Kalman filter algorithm are as follows: Execution status prediction, in, represents the state vector prediction at the kth time step, represents the state vector of the previous time step, A represents the state transfer matrix, B represents the control matrix, and u k represents the control input vector; Error covariance prediction: in, represents the predicted error covariance matrix, A T represents the transpose of the state transfer matrix A, Q represents the process noise covariance matrix; Define the measurement matrix H, Calculate the Kalman gain matrix K k , Among them, H T represents the transpose of the measurement matrix H, R represents the measurement noise covariance matrix, represents the inverse matrix of the measurement prediction error covariance; Execution status update: in, represents the updated state vector, z k represents the observed data at the kth time step, Indicates measurement innovation item; Perform error covariance update, Among them, P k represents the updated error covariance matrix, and I represents the identity matrix.

3. The vehicle calibration information data management system according to claim 1, characterized in that: The method steps for optimizing the measurement noise covariance matrix R are as follows: Define the measurement noise covariance matrix R as a diagonal matrix, in, represents the basic measurement error variance in the x direction, represents the basic measurement error variance in the y direction, represents the basic measurement error variance in the z direction; The measurement noise influence coefficient F is obtained based on the size S, volume V, dispersion D, illumination factor L, and occlusion factor O of the obstacle: According to the measurement noise influence coefficient F Optimization is performed to regain the optimized measurement noise covariance matrix R.

4. The vehicle calibration information data management system according to claim 3, characterized in that: The algorithm expression of the measurement noise influence coefficient F is: F=(1+α s ·S+β v ·V+γ d ·D+d l ·(1-L)+δ o ·O) Among them, α s , β v , γ d , δ l and δ o are the corresponding adjustment coefficients respectively.

5. The vehicle calibration information data management system according to claim 1, characterized in that: The method for determining the dispersion is: Assume that the obstacle consists of n points {(x i ,y i , z i )|i=1,2,···,n}, n represents the total number of points in the point cloud, (x i ,y i , z i ) represents the three-dimensional coordinates of the i-th point; Calculate the centroid of the point cloud; The centroid is used to calculate the degree of deviation of each point from the centroid, and the initial dispersion Dσ is defined as the variance or standard deviation of the Euclidean distance of all points from the centroid; The shape, geometric features, density distribution of point clouds, illumination and occlusion of obstacles are introduced to jointly optimize the initial dispersion Dσ and obtain the dispersion D.

6. The vehicle calibration information data management system according to claim 5, characterized in that: The centroid of the point cloud is calculated according to the following relationship: in, Represents the centroid coordinates of the point cloud.

7. The vehicle calibration information data management system according to claim 6, characterized in that: The initial dispersion Dσ is calculated according to the following relationship:

8. The vehicle calibration information data management system according to claim 7, characterized in that: The dispersion D is calculated according to the following relationship: Among them, aspect represents the ratio of the major axis to the minor axis of the obstacle, ρ represents the density of the point cloud, and α, β, and υ represent the weight factors adjusted according to actual needs. The weight reflecting the light intensity, The weight reflecting the degree of occlusion, The weight that reflects the density of the point cloud, and n represents the total number of points in the point cloud cluster.

9. The vehicle calibration information data management system according to claim 7, characterized in that: The corner point detection algorithm determines the corner points in the image data, including: Gradient calculation; Construct the autocorrelation matrix M; Calculate the corner point response value R; R=det(M)-k(trace(M)) 2 in, is the determinant of the matrix M, I x and I y Represent the gradient of the image in the x and y directions respectively, is the trace of the matrix M, k represents an empirical constant; When the corner point response value R is greater than the threshold, it is judged as a corner point.

10. The vehicle calibration information data management system according to claim 9, characterized in that: The method for refining the corner points is: If the detected corner point coordinates are integer pixels (x i ,y i ), and assume that the actual position of the corner point is (x i +Δx,y i +Δy), construct the error function E of the local gray value for optimization, Among them, E(Δx, Δy) represents the optimization error function, ω(u, v) represents the window function, I(x, y) represents the image grayscale value, and (u, v) represents the offset in the window W. By minimizing the error function E, the sub-pixel displacement Δx and Δy are solved, and the sub-pixel feature point coordinates (u i +Δx,v i +Δy), thereby regenerating the feature point coordinates (u i ,v i ).

Images