An information data management system for vehicle calibration

Abstract

The present invention relates to the technical field of vehicle calibration, and particularly relates to an information data management system for vehicle calibration. By fusing the data collected by a vision sensor and a millimeter-wave radar, and introducing the size, volume, dispersion degree of obstacles, as well as the illumination condition and occlusion situation in the image data to optimize the measurement noise covariance matrix, so as to determine the position of obstacles and judge the sensor calibration deviation in a real-time environment; and when calibration is required, corner points in the image data are extracted through a corner detection algorithm, and corner point refinement is introduced to optimize the extraction of feature points in the image data, so as to combine the distance and angle information measured by the millimeter-wave radar, in order to realize the calibration of the vision sensor and the millimeter-wave radar.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle calibration, and particularly relates to an information data management system for vehicle calibration. Background Art

[0002] Vision sensors (such as cameras) and millimeter-wave radars on vehicles are important sensors. The vision sensor provides image information of obstacles, and the millimeter-wave radar provides distance and speed information of obstacles. However, errors may occur during the installation and calibration of these sensors, resulting in inaccurate data fusion in actual driving scenarios.

[0003] Among them, actual environmental factors (such as lighting conditions, occlusion situations, etc.) will affect the data accuracy of the vision sensor, and factors such as the shape, volume, and dispersion of obstacles will affect the measurement data stability of the millimeter-wave radar. Therefore, it is currently impossible to improve the accuracy of data fusion and ensure the reliability of the fusion result in various complex environments, that is, accurate and stable processing cannot be achieved between information data, thus affecting the accurate judgment of whether the vision sensor and millimeter-wave radar on the vehicle are calibrated normally.

[0004] In addition, after determining whether the calibration is normal, how to accurately calibrate the vehicle vision sensor and millimeter-wave radar, and perform calibration analysis and data security management during the calibration process are also problems that need to be solved currently. Summary of the Invention

[0005] Aiming at the above-mentioned shortcomings of the prior art, the present invention provides an information data management system for vehicle calibration, which can effectively solve the problems in the prior art that it is not convenient to judge the calibration accuracy of sensors due to actual environmental factors and the structure of obstacles, and affect subsequent calibration.

[0006] To achieve the above objectives, the present invention is realized through the following technical solutions:

[0007] The present invention provides an information data management system for vehicle calibration, which at least includes:

[0008] A data processing module, which acquires the image data of obstacles collected by the vision sensor, extracts the position, size, and shape information of the obstacles, acquires the distance, speed, and angle data of the obstacles measured by the millimeter-wave radar, introduces the Kalman filter algorithm to fuse the measurement data of the vision sensor and the millimeter-wave radar to update the position of the obstacles. During the process of fusing the measurement data of the vision sensor and the millimeter-wave radar, the size, volume, dispersion, lighting conditions, and occlusion situation of the obstacles are introduced to optimize the measurement noise covariance matrix for judging the position of the obstacles.

[0009] Among them, when determining the dispersion degree, the dispersion degree is determined by combining the shape of the obstacle, geometric features, density distribution of the point cloud, and the influence of environmental factors.

[0010] And,

[0011] The sensor calibration module obtains the image data captured by the vision sensor, determines the corner points in the image data according to the corner point detection algorithm, introduces the coordinate of the feature points accurately extracted from the corner points, and determines the distance and angle information measured by the millimeter-wave radar, and establishes the coordinate relationship between the vision sensor and the millimeter-wave radar to determine the rotation matrix and the translation vector , realizing the calibration of the vision sensor and the millimeter-wave radar.

[0012] Furthermore, the method steps of updating the obstacle position by the Kalman filter algorithm are as follows:

[0013] Execute state prediction,

[0014]

[0015] Among them, represents the state vector prediction at the th time step, represents the state vector of the previous time step, represents the state transition matrix, represents the control matrix, represents the control input vector;

[0016] Error covariance prediction:

[0017]

[0018] Among them, represents the predicted error covariance matrix, represents the transpose of the state transition matrix of represents the process noise covariance matrix;

[0019] Define the measurement matrix ,

[0020] Calculate the Kalman gain matrix ,

[0021]

[0022] Among them, represents the transpose of the measurement matrix of represents the measurement noise covariance matrix, represents the inverse matrix of the measurement prediction error covariance;

[0023] Execution status update:

[0024]

[0025] Among them, represents the updated state vector, represents the th observation data at the time step, represents the measurement innovation term;

[0026] Perform error covariance update,

[0027]

[0028] Among them, represents the updated error covariance matrix, represents the identity matrix.

[0029] Furthermore, the method steps for optimizing the measurement noise covariance matrix are as follows:

[0030] Define the measurement noise covariance matrix as a diagonal matrix,

[0031]

[0032] Among them, represents the basic measurement error variance in the direction, represents the basic measurement error variance in the direction, represents the basic measurement error variance in the direction;

[0033] According to the size , volume , dispersion , as well as the lighting factor , and the occlusion factor to obtain the measurement noise influence coefficient :

[0034] According to the measurement noise influence coefficient to optimize to re-obtain the optimized measurement noise covariance matrix .

[0035] Furthermore, the algorithm expression of the measurement noise influence coefficient is:

[0036]

[0037] Among them, and are the corresponding adjustment coefficients respectively.

[0038] The method for determining the dispersion degree is as follows:

[0039] Assume that the obstacle is represented by points indicating that, represents the total number of points in the point cloud, represents the -th three-dimensional coordinate of the point;

[0040] Calculate the centroid of the point cloud;

[0041] Use the centroid to calculate the degree of deviation of each point from the centroid, and define the initial dispersion degree as the variance or standard deviation of the Euclidean distances between all points and the centroid;

[0042] Introduce the shape, geometric features of the obstacle, the density distribution of the point cloud, lighting, and occlusion to jointly optimize the initial dispersion degree to obtain the dispersion degree .

[0043] Furthermore, calculate the centroid of the point cloud according to the following relationship,

[0044]

[0045] where, represents the centroid coordinates of the point cloud.

[0046] Furthermore, the initial dispersion degree is calculated according to the following relationship,

[0047]

[0048] Furthermore, the dispersion degree is calculated according to the following relationship,

[0049] , where, represents the ratio of the major axis to the minor axis of the main axis of the obstacle, represents the density of the point cloud, respectively represent the weight factors adjusted according to actual needs, reflects the weight of the lighting intensity, reflects the weight of the occlusion degree, reflects the weight of the point cloud density, represents the total number of points in the point cloud cluster.

[0050] Furthermore, the corner detection algorithm determines the corners in the image data, including,

[0051] Gradient calculation;

[0052] Construct an autocorrelation matrix ;

[0053] Calculate the corner response value ;

[0054]

[0055] wherein, is the determinant of the matrix ; and respectively represent the gradients of the image in the and directions; is the trace of the matrix ; represents an empirical constant;

[0056] When the corner response value is greater than the threshold, it is determined as a corner point.

[0057] Furthermore, the method for corner point refinement is as follows:

[0058] If the detected corner point coordinates are integer pixels , and assuming the true position of the corner point is at , construct an error function of the local gray value for optimization,

[0059]

[0060] wherein, represents the optimized error function, represents the window function, represents the image gray value, represents the offset in the window . By minimizing the error function , solve for the sub-pixel displacement , and obtain the sub-pixel feature point coordinates , and thus regenerate the feature point coordinates .

[0061] The technical solution provided by the present invention has the following beneficial effects compared with the known prior art:

[0062] By fusing the data collected by the vision sensor and the millimeter-wave radar, and introducing the size, volume, and dispersion of the obstacle, as well as the illumination conditions and occlusion situations in the image data, optimize the measurement noise covariance matrix to determine the position of the obstacle and judge the sensor calibration deviation in the real-time environment;

[0063] And when calibration is required, corner points in the image data are extracted through a corner point detection algorithm, and corner point refinement is introduced to optimize the extraction of feature points in the image data, so as to combine the distance and angle information measured by the millimeter-wave radar, in order to achieve the calibration of the vision sensor and the millimeter-wave radar. Brief Description of the Drawings

[0064] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention, and those of ordinary skill in the art can obtain other drawings based on these drawings without creative efforts.

[0065] Figure 1 It is the overall module block diagram of the present invention. Detailed Embodiments

[0066] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention.

[0067] In the advanced driver assistance system (ADAS) of modern vehicles, the calibration of vision sensors and millimeter-wave radars is an essential link.

[0068] In order to ensure the accuracy and reliability of the calibration data, it is necessary to construct an information data management scheme for vehicle calibration, effectively collect, store, analyze, and manage the information during the calibration process, and promote multi-sensor fusion calibration.

[0069] The following further describes the present invention with reference to the embodiments.

[0070] Embodiment 1 (refer to Figure 1 ): An information data management system for vehicle calibration includes at least:

[0071] A data processing module, after the calibration of the vehicle vision sensor and the millimeter-wave radar, obtains the image data of the obstacles collected by the vision sensor, and extracts the position, size, and shape information of the obstacles through a target detection algorithm (such as YOLO, SSD, etc.); obtains the data such as the distance, speed, and angle of the obstacles measured by the millimeter-wave radar;

[0072] Introduce the Kalman filter algorithm to update the position estimate of obstacles by combining the measurement data of visual sensors and millimeter-wave radars, including:

[0073] Initial step:

[0074] Set the initial state vector representing the initial position coordinates of the obstacle (obtained from the initial measurements of the visual sensor and millimeter-wave radar), representing the initial velocity of the obstacle (obtained from the millimeter-wave radar data), representing the transpose symbol;

[0075] Initialize the error covariance matrix , representing the uncertainty of the initial state, representing the identity matrix.

[0076] Prediction step:

[0077] Perform state prediction,

[0078]

[0079] where, representing the state vector prediction at the th time step, representing the state vector at the previous time step, representing the state transition matrix, used to describe how the state changes over time, representing the control matrix, used to describe the impact of control inputs on the state, representing the control input vector, representing the control signal at the th time step, such as acceleration;

[0080] Error covariance prediction:

[0081]

[0082] where, representing the predicted error covariance matrix, used to describe the error prediction at the th time step, representing the state transition matrix transpose of, representing the process noise covariance matrix, used to describe the uncertainty in the state model, reflecting the noise caused by factors such as modeling errors and external environmental impacts.

[0083] Measurement update step, adjust the predicted state according to the data from the camera and millimeter-wave radar:

[0084] Define the measurement matrix , Map the state vector to the observation space, assuming that the observation only provides the position information of the obstacle:

[0085]

[0086] Among them, Extract the position;

[0087] Calculate the Kalman gain matrix , which is used to weight between prediction and observation, balance the uncertainty of prediction and observation data, and improve the fusion accuracy:

[0088]

[0089] Among them, represents the transpose of the measurement matrix , represents the measurement noise covariance matrix, which is used to describe the degree of uncertainty and noise introduced in the sensor measurement process, represents the inverse matrix of the measurement prediction error covariance;

[0090] Perform state update (update the state of the obstacle using the Kalman gain and measurement innovation):

[0091]

[0092] Among them, represents the updated state vector, which is used to describe the fusion result at the th time step (i.e., the estimated position, speed, etc. of the fused obstacle), represents the observation data at the th time step, which is the fused observation value from the camera and millimeter-wave radar, represents the measurement innovation term, which is used to describe the difference between the observed value and the predicted value;

[0093] Perform error covariance update to adjust the error estimate and ensure higher accuracy in the next prediction:

[0094]

[0095] Among them, represents the updated error covariance matrix, represents the identity matrix, which is used to maintain the consistency of matrix dimensions. Thus, based on the above updated state, the final position and speed of the obstacle can be determined, and the position information measured by the vision sensor and millimeter-wave radar for the obstacle can be compared to determine whether there is a calibration anomaly in the vision sensor and millimeter-wave radar. If there is an anomaly, recalibrate the vision sensor and millimeter-wave radar.

[0096] In the above solution, in order to more precisely fuse the data of the camera (i.e., the vision sensor) and the millimeter-wave radar, the size and volume of the obstacle are introduced (the size and volume of the obstacle determine the visibility and distinctness of the target in the sensor, directly affecting the measurement accuracy of the vision sensor and the millimeter-wave radar. For the millimeter-wave radar, a large obstacle has a higher reflected signal intensity, smaller distance and speed measurement errors, and a reduced variance of the measurement noise, while a small obstacle increases it), the dispersion (the dispersion of the obstacle refers to whether the shape of the obstacle is regular and uniform. For irregular, complex-shaped or multi-dispersed targets, the measurement error is usually larger. For the millimeter-wave radar, for dispersed obstacles (such as a grove of trees or a pile of rubble), the signal reflection is chaotic, and the distance measurement error and noise increase), the lighting condition (for the vision sensor, in low-light or backlight conditions, the image noise increases, and feature extraction is difficult, resulting in an increased measurement error), and the occlusion situation (for the vision sensor, occlusion will cause some feature points to be lost or misdetected, increasing the error of vision measurement) to optimize the measurement noise covariance matrix so as to improve the robustness of the algorithm in different scenarios for the above data processing. In this embodiment, the steps for optimizing the measurement noise covariance matrix are as follows:

[0097] Define the basic measurement noise covariance matrix as a diagonal matrix, where each diagonal element represents the noise variance in different measurement dimensions:

[0098]

[0099] where, represents the basic measurement error variance in the direction, represents the basic measurement error variance in the direction, represents the basic measurement error variance in the direction;

[0100] According to the size , volume , dispersion of the obstacle, as well as the lighting factor and the occlusion factor obtain the measurement noise influence coefficient :

[0101]

[0102] where, and are the corresponding adjustment coefficients respectively. Therefore, according to the measurement noise influence coefficient For optimization, there is

[0103]

[0104] wherein represents the optimized measurement noise variance. Therefore, according to the optimized measurement noise covariance matrix is obtained , in the above, by introducing the size, volume and dispersion of obstacles, the measurement accuracy of the vision sensor and millimeter-wave radar for obstacles is directly optimized. This is because obstacles with large sizes and high dispersions tend to increase measurement uncertainty. By dynamically adjusting the measurement errors brought by these characteristics can be compensated, and the accuracy of obstacle position estimation can be improved;

[0105] Moreover, the illumination conditions and occlusion situations are important factors affecting the performance of the vision sensor. In the case of low illumination or strong occlusion, the measurement noise of the vision sensor, i.e., the camera, will be greater. By optimizing , the errors can be reduced in these complex environments, and the stability of the Kalman filter algorithm can be improved.

[0106] Furthermore, the method for determining the dispersion in the above solution is as follows:

[0107] Assume that the obstacle is represented by points , represents the total number of points in the point cloud, represents the three-dimensional coordinates of the -th point. These points are usually collected by sensors such as vision sensors (cameras) or millimeter-wave radars;

[0108] Calculate the centroid of the point cloud,

[0109]

[0110] wherein represents the centroid coordinates of the point cloud;

[0111] Use the centroid to calculate the deviation degree of each point relative to the centroid, and define the initial dispersion as the variance or standard deviation of the Euclidean distance between all points and the centroid. Then there is

[0112]

[0113] wherein, for the initial dispersion the larger it is, the more dispersed the obstacle points are distributed and the more complex the shape is; the smaller the initial dispersion is, the more concentrated the point distribution is.

[0114] Among the above, considering the initial dispersion degree of the obstacle obtained from the variance calculation based on the point cloud , it is also affected by the shape of the obstacle (obstacles of different shapes, such as spheres, cubes, cuboids, etc.), geometric features (the shape of the obstacle is ellipsoidal or irregular), and the density distribution of the point cloud (the dense part and the sparse part of the obstacle may correspond to different dispersion degrees respectively). Therefore, in this embodiment, by introducing the shape of the obstacle, geometric features, and density distribution of the point cloud, the initial dispersion degree is further optimized and determined, and the method is as follows:

[0115] Based on the vision sensor and millimeter-wave radar, obtain point cloud data, perform filtering, denoising, and segmentation to ensure that the point cloud information in the dataset is as accurate as possible;

[0116] According to the preset features of the obstacle, filter out irrelevant point cloud data, for example, background stray points or point clouds of irrelevant objects;

[0117] According to the spatial distribution of the point cloud, divide it into multiple small regions or clusters, and use the k-means clustering algorithm or DBSCAN algorithm to cluster the point cloud;

[0118] Calculate the density of the points in each cluster , for example, obtain the density of this area by the number of points in each cluster and the volume of the cluster, and the density calculation formula is:

[0119]

[0120] Among them, represents the number of points in the cluster , represents the cluster in the volume.

[0121] Perform shape fitting on the point cloud data in each cluster to determine the geometric shape of the obstacle. Common methods include fitting the minimum enclosing sphere or ellipsoid. For obstacles with irregular shapes, use principal component analysis (PCA) to calculate the main principal axis direction;

[0122] Assume that the point cloud data is a matrix, where each row represents a three-dimensional space point, use PCA to calculate the principal components of the point cloud (i.e., the principal axis direction), and calculate the ratio of the major axis to the minor axis of the principal axis ,

[0123]

[0124] Among them, are the eigenvalues of the point cloud in two principal axis directions respectively, reflecting the degree of expansion of the point cloud in each direction.

[0125] Introduce the influence of light and occlusion (environment), adjust the measurement noise weight of each point. In areas with poor lighting or more occlusion, the dispersion may increase. Therefore, calculate the noise weight of each point , introduce the ambient light intensity and the degree of occlusion , so that the noise weight is adjusted according to the change of light,

[0126]

[0127] wherein, is the adjustment factor;

[0128] Thus, the above is re-obtained, that is, the final dispersion ,

[0129]

[0130] wherein, represents the density of the point cloud, which reflects the data distribution of the obstacles collected by the sensor in space, and can indirectly describe the complexity or measurement stability of the obstacles. The higher the point cloud density, usually the more detailed and accurate the measurement of the obstacle by the sensor, while the low-density point cloud may indicate sparse or inaccurate measurement data, respectively represent the weight factors adjusted according to actual needs, the weight reflecting the light intensity, the weight reflecting the degree of occlusion, the weight reflecting the point cloud density, represents the total number of points in the point cloud cluster.

[0131] In the above, a method for judging whether the calibration of the vision sensor and the millimeter-wave radar is normal based on the Kalman filter algorithm is given. Therefore, if there is an abnormal calibration, the sensor calibration module re-calibrates the vision sensor and the millimeter-wave radar. The method includes:

[0132] Use a specific calibration scenario, including calibration objects (reflective balls, checkerboards or obstacles of known size) with known positions and sizes, to ensure that they can be captured by both the vision sensor and the millimeter-wave radar at the same time (currently usually a well-known method, which will not be elaborated here);

[0133] Obtain the data collected by the vision sensor and the millimeter-wave radar:

[0134] In the calibration scenario, collect the image data of the calibration object captured by the vision sensor and the distance and angle information of the calibration object collected by the millimeter-wave radar, where:

[0135] Extract the coordinate of feature points from the image data captured by the vision sensor ;

[0136] Based on the distance and angle information measured by the millimeter-wave radar, obtain the three-dimensional coordinates of the reflected calibration object (which can be directly measured by the millimeter-wave radar, so it will not be elaborated in this embodiment);

[0137] The method for extracting the coordinate of feature points from the image data captured by the vision sensor is as follows:

[0138] The vision sensor captures the image data of multiple calibration objects with known sizes from different angles;

[0139] In each image data, extract the feature points on the calibration object (represented as , which are the pixel coordinates in the image), that is, the corner points of the checkerboard. Among them, the method for determining the feature points on the calibration board includes:

[0140] Judge the corner points in the image data through the corner detection algorithm, then

[0141] Gradient calculation (the gradient of the image in and directions):

[0142]

[0143] Among them, and respectively represent the gradient of the image in and directions, and respectively represent the partial derivatives of the grayscale image;

[0144] Construct the autocorrelation matrix :

[0145]

[0146] Among them, and respectively represent and the sum of the squares of the gradients in the directions, represents and the product sum of the gradients in the directions;

[0147] Calculate the corner response value :

[0148]

[0149] Among them, is the determinant of the matrix , is the trace of the matrix , represents an empirical constant;

[0150] Therefore, when the corner response value is greater than the threshold, it is judged as a corner point.

[0151] Considering that the detection results given by the corner detection algorithm are at integer pixel coordinates. However, in practice, the corner position is very likely to fall between pixels. Therefore, it is necessary to refine it at the sub-pixel level. Then,

[0152] If the detected corner coordinates are integer pixels , and assuming that the true position of the corner is at , construct an error function of the local gray value to optimize it,

[0153]

[0154] where represents the optimized error function, represents a window function used to smooth the local area, represents the image gray value, represents the window in the offset. Thus, by minimizing the error function , solve for the sub-pixel displacement to obtain a more accurate corner position, that is, the sub-pixel feature point coordinates , reducing the accumulation of systematic errors caused by detection errors. Therefore, based on the currently obtained regenerate , and then, form an accurate extraction of the feature points of the image data.

[0155] Establish the coordinate relationship between the vision sensor and the millimeter-wave radar:

[0156] Including defining the coordinate systems of the vision sensor and the millimeter-wave radar as represent the image coordinate system and the sensor coordinate system respectively. Then, there are:

[0157] Points in the vision coordinate system, ;

[0158] Points in the radar coordinate system, ;

[0159] Describe the relationship between the two through the rotation matrix and the translation vector ,

[0160]

[0161] Among them, represents the rotation matrix from the radar coordinate system to the visual coordinate system, and represents the translation vector from the radar coordinate system to the visual coordinate system;

[0162] Obtain the matching point set of the visual sensor and the millimeter-wave radar to get multiple groups of corresponding point pairs , which respectively represent the visual sensor coordinates and the millimeter-wave radar coordinates;

[0163] Solve the rotation matrix (the rotation matrix from the visual sensor to the radar coordinate system) and the translation vector (the translation vector from the visual sensor to the radar coordinate system) (since it is a well-known conventional technique to solve by the least squares method, that is, this embodiment will not elaborate), so as to minimize the error between the feature point pairs of the visual sensor and the millimeter-wave radar. The corresponding optimization formula is:

[0164]

[0165] Among them, represents the Euclidean norm, that is, the length of the vector. Thus, the optimized rotation matrix and the translation vector can be used to convert and align the data of the visual sensor and the millimeter-wave radar. Furthermore, check whether the measurement results of the visual sensor and the millimeter-wave radar for the same calibration object are consistent in the unified coordinate system after conversion, so as to realize the calibration of the visual sensor and the millimeter-wave radar.

[0166] It also includes:

[0167] A calibration data storage module for storing the calibration information data (acquisition time, location, environmental conditions, used equipment, parameter settings (such as the resolution and focal length of the visual sensor, the frequency band and angle range of the millimeter-wave radar, etc.)) generated during the calibration of the above vehicle into the file database respectively (where a preset encryption key is used during storage), and encrypting the file database with the encryption key.

[0168] The method for setting the encryption key is:

[0169] By obtaining the local influence information during the vehicle calibration process, including dimensions , volume , dispersion , as well as the illumination factor , and the occlusion factor ;

[0170] And sort the above impact information from large to small (based on size (the sum of length, width and height), volume , dispersion , and the light factor , and the occlusion factor The data size is sorted to obtain the affected information sequence group again;

[0171] Therefore, based on the corresponding local impact information in the impact information sequence group, the encryption key is collected one by one, such as the impact information sequence group is volume (30) Light factor (29) Occlusion factor (25) Size (23) and dispersion (10), constitute the encryption key 3029252310, and define the encryption key through the impact information generated during the calibration process so that the calibrator can remember it.

[0172] Monitor the decryption key input to the file database, and split the input decryption key into segments to obtain the local impact information volume in the impact information sequence group. , Light Factor , occlusion factor ,size and dispersion The corresponding segment decryption key data is used to determine whether they are the same. If they are the same, the security restrictions on the file database are lifted;

[0173] On the contrary, if they are not the same, the segment decryption key data corresponding to the data in the impact information sequence group is obtained, and a judgment condition is set (taking into account the calibration personnel forgetting). If the correct local impact information (such as volume, illumination factor and size, which is 3) corresponding to the segment decryption key data in the input decryption key accounts for a larger proportion than the wrong local impact information (such as occlusion factor and dispersion, which is 2), a key guidance instruction is generated (including generating a pop-up window indicating that the encryption key of the current file database has been switched, and regenerating key data based on the wrong local impact information and inputting it into the encryption key to obtain a new encryption key, and replacing the original encryption key with the new encryption key), wherein the method for regenerating key data based on the wrong local impact information is:

[0174] Get the start time (e.g., 08:00 on March 15) and end time (e.g., 11:00 on March 15) of the vehicle's visual sensor and millimeter-wave radar calibration;

[0175] Generate fusion time data by aligning and superimposing the start time and the end time (063019);

[0176] Obtain the positions corresponding to the local impact information of the errors in the impact information sequence group respectively, extract data in the fusion time data respectively and input it into the positions corresponding to the initial encryption key (i.e., equivalent to the impact information sequence group) to form a new encryption key, where:

[0177] The method of extracting data in the fusion time data is

[0178] Obtain the position marks corresponding to the local impact information of the errors in the impact information sequence group, determine the extraction starting point based on this position mark, and determine the extraction quantity based on the quantity of the correct local impact information corresponding to the local impact information of the errors. Thus, based on the starting point and the extraction quantity, extract data sequentially from the fusion time data (the order can be from left to right, or from right to left. If the starting point is 2 and the extraction quantity is 3, then what is extracted from 063019 is 630), and fill it into the corresponding position in the initial encryption key after extraction. Therefore, for other (remaining) local impact information of the errors, the method of extracting data in the fusion time data and inputting it into the initial encryption key (impact information sequence group) is carried out by analogy. Furthermore, not only can the security protection of the file database be realized, but also the encryption key pre-constructed by the local impact information is convenient for the calibration personnel to remember. At the same time, considering the situation where it is inconvenient for the calibration personnel to log in to the file database to obtain data due to their own forgetting of the encryption key, redefine the encryption key by executing the start and time of the visual sensor and millimeter-wave radar calibration of this vehicle, so that the calibration personnel can remember according to the past data and fill in the corresponding correct decryption key. At the same time, when the initially input original decryption key is incorrect, a risk emergency protection for the file database is formed through the switching of the encryption key (equal to the decryption key).

[0179] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

1. An information data management system for vehicle calibration, comprising: A data processing module that acquires image data of obstacles collected by a vision sensor, extracts the position, size, and shape information of the obstacles, acquires the distance, speed, and angle data of the obstacles measured by a millimeter-wave radar, and introduces a Kalman filtering algorithm to fuse the measurement data of the vision sensor and the millimeter-wave radar to update the position of the obstacles, characterized in that In the process of fusing the measurement data of the vision sensor and the millimeter-wave radar, the size, volume, dispersion of the obstacle, as well as the illumination condition and occlusion situation in the image data are introduced to optimize the measurement noise covariance matrix to judge the position of the obstacle; When determining the dispersion, the dispersion is clarified by combining the shape of the obstacle, geometric features, density distribution of the point cloud, and the influence of environmental factors; And The sensor calibration module acquires the image data captured by the vision sensor, determines the corner points in the image data according to the corner detection algorithm, introduces the coordinate of the feature points extracted by corner point refinement, and determines the distance and angle information measured by the millimeter-wave radar, establishes the coordinate relationship between the vision sensor and the millimeter-wave radar, and determines the rotation matrix and the translation vector , and realizes the calibration of the vision sensor and the millimeter-wave radar; Dispersion The method for determination is as follows: Assume that the obstacle is represented by points . Let represent the total number of points in the point cloud, and represent the three-dimensional coordinates of the -th point. Calculate the centroid of the point cloud; Among them, represents the centroid coordinates of the point cloud; Use the centroid to calculate the degree of deviation of each point from the centroid and define the initial dispersion Is the variance or standard deviation of the Euclidean distance between all points and the centroid, then there is The initial dispersion of the obstacle is calculated based on the variance of the point cloud , and it is also affected by the shape, geometric features of the obstacle and the density distribution of the point cloud. The shape, geometric features of the obstacle and the density distribution of the point cloud are introduced to further optimize and determine the initial dispersion. The method is as follows: Based on the vision sensor and the millimeter-wave radar, acquire point cloud data, and perform filtering, denoising, and segmentation; Filter out irrelevant point cloud data according to the preset features of the obstacles; According to the spatial distribution of the point cloud, divide it into multiple small regions or clusters, and use the k-means clustering algorithm or the DBSCAN algorithm to perform clustering of the point cloud; Calculate the density of points within each cluster , and the density calculation formula is as follows: Among them, represents the number of points in the cluster, represents the volume in the cluster; Perform shape fitting on the point cloud data within each cluster to determine the geometric shape of the obstacle. Common methods include fitting the minimum bounding sphere or ellipsoid. For obstacles with irregular shapes, principal component analysis is used to calculate the main principal axis direction; Assume point cloud data is a matrix, where each row represents a three-dimensional spatial point. Use PCA to calculate the principal components of the point cloud and calculate the ratio of the major axis to the minor axis of the principal axis , Among them, are the eigenvalues of the point cloud in the two main axis directions, respectively, reflecting the degree of expansion of the point cloud in each direction; Introduce the influence of light and occlusion, adjust the measurement noise weight of each point. In areas with poor lighting or more occlusion, the dispersion may increase. Therefore, calculate the noise weight of each point , introduce the ambient light intensity and the degree of occlusion , so that the noise weight is adjusted according to the change of light Among them, is an adjustment factor; Thus, the above-mentioned, that is, the final dispersion degree, is obtained again. , Among them, represents the density of the point cloud, respectively represent the weight factors adjusted according to actual requirements, the weight reflecting the light intensity, the weight reflecting the occlusion degree, the weight reflecting the point cloud density, represents the total number of points in the point cloud cluster; Measurement noise covariance matrix The method steps for optimization are as follows: Define the measurement noise covariance matrix as a diagonal matrix, Among them, represents the basic measurement error variance in the direction, represents the basic measurement error variance in the direction, represents the basic measurement error variance in the direction; According to the size of the obstacle , volume , dispersion , and the lighting factor , as well as the occlusion factor obtain the measurement noise influence coefficient : According to the measurement noise influence coefficient for optimization is carried out to re-obtain the optimized measurement noise covariance matrix .

2. The information data management system for vehicle calibration according to claim 1, wherein The method steps for the Kalman filtering algorithm to update the obstacle position are: Execute state prediction; Among them, represents the state vector prediction at the th time step, represents the state vector of the previous time step, represents the state transition matrix, represents the control matrix, represents the control input vector; Error covariance prediction: Among them, represents the predicted error covariance matrix, represents the state transition matrix transpose, represents the process noise covariance matrix; Define the measurement matrix , Calculate the Kalman gain matrix , Among them, represents the transpose of the measurement matrix , represents the measurement noise covariance matrix, represents the inverse matrix of the measurement prediction error covariance; Execute state update: Among them, represents the updated state vector, represents the observation data at the th time step, and represents the measurement innovation term; Perform error covariance update; Among them, represents the updated error covariance matrix, represents the identity matrix.

3. The information data management system for vehicle calibration according to claim 1, characterized in that, The measurement noise influence coefficient has the following algorithmic expression: wherein, and are the corresponding adjustment coefficients respectively.

4. An information data management system for vehicle calibration according to claim 1, characterized in that, The corner detection algorithm determines the corners in the image data, including Gradient calculation; Construct an autocorrelation matrix ; Calculate the corner response value ; Among them, is the determinant of the matrix . and respectively represent the gradients of the image in the and directions. is the trace of the matrix . represents an empirical constant; When the corner response value is greater than the threshold, it is judged as a corner point.

5. The information data management system for vehicle calibration according to claim 4, characterized in that, The method for corner refinement is: If the detected corner coordinates are integer pixels , and assuming the true position of the corner is at , construct an error function for the local gray value for optimization. Among them, represents the optimized error function, represents the window function, represents the image gray value, represents the offset in the window . By minimizing the error function , the sub-pixel displacement is solved, and the sub-pixel feature point coordinates are obtained. From this, the feature point coordinates are regenerated.

Images