An AGV precise positioning control method and device combining vision and laser radar, equipment and medium

Abstract

The application discloses an AGV accurate positioning control method and device fusing vision and laser radar, equipment and medium, relates to the AGV accurate positioning technical field, and comprises the following steps: acquiring historical vision and laser radar data and corresponding AGV actual data under different environments, calculating multi-dimensional deviation and mean square error as first process noise covariance;Based on the sensor accuracy, set the first observation noise covariance, and generate a plurality of second observation noise covariance through rotation;The first process noise covariance is split and rotated to generate a plurality of second process noise covariance, and the third process noise covariance is obtained through cartesian product;Randomly combine observation and process noise covariance to form filter control array, applied to fusion positioning and multi-dimensional evaluation, select the best group as the second filter control array for AGV positioning, solve the problem of insufficient accuracy and stability in the AGV positioning of fusion vision and laser radar.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of AGV precise positioning, more particularly, the present application relates to a kind of AGV precise positioning control method, device, equipment and medium of fusion vision and laser radar. BACKGROUND

[0002] With the continuous development of automation technology, the application of automatic guided vehicle (AGV) in industrial, logistics and warehousing fields is becoming more and more widespread. AGV realizes material transportation, cargo distribution and other tasks through autonomous navigation and positioning. Precise positioning is one of the key technologies for AGV autonomous operation, which usually relies on multiple sensors for environment perception and positioning. Existing AGV positioning methods mainly include data fusion based on vision, laser radar, inertial measurement unit and other sensors. Laser radar is widely used in AGV positioning due to its high-precision distance measurement and stability, but its limitations in handling dynamic obstacles, transparent objects and other aspects make it challenging to use laser radar alone. Vision sensors (such as cameras, depth cameras) can provide rich environmental information such as object recognition, scene understanding and spatial structure features, but they are easily disturbed in complex environments such as low light or occlusion. Therefore, fusion technology based on vision and laser radar has become an effective solution for AGV precise positioning.

[0003] By combining the advantages of laser radar and vision sensors, the shortcomings of each are compensated. Laser radar provides accurate environmental depth information and can effectively identify obstacles in complex three-dimensional space; while vision sensors supplement the scene features that laser radar cannot obtain by acquiring environmental texture, color and object shape information. By fusing the data of both, AGV can achieve more robust and accurate positioning in different lighting conditions and environments. In addition, laser radar has an advantage in recognizing complex geometric shapes, while vision sensors can further improve the accuracy and stability of positioning through object recognition, marker detection and other methods. The fusion method not only improves the positioning accuracy of AGV, but also enhances its ability to adapt to complex environments, especially in dynamic environments and variable lighting conditions.

[0004] The existing AGV positioning method that fuses vision and laser radar has the following shortcomings:

[0005] In AGV precise positioning, a filtering algorithm is usually used to fuse data from different sensors. There are two key parameters in the filter - process noise covariance Q and observation noise covariance R, and the control of these two key points plays a crucial role in the precise positioning of vision and laser radar fusion.

[0006] When the AGV is in an unstable environment (e.g., slippery, sudden turns, etc.), Q needs to be set larger so that the system can tolerate larger state prediction errors. Conversely, if the AGV is driving in a stable environment, Q can be set smaller to improve the credibility of the motion model. Although the lidar and vision sensors provide real-time data for localization, in cases where the lidar does not have enough reflection information or the vision information is ambiguous, the system needs to rely on Q to maintain stable motion prediction.

[0007] For the vision sensor, R is related to the image quality, lighting conditions, and field-of-view occlusion. If the vision sensor is greatly affected by changes in lighting or occlusion, R should be increased, indicating that the vision data is less trustworthy. For the lidar, R is related to the sparsity of the point cloud, reflectivity of the surface, etc. In sparse or poorly reflective environments, R should be increased, indicating that the lidar observations may be unreliable. When the environmental lighting conditions are good and the vision data is reliable, R can be appropriately reduced, increasing the influence of the vision sensor.

[0008] When Q is set too large, the system's tolerance for prediction model uncertainty increases, and the filter will rely more on vision and lidar observation data for correction. In unstable environments or when there is noise in the vision and lidar, this can lead to excessive reliance on vision and lidar data, making it prone to positioning drift or mismatch;

[0009] When Q is set too small, the system's trust in the prediction model is too high, and the filter will ignore the influence of vision and lidar data, which can cause the positioning system to respond slowly when encountering sudden changes or large environmental changes, making it unable to correct the position in time.

[0010] When R is set too large, it indicates that the system has a high level of distrust in sensor data, and the filter will reduce its reliance on sensor observations. At this time, even if the sensor data is accurate, the system will have difficulty fully correcting the localization, leading to reduced localization accuracy.

[0011] When R is set too small, it indicates that the system has excessive trust in sensor data, which can lead to incorrect localization corrections in cases where sensor information is noisy or unreliable, resulting in unstable localization.

[0012] In different environments, the same Q and R have different effects on lidar and vision sensors. Lidar and vision sensors have different error characteristics and adaptability, lidar is more accurate in depth measurement and suitable for complex geometric environment, but it is easily limited when the dynamic changes are large; while vision sensors are good at providing environmental features, but are easily disturbed in low light or occlusion conditions. When Q and R use fixed default settings, these differences cannot be fully considered, which may lead to over-reliance on information from a certain sensor or ignore its potential errors, thereby affecting the positioning accuracy of the AGV.

[0013] The prior art has the following disadvantages:

[0014] The existing AGV positioning based on fusion of lidar and vision using filtering algorithm cannot fully distinguish the error characteristics of process noise covariance and observation noise covariance due to insufficient setting, thereby affecting the positioning accuracy and system stability.

[0015] To solve the above problems, the present application provides a solution. SUMMARY

[0016] In order to overcome the above-mentioned defects of the prior art, the embodiments of the present application provide an AGV precise positioning control method, device, equipment and medium for fusing vision and lidar, which dynamically generates and optimizes the best process noise covariance and observation noise covariance in different environments for controlling the fusion positioning of vision and lidar, so as to solve the problems of insufficient positioning accuracy and stability of AGV fusing vision and lidar.

[0017] To achieve the above object, the present application provides the following technical solutions:

[0018] The application discloses an AGV precise positioning control method combining vision and laser radar, and comprises the following steps: acquiring historical vision and laser radar data and corresponding actual AGV data in different environments, calculating deviations in different dimensions, and calculating mean square errors as first process noise covariance; acquiring precision data of the vision and laser radar sensors, and analyzing and setting first observation noise covariance; rotating the first observation noise covariance for several times to generate several second observation noise covariances; splitting the first process noise covariance into two-dimensional relations of a first number of groups, rotating the plane of each two-dimensional relation for several times, generating a first number of second process noise covariances for each rotation, and performing Cartesian product on the first number of second process noise covariances to obtain a third process noise covariance; randomly combining the several second observation noise covariances and the several third process noise covariances to generate a plurality of first filter control arrays, wherein the filter control array comprises an observation noise covariance and a process noise covariance; applying the plurality of first filter control arrays to AGV positioning based on a filter algorithm combining laser radar and vision, and performing multi-dimensional evaluation on each application, and setting the first filter control array with the best evaluation result as a second filter control array for AGV positioning.

[0019] In a preferred embodiment, the vision and laser radar data and the corresponding actual AGV data both comprise data in three dimensions of lateral position coordinates, longitudinal position coordinates and speed; the calculation of deviations in different dimensions and the calculation of mean square errors as first process noise covariance are specifically: time sequencing of the data, calculating the deviations of the lateral position coordinates, the longitudinal position coordinates and the speed at each time point, wherein the deviation is the difference between the historical vision and laser radar data and the actual AGV data in each dimension with a direction; calculating the mean square errors between the deviations in different times in each dimension; and arranging the mean square errors in three dimensions from top to bottom diagonally to construct a first matrix as the first process noise covariance.

[0020] In a preferred embodiment, the acquisition of the precision data of the vision and laser radar sensors and the analysis and setting of the first observation noise covariance are specifically: the precision data comprise a re-projection error of a vision device, a camera focal length and a standard working distance; the precision data further comprise an angle precision of a laser radar device and a standard measurement distance; based on a pinhole imaging model, the re-projection error is converted into a lateral position noise standard deviation, and the square is taken to obtain an observation noise variance of the vision device in the lateral direction; in the longitudinal direction, the angle precision is converted into a longitudinal position noise standard deviation, and the square is taken to obtain an observation noise variance of the laser radar device in the longitudinal direction; and the observation noise variances of the vision device in the lateral direction and the laser radar device in the longitudinal direction are arranged in a diagonal line to construct a second matrix as the first observation noise covariance.

[0021] In a preferred embodiment, the splitting of the first process noise covariance into a first number of two-dimensional relationships is specifically: selecting any two dimensions from the multi-dimensional first process noise covariance to construct a two-dimensional plane; rotating each two-dimensional plane by a plurality of different radian angles, specifically: setting a radian set, constructing a rotation matrix for each two-dimensional plane, the dimension of the rotation matrix being the same as the dimension of the first process noise covariance, and setting the non-zero elements outside the corresponding two-dimensional plane as 1; applying the rotation matrix to the first process noise covariance to generate a second process noise covariance.

[0022] In a preferred embodiment, the application of the plurality of first filter control arrays to the fusion of laser radar and vision based AGV positioning based on the filtering algorithm is specifically: obtaining the AGV state, the AGV state including position coordinates and orientation angle, and constructing a motion model in combination with the process noise covariance in the first filter control array; constructing an observation model in combination with the observation noise covariance in the first filter control array; and fusing the data of vision and laser radar through the motion model and the observation model through the filtering algorithm.

[0023] The application discloses an AGV accurate positioning control device fusing vision and laser radar, which comprises a first process noise covariance acquisition module, a first observation noise covariance acquisition module, a two-dimensional rotation module, a multi-dimensional rotation module, a control signal candidate module and a control module.

[0024] An electronic device comprises at least one processor and a memory connected with the at least one processor in communication; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the AGV accurate positioning control method fusing vision and laser radar.

[0025] A computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the AGV accurate positioning control method fusing vision and laser radar.

[0026] The application discloses an AGV accurate positioning control method, device, equipment and medium fusing vision and laser radar.

[0027] The application ensures that the setting of the first process noise covariance has true scene representativeness and reliability by accurately calculating the deviation and obtaining the mean square error of the historical vision and lidar data and the actual AGV data in different environments; and the first observation noise covariance is set based on the sensor accuracy data, and then diversified observation and process noise covariance combinations are generated through rotation transformation and multi-dimensional plane splitting, which can systematically cover different noise correlations and distribution characteristics, thereby batch generating several representative first filtering control arrays, providing a rich and effective candidate set for subsequent screening of the optimal combination, and finally significantly improving the adaptability and positioning accuracy of filtering fusion. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1 A flowchart of the AGV precise positioning control method of the application fusing vision and lidar is shown in the figure.

[0029] Figure 2 A structural diagram of the AGV precise positioning control device of the application fusing vision and lidar is shown in the figure. DETAILED DESCRIPTION

[0030] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the application.

[0031] Embodiment 1, Figure 1 The AGV precise positioning control method of the application fusing vision and lidar is given, including the following steps:

[0032] S1, historical vision and lidar data and corresponding actual AGV data in different environments are obtained, a plurality of different dimension deviations are calculated, and the mean square error is calculated as the first process noise covariance;

[0033] In this embodiment, the historical vision and lidar data and the corresponding actual AGV data both include data of three dimensions of lateral position coordinates, longitudinal position coordinates and speed;

[0034] In this embodiment, the plurality of different dimension deviations are calculated, and the mean square error is calculated as the first process noise covariance, specifically:

[0035] The data is time-sequenced, at each time point, the deviations of the lateral position coordinates, the longitudinal position coordinates and the speed are calculated, and the deviation is the difference value of each dimension of the historical vision and lidar data and the actual AGV data with direction.

[0036] Calculate the mean square error between the deviations of several different times of each dimension respectively;

[0037] Arrange the mean square errors of the three dimensions from top to bottom diagonally to construct the first matrix as the first process noise covariance.

[0038] It should be noted that the following is the feasible formula for obtaining the difference value of each dimension corresponding to the direction:

[0039] ;

[0040] ;

[0041] ;

[0042] In the formula, is the deviation of the horizontal position coordinate dimension; is the deviation of the longitudinal position coordinate dimension; is the deviation of the speed dimension; , , is the actual AGV data of each dimension; , , is the visual and laser radar data of each dimension.

[0043] It should be noted that the following is the feasible formula for calculating the mean square error:

[0044] ;

[0045] In the formula, N is the total number of time points after time series, and t is the time point label.

[0046] The other mean square errors are the same.

[0047] It should be noted that the following is the feasible expression of the first process noise covariance:

[0048] ;

[0049] In the formula, is the mean square error, and the subscript corresponds to different dimensions.

[0050] It should be noted that under different environmental conditions, specifically, different speeds, accelerations and obstacle densities.

[0051] It should be noted that data timing refers to the alignment of data in three dimensions of horizontal position coordinates, vertical position coordinates and speed on the same time axis to ensure that the data in each dimension has a corresponding relationship at the same time point, avoiding deviation and error caused by inconsistent time. In the timing process, the generated time points are equidistantly segmented to ensure the uniformity of the data, so as to better evaluate the process noise covariance.

[0052] It should be noted that when calculating the deviation, the deviation should not only be the calculation of the numerical difference, but also the directionality of the data in each dimension, that is, the directionality of the data error should be considered. For example, for position error, the deviation should be the directional difference between the actual position and the measured position, not just the numerical difference.

[0053] It should be noted that when calculating the process noise covariance, the independent error of each dimension should be evaluated one by one, and then these independent errors are combined into a covariance matrix. The construction of the covariance matrix requires that the MSE of each dimension be used as the diagonal element of the matrix, and in appropriate cases, the correlation between dimensions (if any) should be considered. If the error between each dimension is independent, the covariance matrix will be a diagonal matrix; if there is a correlation between the errors, non-diagonal elements can be filled as needed.

[0054] It should be noted that when calculating the mean square error, if there are outliers in the experimental data (such as measurement errors or abnormal points caused by extreme environmental changes), these data points should be identified and excluded to avoid affecting the calculation accuracy of the covariance matrix. Obvious measurement values can be removed by outlier detection techniques such as IQR method or Z-score method.

[0055] S2, obtain precision data of visual and lidar sensors, analyze and set a first observation noise covariance;

[0056] In this embodiment, the precision data of the visual and lidar sensors is obtained, analyzed and set to a first observation noise covariance, specifically:

[0057] The precision data includes the re-projection error of the visual device, the camera focal length and the standard working distance;

[0058] The precision data also includes the angle accuracy of the lidar device and the standard measurement distance;

[0059] Based on the pinhole imaging model, the re-projection error is converted into a horizontal position noise standard deviation, and squared to obtain the observation noise variance of the visual device in the horizontal direction;

[0060] In the vertical direction, the angle accuracy is converted into a vertical position noise standard deviation, and squared to obtain the observation noise variance of the lidar device in the vertical direction;

[0061] The observation noise variance of the visual device in the lateral direction and the observation noise variance of the laser radar device in the longitudinal direction are arranged diagonally to construct a second matrix as the first observation noise covariance.

[0062] It should be noted that the following is a calculation formula of the feasible lateral position noise standard deviation:

[0063] In the formula, the lateral position noise standard deviation is the standard working distance is the camera focal length is the re-projection error is.

[0064] It should be noted that the following is a calculation formula of the feasible longitudinal position noise standard deviation:

[0065] In the formula, the longitudinal position noise standard deviation is the standard measurement distance is the angle accuracy is.

[0066] It should be noted that the following is an expression form of the feasible first observation noise covariance:

[0067] ;

[0068] It should be noted that the accuracy data is obtained from device manuals and manufacturers.

[0069] It should be noted that in many AGV fusion positioning applications, the accuracy advantages of different sensors in different directions are not the same. The visual system is usually more accurate in the horizontal direction (x direction) because it relies on the horizontal distribution of image features to determine the position, but the depth direction accuracy is limited (such as affected by light and shielding); the laser radar has higher accuracy in the longitudinal direction (y direction or depth direction) because it directly measures the distance and has small longitudinal error, but the lateral position needs to rely on the scanning angle calculation, which is relatively less accurate than the longitudinal direction; the present application constructs the observation noise covariance matrix, uses the main observation accuracy of the visual system in the x direction, and uses the main observation accuracy of the laser radar in the y direction, so that the filter trusts the data of each sensor in the direction that it is good at and weakens the influence in the direction that it is not good at.

[0070] It should be noted that in AGV positioning, x and y are two plane directions in the world coordinate system, x is the lateral direction, y is the longitudinal driving direction, and the visual in the x direction means that the visual is more reliable in the lateral position in the current coordinate system; the laser radar in the y direction is the same.

[0071] S3, rotating the first observation noise covariance by several different radian angles to generate several second observation noise covariances.

[0072] In the present example, the first observation noise covariance is rotated by several different radian angles to generate several second observation noise covariances, specifically:

[0073] A set of radian angles is set, for each radian angle in the set of radian angles, a two-dimensional rotation matrix is constructed;

[0074] Each two-dimensional rotation matrix is applied to the first observation noise covariance to generate several second observation noise covariances.

[0075] It should be noted that the set of radian angles is a set of slight rotation radian angles within a predefined range;

[0076] It should be noted that the following is a feasible two-dimensional rotation matrix of the observation noise covariance:

[0077] wherein, is the radian angle.

[0078] It should be noted that the following is a feasible formula for applying each two-dimensional rotation matrix to the first observation noise covariance:

[0079] wherein, is the second observation noise covariance corresponding to .

[0080] The following is an example of the specific calculation process of the above .

[0081] Let , , , , then

[0082] ; ; ; and the calculation results of are the same.

[0083] It should be noted that the purpose of two-dimensional rotation is to simulate slight alignment errors or sensor calibration deviations between the observation coordinate system and the filter, and to introduce controlled off-diagonal terms (correlations) in R through small-angle rotation, so as to evaluate and enhance the robustness of the fusion algorithm to sensor alignment errors;

[0084] It should be noted that the candidate observation noise covariance generated for each rotation angle is obtained by similarity transformation, which maintains the symmetry and positive definiteness of the matrix and keeps the trace of the matrix unchanged, facilitating subsequent comparison with the initial energy distribution.

[0085] It should be noted that the value of the radian is determined in combination with the calibration accuracy and installation tolerance of the specific sensor: if the typical installation error of the camera and the radar is known, the radian set covers the interval to reflect the real working condition.

[0086] It should be noted that the sampling strategy of the radian set is not fixed, and equal interval sampling, symmetric sampling or random sampling based on Latin hypercube can be used. If the goal is to cover typical error situations, symmetric equal interval is mainly used, and if higher dimensional randomization is required, random sampling is used.

[0087] S4, the first process noise covariance is split into a first number of two-dimensional relationships, each two-dimensional relationship is rotated by a different radian several times, each radian generates a first number of second process noise covariances, and a Cartesian product of the first number of second process noise covariances is obtained. A third process noise covariance.

[0088] In this embodiment, the first process noise covariance is split into a first number of two-dimensional relationships, specifically:

[0089] The multi-dimensional arbitrary two-dimensional of the first process noise covariance is selected and a two-dimensional plane is constructed.

[0090] In this embodiment, each two-dimensional relationship plane is rotated by a different radian several times, specifically:

[0091] A radian set is set, a rotation matrix is constructed for each two-dimensional plane, the dimension of the rotation matrix is the same as the dimension of the first process noise covariance, and the non-zero elements outside the corresponding two-dimensional plane are set to 1.

[0092] The rotation matrix is applied to the first process noise covariance to generate a second process noise covariance defined by the radian set.

[0093] It should be noted that the first number is the number of different two-dimensional planes that can be split in the first process noise covariance, for example, the first number in this embodiment is 3.

[0094] It should be noted that the following is a feasible rotation matrix of the x and y two-dimensional plane:

[0095] , The rotation radian of the x and y two-dimensional plane.

[0096] The following is the second process noise covariance obtained by rotating the first process noise covariance using the feasible rotation matrix of the x and y two-dimensional plane:

[0097] .

[0098] It should be noted that the following is the rotation matrix of the feasible x and v two-dimensional plane:

[0099] , The rotation radian of the x and v two-dimensional plane.

[0100] The following is the second process noise covariance obtained by rotating the first process noise covariance using the feasible rotation matrix of the x and v two-dimensional plane:

[0101] .

[0102] It should be noted that the following is the rotation matrix of the feasible y and v two-dimensional plane:

[0103] , The rotation radian of the y and v two-dimensional plane.

[0104] It should be noted that any of the above orthogonal similarity transformations maintains the symmetry and positive definiteness of Q and R, maintains the eigenvalues and trace, and only changes the size of the related items under the coordinate axis (i.e. non-diagonal elements).

[0105] The following is the calculation formula of the third process noise covariance:

[0106] .

[0107] It should be noted that the purpose of decomposing the first process noise covariance into several two-dimensional relationships is to separately introduce controlled correlations between different state quantities, rather than introducing global coupling in the entire matrix at once, so that the influence of noise interaction between two state quantities on the filtering performance can be more accurately analyzed and controlled.

[0108] It should be noted that the number of two-dimensional relationships is equal to the number of different state pairs that can be combined in the process noise covariance matrix, for example, when Q is a three-dimensional state (lateral position, longitudinal position, speed), three two-dimensional planes can be combined (lateral-longitudinal, lateral-speed, longitudinal-speed), i.e. the first number is 3.

[0109] It should be noted that when constructing a rotation matrix for each two-dimensional plane, the dimension of the rotation matrix must be the same as the original Q, and the unit matrix structure (non-zero elements are 1) must be maintained in the unrotated dimension, to ensure that the rotation only acts on the target two-dimensional subspace and does not affect the noise distribution of other dimensions.

[0110] It should be noted that in numerical implementation, positive definite check (minimum eigenvalue greater than a set threshold) and symmetry verification should be performed for each generated Q, and a small regularization term should be added if necessary to prevent numerical instability caused by rotation or combination.

[0111] It should be noted that in practical application, the number of third process noise covariances generated by Cartesian product may increase exponentially, and the size of the angle set will be set according to actual computing resources and analysis targets to avoid excessive computing overhead or sample redundancy.

[0112] S5, randomly combining a plurality of second observation noise covariances and a plurality of third process noise covariances to generate a plurality of groups of first filter control arrays, the filter control array comprising an observation noise covariance and a process noise covariance.

[0113] S6, applying a plurality of groups of first filter control arrays to fusion laser radar and vision based AGV positioning based on a filtering algorithm respectively, and performing multi-dimensional evaluation on each application, and setting the first filter control array with the best evaluation result as the second filter control array for AGV positioning.

[0114] In this embodiment, the application of a plurality of groups of first filter control arrays to fusion laser radar and vision based AGV positioning based on a filtering algorithm is specifically:

[0115] Obtaining an AGV state, the AGV state comprising a position coordinate and an orientation angle, and constructing a motion model in combination with a process noise covariance in the first filter control array;

[0116] The AGV state combines the observation noise covariance in the first filter control array to construct an observation model;

[0117] The data of vision and laser radar are fused through the motion model and the observation model through the filtering algorithm.

[0118] It should be noted that the following is a feasible motion model:

[0119] In the formula, A is a state transition matrix; B is a control input matrix; is a control input (such as speed, steering angle); is process noise, which follows a normal distribution with a mean of zero, , is the AGV state, and k is the discrete time step index.

[0120] It should be noted that the following is a feasible observation model:

[0121] In the formula, is an observation matrix, describing the relationship between the state and the observation; is an observation noise, following a normal distribution with mean zero, .

[0122] It should be noted that the motion model predicts the state at the next time according to the state at the last time and the control input; the observation model is used to describe the relationship between the sensor measurement and the true state, and the difference (residual) between the measurement and the predicted value is used to correct the predicted state to obtain the updated state.

[0123] It should be noted that the construction method of the state transition matrix A can be obtained according to the kinematic model of the AGV (such as the differential drive model or the Ackermann steering model).

[0124] It should be noted that the determination of the observation matrix depends on the observation type of the sensor and the definition of the state vector, for example, visual measurement can directly provide the lateral and longitudinal coordinates of the AGV, and laser radar measurement can provide the longitudinal position or the distance to the obstacle.

[0125] It should be noted that the data fusion process based on the above motion model and observation model is realized by using a filtering algorithm (such as extended Kalman filter EKF, unscented Kalman filter UKF), and the implementation process (prediction-update cycle) of this kind of filtering algorithm belongs to mature prior art, which will not be repeated here.

[0126] It should be noted that in the fusion process, the data of vision and laser radar are mapped to the same state space through the observation model, and the weighted fusion is completed by combining the predicted state and the covariance matrix, and the fusion weight is automatically calculated by the filtering algorithm, which ensures the optimal estimation under different sensor accuracy conditions.

[0127] It should be noted that when performing multi-dimensional evaluation on each group, the positioning accuracy, trajectory smoothness, convergence speed and other indicators can be combined for comprehensive scoring, and the evaluation indicators and calculation methods can be set according to application requirements, not limited to a single evaluation standard, and not the focus of the present application, so it is not limited.

[0128] The present application can adaptively select the optimal filtering control parameter in actual AGV positioning by filtering fusion and multi-dimensional evaluation of different candidate process noise covariances and observation noise covariances, thereby improving the positioning accuracy and stability; at the same time, the complementary observation characteristics of vision and laser radar are utilized, and the fusion weight is automatically calculated by the filtering algorithm, which effectively suppresses the influence of single sensor error on the positioning result, improves the robustness and convergence speed in different environmental conditions, and ensures the reliable operation of AGV in complex scenes.

[0129] Example 2, Figure 2The application provides an AGV accurate positioning control device fusing vision and laser radar, which comprises a first process noise covariance acquisition module, a first observation noise covariance acquisition module, a two-dimensional rotation module, a multi-dimensional rotation module, a control signal candidate module and a control module.The first process noise covariance acquisition module is used for acquiring historical vision and laser radar data and corresponding actual AGV data in different environments, calculating deviations in different dimensions, and calculating mean square errors as the first process noise covariance.The first observation noise covariance acquisition module is used for acquiring precision data of the vision and laser radar sensor, and analyzing and setting the first observation noise covariance.The two-dimensional rotation module is used for rotating the first observation noise covariance for several times to generate several second observation noise covariances.The multi-dimensional rotation module is used for splitting the first process noise covariance into a first number of two-dimensional relations, rotating the plane of each two-dimensional relation for several times, generating a first number of second process noise covariances for each radian, and performing Cartesian product on the first number of second process noise covariances to obtain a third process noise covariance.The control signal candidate module is used for randomly combining the several second observation noise covariances and the several third process noise covariances to generate a first number of first filter control arrays, wherein the filter control array comprises an observation noise covariance and a process noise covariance.The control module is used for applying the first number of first filter control arrays to AGV positioning based on the fusion of laser radar and vision based on a filtering algorithm, performing multi-dimensional evaluation on each application, setting the first filter control array with the best evaluation result as a second filter control array, and using the second filter control array for AGV positioning.

[0130] The application further comprises an electronic device, which comprises:

[0131] at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the AGV accurate positioning control method fusing vision and laser radar.

[0132] The application further comprises a computer readable storage medium storing a computer program, and the computer program is executed by a processor to implement the AGV accurate positioning control method fusing vision and laser radar.

Claims

1. A precise positioning control method for AGVs integrating vision and lidar, characterized in that, include: Acquire historical visual and lidar data and corresponding actual AGV data under several different environments, calculate the deviations in several different dimensions, and calculate the mean square error as the first process noise covariance. Acquire accuracy data from visual and lidar sensors, and analyze and set the first observation noise covariance; The first observation noise covariance is rotated several times by different radians to generate several second observation noise covariances; The first process noise covariance is decomposed into a first number of two-dimensional relationships. The plane of each two-dimensional relationship is rotated several times with different radians. Each radian generates a first number of second process noise covariances. A third process noise covariance is obtained by performing a Cartesian product on the first number of second process noise covariances. Several second observation noise covariances and several third process noise covariances are randomly combined to generate several sets of first filter control arrays, wherein the filter control array includes an observation noise covariance and a process noise covariance; Several sets of first filter control arrays are applied to AGV positioning based on the fusion of LiDAR and vision using filtering algorithms. Each set of applications is evaluated in multiple dimensions. The first filter control array with the best evaluation result is set as the second filter control array for AGV positioning.

2. The AGV precise positioning control method integrating vision and lidar according to claim 1, characterized in that, The visual and lidar data, as well as the corresponding actual AGV data, both include data in three dimensions: lateral position coordinates, longitudinal position coordinates, and speed. The calculation of deviations in several different dimensions, and the calculation of the mean square error as the first process noise covariance, are specifically as follows: The data is time-series analyzed, and at each time point, the deviations of the horizontal position coordinates, vertical position coordinates, and speed are calculated. The deviations are the directional differences between the historical visual and lidar data and the actual AGV data in each dimension. Calculate the mean square error between the deviations at several different times for each dimension; The mean squared errors of the three dimensions are arranged diagonally from top to bottom to construct the first matrix, which serves as the first process noise covariance.

3. The AGV precise positioning control method integrating vision and lidar according to claim 2, characterized in that, The acquisition of accuracy data from visual and lidar sensors, and the analysis and setting of the first observation noise covariance, specifically involves: The accuracy data includes the reprojection error of the vision device, the camera focal length, and the standard working distance. The accuracy data also includes the angular accuracy of the lidar equipment and the standard measurement distance; Based on the pinhole imaging model, the reprojection error is converted into the standard deviation of lateral position noise, and the square is calculated to obtain the observation noise variance of the vision device in the lateral direction. In the longitudinal direction, the angular accuracy is converted into the longitudinal position noise standard deviation, and the square is calculated to obtain the observation noise variance of the lidar device in the longitudinal direction; The observation noise variance of the vision device in the horizontal direction and the observation noise variance of the lidar device in the vertical direction are arranged diagonally to construct a second matrix as the first observation noise covariance.

4. The AGV precise positioning control method integrating vision and lidar according to claim 3, characterized in that, The decomposition of the noise covariance of the first process into a two-dimensional relationship of a first set of quantities is as follows: Arbitrarily select two dimensions from the multidimensional noise covariance of the first process to construct a two-dimensional plane; For each set of two-dimensional relationships, the plane is rotated several times by different degrees, specifically as follows: Define a set of radians, construct a rotation matrix for each two-dimensional plane, the dimension of the rotation matrix is ​​the same as the dimension of the noise covariance of the first process, and set the non-zero elements outside the corresponding two-dimensional plane to 1; The rotation matrix is ​​applied to the first process noise covariance to perform rotations limited by the radian set, thereby generating the second process noise covariance.

5. The AGV precise positioning control method integrating vision and lidar according to claim 4, characterized in that, The step of applying several sets of first filter control arrays to AGV positioning based on the fusion of LiDAR and vision using a filtering algorithm is as follows: The AGV status is obtained, including position coordinates and orientation angle, and a motion model is constructed by combining the process noise covariance in the first filter control array. The AGV status is combined with the observation noise covariance in the first filter control array to construct an observation model; The data from vision and lidar are fused using a filtering algorithm through motion and observation models.

6. An apparatus for using the AGV precise positioning control method fusion vision and lidar as described in any one of claims 1-5, characterized in that, It includes a first process noise covariance acquisition module, a first observation noise covariance acquisition module, a two-dimensional rotation module, a multi-dimensional rotation module, a control signal candidate module, and a control module; The first process noise covariance acquisition module is used to acquire historical visual and lidar data and corresponding actual AGV data under several different environments, calculate the deviation in several different dimensions, and calculate the mean square error as the first process noise covariance. The first observation noise covariance acquisition module is used to acquire the accuracy data of the vision and lidar sensors and analyze and set the first observation noise covariance. A two-dimensional rotation module is used to rotate the first observation noise covariance by several different radians to generate several second observation noise covariances. The multidimensional rotation module is used to decompose the first process noise covariance into a first number of two-dimensional relationships, rotate the plane of each two-dimensional relationship several times with different radians, generate a first number of second process noise covariances with each radian, and perform a Cartesian product on the first number of second process noise covariances to obtain a third process noise covariance. The control signal candidate module is used to randomly combine several second observation noise covariances and several third process noise covariances to generate several sets of first filter control arrays, wherein the filter control array includes an observation noise covariance and a process noise covariance; The control module is used to apply several sets of first filter control arrays to AGV positioning based on the fusion of LiDAR and vision using filtering algorithms, and to perform multi-dimensional evaluation on each application. The first filter control array with the best evaluation result is set as the second filter control array for AGV positioning.

7. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the AGV precise positioning control method that integrates vision and lidar as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the AGV precise positioning control method that integrates vision and lidar as described in any one of claims 1 to 5.

Images