A deep perception method for unmanned engineering vehicle in complex operation environment

Abstract

The application discloses a kind of complex operation environment under unmanned engineering vehicle's deep perception method, it is related to vehicle automatic driving field, its steps include:1 using laser radar scans operation environment, establishes work scene two-dimensional map;2 using vehicle-mounted industrial camera shoots the road in front of vehicle, in combination with machine vision algorithm identifies road type;3 using vehicle-mounted sensor obtains vehicle dynamics parameter, input is established vehicle dynamics model, carries out parameter identification;4 the machine vision pavement recognition result obtained with dynamics identification result is fused, determines the road type in front.This application can realize engineering vehicle in complex operation environment including the map establishment of road surface information, and realizes the deep perception to environment, to lay the foundation for automatic operation and automatic driving.

Description

Technical Field

[0001] This invention relates to the field of unmanned driving and automated operation technology for engineering vehicles in the context of automotive intelligence, and specifically to a deep perception method for unmanned engineering vehicles in complex operating environments. Background Technology

[0002] Engineering vehicles, including excavators, loaders, and forklifts, are essential operating equipment widely used in industry, agriculture, construction, water conservancy, and logistics, playing a vital role in national socio-economic development and national defense. With the widespread application of engineering vehicles across various fields, their operating environments are becoming increasingly complex and diverse, bringing higher safety risks and operational challenges. Traditional manual driving methods clearly have limitations in these complex operating environments, as subjective human factors can influence the driver's judgment and decision-making, leading to accidents. The introduction of autonomous driving technology offers a new approach to solving these problems.

[0003] Building on the successful unmanned operation of Automated Guided Vehicles (AGVs) indoors, extending this concept to complex outdoor environments, such as mines and construction sites, becomes particularly crucial. These outdoor environments not only contain more unknown obstacles and terrain variations but are also affected by different road surface conditions, increasing the difficulty of perception and decision-making. Therefore, an innovative approach is needed to enhance the autonomy and adaptability of engineering vehicles in these environments.

[0004] The main challenges currently faced by unmanned engineering vehicles in complex outdoor environments include:

[0005] ① Environmental perception dimension: Unlike road vehicles, engineering vehicles often travel on unpaved roads. Traditional autonomous vehicle environmental perception algorithms do not consider the adhesion conditions between the wheels and the road surface, as well as the impact of road surface type on the lateral / longitudinal stability of the vehicle.

[0006] ② Decision-making ability in the face of sudden environmental changes: Changes in the outdoor environment are more unpredictable, such as gradual or sudden changes in road conditions, requiring vehicles to have the ability to adapt and make decisions quickly in order to cope with changes.

[0007] In summary, existing technologies have a technical problem in accurately perceiving road conditions for unmanned engineering vehicles in complex terrain environments. Summary of the Invention

[0008] This invention addresses the shortcomings of existing technologies by proposing a method for deep environmental perception of unmanned engineering vehicles in complex operating environments. The method aims to deeply perceive different road surface types and soil conditions in the operating environment, thereby achieving unmanned driving that balances the operational efficiency and stability of engineering vehicles.

[0009] To achieve the above-mentioned objectives, the present invention adopts the following technical solution:

[0010] The depth perception method for unmanned engineering vehicles in complex working environments, as described in this invention, is characterized by the following steps:

[0011] S1. Install intelligent sensors on engineering vehicles for working environment perception, including: installing lidar on the top of the engineering vehicle, installing inertial measurement unit in the cab, installing wheel speed sensors on the wheels, and installing industrial cameras in front of the wheels.

[0012] S2. Collect image datasets of different types of soil pavement and label the soil types of the pavement to obtain labeled image datasets, which are used to train the road recognition model.

[0013] S3. Using equation (1), establish the kinematic equations of the engineering vehicle:

[0014]

[0015] In equation (1), X b =[x f ,y f ,θ f ,δ] T For the front axle pose of the engineering vehicle, x f and y f Here are the coordinates of the center of the front axle of the engineering vehicle. The velocity of the center of the front axle of the engineering vehicle along the x-axis. Let θ be the velocity of the center of the front axle of the engineering vehicle along the y-axis. f This refers to the heading angle of the front axle of the engineering vehicle. Let δ be the angular velocity of the front axle of the engineering vehicle, and δ be the turning bend angle of the engineering vehicle. Indicates the turning angular velocity; l r and l f These are the front and rear overhangs of the engineering vehicle, respectively, and v is the speed of the engineering vehicle.

[0016] Define augmented state vector In the formula, n, Let X be the soil subsidence index and the angle of internal friction. Then the differential of X is denoted as... right After discretization, the state estimation model of the unscented Kalman filter is obtained;

[0017] S4. The trigonometric method is used to simplify the dynamic model of wheel-ground interaction, and an analytical model of wheel-ground interaction estimated in real time is obtained, which is used as the observation model for unscented Kalman filtering.

[0018] S5. Use the 2D SLAM mapping method based on the Cartographer algorithm to build a two-dimensional map of the working environment of the engineering vehicle.

[0019] S6. Based on the two-dimensional map, perform AMCL positioning on the engineering vehicle to determine the road surface position (x,y) of the engineering vehicle on the two-dimensional map at time t.

[0020] S7. Use the industrial camera to capture the road surface image in front of the engineering vehicle at time t in the working environment, and input the road surface image in front at time t into the trained road recognition model for recognition, to obtain the road surface soil type and the confidence probability p(t) of the recognition result in the road surface image in front at time t.

[0021] S8. Obtain the soil mechanical parameters at time t: the settlement index n(x,y) and the internal friction angle at the road surface position (x,t) of the engineering vehicle. The estimated value:

[0022] S8.1. The wheel speed signal of the engineering vehicle collected by the wheel speed sensor is used to calculate the vehicle speed v(t) at time t; and the heading angle θ of the engineering vehicle at time t is calculated using the magnetometer signal of the engineering vehicle collected by the inertial measurement unit. f (t);

[0023] S8.2. The vehicle speed v(t), road surface position (x, y), and heading angle θ at time t are given. f The bending angle δ(t) read by the vehicle-mounted sensor is input into the state estimation model to obtain the estimated value of the soil mechanical parameter subsidence index n(x,y) and internal friction angle at time t. The estimated value;

[0024] S9. Obtain the observed values ​​of the soil mechanical parameters, settlement index n(x,y), and internal friction angle, at time t when the vehicle is at its road surface position (x,y). Observations:

[0025] S9.1. Based on the acceleration signal of the engineering vehicle at time t collected by the inertial measurement unit, obtain the acceleration a(t) of the front and rear frames of the engineering vehicle at time t, and calculate the vertical force F of the wheels of the engineering vehicle at time t according to the formula for calculating the vertical force of the wheels of articulated vehicles. z (t);

[0026] S9.2. Based on the wheel speed signal and the power signal of the engineering vehicle, calculate the wheel torque T(t) of the engineering vehicle at time t using the motor torque-power relationship, and calculate the longitudinal force F of the wheel according to the driving torque-driving force relationship. x (t);

[0027] S9.3 Input the wheel vertical force, longitudinal force, and torque obtained at time t into the observation model of the unscented Kalman filter, and obtain the observed values ​​of the soil mechanical parameters, settlement index n(x,y), and internal friction angle at time t. Observed values;

[0028] S10. Using the unscented Kalman filter algorithm, estimate and observe the soil mechanical parameters at time t, including the subsidence index n(x,y) and the internal friction angle. The estimated and observed values ​​are processed to obtain the filtered value of the soil mechanical parameters, the subsidence index n(x,y), and the internal friction angle at time t. The filtered value is used as the road surface type estimation result based on vehicle dynamics;

[0029] S11. The road surface soil type and the confidence probability p(t) of the identification result in the road surface image at time t are fused with the road surface type estimation result to obtain the final road surface type identification result of the engineering vehicle at the road surface position (x,y) at time t, so as to realize the perception of the working environment of the engineering vehicle at the road surface position (x,y) at time t.

[0030] The depth perception method for unmanned engineering vehicles in complex working environments described in this invention is also characterized in that step S4 includes:

[0031] S4.1 Constructing a wheel-land interaction model using equations (3)-(5):

[0032]

[0033]

[0034]

[0035] In equations (3)-(5), F x For the longitudinal force of the wheels of the engineering vehicle, F z τ is the vertical force of the engineering vehicle's wheel, T is the driving torque of the engineering vehicle's wheel, θ is the contact angle between the engineering vehicle's wheel and the road surface, θ1 is the wheel's approach angle, θ2 is the wheel's departure angle, τ is the tangential stress along the soil, σ is the normal stress of the soil, r is the wheel radius of the engineering vehicle, and b is the wheel width of the engineering vehicle.

[0036] S4.2. Define the fitting function using equations (6) and (7):

[0037]

[0038]

[0039] In equations (6) and (7), σ1(θ) and σ2(θ) represent the normal stresses in different angular ranges, and τ1(θ) and τ2(θ) represent the tangential stresses in different angular ranges. Let represent the three undetermined coefficients for calculating σ1(θ). Let represent the three undetermined coefficients for calculating σ²(θ). Let represent the three undetermined coefficients for calculating τ1(θ). These represent the three undetermined coefficients for calculating τ2(θ); The maximum normal stress angle;

[0040] S4.3 Calculate the wheel entry angle θ using equations (7) and (8). 1m and the departure angle θ 2m :

[0041]

[0042]

[0043] S4.4 Calculate the coefficients in the fitted function using equations (9) and (10):

[0044]

[0045]

[0046] In equations (9) and (10), This represents the maximum normal stress and the intermediate approach angle θ. 1m The proportionality coefficient of the normal stress at that point, and Represents the maximum normal stress and the intermediate departure angle θ 2m The proportionality coefficient of the normal stress at that point, and This represents the proportionality coefficient between the maximum tangential stress and the tangential stress at the intermediate approach angle, and This represents the proportionality coefficient between the maximum normal stress and the tangential stress at the intermediate departure angle, and σ m The contact angle is θ m The normal stress value at that time, σ 1m The contact angle is θ 1m The normal stress value at time τm The contact angle is θ m The normal stress value at time τ 1m The contact angle is θ 1m The normal stress value at that time.

[0047] S4.5, The coefficients calculated in S4.4 and Substituting these values ​​into the fitting function, we obtain a fitting function with definite coefficients:

[0048]

[0049]

[0050]

[0051]

[0052] S4.6. Obtain the wheel force vector using equation (16).

[0053]

[0054] In equation (16), I(θ) represents the wheel-ground contact angle matrix, and X represents the soil stress vector, and σ represents the normal stress of the soil, including σ1(θ) and σ2(θ); τ represents the tangential stress of the soil, including τ1(θ) and τ2(θ).

[0055] S4.7 Substituting equations (12)-(15) into equation (16), we obtain the analytical model of wheel-land interaction based on triangular approximation simplification using equation (17):

[0056] F = rb(F1 - F2)X m (17)

[0057] In equation (11), X m Denotes the maximum stress vector, and X m =[σ m ,τ m ] T Where F1 represents the first force calculation matrix, F2 represents the second force calculation matrix, and we have:

[0058] F1=[I(θ m )K 11 (θ σ1 ,θ τ1 )-I(θ1)]K 21 (θ σ1 ,θ τ1(18)

[0059] F2=[I(θ m )K 12 (θ σ2 ,θ τ2 )-I(θ2)]K 22 (θ σ2 ,θ τ2 (19)

[0060] In formula (12), I(θ m ) indicates that the contact angle is θ m Let I(θ1) represent the wheel-ground contact angle matrix when the contact angle is the entry angle θ1, and I(θ2) represent the wheel-ground contact angle matrix when the contact angle is the departure angle θ2. p1 Indicates the entry characteristic angle, θ p2 K represents the departure characteristic angle. 11 K 12 K represents two first stress characteristic matrices. 21 K 22 Let represent two second stress characteristic matrices, and we have:

[0061] θ σ1 =2arccosλ σ1 (20)

[0062] θ τ1 =2arccosλ τ1 (twenty one)

[0063] θ σ2 =2arccosλ σ2 (twenty two)

[0064] θ τ2 =2arccosλ τ2 (twenty three)

[0065]

[0066]

[0067]

[0068]

[0069] S4.8. Using equation (16), the simplified wheel driving torque T of the engineering vehicle is obtained:

[0070] T = r 2 b(T D1 -T D2 )τ m (28)

[0071] In equation (16), T D1 T D2 This represents two torque calculation matrices, and

[0072] The fusion in step 11 is performed as follows:

[0073] Step S11.1: Set a confidence probability threshold p std ;

[0074] Step S11.2: Based on the soil mechanical parameter comparison table for typical road surfaces, look up the range of soil mechanical parameters corresponding to the road surface soil type in the road surface image at time t obtained in step S7; and determine the settlement index n(x,y) and internal friction angle of the soil mechanical parameters obtained in step S10. If the filtered value is within the range of the queried soil mechanical parameters, then the soil type identification result of the vehicle at position (x,y) at time t obtained in step S7 is correct, and it is set as the final road surface type identification result of the engineering vehicle at position (x,y) at time t; otherwise, proceed to step S11.3.

[0075] Step S11.3: Combine the confidence probability p(t) corresponding to the soil type of the vehicle at position (x,y) at time t obtained in step S7 with the confidence probability threshold p std Compare, if p(t) is lower than p std If the soil type at time t identified in step S7 is incorrect, then step S10.4 is executed; otherwise, step S11.5 is executed.

[0076] Step S11.4: Based on the soil mechanical parameters (settlement index n(x,y) and internal friction angle) of the engineering vehicle at the road surface position (x,y) at time t estimated in step S10, The filtered value is used to look up the soil mechanical parameters of typical road surfaces in the table to determine the soil type of the engineering vehicle at the road surface position (x,y) at time t, and this is used as the final road surface type identification result.

[0077] Step S11.5: Using the probability density function truncation method, the soil mechanical parameters corresponding to the soil type at the road surface position (x,y) of the engineering vehicle at time t identified in step S7, and the soil mechanical parameters at the road surface position (x,y) of the engineering vehicle at time t estimated in step S10, namely the settlement index n(x,y) and the internal friction angle, are analyzed. The filtered values ​​are fused together, and the fusion result is used as the final road surface type identification result of the engineering vehicle at the road surface position (x,y) at time t.

[0078] The present invention provides an electronic device, including a memory and a processor, wherein the memory is used to store a program that supports the processor in executing the depth sensing method, and the processor is configured to execute the program stored in the memory.

[0079] The present invention discloses a computer-readable storage medium on which a computer program is stored, wherein the computer program is executed by a processor to perform the steps of the depth perception method.

[0080] Compared with existing technologies, the beneficial effects of this invention are reflected in:

[0081] 1. In the environmental perception stage of unmanned driving technology, this invention, based on the characteristics of engineering vehicles often working on unpaved roads such as mud, loam, and clay and the complex working environment, not only accurately detects the location of obstacles, but also surveys the soil and road surface types in the working environment, so that relevant technicians can determine the vehicle's maximum lateral acceleration, longitudinal acceleration and other control indicators according to the road surface type, thus realizing the deep environmental perception of unmanned engineering vehicles in complex working environments.

[0082] 2. This invention simplifies the traditional wheel-ground interaction model (Bekker, Wong, Janosi model) commonly used in vehicle engineering based on the trigonometric method, effectively avoiding the complex integral calculations of the original model. While ensuring the accuracy of the analysis, the calculation speed is greatly enhanced, making it possible to use the vehicle to estimate the road surface type in real time.

[0083] 3. This invention deeply integrates the soil mechanical parameter identification method based on machine vision and vehicle dynamics, which solves the shortcomings of the former in terms of poor robustness and low accuracy, and the latter in terms of poor universality and reliance on models. It combines the advantages of the former in terms of early prediction and the latter in terms of high reliability, so as to achieve accurate and fast identification of road surface types. Attached Figure Description

[0084] Figure 1 This is a flowchart of the overall algorithm of the present invention;

[0085] Figure 2 This is a kinematic model diagram of an engineering vehicle using an off-road forklift as an example, according to the present invention.

[0086] Figure 3 This is a diagram illustrating the interaction between the wheel and the soft road surface.

[0087] Figure 4 This is a flowchart of the machine vision and dynamics data fusion process of the present invention. Detailed Implementation

[0088] In this embodiment, a depth perception method for unmanned engineering vehicles operating in complex environments is described, see [reference needed]. Figure 1 It is done by following these steps:

[0089] S1. Before starting the work environment perception, install intelligent sensors on the engineering vehicle: install a lidar on the top of the engineering vehicle, an inertial measurement unit in the cab, wheel speed sensors on the wheels, and an industrial camera in front of the vehicle.

[0090] In practice, an electric off-road forklift is selected as the vehicle, and a lidar is installed on the top of the forklift frame, an industrial camera is installed on the fork mast, and an inertial measurement unit is fixedly installed on the cab seat. All the installed smart sensors are connected to the embedded computing platform (Nvidia Jetson Xavier NX, etc.) in the cab.

[0091] S2. Collect image datasets of different types of soil pavement and label the soil types of the pavement to obtain labeled image datasets, which are used to train the road recognition model.

[0092] In practice, soft soil pavements are categorized into three types: clay, loam, and sand. A large amount of image data is acquired through on-site photography and divided into training, validation, and test sets. Each image is labeled with the correct pavement type. After preprocessing, the images are input into a CNN model. The CNN model is trained using the training set, a loss function and optimizer are defined, and the model weights are iteratively adjusted to minimize the loss.

[0093] S3. Using equation (1), establish the kinematic equations of the engineering vehicle:

[0094]

[0095] In equation (1), X b =[x f ,y f ,θ f ,δ] T For the front axle pose of the engineering vehicle, x f and y f Here are the coordinates of the center of the front axle of the engineering vehicle. The velocity of the center of the front axle of the engineering vehicle along the x-axis. Let θ be the velocity of the center of the front axle of the engineering vehicle along the y-axis. f This refers to the heading angle of the front axle of the engineering vehicle. Let δ be the angular velocity of the front axle of the engineering vehicle, and δ be the turning bend angle of the engineering vehicle. Indicates the turning angular velocity; l r and l f These are the front and rear overhangs of the engineering vehicle, respectively, and v is the speed of the engineering vehicle.

[0096] Define augmented state vector In the formula, n, Let X be the soil subsidence index and the angle of internal friction. Then the differential of X is denoted as... right After discretization, the state estimation model of the unscented Kalman filter is obtained;

[0097] In specific implementation, establish such as Figure 2 The kinematic model of the off-road forklift shown.

[0098] Define the vehicle speed v as v = v f Then the coordinates of the midpoint O of the front bridge are... f for:

[0099]

[0100] Where, θ f This is the heading angle of the front axle. According to... Figure 2 Equation (1) can be obtained from the geometric relationships in the equation.

[0101] S4. The trigonometric method is used to simplify the dynamic model of wheel-ground interaction, and an analytical model of wheel-ground interaction estimated in real time is obtained, which is used as the observation model for unscented Kalman filtering.

[0102] S4.1 Constructing a wheel-land interaction model using equations (3)-(5):

[0103]

[0104]

[0105]

[0106] In equations (3)-(5), F x For the longitudinal force of the wheels of the engineering vehicle, F z τ is the vertical force of the engineering vehicle's wheel, T is the driving torque of the engineering vehicle's wheel, θ is the contact angle between the engineering vehicle's wheel and the road surface, θ1 is the wheel's approach angle, θ2 is the wheel's departure angle, τ is the tangential stress along the soil, σ is the normal stress of the soil, r is the wheel radius of the engineering vehicle, and b is the wheel width of the engineering vehicle.

[0107] S4.2. Define the fitting function using equations (6) and (7):

[0108]

[0109]

[0110] In equations (6) and (7), σ1(θ) and σ2(θ) represent the normal stresses in different angular ranges, and τ1(θ) and τ2(θ) represent the tangential stresses in different angular ranges. Let represent the undetermined coefficients for calculating σ1(θ). Let represent the undetermined coefficients for calculating σ²(θ). Let each represent the undetermined coefficients for calculating τ1(θ). Let represent the undetermined coefficients for calculating τ2(θ); The maximum normal stress angle;

[0111] S4.3 Calculate the wheel entry angle θ using equations (7) and (8). 1m and the departure angle θ 2m :

[0112]

[0113]

[0114] S4.4 Calculate the coefficients in the fitted function using equations (9) and (10):

[0115]

[0116]

[0117] In equations (9) and (10), This represents the maximum normal stress and the intermediate approach angle θ. 1m The proportionality coefficient of the normal stress at that point, and Represents the maximum normal stress and the intermediate departure angle θ 2m The proportionality coefficient of the normal stress at that point, and This represents the proportionality coefficient between the maximum tangential stress and the tangential stress at the intermediate approach angle, and This represents the proportionality coefficient between the maximum normal stress and the tangential stress at the intermediate departure angle, and σ m The contact angle is θ m The normal stress value at that time, σ 1m The contact angle is θ 1m The normal stress value at time τ m The contact angle is θ m The tangential stress value at time τ 1m The contact angle is θ 1m The tangential stress value at that time.

[0118] S4.5, The coefficients calculated in S4.4 and Substituting these values ​​into the fitting function, we obtain a fitting function with definite coefficients:

[0119]

[0120]

[0121]

[0122]

[0123] S4.6. Rewrite equations (3) and (4), and use equation (10) to obtain the wheel force vector.

[0124]

[0125] In equation (16), I(θ) represents the wheel-ground contact angle matrix, and X represents the soil stress vector, and σ represents the normal stress of the soil, which includes two parts: σ1(θ) and σ2(θ); τ represents the tangential stress of the soil, which includes two parts: τ1(θ) and τ2(θ).

[0126] S4.7 Substituting equations (12)-(15) into equation (16), we obtain the analytical model of wheel-land interaction based on triangular approximation simplification using equation (11):

[0127] F = rb(F1 - F2)X m (17)

[0128] In equation (11), X m Denotes the maximum stress vector, and X m =[σ m ,τ m ] T Where F1 represents force calculation matrix 1, F2 represents force calculation matrix 2, and we have:

[0129] F1=[I(θ m )K 11 (θ σ1 ,θ τ1 )-I(θ1)]K 21 (θ σ1 ,θ τ1 (18)

[0130] F2=[I(θ m )K 12 (θ σ2 ,θ τ2)-I(θ2)]K 22 (θ σ2 ,θ τ2 (19)

[0131] In formula (12), I(θ m ) indicates that the contact angle is θ m Let I(θ1) represent the wheel-ground contact angle matrix when the contact angle is the entry angle θ1, and I(θ2) represent the wheel-ground contact angle matrix when the contact angle is the departure angle θ2. p1 Indicates the entry characteristic angle, θ p2 K represents the departure characteristic angle. 1i Represents the stress characteristic matrix 1, K 2i Let stress characteristic matrix 2 represent the stress characteristic matrix, and we have:

[0132] θ σ1 =2arccosλ σ1 (20)

[0133] θ τ1 =2arccosλ τ1 (twenty one)

[0134] θ σ2 =2arccosθ σ2 (twenty two)

[0135] θ τ2 =2arccosλ τ2 (twenty three)

[0136]

[0137]

[0138]

[0139]

[0140] S4.8. Using equation (16), the simplified wheel driving torque T of the engineering vehicle is obtained:

[0141] T = r 2 b(T D1 -T D2 )τ m (28)

[0142] In equation (16), T D1 T D2 This represents the torque calculation matrix, and

[0143] In specific implementation, a wheel-ground interaction model based on the pressure-bearing model and the Janosi shear model, derived from the theories of Bekker, Wong, et al., is adopted. Figure 3 As shown. In the model, we have equations (29)-(31):

[0144]

[0145]

[0146]

[0147] The model characterizes soil mechanical parameters (compression characteristic parameters: cohesive modulus k). c internal friction modulus Settlement index n; Shear characteristic parameters: soil cohesion c, soil internal friction angle Soil shear modulus K) and wheel mechanical parameters (resultant force F in the horizontal direction) x The resultant force F in the vertical direction z The relationship between the driving torque T and the subsidence index n and the soil internal friction angle. It is the dominant parameter of the wheel-ground interaction. In the subsequent filtering process, other soil mechanical parameters are assigned empirical values ​​and treated as constants.

[0148] S5. Use the 3D SLAM mapping method based on the Cartographer algorithm to build a two-dimensional map of the working environment of the engineering vehicle.

[0149] In practice, the computing platform calls the ROS Melodic Cartographer package, inputs LiDAR data, and generates a two-dimensional map of the working environment by having a manually driven vehicle scan the entire working environment.

[0150] S6. Based on the two-dimensional map, perform AMCL positioning on the engineering vehicle to determine the road surface position (x,y) of the engineering vehicle on the two-dimensional map at time t.

[0151] In practice, the computing platform calls the amcl function package in ROS Melodic, inputs the odometer information generated by the wheel speed sensor, the laser scanning information generated by the lidar, and the two-dimensional map obtained by S5, and obtains the estimated current pose of the vehicle in the map coordinate system.

[0152] S7. Use an industrial camera to capture an image of the road surface in front of the engineering vehicle at time t in the working environment, and input the image of the road surface in front at time t into the trained road recognition model for recognition, to obtain the road surface soil type and the confidence probability p(t) of the recognition result in the image of the road surface in front at time t.

[0153] In practice, images of the road surface ahead at different times are input into the road surface recognition model in step S2 to obtain the road surface soil type and recognition result confidence probability p(t) in the road surface image ahead at time t.

[0154] S8. Obtain the soil mechanical parameters at time t: the settlement index n(x,y) and the internal friction angle at the road surface position (x,y) of the engineering vehicle. The estimated value:

[0155] S8.1. The wheel speed signals of the engineering vehicle collected by the wheel speed sensors are used to calculate the overall vehicle speed v(t) at time t; and the heading angle θ of the engineering vehicle at time t is calculated using the magnetometer signal of the engineering vehicle collected by the inertial measurement unit. f (t);

[0156] In practice, the wheel speed information is obtained by differentiating the pulse signal from the wheel speed sensor. Then, combined with the wheel radius, the vehicle speed is calculated using the formula v = ωr. The heading angle of the engineering vehicle is directly measured using the built-in algorithm of the inertial measurement unit.

[0157] S8.2, the vehicle speed v(t) and heading angle θ at time t. f The bending angle δ(t) read by the vehicle-mounted sensor is input into the state estimation model to obtain the estimated value of the soil mechanical parameters subsidence index n(x,y) and internal friction angle at time t. The estimated value;

[0158] In practice, a suitable time step is selected, and the vehicle speed v(t) and heading angle θ at time t are recorded. f Substituting the values ​​of the bending angle δ(t) read by the vehicle-mounted sensor into the state estimation model, we obtain the estimated values ​​of the soil mechanical parameters, the subsidence index n(x,y), and the internal friction angle at time t. The estimated value;

[0159] S9. Obtain the observed values ​​of the soil mechanical parameters, settlement index n(x,y), and internal friction angle, at time t when the vehicle is at its road surface position (x,y). Observations:

[0160] S9.1. Based on the acceleration signal of the engineering vehicle at time t collected by the inertial measurement unit, obtain the acceleration a(t) of the front and rear frames of the engineering vehicle at time t, and calculate the vertical force F of the wheels of the engineering vehicle at time t according to the formula for calculating the vertical force of the wheels of articulated vehicles. z (t);

[0161] S9.2. Based on the wheel speed signal and the power signal of the engineering vehicle, calculate the wheel torque T(t) of the engineering vehicle at time t using the motor torque-power relationship, and calculate the longitudinal force F of the wheel according to the driving torque-driving force relationship. x (t);

[0162] In practice, since direct measurement of wheel vertical force, longitudinal force and driving torque requires expensive six-dimensional force sensors, indirect measurement is performed using an IMU, speed sensor and current clamp.

[0163] The calculation of the vertical force on the wheels of articulated vehicles can be expressed by equation (30):

[0164]

[0165] In the formula, M is the total mass of the forklift; B is the wheelbase; h is the height of the center of gravity; b2 is the distance from the center of gravity of the rear frame to the center of the rear axle; k1 = B e / B, B e k2 is the distance from the rear axle hinge point to the inner side of the rear wheel; k2 = b2 / (l f +l r ), l f and l r These are the distances from the articulation point to the midpoints of the front and rear axles, respectively; a' x =(m f a x1 +m r a x2 cosδ-m r a y2 sinδ) / M,m f and m r These are the masses of the front and rear frames, respectively. x1 and a x2 These are the longitudinal accelerations of the front and rear frames, a and a', respectively. y2 δ is the lateral acceleration of the rear frame; δ is the bend angle between the front and rear frames. When the forklift is traveling in a straight line, δ = 0; a' y =(m f a y1 +m r a y2 cosδ-m r a x2 sinδ) / M,a y1 Let be the lateral acceleration of the front frame. As can be seen from equation (30), as long as the acceleration of the front and rear frames is measured using an inertial measurement unit, the vertical force of the wheels can be estimated.

[0166] The example vehicle is directly driven by an electric motor. As can be seen from the following formula (31), if the motor torque can be measured, the wheel torque can be calculated.

[0167]

[0168] T = T M ·i0·r (32)

[0169]

[0170] P M =U·I (34)

[0171] In equations (31)-(34), T M P represents the motor torque. M n represents the motor power. M The motor speed is given by n; T is the wheel torque; i0 is the motor reduction ratio; n W U is the wheel speed; U is the power supply voltage; I is the motor current.

[0172] In the experiment, the wheel speed was measured by a speed sensor. A Hall single-channel speed sensor of model SJ1091H was selected. The motor speed could be calculated by equation (33). The motor current could be measured by a current clamp. The motor power could be calculated by equation (34). The motor torque could be calculated by substituting the measured motor power and motor speed into equation (31).

[0173] The formula for calculating the longitudinal force of a wheel is as follows:

[0174]

[0175] S9.3 Input the wheel vertical force, longitudinal force, and torque obtained at time t into the observation model of the unscented Kalman filter, and obtain the observed values ​​of the soil mechanical parameters, settlement index n(x,y), and internal friction angle at time t. Observed values;

[0176] In specific implementation, the forward Euler method is used to discretize the wheel-ground interaction model simplified by the trigonometric method, and an appropriate time step is selected to represent F. x (t), F z Substituting T(t) and T(t) into the simplified model, we obtain the observed values ​​of the soil mechanical parameters, the subsidence index n(x,y) and the internal friction angle at time t. The observed values.

[0177] S10. Using the unscented Kalman filter algorithm, estimate and observe the soil mechanical parameters at time t, including the subsidence index n(x,y) and the internal friction angle. The estimated and observed values ​​are processed to obtain the filtered value of the soil mechanical parameters, the subsidence index n(x,y), and the internal friction angle at time t. The filtered value is used as the road surface type estimation result based on vehicle dynamics;

[0178] In practice, the state estimation model and the observation model are transformed into the general form of Kalman filtering as shown in equation (36):

[0179]

[0180] Where f(·) is the state function of the system, which can be obtained by discretizing the state equation in equation (1) using the forward Euler method; h(·) is the measurement function, which can be described by equation (17). W is the process noise, and V is the observation noise. They are uncorrelated Gaussian white noise with a mean of 0, and their covariances are respectively:

[0181]

[0182] Following the general steps of unscented Kalman filtering—initializing the filter value, updating the time, updating the measurement, and updating the filter—the filtered value of the soil mechanical parameter settlement index n(x,y) and the internal friction angle at time t can be obtained. The filtered value.

[0183] S11, such as Figure 4 As shown, the road surface soil type and the confidence probability p(t) of the identification result in the road surface image at time t are fused with the road surface type estimation result to obtain the final road surface type identification result of the engineering vehicle at the road surface position (x,y) at time t, so as to realize the perception of the working environment of the engineering vehicle at the road surface position (x,y) at time t.

[0184] Step S11.1: Set a confidence probability threshold p std ;

[0185] Step S11.2: Based on the soil mechanical parameter comparison table for typical road surfaces, look up the range of soil mechanical parameters corresponding to the road surface soil type in the road surface image at time t obtained in step S7; and determine the settlement index n(x,y) and internal friction angle of the soil mechanical parameters obtained in step S10. If the filtered value is within the range of the queried soil mechanical parameters, then the soil type identification result of the vehicle at position (x,y) at time t obtained in step S7 is correct, and it is set as the final road surface type identification result of the engineering vehicle at position (x,y) at time t; otherwise, proceed to step S11.3.

[0186] In practical implementation, the following reference value range table can be derived by referring to existing literature:

[0187]

[0188] If the settlement index n and internal friction angle estimated in step S10 are... If all the results fall within the range of the road surface type identification results in step two, then the identification results are considered consistent, and the current vehicle coordinates and road surface type features are stored.

[0189] Step S11.3: Combine the confidence probability p(t) of the soil type corresponding to the vehicle at position (x,y) at time t obtained in step S7 with the confidence probability threshold p std Compare, if p(t) is lower than p std If the soil type at time t identified in step S7 is incorrect, then step S11.4 is executed; otherwise, step S11.5 is executed.

[0190] Step S11.4: Based on the soil mechanical parameters (settlement index n(x,y) and internal friction angle) of the engineering vehicle at the road surface position (x,y) at time t estimated in step S10, The filtered value is used to look up the soil mechanical parameters of typical road surfaces in the table to determine the soil type of the engineering vehicle at the road surface position (x,y) at time t, and this is used as the final road surface type identification result.

[0191] In specific implementation, when the confidence probability p in step S6 res Below the threshold p std If the road surface features are not obvious and the visual recognition results are inaccurate, the dynamic recognition results are directly adopted and the road surface type is determined by referring to a table.

[0192] Step S11.5: Using the probability density function truncation method, the soil mechanical parameters corresponding to the soil type at the road surface position (x,y) of the engineering vehicle at time t identified in step S7, and the soil mechanical parameters at the road surface position (x,y) of the engineering vehicle at time t estimated in step S10, namely the settlement index n(x,y) and the internal friction angle, are analyzed. The filtered values ​​are fused together, and the fusion result is used as the final road surface type identification result of the engineering vehicle at the road surface position (x,y) at time t.

[0193] In specific implementation, when the confidence probability p in step S6 res Above the threshold p std If the dynamic estimation result is inaccurate, then the constraint conditions are designed based on the range (a,d) of typical parameter values ​​for the identified road surface type. res ,b(2-p res The corrected subsidence index n and internal friction angle are obtained using the probability density function truncation method. Determine the road surface type by referring to the table and store it along with the coordinates.

[0194] In this embodiment, an electronic device includes a memory and a processor. The memory stores a program that supports the processor in executing the above-described method, and the processor is configured to execute the program stored in the memory.

[0195] In this embodiment, a computer-readable storage medium stores a computer program, which is executed by a processor to perform the steps of the above method.

Claims

1. A depth perception method for unmanned engineering vehicles operating in complex environments, characterized in that, It is done according to the following steps: S1. Install intelligent sensors on engineering vehicles for working environment perception, including: installing lidar on the top of the engineering vehicle, installing inertial measurement unit in the cab, installing wheel speed sensors on the wheels, and installing industrial cameras in front of the wheels. S2. Collect image datasets of different types of soil and road surfaces and label the soil types of the road surfaces to obtain labeled image datasets, which are used to train the road recognition model; S3. Use equation (1) to establish the kinematic equations of the engineering vehicle: (1) In equation (1), This refers to the front axle position of the engineering vehicle. and Here are the coordinates of the center of the front axle of the engineering vehicle. For the center of the front axle of the engineering vehicle Axial velocity, For the center of the front axle of the engineering vehicle Axial velocity, This refers to the heading angle of the front axle of the engineering vehicle. The angular velocity of the front axle of the engineering vehicle. For the steering bend angle of engineering vehicles, Indicates the turning angular velocity; and These are the front and rear overhangs of the engineering vehicle, respectively. The speed of the engineering vehicle; Define augmented state vector In the formula, Let the subsidence index and internal friction angle of the soil be the subsidence index and the internal friction angle, respectively. The differential is denoted as ;right After discretization, the state estimation model of the unscented Kalman filter is obtained; S4. The trigonometric method is used to simplify the dynamic model of wheel-ground interaction, and an analytical model of wheel-ground interaction estimated in real time is obtained, which is used as the observation model of unscented Kalman filtering. S5. Use the 2D SLAM mapping method based on the Cartographer algorithm to create a two-dimensional map of the working environment of the engineering vehicle; S6. Based on the two-dimensional map, perform AMCL positioning on the engineering vehicle to determine... Location of engineering vehicles on the road surface in a 2D map ; S7. Use the industrial camera to film the work environment. Real-time images of the road ahead of the engineering vehicle, and The image of the road surface ahead at any given time is input into the trained road recognition model for identification, resulting in... Road surface soil type and confidence probability of recognition results in the road surface image ahead at time [time]. ; S8, Obtain Construction vehicles are always on the road. Soil mechanical parameters at the location, subsidence index and internal friction angle The estimated value: S8.

1. The wheel speed signal of the engineering vehicle collected by the wheel speed sensor is used to calculate... Vehicle speed at any given moment ; and utilize the data collected by the inertial measurement unit The magnetometer signal of the engineering vehicle is used for calculation. The heading angle of the engineering vehicle at all times (t); S8.2, the speed of the entire vehicle at time t. (t), Road surface location Heading angle (t) The bending angle read by the vehicle-mounted sensor Input the data into the state estimation model and obtain the soil mechanical parameter subsidence index at time t. The estimated value and internal friction angle The estimated value; S9. Obtain the vehicle's position on the road at time t. Soil mechanical parameters, subsidence index Observed values ​​and internal friction angle Observations: S9.1, Based on the data collected by the inertial measurement unit Acceleration signals of engineering vehicles at all times are acquired. The acceleration a(t) of the front and rear frames of the engineering vehicle at any given time is calculated, and the vertical force of the wheels of the articulated vehicle is calculated using the formula for this purpose. Vertical force on the wheels of construction vehicles ; S9.

2. Based on the wheel speed signal and the power signal of the engineering vehicle, calculate using the motor torque-power relationship. Wheel torque of engineering vehicles at all times And based on the driving torque-driving force relationship, calculate the longitudinal force of the wheel. ; S9.3, will The vertical force, longitudinal force, and torque of the wheel obtained at each moment are input into the observation model of the unscented Kalman filter, and the results are obtained. Soil mechanical parameters at any time, subsidence index Observed values ​​and internal friction angle Observed values; S10. Use the unscented Kalman filter algorithm to evaluate the soil mechanical parameters, specifically the subsidence index, at time t. The estimated and observed values ​​and the internal friction angle The estimated and observed values ​​are processed to obtain the subsidence index, a soil mechanical parameter at time t. Filter value and internal friction angle The filtered value is used as the road surface type estimation result based on vehicle dynamics; S11, will Road surface soil type and confidence probability of recognition results in the road surface image ahead at time [time]. By fusing the results with the road surface type estimation, we obtain Construction vehicles are always on the road. The final road surface type identification result, in order to achieve the following: Construction vehicles are always on the road. Perception of the work environment.

2. The depth perception method for unmanned engineering vehicles in complex operating environments according to claim 1, characterized in that, Step S4 includes: S4.1 Constructing a wheel-land interaction model using equations (3)-(5): (3) (4) (5) In equations (3)-(5), For the longitudinal force of the wheels of engineering vehicles, The vertical force on the wheels of the engineering vehicle. For the wheel drive torque of engineering vehicles, This indicates the contact angle between the wheels of the engineering vehicle and the road surface. The wheel approach angle of the engineering vehicle; The departure angle of the wheels of the engineering vehicle; For the tangential stress along the soil, For soil normal stress, This refers to the wheel radius of the engineering vehicle. The width of the wheels of the engineering vehicle; S4.

2. Define the fitting function using equations (6) and (7): (6) (7) In equations (6) and (7), , These represent the normal stresses within different angular ranges. , These represent the tangential stresses within different angular ranges. They represent calculations respectively. The three undetermined coefficients, They represent calculations respectively. The three undetermined coefficients, They represent calculations respectively. The three undetermined coefficients, They represent calculations respectively. The three undetermined coefficients; The maximum normal stress angle; S4.3 Calculate the wheel entry angle using equations (8) and (9). and the middle departure angle : (8) (9) S4.4 Calculate the coefficients in the fitted function using equations (10) and (11): (10) (11) In equations (10) and (11), Indicates the maximum normal stress and the intermediate approach angle The proportionality coefficient of the normal stress at that point, and ; Represents the maximum normal stress and the intermediate departure angle The proportionality coefficient of the normal stress at that point, and ; This represents the proportionality coefficient between the maximum tangential stress and the tangential stress at the intermediate approach angle, and ; This represents the proportionality coefficient between the maximum tangential stress and the tangential stress at the intermediate departure angle, and ; Indicates the contact angle as The normal stress value at that time, Indicates the contact angle as The normal stress value at that time, Indicates the contact angle as The tangential stress value at that time, Indicates the contact angle as The tangential stress value at that time; S4.5, The coefficients calculated in S4.4 , and , Substituting these into the fitting function, we can obtain the fitting function with definite coefficients using equations (12)-(15): (12) (13) (14) (15) S4.

6. Obtain the wheel force vector using equation (16). : (16) In equation (16), This represents the wheel-ground contact angle matrix, and ; Represents the soil stress vector, and , Represents the normal stress of the soil, including , ; This represents the tangential stress in the soil, including , ; S4.7 Substituting equations (12)-(15) into equation (16), we obtain the analytical model of wheel-land interaction based on triangular approximation simplification using equation (17): (17) In equation (17), Denotes the maximum stress vector, and ,in This represents the matrix for calculating the first force. Let the matrix for calculating the second force be denoted as , and we have: (18) (19) In equations (18)-(19), Indicates the contact angle as Wheel-ground contact angle matrix at time, The contact angle is the entry angle. Wheel-ground contact angle matrix at time, The contact angle is the departure angle. Wheel-ground contact angle matrix at time, , Represents two first stress characteristic matrices. , Let represent two second stress characteristic matrices, and we have: (20) (21) (22) (23) (24) (25) (26) (27) S4.

8. Using equation (28), a simplified wheel driving torque of the engineering vehicle is obtained. : (28) In equation (28), This represents two torque calculation matrices, and , .

3. The depth perception method for unmanned engineering vehicles in complex operating environments according to claim 1, characterized in that, The fusion in step 11 is performed as follows: Step S11.1: Set a confidence probability threshold. ; Step S11.2: Based on the soil mechanical parameters comparison table for typical road surfaces, look up the values ​​obtained in step S7. The range of soil mechanical parameters corresponding to the soil type of the road surface in the image ahead at any given time; And determine the subsidence index, a soil mechanical parameter, in step S10. and internal friction angle Check whether the filtered value is within the range of the queried soil mechanical parameters; if it is, it indicates that the value obtained in step S7 is within the range of the queried soil mechanical parameters. Vehicle location at all times The soil type identification result is correct, and it is set as the location of the engineering vehicle on the road surface at time t. The final road surface type identification result; otherwise, proceed to step S11.3; Step S11.3: Obtain the vehicle's position at time t obtained in step S7. Confidence probability corresponding to soil type With the confidence probability threshold If a comparison is made, Below This indicates that step S7 identified... Construction vehicles are always on the road. The soil type is incorrect; proceed to step S10.

4. Otherwise, proceed to step S11.5; Step S11.4: Based on the estimated position of the engineering vehicle on the road surface at time t in step S10. Soil mechanical parameters, subsidence index and internal friction angle The filtered value is used to consult a table of soil mechanical parameters for typical road surfaces to determine the position of the engineering vehicle on the road surface at time t. The soil type was determined and used as the final pavement type identification result; Step S11.5: Apply the probability density function truncation method to the data identified in step S7. Construction vehicles are always on the road. The soil mechanical parameters corresponding to the soil type and the estimated position of the engineering vehicle on the road surface at time t in step S10. Soil mechanical parameters, subsidence index and internal friction angle The filtered values ​​are fused together, and the fused result is used as... Construction vehicles are always on the road. The final road surface type identification result.

4. An electronic device, comprising a memory and a processor, characterized in that, The memory is used to store a program that supports the processor in executing any of the depth perception methods of claims 1-3, and the processor is configured to execute the program stored in the memory.

5. A computer-readable storage medium storing a computer program thereon, characterized in that, The computer program is executed by the processor to perform the steps of any of the depth perception methods described in claims 1-3.

Images