Collision prevention system and method in cabin of unmanned loader and loader

Abstract

The invention discloses an unmanned loader in-cabin collision prevention system and method and a loader, and belongs to the technical field of loader collision prevention. The unmanned loader in-cabin collision prevention system comprises an upper computer, a rotating bucket oil cylinder displacement sensor, a movable arm oil cylinder displacement sensor, a hinge angle sensor and a motor controller which are electrically connected with an electronic control unit; the motor controller is electrically connected with the hub motor, the upper computer is electrically connected with the at least one camera and the at least one laser radar, and the hinge angle sensor is arranged at the hinge point of a front vehicle body and a rear vehicle body of the loader. The self-adaptive safety area of the cabin is constructed in real time through the laser radar, the distance between the loader and the cabin boundary and the distance between the loader and key components such as a grab bucket are dynamically monitored through fusion of the laser radar and the camera, and the four hub motors of the loader are controlled through the electronic control unit so that real-time path tracking operation of the loader can be achieved. And the safety and the reliability of the unmanned loader in the cabin in the operation process are effectively improved.

Description

Technical Field

[0001] The invention discloses a system and method for preventing collision in a cabin of an unmanned loader and a loader, belonging to the technical field of preventing collision of loaders. Background Art

[0002] In port and ship loading and unloading operations, loaders are one of the key equipment to improve operating efficiency. Since the space inside the cabin is relatively closed and the structure is complex, there are many obstacles, such as bulkheads, materials, grabs and other operating equipment. Loaders face complex environmental and safety challenges when operating in the cabin. In addition, loader drivers need to perform long-term, high-intensity operations in a complex environment, which not only easily leads to operator fatigue, but may also cause safety accidents due to human errors. Therefore, unmanned loaders, as a typical intelligent product of engineering machinery, are expected to reduce the labor burden of operators and improve operational safety, which is a new trend in the development of the industry.

[0003] Due to the variability of the working environment in the cabin, the active safety system of the loader must be able to adapt to environmental changes in real time and dynamically adjust the safety area to ensure the safety of the operation. At present, among the existing technologies at home and abroad, the technology of the autonomous operation system of the unmanned loader in the cabin is blank, and it is impossible to achieve efficient and safe operation in the complex cabin environment. Summary of the invention

[0004] In view of the defects of the prior art, the present invention proposes a system, method and loader for preventing collision in the cabin of an unmanned loader. Through integrated multi-sensor technology, intelligent algorithms and adaptive control strategies, efficient and safe operations in complex cabin environments can be achieved, the risk of safety accidents can be reduced, and operating efficiency can be improved.

[0005] The technical solution of the present invention is as follows:

[0006] According to a first aspect of an embodiment of the present invention, there is provided an unmanned loader cabin collision prevention system, comprising a host computer electrically connected to an electronic control unit, a bucket cylinder displacement sensor, a boom cylinder displacement sensor, an articulation angle sensor and a motor controller, wherein the motor controller is electrically connected to a wheel hub motor, the host computer is electrically connected to at least one camera and at least one laser radar, respectively, and the articulation angle sensor is arranged at the articulation point of the front and rear bodies of the loader.

[0007] Further, the motor controller includes: a front body right motor controller, a front body left motor controller, a rear body right motor controller, and a rear body left motor controller; the wheel hub motor includes: a front body right wheel hub motor electrically connected to the front body right motor controller, a front body left wheel hub motor electrically connected to the front body left motor controller, a rear body right wheel hub motor electrically connected to the rear body right motor controller, and a rear body left wheel hub motor electrically connected to the rear body left motor controller.

[0008] According to a second aspect of an embodiment of the present invention, a method for preventing collision in a cabin of an unmanned loader is provided, which is applied to the system for preventing collision in a cabin of an unmanned loader according to the first aspect, comprising:

[0009] The host computer obtains cabin interior point cloud data, obstacle point cloud data, cabin interior image data and loader dynamic position data through the camera and the laser radar respectively, determines initial path point information according to the cabin interior point cloud data and the loader dynamic position data, and sends it to the electronic control unit;

[0010] The electronic control unit controls the motor controller to drive according to the initial path point information and sends the current vehicle speed to the host computer;

[0011] The host computer executes a bulkhead detection strategy according to the current vehicle speed, and executes a pile and grab detection strategy according to the obstacle point cloud data and the cabin interior image data. The bulkhead detection strategy and the pile and grab detection strategy are used to control the electronic control unit to execute the target action.

[0012] Further, the determining of the initial path point information according to the cabin internal point cloud data and the loader dynamic position data includes:

[0013] Preprocessing and extracting the cabin interior point cloud data to obtain point cloud data of a first area of ​​interest;

[0014] Based on an adaptive ground estimation algorithm, the point cloud data of the first area of ​​interest is filtered out to obtain the point cloud data of the second area of ​​interest;

[0015] The point cloud data of the second area of ​​interest is filtered out by the loader body through convex hull calculation, point cloud clipping and dynamic rotation transformation to obtain point cloud data of the third area of ​​interest;

[0016] Extracting the point cloud data of the third region of interest based on the Alpha Shape algorithm to obtain point cloud boundary data of the cabin;

[0017] Obtaining a cabin collision boundary according to safety parameters and point cloud boundary data of the cabin;

[0018] Initial path point information is determined according to the cabin collision boundary.

[0019] Furthermore, the bulkhead detection strategy includes:

[0020] determining a safe distance according to the current vehicle speed;

[0021] The four real-time postures of the detection points of the front and rear bodies of the loader are obtained respectively by the articulation angle sensor, and four actual distances from the detection points of the front and rear bodies of the loader to the collision boundary of the cabin are obtained according to the four real-time postures of the detection points of the front and rear bodies of the loader, and the minimum distance is obtained according to the four actual distances;

[0022] Determine whether the minimum distance is greater than the safety distance:

[0023] Yes, execute the adaptive correction strategy for the vehicle boundary;

[0024] No, there is a risk of collision for the loader, and a signal to take braking measures is sent to the electronic control unit.

[0025] Furthermore, the material pile and grab bucket detection strategy includes:

[0026] Performing clustering processing on the obstacle point cloud data to obtain a plurality of obstacle areas with cluster numbers;

[0027] Performing bounding box fitting on each of the plurality of obstacle regions with cluster numbers to obtain eight boundary points of the 3D bounding box;

[0028] Preprocessing the cabin interior image data to obtain corrected image data, and processing the corrected image data using a deep learning algorithm to obtain a two-dimensional detection frame;

[0029] The eight boundary points of the two-dimensional detection frame and the 3D bounding frame are fused and matched, and the real-time distance between the edge of the material and the loader, the shortest actual distance between the grab bucket and the loader, and the actual vertical distance between the grab bucket and the loader are obtained respectively;

[0030] Determine whether the shortest actual distance between the grab bucket and the loader is greater than a preset safety threshold:

[0031] Yes, proceed to the next step;

[0032] If no, a braking command is sent to the electronic control unit, and a loader waiting time determination strategy is executed;

[0033] Determine whether the actual vertical distance between the grab bucket and the loader is greater than a preset safety threshold in the vertical direction:

[0034] Yes, execute the adaptive vehicle boundary correction strategy;

[0035] No, a braking instruction is sent to the electronic control unit, and the loader waiting time judgment strategy is executed.

[0036] Furthermore, the loader waiting time judgment strategy includes:

[0037] Determine whether the loader waiting time is greater than the grab bucket operation time safety threshold:

[0038] Yes, re-execute the bulkhead detection strategy and the stockpile and grab detection strategy;

[0039] If no, the loader continues to wait until the loader waiting time is greater than the grab bucket operation time safety threshold.

[0040] Furthermore, the adaptive correction vehicle boundary strategy includes:

[0041] According to the bucket cylinder displacement sensor and the boom cylinder displacement sensor, the bucket hydraulic cylinder length and the boom hydraulic cylinder length are respectively obtained;

[0042] Determine whether the bucket hydraulic cylinder length and the boom hydraulic cylinder length both meet the working range conditions:

[0043] Yes, adaptively correct the loader body boundary;

[0044] If no, proceed to the next step;

[0045] Obtain the real-time distance between the material boundary point set and the loader boundary detection point and the relative speed between the loader and the material in front of the vehicle body, and determine whether the real-time distance between the material boundary point set and the loader boundary detection point is less than or equal to the preset safety distance and the relative speed between the loader and the material in front of the vehicle body is greater than 0:

[0046] Yes, the loader is at risk of collision, and a signal for taking braking measures is sent to the electronic control unit;

[0047] If no, proceed to the next step;

[0048] Determine whether the loader has reached the end point:

[0049] Yes, end the process;

[0050] No, execute the bulkhead detection strategy and the stockpile and grab detection strategy.

[0051] Furthermore, after the adaptive correction of the loader body boundary is performed, the method further includes:

[0052] Obtain the real-time distance between the rear vehicle body material boundary and the loader boundary detection point, and determine that the real-time distance between the rear vehicle body material boundary and the loader boundary detection point is less than or equal to a preset safety distance:

[0053] Yes, the loader is at risk of collision, and a signal for taking braking measures is sent to the electronic control unit;

[0054] If no, proceed to the next step;

[0055] Determine whether the loader has reached the end point:

[0056] Yes, end the process;

[0057] No, execute the bulkhead detection strategy and the stockpile and grab detection strategy.

[0058] According to a third aspect of an embodiment of the present invention, there is provided a loader, comprising: a loader body and the unmanned loader cabin collision prevention system according to the second aspect.

[0059] The present invention provides a system and method for preventing collision in a cabin of an unmanned loader, and a loader, the beneficial effects of which are:

[0060] (1) The unmanned loader cabin collision prevention system consists of a host computer, laser radar, camera, electronic control unit and other modules, which can be flexibly configured according to different operation scenarios. This modular design facilitates rapid promotion and application to other unmanned driving scenarios, such as docks, mining areas, mixing plants, etc., and has good practical value and market prospects;

[0061] (2) The adaptive safety zone of the cabin is constructed in real time through LiDAR. The distance between the loader and the cabin boundary and key components such as the grab bucket is dynamically monitored by LiDAR and camera fusion. The four wheel hub motors of the loader are controlled by the electronic control unit (ECU) to achieve real-time path tracking of the loader, effectively improving the safety and reliability of the unmanned loader operation in the cabin.

[0062] (3) Based on the data collected in real time by the articulation angle sensor and lidar, the point cloud boundary extraction based on the Alpha Shape algorithm was used to generate the dangerous collision boundary in the cabin and filter out the loader's own body, thereby improving the robustness of the loader in preventing collisions.

[0063] (4) The displacement information collected by the hydraulic cylinder displacement sensor is used to adaptively correct the loader body boundary, ensuring the practicality and safety of the unmanned loader during reversing operations;

[0064] (5) A dynamic update mechanism based on adaptive computing is introduced. By adjusting the safety distance and path planning strategy of the cabin operation boundary in real time, advanced algorithms are used in combination with the electronic control unit (ECU) to conduct in-depth analysis of multi-sensor fusion data, dynamically calculate the changes in the cabin boundary and the operating environment, and adaptively optimize the obstacle position and operating boundary, more accurate obstacle avoidance and path optimization can be achieved, thereby enhancing the intelligent decision-making ability and operating efficiency of the unmanned loader in cabin operations.

[0065] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 The present invention is a schematic block diagram of the structure of a collision prevention system in a cabin of an unmanned loader according to an exemplary embodiment.

[0067] Figure 2 The figure is a schematic diagram showing the installation position of a collision prevention system in a cabin of an unmanned loader according to an exemplary embodiment.

[0068] Figure 3 The figure is a schematic diagram showing the installation position of a collision prevention system in a cabin of an unmanned loader according to an exemplary embodiment.

[0069] Figure 4 The present invention is a flow chart of a method for preventing collision in a cabin of an unmanned loader according to an exemplary embodiment.

[0070] Figure 5 The present invention is a flow chart of a method for preventing collision in a cabin of an unmanned loader according to an exemplary embodiment.

[0071] Figure 6 It is a schematic diagram of the collision boundary and operation effect in a method for preventing collision in a cabin of an unmanned loader according to an exemplary embodiment.

[0072] Figure 7 It is a schematic diagram of the side view field of view of a laser radar and a camera in a method for preventing collision in a cabin of an unmanned loader according to an exemplary embodiment.

[0073] Figure 8 It is a schematic diagram of the overhead field of view of a laser radar and a camera in a method for preventing collision in a cabin of an unmanned loader according to an exemplary embodiment. DETAILED DESCRIPTION

[0074] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0075] In the description of the present invention, it should be noted that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside", etc. indicate directions or positional relationships based on the directions or positional relationships shown in the accompanying drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operated in a specific direction. Therefore, they cannot be understood as limitations on the present invention.

[0076] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0077] Embodiment 1: A collision prevention system in a cabin of an unmanned loader is shown according to an exemplary embodiment. Figure 1 , Figure 2 and Figure 3 As shown, it includes a host computer 2 electrically connected to the electronic control unit 1, a bucket cylinder displacement sensor 11, a boom cylinder displacement sensor 15, an articulation angle sensor 18 and a motor controller, the motor controller is electrically connected to the wheel hub motor, the host computer 2 is electrically connected to at least one camera and at least one laser radar respectively, the articulation angle sensor 18 is arranged at the hinge point of the front and rear bodies of the loader, and multiple cameras and laser radars are installed directly above the loader cab, wherein the cameras include: a first camera 3, a second camera 5, a third camera 7 and a fourth camera 9, and the laser radars include: a first laser radar 4, a second laser radar 6, a third laser radar 8 and a fourth laser radar 10.

[0078] The motor controller includes: a front body right motor controller 12, a front body left motor controller 16, a rear body right motor controller 19, and a rear body left motor controller 21. The wheel hub motor includes: a front body right wheel hub motor 13 electrically connected to the front body right motor controller 12, a front body left wheel hub motor 17 electrically connected to the front body left motor controller 16, a rear body right wheel hub motor 20 electrically connected to the rear body right motor controller 19, and a rear body left wheel hub motor 22 electrically connected to the rear body left motor controller 21.

[0079] Embodiment 2: A method for preventing collision in a cabin of an unmanned loader is shown according to an exemplary embodiment. Figure 4 , Figure 5 , Figure 6 , Figure 7 and Figure 8 As shown, the specific steps are as follows:

[0080] Step S10, the host computer obtains cabin interior point cloud data, obstacle point cloud data, cabin interior image data and loader dynamic position data through the camera and the laser radar respectively, determines initial path point information according to the cabin interior point cloud data and the loader dynamic position data and sends it to the electronic control unit, and the specific steps are as follows:

[0081] First, the electronic control unit has a point cloud region of interest Roi [0] -Roi [5] , safety parameters a, b, c, a', b', c', safety distance threshold d s , the four vertices of the vehicle body and the hinge point (d x0 ,d y0 )'s relative position (d x1 ,d y1 ),(d x2 ,d y2 ),(d x3 ,d y3 ),(d x4 ,d y4 ), the loader's own height h top , the actual distance safety threshold L between the grab bucket and the loader sg , the safety distance threshold L between the grab bucket and the material boundary sc , grab bucket and loader vertical safety threshold L stop , End point error threshold E dth , the grab bucket's operating time safety threshold T s The electronic control unit reads the relevant parameters of the articulation angle sensor, bucket cylinder displacement sensor, boom cylinder displacement sensor and the speed values ​​of each hub motor.

[0082] Step S11, the host computer obtains cabin interior point cloud data, obstacle point cloud data, cabin interior image data and loader dynamic position data through the camera and the laser radar respectively, pre-processes and extracts the cabin interior point cloud data, and obtains point cloud data of the first area of ​​interest, the specific content is as follows:

[0083] The purpose of downsampling the point cloud data inside the cabin is to evenly group the point cloud data by voxel size to reduce the amount of point cloud data. i ,y i ,z i ) will be mapped into a voxel, and the size of the voxel determines the grouping range of each point. If the coordinates of two points fall in the same voxel, they will be classified into the same group, as shown in the formula:

[0084] L x =0.15, L y =0.15, L z =0.15

[0085] Where, L x , L y , L z is the size of the voxel grid, all point cloud points will be voxelized and grouped according to these sizes;

[0086] Remove outliers from the point cloud and calculate the distance d from the k neighboring points around the target point to the target point in the statistical filter. i , and calculate the average value μ of these distances, that is:

[0087]

[0088] μ=d

[0089]

[0090] d′=μ+σM

[0091] In the formula, p is the target point, k is the number of neighborhood points when calculating the mean; M is the standard deviation multiple; σ is the standard deviation, which indicates the degree of dispersion of the neighborhood distance of the current point p; d is the average neighborhood distance of the current point, and μ is the average neighborhood distance of all points;

[0092] If d is greater than d′, that is, the average neighborhood distance d of the target point p exceeds the threshold, then the point p is considered an outlier and is removed; at this time, points whose distance from the average value exceeds a certain standard deviation will be removed to ensure that only points that meet the specific distribution are retained in the point cloud data;

[0093] Extract the point cloud data of the first area of ​​interest, using conditional filters and pass filters, namely:

[0094]

[0095] In the formula, x min =Roi [0] ,x max =Roi [1] ,y min =Roi [2] ,y max =Roi [3] ,z min =Roi [4] ,z max =Roi [5] .

[0096] Step S12, based on the adaptive ground estimation algorithm, the point cloud data of the first area of ​​interest is filtered out to obtain the point cloud data of the second area of ​​interest, and the specific content is as follows:

[0097] Data normalization, first convert the point cloud data of the first area of ​​interest into a polar coordinate system. Polar coordinates divide the three-dimensional point cloud into several channels and distance regions. Points are assigned to grids according to their distance and angle. Gridding localizes the point cloud to facilitate the subsequent calculation of the features of each area, where:

[0098]

[0099] Where d is the distance from the point to the origin; C p To normalize the angle of the point to the interval [0,1]; B p To normalize the radial distance of the point to the interval [0,1]; r min is the minimum radial distance from the origin in the point cloud data; r max is the maximum radial distance from the origin in the point cloud data;

[0100] Feature extraction: For each grid cell, the minimum height value of the points in the cell is calculated as a preliminary ground feature indicator. At the same time, a Gaussian distribution smoothing kernel K(x) is generated to smooth the height value of the grid, that is:

[0101]

[0102] Where x is the height of a grid, μ is the mean of the Gaussian distribution, σ is the standard deviation of the Gaussian distribution, and exp is the natural exponential function e x ;

[0103] Gaussian smoothing H s (i) Use the obtained Gaussian kernel K(x) to perform Gaussian smoothing on the height value of each grid to reduce the impact of noise on the ground height estimation, that is:

[0104]

[0105] In the formula, k is the smoothing range, K(ji) is the Gaussian kernel function weight, and H(j) is the original height value;

[0106] Height difference calculation H dif , calculate the height difference between each cell and its adjacent cells, retaining relatively large differences. The height difference in the ground area is usually small, while the height difference of obstacles or non-ground points is large. Therefore, the height difference can be used as an important indicator to determine whether it is on the ground; that is:

[0107] H dif =max(H(i)-H(i-1),H(i)-H(i+1))

[0108] Determine the ground, combined with the smooth height (H s ), height difference (H dif ), and the set threshold (H min_max ), determine whether each unit is a ground point; the ground area usually has a flat height (small height difference) and the height value is within a certain range, so if the smooth height and height difference are both less than the threshold, the unit is considered to be the ground;

[0109] Median filtering takes the median of the height of the four adjacent ground points around the non-ground point and updates it to the ground point for median filtering. Median filtering fills isolated points with the statistical values ​​of the surrounding ground points to further improve the robustness of ground judgment, that is:

[0110]

[0111] In the formula, H m is the median height, H sort [1] is the lowest height value after sorting, H sort [2] is the second lowest height value after sorting;

[0112] Mean filtering detects isolated points among ground points. When an obvious abnormal value of height is detected, it is replaced by the average value of adjacent heights to filter outliers, that is:

[0113]

[0114] In the formula, H new Indicates the current point height updated with the average height value of the adjacent points, which is used to filter outliers; H sd It represents the sum of the height values ​​of all the surrounding adjacent points, and n represents the total number of surrounding adjacent points;

[0115] Ground judgment and removal,For each point cloud cell, determine whether it is the ground based on the height difference and height value. The ground judgment condition is: if Hsmoothed <tH max And H diff <tH diff , then it is marked as the ground, where H smoothed is the smoothed height value; H diff is the height difference between adjacent cells; tH max is the preset maximum height value; tH diff is the maximum height difference allowed;

[0116] Step S14, filtering out the body of the loader from the point cloud data of the second area of ​​interest by convex hull calculation, point cloud clipping and dynamic rotation transformation to obtain point cloud data of the third area of ​​interest, effectively filtering out the point cloud data of the loader itself, thereby avoiding interference with the identification of dangerous areas and operating areas in the cabin, the specific contents are as follows:

[0117] Define the input point set: The input point set Ps is the coordinates of the four vertices of a quadrilateral. Each point is stored in the point cloud using the following formula;

[0118] P s ={(x 1 ,y 1 ,z 1 ),(x 2 ,y 2 ,z 2 ),(x 3 ,y 3 ,z 3 ),(x 4 ,y 4 ,z 4 )}

[0119] Convex hull calculation: The convex hull is constructed by points on a two-dimensional plane. The input is the x,y coordinate projection of the above four points. The projection formula is:

[0120] P proj ={(x k ,y k )|k=1,2,3,4}

[0121] Using the two-dimensional coordinate point set P proj Construct the convex hull:

[0122] C(P proj )={(x i ,y i )|i=1,2,...,n}

[0123] Where n is the number of convex hull boundary points, which is calculated by the convex hull algorithm from the point set in the point cloud;

[0124] Point cloud clipping, the clipping rule is based on whether the point falls inside or on the boundary of the convex hull area; for each point (x, y, z) in the point cloud, it is projected onto a two-dimensional plane:

[0125] P c =(x,y)

[0126] The following conditions determine whether a point is retained:

[0127]

[0128] If In=False, keep the point (x,y,z); if In=True, remove the point (x,y,z);

[0129] The rotation transformation of each point (x, y) in the front body area when the loader turns is calculated as follows:

[0130]

[0131] Where (x′, y′) is the coordinate of each point after rotation, and θ is the rotation angle (expressed in radians). The formula is:

[0132]

[0133] In the formula, γ α It is the articulation angle, which is obtained by the angle sensor installed at the articulation point of the loader.

[0134] Step S15, extracting the point cloud data of the third area of ​​interest based on the Alpha Shape algorithm to obtain the point cloud boundary data of the cabin. The Alpha Shape algorithm can accurately extract the boundary of the cabin according to the distribution characteristics of the point cloud data, providing a basis for the subsequent generation of the danger zone. The specific contents are as follows:

[0135] Calculation of the extent of the point cloud in the X and Y axis directions:

[0136] X-axis range:

[0137] p x_min =min{x i |(x i ,y i ,z i )∈P c}

[0138] p x_max =max{x i |(x i ,y i ,z i )∈P c}

[0139] Y-axis range:

[0140] p y_min =min{y i |(x i ,y i ,z i )∈P c}

[0141] p y_max =max{y i |(x i ,y i ,z i )∈P c}

[0142] Where P c Represents the input point cloud data;

[0143] Resolution division and unit width:

[0144] X-axis resolution unit width:

[0145]

[0146] Y-axis resolution unit width:

[0147]

[0148] In the formula, R e It refers to the spatial resolution or grid size of the unit when meshing the point cloud;

[0149] Point cloud unit index calculation:

[0150] X-axis index, for a point (x i ,y i ,z i ), whose index is:

[0151]

[0152] Y-axis index, for a point (x i ,y i ,z i ), whose index is:

[0153]

[0154] Boundary point extraction:

[0155] X-axis boundary points, at each index I dx , calculate the minimum and maximum y values ​​respectively:

[0156]

[0157] Y-axis boundary points, at each index I dy , calculate the minimum and maximum x values ​​separately:

[0158]

[0159] In the formula, I dx is the index value, indicating the grid unit of the point cloud data in the X-axis direction; dy is the index value, indicating the grid unit of the point cloud data in the Y-axis direction;

[0160] The boundary point set is merged, that is, the final boundary point cloud is the union of the boundary point clouds in the X and Y directions, that is:

[0161] B p =B p_x +B p_y

[0162] In the formula, B p represents the boundary point set of the cabin, B p_x represents the boundary point set in the X direction, B p_y Represents the boundary point set in the Y direction.

[0163] Step S16, obtaining the cabin collision boundary according to the safety parameters and the point cloud boundary data of the cabin, the specific contents are as follows:

[0164] Adjust point coordinates:

[0165] Adjusted minimum point coordinates:

[0166] p x ' _min =p x_min +a

[0167] p′ y_min =p y_min +b

[0168] p′ z_min =p z_min +c

[0169] The adjusted maximum point coordinates:

[0170] p x ' _max =p x_max +a′

[0171] p′ y_max =p y_max +b′

[0172] p′ z_max =p z_max +c′

[0173] In the formula, a, b, c, a', b', c' are determined according to the actual working conditions;

[0174] According to the adjusted coordinates of the minimum and maximum points in the XY direction, the coordinates of the four collision boundary vertices can be obtained. The four vertices are connected in a clockwise direction to obtain a rectangle composed of four line segments, which is the generated cabin danger boundary. Its equations can be defined as follows:

[0175]

[0176] Step S17, determining the initial path point information according to the cabin collision boundary, the specific content is as follows:

[0177] The host computer plans an initial driving path for the loader based on the starting point according to the cabin environment point cloud data collected in real time by the laser radar and the dynamic position data of the loader, and adopts an algorithm based on the combination of global path planning and local obstacle avoidance, and then calculates the planned path point information x ref ,y ref , δ ref Transmitted to the electronic control unit.

[0178] Step S20, the electronic control unit controls the motor controller to drive according to the initial path point information and sends the current vehicle speed to the host computer. The specific steps are as follows:

[0179] The calculation process of vehicle speed v is as follows:

[0180]

[0181] In the formula, w 1 、w 2 、w 3 、w 4 are the wheel speeds of the wheels corresponding to the left wheel hub motor of the front body, the right wheel hub motor of the front body, the left wheel hub motor of the rear body, and the right wheel hub motor of the rear body; R is the tire radius; g is the acceleration of gravity; t is the time; v f is the driving speed; v s is the braking speed.

[0182] Step S30, the host computer executes the bulkhead detection strategy according to the current vehicle speed, and executes the pile and grab bucket detection strategy according to the obstacle point cloud data and the cabin interior image data. The bulkhead detection strategy and the pile and grab bucket detection strategy are used to control the electronic control unit to execute the target action. The specific steps are as follows:

[0183] The bulkhead detection strategy described above includes:

[0184] Step S31, determining the safety distance L according to the current vehicle speed d , the specific process is as follows:

[0185]

[0186] Where v is the current vehicle speed, t r is the reaction time, i.e. the delay time before the host computer detects danger and triggers braking; a max is the maximum acceleration, which is determined by the performance of the loader; a b It is the vehicle's braking deceleration, reflecting the deceleration ability when the vehicle speed drops to zero, and is determined by the loader's braking performance parameters.

[0187] Step S32, four real-time postures of the detection points of the front and rear bodies of the loader are obtained respectively by the articulation angle sensor, four actual distances from the detection points of the front and rear bodies of the loader to the collision boundary of the cabin are obtained according to the four real-time postures of the detection points of the front and rear bodies of the loader, and the minimum distance is obtained according to the four actual distances. The specific process is as follows:

[0188] The actual distance between the loader and the cabin boundary is calculated as follows;

[0189] In the vehicle coordinate system, the real-time coordinates of the four detection points are as follows:

[0190]

[0191] The generated cabin danger boundary is defined as four line segments, and their equations are:

[0192]

[0193] Calculate the distance L from each detection point to the four dangerous boundaries aij (i represents the detection point, j represents the danger boundary), the distance formula is as follows:

[0194]

[0195] Where, i∈{1,2,3,4},j∈{1,2,3,4}.

[0196] Find the minimum value among these distances: L amin =min{L aij |i=1,2,3,4;j=1,2,3,4}

[0197] The obtained L amin It indicates the minimum distance from any of the four detection points of the loader to the dangerous boundary inside the cabin.

[0198] Step S33, determine whether the minimum distance is greater than the safety distance:

[0199] Yes, execute the adaptive correction strategy for the vehicle boundary;

[0200] No, the loader has a collision risk, and sends a braking signal to the electronic control unit. The electronic control unit receives the cabin danger boundary detection signal from the host computer, and makes each wheel hub motor take braking measures according to the signal.

[0201] The above-mentioned stockpile and grab detection strategies include:

[0202] Step S34, clustering the obstacle point cloud data to obtain multiple obstacle areas with cluster numbers, the specific content is as follows:

[0203] Calculate the grid to which the point belongs, and determine its position in the grid based on the coordinates of the point. The offset calculation formula of the point in the grid is:

[0204]

[0205] Where, L g is the side length of the grid;

[0206] Determine whether the point is within the grid range, that is:

[0207] x C ≥0 and x C <L g

[0208] y C ≥0 and y C <L g

[0209] Offset the point cloud coordinates to the grid origin, that is, the lower left corner, and then calculate the grid cell index to which the point belongs. The conversion formula is:

[0210]

[0211] Where N g is the number of mesh divisions;

[0212] The propagation of grid numbers (processing surrounding grids during clustering) is done by traversing a grid with a side length of L for all points in a grid. g2 The neighborhood of , the offset calculation formula is:

[0213]

[0214] Determine whether the neighborhood is out of bounds:

[0215] (x+k XI )≥0 and (x+k XI )<N g

[0216] (y+k YI )≥0 and (y+kYI )<N g

[0217] If a point in the neighborhood grid is not processed, then

[0218] cartesianData[x+k XI ][y+k YI ]=-1

[0219] Where cartesianData is a two-dimensional array representing the grid divided in the Cartesian coordinate system. Each element corresponds to a grid unit and records the state or attribute of the unit.

[0220] Initialization and statistics: First, count the number of grid points. Each time a point cloud point is traversed, the number of points in the corresponding grid unit is increased by 1, that is:

[0221] N g [x I ][y I ]=N g [x I ][y I ]+1

[0222] Secondly, the grid threshold is judged. If the number of points in a grid exceeds the threshold, that is:

[0223] gridNum[x I ][y I ]>1

[0224] Then the grid is marked as an obstacle:

[0225] cartesianData[x I ][y I ]=-1

[0226] Propagate the cluster number. If a grid belongs to an obstacle, call the recursive search function to propagate the cluster number, that is:

[0227] cartesianData[x][y]=Id c

[0228] Where, Id c is an integer representing the number of the current cluster. Each obstacle area cluster is assigned a unique number. After each obstacle area cluster is completed, the new cluster number is automatically increased, that is:

[0229] Id c =Id c +1.

[0230] Step S35, performing bounding box fitting on the plurality of obstacle regions with cluster numbers to obtain eight boundary points of the 3D bounding box, the specific contents are as follows:

[0231] Extract the center point of the bounding box, that is, calculate the center point of the clustered object point cloud (x c ,y c ,z c ):

[0232]

[0233] In the formula, (x i ,y i ,z i ) is the coordinate of the i-th point in the point cloud, and n is the number of points in the point cloud;

[0234] The main direction calculation is to extract the main direction of the point cloud data through principal component analysis (PCA) to obtain the covariance matrix of the point cloud:

[0235]

[0236] In the formula, is the point (x i ,y i ,z i ) and the center point (x c ,y c ,z c )’s offset vector, [x i -x c y i -y c z i -z c ] is the transpose of the offset vector;

[0237] The above formula can be expanded and simplified, namely:

[0238]

[0239] Perform eigenvalue decomposition on C:

[0240]

[0241] In the formula, is the eigenvector, indicating the main direction; j is the eigenvalue, indicating the variance of the main direction;

[0242] Calculate the length, width and height of the bounding box by projecting the point cloud onto the main direction axis and calculating the minimum and maximum values ​​in each direction:

[0243] L x =max(x′)-min(x′),Ly =max(y′)-min(y′),L z =max(z′)-min(z′)

[0244] Where x', y', z' are the coordinates of the point cloud after projection to the main direction; L x , L y , L z Respectively represent the length, width and height of the bounding box;

[0245] Calculate the rotation angle α of the bounding box relative to the world coordinate system in order to grasp the directional characteristics of the target object. The rotation angle can be calculated by the angle between the main direction vector and the X-axis:

[0246]

[0247] In the formula, v 1,x The component of the principal direction vector in the x-axis direction, v 1,y is the component of the principal direction vector in the y-axis direction;

[0248] Determine the eight boundary points of the 3D bounding box, namely:

[0249]

[0250] Step S36, preprocessing the cabin interior image data to obtain corrected image data, and processing the corrected image data using a deep learning algorithm to obtain a two-dimensional detection frame, the specific contents are as follows:

[0251] The host computer preprocesses the image data of the interior of the cabin and corrects the image through radial distortion correction and tangential distortion correction algorithms to eliminate the influence of distortion on image quality. For radial distortion, the correction formula is as follows:

[0252]

[0253] In the formula, k 1 ,k 2 ,k 3 is the radial distortion parameter, (x c ,y c ) is the coordinate value before distortion correction, (x' c ,y' c ) is the coordinate value after distortion correction, r represents the radial distance from the pixel to the image distortion center (usually the optical center of the image, i.e., the lens center);

[0254] For tangential distortion, the correction formula is as follows:

[0255]

[0256] In the formula, p 1 ,p 2 is the tangential distortion parameter, (x c ,y c ) is the coordinate value before distortion correction, (x' c ,y' c ) is the coordinate value after distortion correction.

[0257] The corrected image data is processed using a deep learning algorithm to identify target objects such as grabs and materials in the image.

[0258] Step S37, the eight boundary points of the two-dimensional detection frame and the 3D bounding frame are fused and matched, and the real-time distance between the edge of the material and the loader, the shortest actual distance between the grab and the loader, and the actual vertical distance between the grab and the loader are obtained respectively. The specific contents are as follows:

[0259] The 3D bounding box generated by point cloud clustering is fused and matched with the 2D detection box obtained by deep learning training;

[0260] The algorithm module of the host computer extracts the edge of the material at the identified rear body position and calculates the real-time distance L between it and the rear body of the loader. c ;

[0261] The host computer calculates the actual distance L between the grab bucket and the loader based on high-precision lidar sensor data and precise geometric relationship model. g And the actual vertical distance L top The shortest actual distance between the grab bucket and the loader is L g The calculation formula is as follows:

[0262] In the formula, (X g ,Y g ,Z g ) is the lowest point of the grab bucket, (X 1 ,Y 1 )(X 2 ,Y 2 ),(X 3 ,Y 3 ),(X 4 ,Y 4 ) are the four detection point positions of the loader;

[0263] In addition, the actual vertical distance L of the grab bucket needs to be calculated to avoid collision with the loader when the grab bucket is lowered. top :

[0264] L top =h top +h g

[0265]

[0266] In the formula, h g It is the distance between the lowest point of the grab bucket and the highest point of the loader itself.

[0267] Step S38, determining the shortest actual distance L between the grab bucket and the loader g Is it greater than the preset safety threshold L? sg :

[0268] Yes, proceed to the next step;

[0269] If no, a braking command is sent to the electronic control unit, and the loader waiting time judgment strategy is executed;

[0270] Step S39, determine the actual vertical distance L between the grab bucket and the loader top Is it greater than the preset safety threshold L in the vertical direction? stop :

[0271] Yes, execute the adaptive vehicle boundary correction strategy;

[0272] No, a braking command is sent to the electronic control unit, and a loader waiting time judgment strategy is executed.

[0273] The above-mentioned loader waiting time judgment strategy includes:

[0274] Step S310, determine whether the loader waiting time T is greater than the grab bucket operation time safety threshold T s :

[0275] Yes, re-execute the bulkhead detection strategy and the stockpile and grab detection strategy;

[0276] No, the loader continues to wait until the loader waiting time T is greater than the grab bucket operation time safety threshold T s .

[0277] Adaptive correction of vehicle boundary strategy, including:

[0278] Step S311, obtaining the bucket hydraulic cylinder length L according to the bucket cylinder displacement sensor and the boom cylinder displacement sensor lc And boom hydraulic cylinder length L la , the specific process is as follows:

[0279] First calculate the length L of the bucket hydraulic cylinder lc According to the telescopic displacement of the hydraulic rod, the length of the bucket hydraulic cylinder is calculated in real time through the displacement sensor. The formula is as follows:

[0280] L lc =L lc0 +ΔLlc

[0281] Where, L lc0 is the initial length of the bucket hydraulic cylinder (the length in the fully retracted state), ΔL lc is the extension of the bucket hydraulic cylinder;

[0282] Secondly, calculate the length L of the boom hydraulic cylinder la , according to the telescopic displacement of the hydraulic rod, the length of the boom hydraulic cylinder is calculated in real time through the displacement sensor, and the formula is as follows:

[0283] L la =L la0 +ΔL la

[0284] Where, L la0 is the initial length of the boom hydraulic cylinder (the length in the fully retracted state), ΔL lc is the extension of the boom hydraulic cylinder;

[0285] Step S312, determine the length L of the bucket hydraulic cylinder lc And boom hydraulic cylinder length L la Whether the working range conditions are met:

[0286] Yes, adaptively correct the loader body boundary;

[0287] If no, proceed to the next step;

[0288] Determine whether the job status is:

[0289] Bucket hydraulic cylinder judgment range:

[0290] L lc_min <L lc <L lc_max

[0291] Where, L lc_min L is the minimum working length of the bucket hydraulic cylinder. lc_max is the maximum working length of the bucket hydraulic cylinder;

[0292] Boom hydraulic cylinder judgment range:

[0293] L la_min <L la <L la_max

[0294] Where, L lc_min is the minimum working length of the boom hydraulic cylinder, L la_max It is the maximum working length of the boom hydraulic cylinder.

[0295] Step S313, obtaining the real-time distance between the material boundary point set and the loader boundary detection point and the relative speed between the loader and the material in front of the vehicle body, and determining whether the real-time distance between the material boundary point set and the loader boundary detection point is less than or equal to a preset safety distance and the relative speed between the loader and the material in front of the vehicle body is greater than 0:

[0296] Yes, the loader is at risk of collision, and a signal is sent to the electronic control unit to take braking measures;

[0297] If no, proceed to the next step;

[0298] Step S314, determine whether the loader has reached the end point:

[0299] Yes, end the process;

[0300] No, execute the bulkhead detection strategy and the stockpile and grab detection strategy.

[0301] The specific calculation process of the real-time distance between the above material boundary point set and the loader boundary detection point is as follows:

[0302] First, the detected material point cloud data is extracted through the edge detection algorithm, denoted as P i (x i ,y i ), where i = 1, 2, ..., N, N is the total number of boundary points; (X 1 ,Y 1 )(X 2 ,Y 2 ),(X 3 ,Y 3 ),(X 4 ,Y 4 ) is the position of the boundary detection point of the loader; then the real-time distance L between the material boundary point set and the loader boundary detection point is calculated c , and judge L c Is it less than or equal to the preset safety distance L? sc , real-time distance L c The calculation formula is as follows:

[0303]

[0304] The specific steps for judging whether the loader has reached the end point are as follows:

[0305] Calculate the end point distance error E d :

[0306]

[0307] In the formula, x s is the horizontal coordinate of the current point; y sis the ordinate of the current point; x tar is the horizontal coordinate of the target end point, y tar is the ordinate of the target end point;

[0308] Determine whether the end position has been reached, that is:

[0309]

[0310] In the formula, T means that the destination has been reached; F means that the destination has not been reached; E dth is the allowable error range.

[0311] After the above-mentioned adaptive correction of the loader body boundary, the following steps are also included:

[0312] Get the real-time distance between the rear body material boundary and the loader boundary detection point, and determine whether the real-time distance between the rear body material boundary and the loader boundary detection point is less than or equal to the preset safety distance:

[0313] Yes, the loader is at risk of collision, and a signal for taking braking measures is sent to the electronic control unit;

[0314] If no, proceed to the next step;

[0315] Determine whether the loader has reached the end point:

[0316] Yes, end the process;

[0317] No, execute the bulkhead detection strategy and the stockpile and grab detection strategy.

[0318] The calculation process of the real-time distance between the rear body material boundary and the loader boundary detection point is as follows:

[0319] First, the detected rear body material point cloud data is extracted through the edge detection algorithm, and the boundary points are recorded as P i (x i ,y i ), where i = 1, 2, ..., N, N is the total number of boundary points; (X 3 ,Y 3 ),(X 4 ,Y 4 )(X 5 ,Y 5 ),(X 6 ,Y 6 ) is the position of the boundary detection point of the rear body of the loader; then calculate the real-time distance L between the boundary of the rear body material and the boundary detection point of the loader c ′, and judge L c Is ' less than or equal to the preset safety distance L? sc ; Real-time distance L c The calculation formula is as follows:

[0320]

[0321] Embodiment 3: A loader according to an exemplary embodiment includes: a loader body and the unmanned loader cabin collision prevention system described in Embodiment 1.

[0322] Although the embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and the embodiments. It can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily realized. Therefore, without departing from the general concept defined by the claims and equivalent scope, the present invention is not limited to the specific details and the illustrations shown and described here.

Claims

1. An unmanned loader cabin collision prevention system, characterized in that: It includes a host computer electrically connected to an electronic control unit, a bucket cylinder displacement sensor, a boom cylinder displacement sensor, an articulation angle sensor and a motor controller, wherein the motor controller is electrically connected to a wheel hub motor, the host computer is electrically connected to at least one camera and at least one laser radar, respectively, and the articulation angle sensor is arranged at the articulation point of the front and rear bodies of the loader.

2. The unmanned loader cabin collision prevention system according to claim 1, characterized in that: The motor controller includes: a front body right motor controller, a front body left motor controller, a rear body right motor controller, and a rear body left motor controller. The wheel hub motor includes: a front body right wheel hub motor electrically connected to the front body right motor controller, a front body left wheel hub motor electrically connected to the front body left motor controller, a rear body right wheel hub motor electrically connected to the rear body right motor controller, and a rear body left wheel hub motor electrically connected to the rear body left motor controller.

3. A method for preventing collision in a cabin of an unmanned loader, applied to the system for preventing collision in a cabin of an unmanned loader according to claim 1 or 2, characterized in that: include: The host computer obtains cabin interior point cloud data, obstacle point cloud data, cabin interior image data and loader dynamic position data through the camera and the laser radar respectively, determines initial path point information according to the cabin interior point cloud data and the loader dynamic position data, and sends it to the electronic control unit; The electronic control unit controls the motor controller to drive according to the initial path point information and sends the current vehicle speed to the host computer; The host computer executes a bulkhead detection strategy according to the current vehicle speed, and executes a pile and grab detection strategy according to the obstacle point cloud data and the cabin interior image data. The bulkhead detection strategy and the pile and grab detection strategy are used to control the electronic control unit to execute the target action.

4. The method for preventing collision in a cabin of an unmanned loader according to claim 3, characterized in that: The determining of the initial path point information according to the cabin internal point cloud data and the loader dynamic position data includes: Preprocessing and extracting the cabin interior point cloud data to obtain point cloud data of a first area of ​​interest; Based on an adaptive ground estimation algorithm, the point cloud data of the first area of ​​interest is filtered out to obtain the point cloud data of the second area of ​​interest; The point cloud data of the second area of ​​interest is filtered out by the loader body through convex hull calculation, point cloud clipping and dynamic rotation transformation to obtain point cloud data of the third area of ​​interest; Extracting the point cloud data of the third region of interest based on the Alpha Shape algorithm to obtain point cloud boundary data of the cabin; Obtaining a cabin collision boundary according to safety parameters and point cloud boundary data of the cabin; Initial path point information is determined according to the cabin collision boundary.

5. The method for preventing collision in a cabin of an unmanned loader according to claim 4, characterized in that: The bulkhead detection strategy includes: determining a safe distance according to the current vehicle speed; The four real-time postures of the detection points of the front and rear bodies of the loader are obtained respectively by the articulation angle sensor, and four actual distances from the detection points of the front and rear bodies of the loader to the collision boundary of the cabin are obtained according to the four real-time postures of the detection points of the front and rear bodies of the loader, and the minimum distance is obtained according to the four actual distances; Determine whether the minimum distance is greater than the safety distance: Yes, execute the adaptive correction strategy for the vehicle boundary; No, there is a risk of collision for the loader, and a signal to take braking measures is sent to the electronic control unit.

6. The method for preventing collision in a cabin of an unmanned loader according to claim 5, characterized in that: The material pile and grab bucket detection strategy includes: Performing clustering processing on the obstacle point cloud data to obtain a plurality of obstacle areas with cluster numbers; Performing bounding box fitting on each of the plurality of obstacle regions with cluster numbers to obtain eight boundary points of a 3D bounding box; Preprocessing the cabin interior image data to obtain corrected image data, and processing the corrected image data using a deep learning algorithm to obtain a two-dimensional detection frame; The eight boundary points of the two-dimensional detection frame and the 3D bounding frame are fused and matched, and the real-time distance between the edge of the material and the loader, the shortest actual distance between the grab bucket and the loader, and the actual vertical distance between the grab bucket and the loader are obtained respectively; Determine whether the shortest actual distance between the grab bucket and the loader is greater than a preset safety threshold: Yes, proceed to the next step; If no, a braking command is sent to the electronic control unit, and a loader waiting time determination strategy is executed; Determine whether the actual vertical distance between the grab bucket and the loader is greater than a preset safety threshold in the vertical direction: Yes, execute the adaptive vehicle boundary correction strategy; No, a braking instruction is sent to the electronic control unit, and the loader waiting time judgment strategy is executed.

7. The method for preventing collision in a cabin of an unmanned loader according to claim 6, characterized in that: The loader waiting time judgment strategy includes: Determine whether the loader waiting time is greater than the grab bucket operation time safety threshold: Yes, re-execute the bulkhead detection strategy and the stockpile and grab detection strategy; If no, the loader continues to wait until the loader waiting time is greater than the grab bucket operation time safety threshold.

8. The method for preventing collision in a cabin of an unmanned loader according to claim 7, characterized in that: The adaptive correction vehicle boundary strategy includes: According to the bucket cylinder displacement sensor and the boom cylinder displacement sensor, the length of the bucket hydraulic cylinder and the length of the boom hydraulic cylinder are respectively obtained; Determine whether the bucket hydraulic cylinder length and the boom hydraulic cylinder length both meet the working range conditions: Yes, adaptively correct the loader body boundary; If no, proceed to the next step; Obtain the real-time distance between the material boundary point set and the loader boundary detection point and the relative speed between the loader and the material in front of the vehicle body, and determine whether the real-time distance between the material boundary point set and the loader boundary detection point is less than or equal to the preset safety distance and the relative speed between the loader and the material in front of the vehicle body is greater than 0: Yes, the loader is at risk of collision, and a signal for taking braking measures is sent to the electronic control unit; If no, proceed to the next step; Determine whether the loader has reached the end point: Yes, end the process; No, execute the bulkhead detection strategy and the stockpile and grab detection strategy.

9. The method for preventing collision in a cabin of an unmanned loader according to claim 7 or 8, characterized in that: After the adaptive correction of the loader body boundary is performed, the method further includes: Obtain the real-time distance between the rear vehicle body material boundary and the loader boundary detection point, and determine that the real-time distance between the rear vehicle body material boundary and the loader boundary detection point is less than or equal to a preset safety distance: Yes, the loader is at risk of collision, and a signal for taking braking measures is sent to the electronic control unit; If no, proceed to the next step; Determine whether the loader has reached the end point: Yes, end the process; No, execute the bulkhead detection strategy and the stockpile and grab detection strategy.

10. A loader, characterized in that: include: A loader body and an unmanned loader cabin collision prevention system as described in claim 1 or 2.

Images