Truck-mounted crane manual arm operation path planning method and system facing narrow space

Abstract

The invention provides a narrow-space-oriented truck-mounted crane manual arm operation path planning method and system, and relates to the technical field of path planning, and the method comprises the steps: carrying out the multi-source information fusion through binocular vision and laser ranging, and building an operation space three-dimensional model; a convolutional neural network is adopted to identify an obstacle, a kinematics constraint is combined to establish an obstacle avoidance cost function, and a bidirectional probability sampling algorithm is utilized to generate and optimize an initial path. Afterwards, a target path is screened by considering the dynamic constraint of the load weight, and dynamic adjustment is carried out based on a comprehensive evaluation function of path length, smoothness and safety degree and a virtual spring damping model; and finally, performing polynomial fitting on the adjusted path, calculating a joint angle and performing local correction according to a kinematic model, and finally outputting a working path meeting constraint conditions. According to the invention, the working efficiency and safety of the truck-mounted crane in a narrow space are effectively improved.

Description

Technical Field

[0001] The invention relates to a path planning technology, and in particular to a method and system for planning a path for a manual boom operation of a truck-mounted crane in a narrow space. Background Art

[0002] Truck-mounted cranes are widely used in logistics, construction, and rescue scenarios. Their operating environments are often complex and changeable, especially in narrow spaces. The operation of manual arms is limited by densely distributed obstacles and limited operating space. Traditional path planning methods are difficult to meet the needs of high-precision obstacle avoidance and dynamic adjustment. In the existing technology, most path planning methods rely on a single sensor and lack multi-source information fusion, resulting in insufficient perception of obstacles in complex environments and inability to accurately identify obstacle attributes and their impact on path planning.

[0003] At the same time, when the boom of a truck-mounted crane is operating in a narrow space, it is necessary to consider the different characteristics of rigid obstacles and deformable obstacles. However, the existing path planning algorithms usually adopt a unified safety distance setting and lack targeted strategies, which makes it difficult to balance the safety and effectiveness of the path planning results. In addition, due to the complex kinematic and dynamic constraints of the boom of a truck-mounted crane, the existing path planning scheme is insufficient in dynamic adjustment and optimization, making it difficult to generate a smooth and optimized path that meets the operation requirements in real time, affecting the operation efficiency and safety.

[0004] Therefore, there is an urgent need for a technical solution for path planning of the manual boom of a truck-mounted crane in narrow spaces. This solution can combine multi-source information perception, obstacle classification and recognition, and real-time path optimization to improve the path planning accuracy, safety, and dynamic adjustment capability of the manual boom of a truck-mounted crane in narrow space operations. Summary of the invention

[0005] The embodiments of the present invention provide a method and system for planning an operation path of a manual boom of a truck-mounted crane in a narrow space, which can solve the problems in the prior art.

[0006] According to a first aspect of the embodiments of the present invention,

[0007] A method for planning the operation path of a manual boom of a truck-mounted crane in a narrow space is provided, comprising:

[0008] The environment image of the working space is collected by binocular stereo vision cameras installed at both ends of the vehicle-mounted crane, the corresponding relationship of the feature points of the environment image is extracted by a feature point matching algorithm, and the depth information is calculated to obtain the first depth information, the working space is scanned by a laser ranging sensor installed on the top of the vehicle-mounted crane to obtain the obstacle contour point cloud, the second depth information is obtained based on the obstacle contour point cloud calculation, the first depth information and the second depth information are multi-source information fused to obtain fused depth information, and a three-dimensional model of the working space is established in combination with the posture information obtained by the vehicle-mounted posture sensor of the vehicle-mounted crane, the three-dimensional model is spatially indexed and data compressed by a voxel partitioning method, and a real-time environment map is output;

[0009] Based on the real-time environment map, a convolutional neural network is used to identify and classify obstacles in the working space to obtain obstacle attribute information, and obstacles are divided into rigid obstacles and deformable obstacles according to the obstacle attribute information. A first safety distance threshold and a second safety distance threshold are set for the rigid obstacles and the deformable obstacles, respectively, and an obstacle avoidance cost function is established in combination with the kinematic constraint parameters of the manual arm of the truck crane, and a bidirectional probability sampling algorithm is used to generate path nodes. Based on the obstacle avoidance cost function, the path nodes are optimally connected to obtain multiple groups of initial paths, and the load weight information of the manual arm of the truck crane is input into the dynamics model, the joint rotation torque of the multiple groups of initial paths is calculated, and the path that meets the dynamic constraints is selected as the target optimization path;

[0010] For the target optimized path, a comprehensive evaluation function including path length evaluation items, motion smoothness evaluation items and obstacle avoidance safety evaluation items is established, a virtual spring damping model is established in the coordinate system of the end effector of the manual arm of the truck crane, the spring stiffness coefficient and the damping coefficient of the virtual spring damping model are adjusted according to the evaluation score calculated by the comprehensive evaluation function, the target optimized path is dynamically adjusted in real time to obtain an adjusted path, a piecewise continuous polynomial function is used to fit the adjusted path to obtain a fitted path, the joint angles of the fitted path are calculated based on the kinematic model of the manual arm of the truck crane, the path segments that do not meet the kinematic constraints are locally corrected, and finally an operation path that meets the constraints is output.

[0011] In an optional embodiment,

[0012] The binocular stereo vision cameras installed at both ends of the vehicle-mounted boom frame are used to collect the environment image of the working space, the feature point correspondence of the environment image is extracted by the feature point matching algorithm, and the depth information is calculated to obtain the first depth information, the laser ranging sensor installed at the top of the vehicle-mounted boom frame is used to scan the working space to obtain the obstacle contour point cloud, the second depth information is obtained according to the obstacle contour point cloud calculation, and the first depth information and the second depth information are multi-source information fused to obtain the fused depth information, including:

[0013] The binocular stereo vision cameras installed at both ends of the vehicle-mounted boom frame are used to collect the working environment image and perform local histogram equalization processing to obtain a pre-processed image, an image autocorrelation matrix is ​​constructed according to the horizontal gradient and the vertical gradient of the pre-processed image, a ratio of the determinant value and the trace of the image autocorrelation matrix is ​​calculated to obtain a corner point response function, a mean value and a standard deviation of a local area of ​​the image are calculated based on the corner point response function to construct an adaptive threshold, and the corner point response function is screened by the adaptive threshold to obtain feature point data;

[0014] Establishing a matching cost function based on spatial geometric constraints for the feature point data, wherein the matching cost function includes a basic matrix constraint term and a normalized cross-correlation term, determining a feature point correspondence relationship according to a minimum value of the matching cost function, projecting the feature point correspondence relationship into an image epipolar coordinate system to calculate a disparity value, and converting the disparity value into first depth information in combination with a camera calibration parameter;

[0015] The laser ranging sensor installed on the top of the vehicle-mounted boom is used to scan the working environment to obtain point cloud data, calculate the local density distribution of the point cloud data, determine the cluster center point based on the local density distribution, calculate the minimum distance from each sampling point to the cluster center point, remove noise points whose distance exceeds the standard distance threshold, and perform regional growth on the remaining point cloud data based on the cluster center to obtain segmented point clouds, and use an iterative closest point algorithm to perform spatial registration and three-dimensional reconstruction on the segmented point clouds to obtain second depth information;

[0016] A state vector including three-dimensional coordinate components and velocity components is established, a state space model is constructed using the first depth information and the second depth information as observation data, a state prediction value is obtained according to the state space model, and a Kalman gain matrix is ​​calculated, the observation data and the state prediction value are fused using the Kalman gain matrix to obtain a state estimation value, a state estimation error between the state estimation value and the observation data is calculated, a measurement confidence of the depth information is calculated according to the state estimation error, a fusion weight coefficient is determined based on the measurement confidence, and the first depth information and the second depth information are fused based on the fusion weight coefficient to obtain fused depth information.

[0017] In an optional embodiment,

[0018] The three-dimensional model of the working space is established by combining the posture information obtained by the posture sensor on the truck crane, and the three-dimensional model is spatially indexed and data compressed by the voxel partitioning method, and the real-time environment map is output, including:

[0019] Acquire the position coordinates and attitude angle data of the truck crane from the vehicle-mounted attitude sensor, construct a transformation matrix from the vehicle coordinate system to the global coordinate system according to the position coordinate data and the attitude angle data, and multiply the transformation matrix with the spatial point set in the fused depth information to obtain the depth point cloud data in the global coordinate system;

[0020] Calculating the spatial range of the depth point cloud data to obtain a spatial bounding box, dividing the spatial bounding box into an initial regular grid, calculating the Euclidean distance between the internal points of each initial regular grid to obtain a spatial distribution density, and adaptively adjusting the grid resolution according to the spatial distribution density to obtain a voxel grid;

[0021] Calculating the spatial centroid and covariance matrix of the point cloud in each grid cell in the voxel grid, performing eigenvalue decomposition on the covariance matrix to obtain eigenvalues ​​and principal direction vectors, using the principal direction vectors as local surface normal vectors, calculating the local curvature according to the ratio of the eigenvalues, and calculating the voxel occupancy probability of the grid cell by counting the number of points in the grid cell;

[0022] Establishing a sliding update mechanism of the time window, projecting the newly acquired depth point cloud data to the global coordinate system through the transformation matrix, spatially registering the projected point cloud data with the voxel grid, updating the state of the grid unit in the overlapping area using the voxel occupancy probability, and applying an attenuation coefficient to the occupancy probability according to the time window range to obtain the updated voxel occupancy probability;

[0023] A spatial distance field function is constructed based on the updated voxel occupancy probability and the local surface normal vector, an isosurface of the spatial distance field function is extracted to obtain an environmental surface mesh model, the environmental surface mesh model is simplified and compressed according to the local curvature, and a real-time environmental map is output.

[0024] In an optional embodiment,

[0025] Based on the real-time environment map, a convolutional neural network is used to identify and classify obstacles in the working space to obtain obstacle attribute information. According to the obstacle attribute information, the obstacles are divided into rigid obstacles and deformable obstacles. A first safety distance threshold and a second safety distance threshold are set for the rigid obstacles and the deformable obstacles, respectively. An obstacle avoidance cost function is established in combination with the kinematic constraint parameters of the truck crane manual arm. A bidirectional probability sampling algorithm is used to generate path nodes. Based on the obstacle avoidance cost function, the path nodes are optimized and connected to obtain multiple groups of initial paths, including:

[0026] A convolutional neural network is used to extract features from a real-time environment map. The convolutional neural network includes a feature extraction backbone network, a feature fusion neck network, and a prediction head network. The feature extraction backbone network extracts multi-scale features through a spatial pyramid pooling module and a channel attention mechanism. The feature fusion neck network uses an adaptive feature pyramid structure and skip connections to fuse features at different levels. The prediction head network outputs the location information, size information, and category information of obstacles through parallel branches.

[0027] Classifying obstacles into rigid obstacles and deformable obstacles according to the category information, and calculating a first safety distance threshold of the rigid obstacles and a second safety distance threshold of the deformable obstacles based on a basic safety distance and an end effector speed, wherein the first safety distance threshold is greater than the second safety distance threshold;

[0028] Obtaining the upper and lower limits of the joint position of the truck crane to obtain the joint position constraint, obtaining the maximum speed of the joint movement to obtain the speed constraint, obtaining the maximum acceleration of the joint movement to obtain the acceleration constraint, and combining the joint position constraint, the speed constraint and the acceleration constraint with the first safety distance threshold and the second safety distance threshold to construct an obstacle avoidance cost function;

[0029] The current posture of the truck-mounted crane is obtained as the starting point of the path planning, and the target working position is obtained as the end point of the path planning. A bidirectional probabilistic path rapid expansion algorithm is used to simultaneously generate a search tree from the starting point and the end point. At each sampling, the target configuration is selected as a sampling node based on a set probability, and the distance from the sampling node to the nearest tree node is calculated. The satisfaction of the joint position constraint, velocity constraint and acceleration constraint of the sampling node is verified, and a candidate path node is generated;

[0030] The candidate path nodes are optimized based on the obstacle avoidance cost function, and the optimized path nodes are connected. When the distance between two connected search tree nodes is less than a preset node distance threshold, multiple groups of initial paths are generated.

[0031] In an optional embodiment,

[0032] Inputting the load weight information of the manual arm of the truck crane into the dynamics model, calculating the joint rotation torques of the multiple groups of initial paths, and selecting the path that satisfies the dynamics constraints as the target optimization path includes:

[0033] A dynamic model of the manual arm of the truck-mounted crane is established, wherein the dynamic model includes a connecting rod angular velocity transmission equation, an angular acceleration transmission equation, and a linear acceleration transmission equation. Based on the dynamic model, a dynamic relationship between adjacent connecting rods is constructed to obtain the connecting rod dynamic state parameters;

[0034] Obtaining the load weight of the truck crane manual arm, substituting the load weight and the connecting rod dynamic state parameters into the dynamic equation of the end effector, and calculating the dynamic parameters of the end effector;

[0035] The dynamic parameters of the end effector are transformed into coordinates through the Jacobian matrix, and the inverse dynamics recursive method is used to calculate the torque of each joint by combining the mass of the connecting rod and the connecting rod resultant force, and the maximum torque constraint of each joint is obtained;

[0036] Node sampling is performed on multiple groups of initial paths, kinematic information of each sampled node is extracted, the kinematic information is substituted into the calculation formula of each joint torque, the torque curve of each initial path is obtained, and a path subset that meets the dynamic constraints is screened out by comparing the torque curve with the joint maximum torque constraint;

[0037] The torque index value and trajectory characteristic value of each path in the path subset are calculated, the path subset is comprehensively evaluated based on the torque index value and trajectory characteristic value, and the path with the highest comprehensive evaluation score is selected as the target optimization path.

[0038] In an optional embodiment,

[0039] For the target optimization path, a comprehensive evaluation function including a path length evaluation item, a motion smoothness evaluation item, and an obstacle avoidance safety evaluation item is established, a virtual spring damping model is established in the coordinate system of the end effector of the manual arm of the truck crane, and the spring stiffness coefficient and the damping coefficient of the virtual spring damping model are adjusted according to the evaluation score calculated by the comprehensive evaluation function. The target optimization path is adjusted dynamically in real time to obtain an adjusted path including:

[0040] An evaluation system is established for the path node position coordinate sequence and the joint angle sequence at the path nodes of the target optimization path. The path length evaluation item is obtained by calculating the weighted combination of the Manhattan distance and the Chebyshev distance between adjacent nodes. The motion smoothness evaluation item is obtained by calculating the linear combination of the first-order difference and the second-order difference of the joint angle at the node. The obstacle avoidance safety evaluation item is calculated by constructing a potential field function that considers the obstacle shape and motion trend. The path length evaluation item, the motion smoothness evaluation item and the obstacle avoidance safety evaluation item are combined using an adaptive weight coefficient to obtain a comprehensive evaluation function.

[0041] A nonlinear virtual spring damping model is established in the coordinate system of the end effector of the manual arm of the truck crane, and a fuzzy control rule is used to adaptively adjust the spring stiffness coefficient and the damping coefficient of the virtual spring damping model according to the evaluation score of the comprehensive evaluation function;

[0042] Substituting the spring stiffness coefficient and the damping coefficient into a virtual spring damping model, calculating the virtual force acting on the path node based on the nonlinear combination of the position deviation and the speed deviation of the path node, making real-time dynamic adjustments to the target optimization path according to the virtual force, and using the fourth-order Runge-Kutta method to solve the node motion equation to obtain the displacement adjustment of the path node;

[0043] Applying the displacement adjustment amount to the path node position coordinates to obtain the adjusted path node position, and determining whether the joint angle, angular velocity and angular acceleration corresponding to the adjusted path node position meet the motion constraints by calculating the Jacobian matrix at the path node;

[0044] When the motion constraint is satisfied, the adjusted path node position is substituted into the comprehensive evaluation function to recalculate the evaluation score. If the evaluation score is greater than the dynamic convergence threshold calculated based on the current number of iterations and has not reached the preset upper limit of the number of iterations, the process returns to the step of adjusting the spring stiffness coefficient and the damping coefficient of the virtual spring damping model until an adjusted path that satisfies the constraint condition is obtained.

[0045] In an optional embodiment,

[0046] The adjusted path is fitted by a piecewise continuous polynomial function to obtain a fitting path. The joint angles of the fitting path are calculated based on the kinematic model of the truck crane boom. The path segments that do not meet the kinematic constraints are locally corrected. The final output of the operation path that meets the constraints includes:

[0047] Acquire mechanical limit values ​​and speed limit values ​​of each joint of the manual arm of the truck crane, wherein the mechanical limit values ​​include upper and lower limits of the joint rotation angle, and the speed limit values ​​include the maximum value of the joint angular velocity; and establish a kinematic constraint condition matrix based on the mechanical limit values ​​and speed limit values;

[0048] Constructing a position component polynomial function that uses multiple control points to determine the shape of the curve, and constructing a spherical linear interpolation function based on a unit quaternion as an attitude component polynomial function; combining the position component polynomial function and the attitude component polynomial function to form a piecewise continuous polynomial function;

[0049] The node position coordinate sequence and the node attitude angle sequence of the adjusted path are used as input data, the coefficients of the position component polynomial function and the attitude component polynomial function are solved by the least square method to obtain the polynomial coefficients, and the fitting path is calculated according to the polynomial coefficients to obtain the position coordinate sequence and the attitude angle sequence of the fitting path;

[0050] A manual arm kinematic model is established based on a link parameter method, and the position coordinate sequence and posture angle sequence of the fitting path are substituted into the manual arm kinematic model to obtain a joint angle sequence, and the joint angle sequence is compared with a kinematic constraint matrix to calculate the angle excess and speed excess of each joint, and the path segment interval that does not meet the constraint condition is determined according to the angle excess and speed excess;

[0051] Constructing a transition curve within the path segment interval, converting the kinematic constraint matrix into the boundary constraint of the transition curve, solving the position coordinates and attitude angle of the transition node that meets the boundary constraint; substituting the transition node into a piecewise continuous polynomial function to obtain a corrected path segment;

[0052] An interpolation method that maintains geometric continuity is used to splice the path segment that meets the constraint conditions with the corrected path segment to obtain an operation path; the operation path is substituted into the kinematic model to solve the joint angle sequence, determine whether the kinematic constraint condition matrix is ​​met, and output the operation path when the constraint conditions are met.

[0053] According to a second aspect of the embodiments of the present invention,

[0054] Provided is a truck-mounted crane manual arm operation path planning system for narrow spaces, including:

[0055] The first unit is used to collect an environmental image of the working space through binocular stereo vision cameras installed at both ends of the vehicle-mounted crane, extract the corresponding relationship of feature points of the environmental image through a feature point matching algorithm and calculate the depth information to obtain first depth information, scan the working space through a laser ranging sensor installed on the top of the vehicle-mounted crane to obtain an obstacle contour point cloud, obtain second depth information based on the obstacle contour point cloud calculation, perform multi-source information fusion on the first depth information and the second depth information to obtain fused depth information, establish a three-dimensional model of the working space in combination with the posture information obtained by the vehicle-mounted posture sensor of the vehicle-mounted crane, perform spatial indexing and data compression on the three-dimensional model through a voxel partitioning method, and output a real-time environmental map;

[0056] The second unit is used to identify and classify obstacles in the working space based on the real-time environment map by using a convolutional neural network to obtain obstacle attribute information, classify the obstacles into rigid obstacles and deformable obstacles according to the obstacle attribute information, set a first safety distance threshold and a second safety distance threshold for the rigid obstacles and the deformable obstacles respectively, establish an obstacle avoidance cost function in combination with the kinematic constraint parameters of the manual arm of the truck crane, generate path nodes by using a bidirectional probability sampling algorithm, optimize and connect the path nodes based on the obstacle avoidance cost function to obtain multiple groups of initial paths, input the load weight information of the manual arm of the truck crane into the dynamics model, calculate the joint rotation torque of the multiple groups of initial paths, and select the path that meets the dynamic constraints as the target optimization path;

[0057] The third unit is used to establish a comprehensive evaluation function including path length evaluation items, motion smoothness evaluation items and obstacle avoidance safety evaluation items for the target optimization path, establish a virtual spring damping model in the coordinate system of the end effector of the manual arm of the truck crane, adjust the spring stiffness coefficient and the damping coefficient of the virtual spring damping model according to the evaluation score calculated by the comprehensive evaluation function, perform real-time dynamic adjustment on the target optimization path to obtain an adjusted path, use a piecewise continuous polynomial function to fit the adjusted path to obtain a fitting path, calculate the joint angles of the fitting path based on the kinematic model of the manual arm of the truck crane, perform local correction on the path segments that do not meet the kinematic constraints, and finally output an operation path that meets the constraints.

[0058] According to a third aspect of the embodiments of the present invention,

[0059] An electronic device is provided, comprising:

[0060] processor;

[0061] a memory for storing processor-executable instructions;

[0062] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0063] According to a fourth aspect of the embodiments of the present invention,

[0064] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0065] In this embodiment, the environmental information of the working space is accurately collected through the coordinated work of the binocular stereo vision camera and the laser ranging sensor, and the depth information is integrated to generate a high-precision real-time environmental map to provide reliable data support for subsequent operations; the convolutional neural network is used to accurately identify and classify the attributes of obstacles, and differentiated safety distances are set for different types of obstacles to improve the flexibility and safety of obstacle avoidance; the path planning process is optimized by combining the bidirectional probability sampling algorithm with the dynamic constraints to ensure the stability and effectiveness of the manual arm of the truck crane in complex environments; the comprehensive evaluation function is combined with the virtual spring damping model to achieve dynamic optimization and adjustment of the path and improve the smoothness and safety of the path; based on the local correction of the kinematic model, it is ensured that the path planning meets the actual operation constraints, and finally an efficient, safe and executable operation path is output, thereby improving the intelligence level and operation efficiency of the truck crane. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 It is a flow chart of a method for planning a manual arm operation path of a truck-mounted crane in a narrow space according to an embodiment of the present invention;

[0067] Figure 2 The present invention is a schematic diagram of the structure of a system for planning a manual arm operation path for a truck-mounted crane in a narrow space according to an embodiment of the present invention. DETAILED DESCRIPTION

[0068] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0069] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0070] Figure 1 FIG. 1 is a flow chart of a method for planning a manual arm operation path of a truck-mounted crane in a narrow space according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0071] S101. Collecting an environmental image of the working space by using binocular stereo vision cameras installed at both ends of the vehicle-mounted crane, extracting the correspondence between feature points of the environmental image by using a feature point matching algorithm and performing depth information calculation to obtain first depth information, scanning the working space by using a laser rangefinder sensor installed at the top of the vehicle-mounted crane to obtain an obstacle contour point cloud, obtaining second depth information based on the obstacle contour point cloud calculation, performing multi-source information fusion on the first depth information and the second depth information to obtain fused depth information, establishing a three-dimensional model of the working space in combination with the posture information obtained by the vehicle-mounted posture sensor of the vehicle-mounted crane, performing spatial indexing and data compression on the three-dimensional model by using a voxel partitioning method, and outputting a real-time environmental map;

[0072] S102. Based on the real-time environment map, a convolutional neural network is used to identify and classify obstacles in the working space to obtain obstacle attribute information, and obstacles are divided into rigid obstacles and deformable obstacles according to the obstacle attribute information. A first safety distance threshold and a second safety distance threshold are set for the rigid obstacles and the deformable obstacles, respectively. An obstacle avoidance cost function is established in combination with the kinematic constraint parameters of the manual arm of the truck crane, and a bidirectional probability sampling algorithm is used to generate path nodes. Based on the obstacle avoidance cost function, the path nodes are optimally connected to obtain multiple groups of initial paths, and the load weight information of the manual arm of the truck crane is input into the dynamics model, the joint rotation torque of the multiple groups of initial paths is calculated, and the path that meets the dynamic constraints is selected as the target optimization path;

[0073] S103. For the target optimized path, a comprehensive evaluation function including path length evaluation items, motion smoothness evaluation items and obstacle avoidance safety evaluation items is established, a virtual spring damping model is established in the coordinate system of the end effector of the manual arm of the truck crane, the spring stiffness coefficient and the damping coefficient of the virtual spring damping model are adjusted according to the evaluation score calculated by the comprehensive evaluation function, the target optimized path is dynamically adjusted in real time to obtain an adjusted path, a piecewise continuous polynomial function is used to fit the adjusted path to obtain a fitted path, the joint angles of the fitted path are calculated based on the kinematic model of the manual arm of the truck crane, local corrections are made to the path segments that do not meet the kinematic constraints, and finally an operation path that meets the constraints is output.

[0074] In an optional implementation, binocular stereo vision cameras installed at both ends of a vehicle-mounted boom frame are used to collect an environmental image of the working space, a feature point matching algorithm is used to extract the corresponding relationship of feature points of the environmental image and depth information is calculated to obtain first depth information, a laser ranging sensor installed at the top of the vehicle-mounted boom frame is used to scan the working space to obtain an obstacle contour point cloud, and second depth information is obtained based on the obstacle contour point cloud calculation, and multi-source information fusion of the first depth information and the second depth information is performed to obtain fused depth information, including:

[0075] The binocular stereo vision cameras installed at both ends of the vehicle-mounted boom frame are used to collect the working environment image and perform local histogram equalization processing to obtain a pre-processed image, an image autocorrelation matrix is ​​constructed according to the horizontal gradient and the vertical gradient of the pre-processed image, a ratio of the determinant value and the trace of the image autocorrelation matrix is ​​calculated to obtain a corner point response function, a mean value and a standard deviation of a local area of ​​the image are calculated based on the corner point response function to construct an adaptive threshold, and the corner point response function is screened by the adaptive threshold to obtain feature point data;

[0076] Establishing a matching cost function based on spatial geometric constraints for the feature point data, wherein the matching cost function includes a basic matrix constraint term and a normalized cross-correlation term, determining a feature point correspondence relationship according to a minimum value of the matching cost function, projecting the feature point correspondence relationship into an image epipolar coordinate system to calculate a disparity value, and converting the disparity value into first depth information in combination with a camera calibration parameter;

[0077] The laser ranging sensor installed on the top of the vehicle-mounted boom is used to scan the working environment to obtain point cloud data, calculate the local density distribution of the point cloud data, determine the cluster center point based on the local density distribution, calculate the minimum distance from each sampling point to the cluster center point, remove noise points whose distance exceeds the standard distance threshold, and perform regional growth on the remaining point cloud data based on the cluster center to obtain segmented point clouds, and use an iterative closest point algorithm to perform spatial registration and three-dimensional reconstruction on the segmented point clouds to obtain second depth information;

[0078] A state vector including three-dimensional coordinate components and velocity components is established, a state space model is constructed using the first depth information and the second depth information as observation data, a state prediction value is obtained according to the state space model, and a Kalman gain matrix is ​​calculated, the observation data and the state prediction value are fused to obtain a state estimation value using the Kalman gain matrix, a state estimation error between the state estimation value and the observation data is calculated, measurement confidence of the depth information is calculated according to the state estimation error, a fusion weight coefficient is determined based on the measurement confidence, and the first depth information and the second depth information are fused based on the fusion weight coefficient to obtain fused depth information.

[0079] For the operation path planning of the boom of a truck-mounted crane in a narrow space, this embodiment proposes a multi-source information fusion depth information acquisition method based on binocular vision and laser ranging to build an accurate environment model.

[0080] Exemplarily, first, binocular stereo vision cameras installed at both ends of the vehicle-mounted boom are used to collect images of the working environment. In order to enhance the image quality, the collected images are subjected to local histogram equalization. Specifically, the image is divided into several small blocks, each of which is subjected to histogram equalization, and then the processed small blocks are spliced ​​into a complete image, thereby improving the contrast and clarity of the image. For example, the image is divided into small blocks of 16×16 pixels, and the histogram of each small block is calculated and equalized.

[0081] Next, feature points are extracted from the preprocessed image. The gradients of the image in the horizontal and vertical directions are calculated to construct the image autocorrelation matrix. Then, the ratio of the determinant value of the image autocorrelation matrix to the trace is calculated to obtain the corner point response function. In order to adaptively select feature points, the mean and standard deviation of the local area of ​​the image (for example, a 5×5 pixel window) are calculated to construct an adaptive threshold. Finally, the points whose corner point response function values ​​are greater than the adaptive threshold are taken as feature points.

[0082] Then, the extracted feature points are matched. A matching cost function based on spatial geometric constraints is established, which contains basic matrix constraints and normalized cross-correlation terms. The correspondence between feature points is determined by minimizing the matching cost function. For example, the basic matrix is ​​iteratively calculated using the RANSAC algorithm, and matching point pairs that do not meet the basic matrix constraints are eliminated. The matched feature points are projected into the image epipolar coordinate system to calculate the disparity value. Combined with the camera calibration parameters (camera focal length, baseline length), the disparity value is converted into the first depth information. Assuming that the coordinates of a feature point in the left camera image are (100, 150), the coordinates of the corresponding feature point in the right camera image are (90, 150), the camera baseline length is 20cm, and the focal length is 50mm, the disparity value is 10 pixels, and the corresponding depth information is 1 meter.

[0083] At the same time, a laser ranging sensor installed on the top of the vehicle-mounted boom is used to scan the working environment to obtain point cloud data. In order to remove noise points, the local density distribution of the point cloud data is calculated. The cluster center point is determined based on the local density distribution. The minimum distance from each sampling point to the cluster center point is calculated, and noise points whose distance exceeds the standard distance threshold (for example, twice the average distance) are eliminated. For example, a point cloud cluster contains 100 points with an average distance of 5 cm. The standard distance threshold is 10 cm, and points that are more than 10 cm away from the cluster center point will be eliminated. Then, regional growth is performed on the remaining point cloud data based on the cluster center to obtain a segmented point cloud. The segmented point cloud is spatially registered and 3D reconstructed using an iterative closest point algorithm to obtain the second depth information.

[0084] Finally, multi-source information fusion is performed. A state vector containing three-dimensional coordinate components and velocity components is established. The first depth information and the second depth information are used as observation data to construct a state space model. The state prediction value is obtained according to the state space model, and the Kalman gain matrix is ​​calculated. The observation data and the state prediction value are fused using the Kalman gain matrix to obtain the state estimation value. The state estimation error between the state estimation value and the observation data is calculated. The measurement confidence of the depth information is calculated based on the state estimation error. The fusion weight coefficient is determined based on the measurement confidence. The first depth information and the second depth information are fused based on the fusion weight coefficient to obtain the final fused depth information. For example, assuming that the measurement confidence of the first depth information is 0.8 and the measurement confidence of the second depth information is 0.9, the fusion weight coefficients can be set to 0.4 and 0.6 respectively.

[0085] In this embodiment, by fusing the depth information of binocular vision and laser ranging sensors, the shortcomings of a single sensor can be effectively compensated, and the accuracy and reliability of depth information can be improved. Binocular vision is suitable for scenes with rich textures, while laser ranging is suitable for structured environments. By fusing the information of the two sensors, the adaptability of the system to different operating environments can be enhanced. Accurate depth information can provide reliable environmental perception information for the path planning of the boom of the truck crane, thereby improving the safety of the operation and avoiding collision accidents.

[0086] In an optional implementation, a three-dimensional model of the working space is established in combination with the posture information obtained by the posture sensor on the truck crane, and the three-dimensional model is spatially indexed and data compressed by a voxel partitioning method, and outputting a real-time environment map includes:

[0087] Acquire the position coordinates and attitude angle data of the truck crane from the vehicle-mounted attitude sensor, construct a transformation matrix from the vehicle coordinate system to the global coordinate system according to the position coordinate data and the attitude angle data, and multiply the transformation matrix with the spatial point set in the fused depth information to obtain the depth point cloud data in the global coordinate system;

[0088] Calculating the spatial range of the depth point cloud data to obtain a spatial bounding box, dividing the spatial bounding box into an initial regular grid, calculating the Euclidean distance between the internal points of each initial regular grid to obtain a spatial distribution density, and adaptively adjusting the grid resolution according to the spatial distribution density to obtain a voxel grid;

[0089] Calculating the spatial centroid and covariance matrix of the point cloud in each grid cell in the voxel grid, performing eigenvalue decomposition on the covariance matrix to obtain eigenvalues ​​and principal direction vectors, using the principal direction vectors as local surface normal vectors, calculating the local curvature according to the ratio of the eigenvalues, and calculating the voxel occupancy probability of the grid cell by counting the number of points in the grid cell;

[0090] Establishing a sliding update mechanism of the time window, projecting the newly acquired depth point cloud data to the global coordinate system through the transformation matrix, spatially registering the projected point cloud data with the voxel grid, updating the state of the grid unit in the overlapping area using the voxel occupancy probability, and applying an attenuation coefficient to the occupancy probability according to the time window range to obtain the updated voxel occupancy probability;

[0091] A spatial distance field function is constructed based on the updated voxel occupancy probability and the local surface normal vector, an isosurface of the spatial distance field function is extracted to obtain an environmental surface mesh model, the environmental surface mesh model is simplified and compressed according to the local curvature, and a real-time environmental map is output.

[0092] Exemplarily, first, the vehicle's position information, including position coordinates and attitude angles, is obtained from the vehicle-mounted attitude sensor of the truck crane. The position coordinates refer to the position of the truck crane in the global coordinate system, and the attitude angle refers to the direction and angle information of the truck crane. These data are directly obtained through the vehicle-mounted sensors, usually including three-dimensional coordinate data and the vehicle's rotation matrix. With this information, a transformation matrix from the vehicle body coordinate system to the global coordinate system can be established. The function of this matrix is ​​to convert the local coordinates of the truck crane into the position and direction in the global coordinate system, thereby providing basic data for the subsequent construction of the spatial model.

[0093] Next, the transformation matrix is ​​calculated with the spatial point set in the fused depth information to obtain the depth point cloud data in the global coordinate system. The depth information comes from the fusion of the laser ranging sensor and the visual sensor, providing the three-dimensional coordinates of each point in the working space. These points are converted to the global coordinate system through the transformation matrix to facilitate subsequent modeling and analysis of the environment.

[0094] Once the depth point cloud data is obtained, its spatial extent can be calculated and a spatial bounding box can be created for it. The spatial bounding box is the smallest cube or rectangular box that contains the point cloud, which can help us quickly determine which areas are valid and which areas can be ignored. Next, this spatial bounding box is divided into an initial regular grid. Each grid represents a certain spatial area, and the density of the grid is adjusted according to the spatial distribution density of the point cloud data. Specifically, by calculating the distance between the points in each grid cell, the spatial distribution density of the area is obtained. Areas with higher density may represent obstacles or important structures, while areas with lower density can use lower resolutions. This adaptive adjustment can optimize the efficiency and accuracy of spatial indexing.

[0095] In each voxel grid, the spatial centroid of all points in the grid, that is, the average position of all points in the area, and the covariance matrix of the area are calculated. The covariance matrix describes the distribution of the point cloud and can provide the main direction of the point cloud. By performing eigenvalue decomposition on the covariance matrix, the main direction vector can be obtained, which represents the main distribution direction of the point cloud. This main direction vector will be used as the local surface normal vector to describe the surface orientation of the area. At the same time, by calculating the ratio of the eigenvalues, the local curvature of the area, that is, the degree of surface curvature, can be estimated. In order to further understand the occupancy of the grid, the number of points in the grid unit is counted to calculate the voxel occupancy probability of the grid unit. The voxel occupancy probability indicates the degree to which the area is occupied by obstacles or objects, and is an important indicator for judging the distribution of obstacles in space.

[0096] In order to maintain the real-time and accuracy of the environmental model, the system establishes a sliding update mechanism for the time window. As new depth point cloud data is acquired, these new data are projected into the global coordinate system through the transformation matrix, and then spatially aligned with the existing voxel grid. The purpose of spatial alignment is to match the new point cloud data with the existing environmental model so that the state of the corresponding area can be updated in the entire model. This update is based on the change of voxel occupancy probability, that is, if the new point cloud data indicates that an obstacle appears or disappears in a certain area, the occupancy probability of the area is adjusted. In addition, according to the range of the time window, an attenuation coefficient is applied to the occupancy probability to reflect that the old data gradually no longer affects the current model.

[0097] Finally, based on the updated voxel occupancy probability and local surface normal vector, a spatial distance field function is constructed. The spatial distance field function describes the distance from each point to the surface of the surrounding obstacles, and is a way to clearly describe the structure of the environment. By extracting the isosurface of the spatial distance field function, the surface mesh model of the environment in the working space can be obtained. In order to simplify and compress these mesh models, the surface mesh model is simplified using local curvature information. The simplification process helps to improve real-time computing efficiency while maintaining high model accuracy. Ultimately, the output real-time environment map can not only accurately reflect the location of obstacles in the working space, but also adapt to the dynamic operation requirements of the truck crane in a small space.

[0098] In this embodiment, accurate three-dimensional modeling of the working space is achieved through the collaborative work of depth information and vehicle-mounted attitude sensors, and a real-time updated environmental map is generated; adaptive voxel grid division and spatial distribution density adjustment enable environmental modeling to achieve a balance between different accuracies and efficiencies; real-time and dynamic performance are ensured through the update mechanism of the sliding time window, and environmental changes are reflected in a timely manner; local surface normal vectors and spatial distance field functions are used for accurate obstacle identification and modeling, and the accuracy and computational efficiency of the environmental map are optimized; by simplifying and compressing the environmental surface grid model, the real-time performance and accuracy of subsequent path planning and control are improved, supporting the truck crane to operate efficiently and safely in complex environments.

[0099] In an optional implementation, based on the real-time environment map, a convolutional neural network is used to identify and classify obstacles in the working space to obtain obstacle attribute information, and obstacles are divided into rigid obstacles and deformable obstacles according to the obstacle attribute information. A first safety distance threshold and a second safety distance threshold are set for the rigid obstacles and the deformable obstacles, respectively, and an obstacle avoidance cost function is established in combination with the kinematic constraint parameters of the manual arm of the truck crane, and a bidirectional probability sampling algorithm is used to generate path nodes. Based on the obstacle avoidance cost function, the path nodes are optimized and connected to obtain multiple groups of initial paths, including:

[0100] A convolutional neural network is used to extract features from a real-time environment map. The convolutional neural network includes a feature extraction backbone network, a feature fusion neck network, and a prediction head network. The feature extraction backbone network extracts multi-scale features through a spatial pyramid pooling module and a channel attention mechanism. The feature fusion neck network uses an adaptive feature pyramid structure and skip connections to fuse features at different levels. The prediction head network outputs the location information, size information, and category information of obstacles through parallel branches.

[0101] Classifying obstacles into rigid obstacles and deformable obstacles according to the category information, and calculating a first safety distance threshold of the rigid obstacles and a second safety distance threshold of the deformable obstacles based on a basic safety distance and an end effector speed, wherein the first safety distance threshold is greater than the second safety distance threshold;

[0102] Obtaining the upper and lower limits of the joint position of the truck crane to obtain the joint position constraint, obtaining the maximum speed of the joint movement to obtain the speed constraint, obtaining the maximum acceleration of the joint movement to obtain the acceleration constraint, and combining the joint position constraint, the speed constraint and the acceleration constraint with the first safety distance threshold and the second safety distance threshold to construct an obstacle avoidance cost function;

[0103] The current posture of the truck-mounted crane is obtained as the starting point of the path planning, and the target working position is obtained as the end point of the path planning. A bidirectional probabilistic path rapid expansion algorithm is used to simultaneously generate a search tree from the starting point and the end point. At each sampling, the target configuration is selected as a sampling node based on a set probability, and the distance from the sampling node to the nearest tree node is calculated. The satisfaction of the joint position constraint, velocity constraint and acceleration constraint of the sampling node is verified, and a candidate path node is generated;

[0104] The candidate path nodes are optimized based on the obstacle avoidance cost function, and the optimized path nodes are connected. When the distance between two connected search tree nodes is less than a preset node distance threshold, multiple groups of initial paths are generated.

[0105] Exemplarily, first, the acquired real-time environment map is preprocessed, such as grayscale, denoising and other operations, to improve image quality and algorithm efficiency. Then, the preprocessed map is input into the trained convolutional neural network for feature extraction and obstacle recognition. The convolutional neural network consists of three parts: feature extraction backbone network, feature fusion neck network and prediction head network. The feature extraction backbone network adopts the ResNet50 network structure, and combines the spatial pyramid pooling module and the channel attention mechanism to extract multi-scale features, such as the edge, texture, shape and other features of the obstacle. The feature fusion neck network adopts an adaptive feature pyramid structure and a jump connection to fuse features of different levels, and fuses the features extracted at different stages in the ResNet50 network to obtain richer obstacle information. The prediction head network contains three parallel branches, which output the location information of the obstacle (the pixel coordinates of the center point of the obstacle), size information (the pixel values ​​of the length and width of the obstacle) and category information (whether the obstacle is a rigid obstacle or a deformable obstacle). Taking an environment map of size 640×480 as an example, suppose an object located at (320, 240), with a size of 10×20 and a category of rigid obstacle is identified.

[0106] Next, according to the category information of the obstacles, they are divided into rigid obstacles and deformable obstacles. For rigid obstacles, a first safety distance threshold is set; for deformable obstacles, a second safety distance threshold is set. The size of these two thresholds is related to the basic safety distance and the end effector speed. Assuming the basic safety distance is 0.5 meters and the end effector speed is 0.1 meters per second, the first safety distance threshold is set to 0.6 meters and the second safety distance threshold is set to 0.55 meters. The first safety distance threshold is greater than the second safety distance threshold because rigid obstacles are not deformable and require a larger safety distance to avoid collision.

[0107] Then, obtain the kinematic constraint parameters of the truck crane, including the upper and lower limits of the joint position, the maximum speed of the joint movement, and the maximum acceleration. For example, assume that the angle range of a joint is 0 to 90 degrees, the maximum speed is 10 degrees / second, and the maximum acceleration is 20 degrees / second². Combine these constraint parameters with the first safety distance threshold and the second safety distance threshold to construct an obstacle avoidance cost function. This cost function is used to evaluate the safety and smoothness of the path. For example, the closer the distance to the obstacle, the higher the cost; the greater the joint movement speed or acceleration, the higher the cost.

[0108] Next, the current posture of the truck crane is obtained as the starting point of the path planning, and the target working position is obtained as the end point of the path planning. The improved bidirectional probabilistic path rapid expansion algorithm is used to simultaneously generate the search tree from the starting point and the end point. At each sampling, the target configuration is selected as the sampling node based on a preset probability (for example, 0.8). The distance from the sampling node to the nearest tree node is calculated, and it is verified whether the sampling node satisfies the joint position constraint, velocity constraint, and acceleration constraint. If satisfied, the node is added to the search tree as a candidate path node. For example, assuming that the joint angle of the current node is 30 degrees and the joint angle of the sampling node is 40 degrees, if the joint angle range is 0 to 90 degrees and the maximum speed is 10 degrees / second, the sampling node satisfies the constraints.

[0109] Finally, the candidate path nodes are optimized based on the obstacle avoidance cost function, for example, the path is smoothed using the gradient descent method, and the optimized path nodes are connected. When the distance between the two connected search tree nodes is less than the preset node distance threshold (for example, 0.1 meters), it is considered that a feasible path is found and the path is added to the initial path set. Finally, multiple groups of initial paths are generated, and the optimal path is selected as the final obstacle avoidance path.

[0110] In this embodiment, by distinguishing between rigid obstacles and deformable obstacles and setting different safety distance thresholds, collisions can be avoided more effectively and operation safety can be improved. By using an improved bidirectional probability sampling algorithm, multiple groups of initial paths can be quickly generated, and optimized through obstacle avoidance cost functions to improve path planning efficiency. Combined with the kinematic constraint parameters of the truck crane, a path that meets the actual motion constraints can be generated, enhancing the executability and adaptability of the path.

[0111] In an optional implementation, inputting the load weight information of the manual arm of the truck crane into the dynamic model, calculating the joint rotation torques of the multiple groups of initial paths, and selecting the path satisfying the dynamic constraints as the target optimization path includes:

[0112] A dynamic model of the manual arm of the truck-mounted crane is established, wherein the dynamic model includes a connecting rod angular velocity transmission equation, an angular acceleration transmission equation, and a linear acceleration transmission equation. Based on the dynamic model, a dynamic relationship between adjacent connecting rods is constructed to obtain the connecting rod dynamic state parameters;

[0113] Obtaining the load weight of the truck crane manual arm, substituting the load weight and the connecting rod dynamic state parameters into the dynamic equation of the end effector, and calculating the dynamic parameters of the end effector;

[0114] The dynamic parameters of the end effector are transformed into coordinates through the Jacobian matrix, and the inverse dynamics recursive method is used to calculate the torque of each joint by combining the mass of the connecting rod and the connecting rod resultant force, and the maximum torque constraint of each joint is obtained;

[0115] Node sampling is performed on multiple groups of initial paths, kinematic information of each sampled node is extracted, the kinematic information is substituted into the calculation formula of each joint torque, the torque curve of each initial path is obtained, and a path subset that meets the dynamic constraints is screened out by comparing the torque curve with the joint maximum torque constraint;

[0116] The torque index value and trajectory characteristic value of each path in the path subset are calculated, the path subset is comprehensively evaluated based on the torque index value and trajectory characteristic value, and the path with the highest comprehensive evaluation score is selected as the target optimization path.

[0117] The manual boom path planning method for a truck-mounted crane aims to optimize the operating efficiency and safety of the truck-mounted crane. The core idea of ​​this method is to integrate the load weight information into the dynamic model and screen out the best path that meets the dynamic constraints by calculating the joint rotation torque of multiple groups of initial paths.

[0118] Exemplarily, first, a dynamic model of the boom of a truck crane is established. The model contains the transmission relationship between the angular velocity, angular acceleration and linear acceleration between the connecting rods. For example, the angular velocity of the first connecting rod will affect the angular velocity of the second connecting rod, and the linear acceleration of the second connecting rod will affect the angular acceleration of the third connecting rod, and so on. Through these transfer equations, the dynamic relationship between adjacent connecting rods can be constructed, and the dynamic state parameters of each connecting rod, such as angular velocity, angular acceleration, linear velocity and linear acceleration, etc., can be obtained. Assuming that the angular velocity of the first connecting rod is 10 degrees per second, through the angular velocity transfer equation, it can be calculated that the angular velocity of the second connecting rod is 8 degrees per second, and so on.

[0119] Next, obtain the load weight of the truck crane boom. Substitute the load weight and the previously calculated connecting rod dynamic state parameters into the dynamic equation of the end effector to calculate the dynamic parameters of the end effector, such as the linear velocity and linear acceleration of the end effector. Assuming the load weight is 100 kg, the angular velocity of the first connecting rod is 10 degrees per second, and the angular velocity of the second connecting rod is 8 degrees per second, the linear velocity of the end effector can be calculated to be 2 meters per second.

[0120] Then, the dynamic parameters of the end effector are transformed into coordinates through the Jacobian matrix. The Jacobian matrix describes the relationship between the end effector velocity and the joint velocity. Through coordinate transformation, the dynamic parameters of the end effector can be converted into dynamic parameters of the joint space. Combined with the mass of the connecting rod and the connecting rod resultant force, the inverse dynamics recursive method is used to calculate the torque of each joint, and the maximum torque constraint of each joint is obtained. For example, it is known that the linear velocity of the end effector is 2 meters per second. Through the Jacobian matrix and inverse dynamics calculation, it can be obtained that the torque of the first joint is 100 Nm, the torque of the second joint is 80 Nm, and the maximum torque constraint of the first joint is 120 Nm, and the maximum torque constraint of the second joint is 100 Nm.

[0121] Subsequently, node sampling is performed on multiple groups of initial paths to extract the kinematic information of each sampled node, such as position, velocity, acceleration, etc. These kinematic information are substituted into the calculation formula of each joint torque to obtain the torque curve of each initial path. Assuming there are two initial paths, the torque curves of the two paths can be obtained by calculation. By comparing the torque curves with the maximum torque constraints of the joints, a subset of paths that meet the dynamic constraints are screened out. For example, if the torque of the first path at some nodes exceeds the maximum torque constraint of the joint, the path is eliminated; if the torque of the second path is always within the maximum torque constraint of the joint, the path is retained.

[0122] Finally, the torque index value and trajectory characteristic value of each path in the path subset are calculated. The torque index values ​​include average torque, maximum torque, etc., and the trajectory characteristic values ​​include path length, average speed, etc. Based on these index values, the path subset is comprehensively evaluated, and the path with the highest comprehensive evaluation score is selected as the target optimization path. For example, the second path has a smaller average torque and a shorter path length, so its comprehensive evaluation score is higher and it is selected as the target optimization path.

[0123] In this embodiment, by pre-screening the path that meets the dynamic constraints, the calculation amount of the subsequent optimization algorithm is reduced, thereby improving the efficiency of path planning. Since the load weight and dynamic constraints are taken into account, the planned path can ensure that the truck crane can complete the task safely and reliably in actual operation. By comprehensively considering the torque index value and the trajectory characteristic value, the path with the best comprehensive performance is selected, thereby improving the operating efficiency and accuracy of the truck crane.

[0124] In an optional implementation, for the target optimization path, a comprehensive evaluation function including a path length evaluation item, a motion smoothness evaluation item, and an obstacle avoidance safety evaluation item is established, a virtual spring damping model is established in the coordinate system of the end effector of the manual arm of the truck crane, and the spring stiffness coefficient and the damping coefficient of the virtual spring damping model are adjusted according to the evaluation score calculated by the comprehensive evaluation function. The target optimization path is dynamically adjusted in real time to obtain an adjusted path, which includes:

[0125] An evaluation system is established for the path node position coordinate sequence and the joint angle sequence at the path nodes of the target optimization path. The path length evaluation item is obtained by calculating the weighted combination of the Manhattan distance and the Chebyshev distance between adjacent nodes. The motion smoothness evaluation item is obtained by calculating the linear combination of the first-order difference and the second-order difference of the joint angle at the node. The obstacle avoidance safety evaluation item is calculated by constructing a potential field function that considers the obstacle shape and motion trend. The path length evaluation item, the motion smoothness evaluation item and the obstacle avoidance safety evaluation item are combined using an adaptive weight coefficient to obtain a comprehensive evaluation function.

[0126] A nonlinear virtual spring damping model is established in the coordinate system of the end effector of the manual arm of the truck crane, and a fuzzy control rule is used to adaptively adjust the spring stiffness coefficient and the damping coefficient of the virtual spring damping model according to the evaluation score of the comprehensive evaluation function;

[0127] Substituting the spring stiffness coefficient and the damping coefficient into a virtual spring damping model, calculating the virtual force acting on the path node based on the nonlinear combination of the position deviation and the speed deviation of the path node, making real-time dynamic adjustments to the target optimization path according to the virtual force, and using the fourth-order Runge-Kutta method to solve the node motion equation to obtain the displacement adjustment of the path node;

[0128] Applying the displacement adjustment amount to the path node position coordinates to obtain the adjusted path node position, and determining whether the joint angle, angular velocity and angular acceleration corresponding to the adjusted path node position meet the motion constraints by calculating the Jacobian matrix at the path node;

[0129] When the motion constraint is satisfied, the adjusted path node position is substituted into the comprehensive evaluation function to recalculate the evaluation score. If the evaluation score is greater than the dynamic convergence threshold calculated based on the current number of iterations and has not reached the preset upper limit of the number of iterations, the process returns to the step of adjusting the spring stiffness coefficient and the damping coefficient of the virtual spring damping model until an adjusted path that satisfies the constraint condition is obtained.

[0130] Exemplarily, first, the target optimization path is preliminarily planned to obtain the node position coordinate sequence on the path and the joint angle sequence corresponding to each node. These data will be used for subsequent path evaluation and adjustment. For example, assuming that the planned path contains 10 nodes, the position coordinates of each node are represented by three-dimensional coordinates (x, y, z), and the joint angle is represented by a vector.

[0131] Next, an evaluation system is established for the planned path. The path length evaluation item is obtained by calculating the weighted sum of the Manhattan distance and Chebyshev distance between adjacent nodes. For example, the Manhattan distance between two adjacent nodes is the sum of the absolute values ​​of the coordinate differences of the two nodes, and the Chebyshev distance is the maximum value of the absolute values ​​of the coordinate differences of the two nodes. The motion smoothness evaluation item is obtained by calculating the linear combination of the first-order difference (i.e., the change in the angle of adjacent nodes) and the second-order difference (i.e., the change in the angle change) of the joint angle at the node. The obstacle avoidance safety evaluation item is calculated by constructing a potential field function that takes into account the shape and movement trend of the obstacle. For example, the closer the distance to the obstacle, the higher the potential field value, indicating that the obstacle avoidance safety is lower. Finally, the three evaluation indicators are combined into a comprehensive evaluation function using adaptive weight coefficients. For example, the initial weight coefficients can be set to be equal and then adjusted according to the actual situation.

[0132] Then, a nonlinear virtual spring-damper model is established in the coordinate system of the end effector of the boom of the truck-mounted crane. The model simulates the behavior of the spring and damper, and can adaptively adjust the spring stiffness coefficient and the damping coefficient according to the comprehensive evaluation score of the path. For example, if the comprehensive evaluation score is low, indicating that the path quality is poor, the spring stiffness coefficient and the damping coefficient are increased to generate a larger correction force. On the contrary, the spring stiffness coefficient and the damping coefficient are reduced. Fuzzy control rules are used to adjust the parameters. For example, three fuzzy sets of "low score", "medium score" and "high score" are set, and the parameters are adjusted according to the fuzzy set to which the score belongs.

[0133] Next, the calculated spring stiffness coefficient and damping coefficient are substituted into the virtual spring damping model, and the virtual force acting on the path node is calculated based on the nonlinear combination of the position deviation (i.e., the difference between the actual position and the planned position) and the speed deviation (i.e., the difference between the actual speed and the planned speed) of the path node. Then, the target optimization path is adjusted dynamically in real time based on the virtual force, and the node motion equation is solved using the fourth-order Runge-Kutta method to obtain the displacement adjustment of the path node.

[0134] The calculated displacement adjustment is applied to the path node position coordinates to obtain the adjusted path node position. Then, by calculating the Jacobian matrix at the path node (a matrix that maps the velocity in the joint space to the velocity in the Cartesian space), it is determined whether the joint angle, angular velocity, and angular acceleration corresponding to the adjusted path node position meet the motion constraints. For example, the range of joint angles, the limit values ​​of angular velocity and angular acceleration, etc.

[0135] If the motion constraints are satisfied, the adjusted path node positions are substituted into the comprehensive evaluation function to recalculate the evaluation score. If the new evaluation score is greater than the dynamic convergence threshold calculated based on the current number of iterations (for example, the threshold gradually decreases as the number of iterations increases) and does not reach the preset upper limit of the number of iterations (for example, the maximum number of iterations is 100), then return to re-adjust the spring stiffness coefficient and damping coefficient of the virtual spring damping model, and repeat the above steps until an adjusted path that satisfies the constraints is obtained.

[0136] In this embodiment, by considering the shape and movement trend of obstacles and using potential field functions to evaluate the safety of obstacle avoidance, the safety of the path can be effectively improved and collisions can be avoided. By considering the first-order and second-order differences of the joint angles, the changes in joint speed and acceleration can be effectively reduced, making the path smoother and reducing mechanical wear. Through the weighted combination of Manhattan distance and Chebyshev distance, the path length can be shortened as much as possible to improve work efficiency while ensuring safety and smoothness.

[0137] In an optional implementation, a piecewise continuous polynomial function is used to fit the adjusted path to obtain a fitting path, the joint angles of the fitting path are calculated based on the kinematic model of the truck crane boom, and the path segments that do not meet the kinematic constraints are locally corrected. Finally, the output of the operation path that meets the constraint conditions includes:

[0138] Acquire mechanical limit values ​​and speed limit values ​​of each joint of the manual arm of the truck crane, wherein the mechanical limit values ​​include upper and lower limits of the joint rotation angle, and the speed limit values ​​include the maximum value of the joint angular velocity; and establish a kinematic constraint condition matrix based on the mechanical limit values ​​and speed limit values;

[0139] Constructing a position component polynomial function that uses multiple control points to determine the shape of the curve, and constructing a spherical linear interpolation function based on a unit quaternion as an attitude component polynomial function; combining the position component polynomial function and the attitude component polynomial function to form a piecewise continuous polynomial function;

[0140] The node position coordinate sequence and the node attitude angle sequence of the adjusted path are used as input data, the coefficients of the position component polynomial function and the attitude component polynomial function are solved by the least square method to obtain the polynomial coefficients, and the fitting path is calculated according to the polynomial coefficients to obtain the position coordinate sequence and the attitude angle sequence of the fitting path;

[0141] A manual arm kinematic model is established based on a link parameter method, and the position coordinate sequence and posture angle sequence of the fitting path are substituted into the manual arm kinematic model to obtain a joint angle sequence, and the joint angle sequence is compared with a kinematic constraint matrix to calculate the angle excess and speed excess of each joint, and the path segment interval that does not meet the constraint condition is determined according to the angle excess and speed excess;

[0142] Constructing a transition curve within the path segment interval, converting the kinematic constraint matrix into the boundary constraint of the transition curve, solving the position coordinates and attitude angle of the transition node that meets the boundary constraint; substituting the transition node into a piecewise continuous polynomial function to obtain a corrected path segment;

[0143] An interpolation method that maintains geometric continuity is used to splice the path segment that meets the constraint conditions with the corrected path segment to obtain an operation path; the operation path is substituted into the kinematic model to solve the joint angle sequence, determine whether the kinematic constraint condition matrix is ​​met, and output the operation path when the constraint conditions are met.

[0144] Exemplarily, first, obtain the kinematic constraint information of the manual arm of the truck crane. This includes the mechanical limit value of each joint, that is, the upper and lower limits of the joint angle, and the speed limit value of each joint, that is, the maximum value of the joint angular velocity. For example, assuming that the manual arm has three joints, the rotation angle range of the first joint is -120 degrees to 120 degrees, and the maximum angular velocity is 10 degrees per second; the rotation angle range of the second joint is -60 degrees to 60 degrees, and the maximum angular velocity is 5 degrees per second; the rotation angle range of the third joint is -90 degrees to 90 degrees, and the maximum angular velocity is 8 degrees per second. These limit values ​​and speed limit values ​​will be used to establish a kinematic constraint matrix, which is used to subsequently determine whether the generated path meets the motion constraints of the manual arm.

[0145] Next, a piecewise continuous polynomial function for path fitting is constructed. The position component uses a polynomial function determined by multiple control points, such as a cubic polynomial, a quintic polynomial, etc. The shape of the curve can be flexibly controlled by adjusting the position of the control points. The attitude component uses a spherical linear interpolation function based on unit quaternions to ensure the smoothness and stability of the attitude interpolation. The position component polynomial function and the attitude component polynomial function are combined to form a piecewise continuous polynomial function for fitting the adjusted path.

[0146] Then, the adjusted path is fitted. The node position coordinate sequence and node attitude angle sequence of the adjusted path are used as input data. The coefficients of the position component polynomial function and the attitude component polynomial function are solved by the least squares method. For example, suppose there are five nodes, each of which has three-dimensional position coordinates and three attitude angles. The coefficients of the polynomial function can be obtained by the least squares method, and then the fitting path can be calculated based on these coefficients to obtain the position coordinate sequence and attitude angle sequence of the fitting path.

[0147] Next, perform kinematic solution and constraint verification. The kinematic model of the manual arm is established based on the link parameter method. Substitute the position coordinate sequence and posture angle sequence of the fitting path into the kinematic model of the manual arm to solve the joint angle sequence. Compare the solved joint angle sequence with the previously established kinematic constraint matrix to calculate the angle excess and speed excess of each joint. For example, if the rotation angle of the first joint at a certain moment is 130 degrees, its angle excess is 10 degrees. Determine the path segment interval that does not meet the constraint conditions based on the angle excess and speed excess.

[0148] For the path segments that do not meet the constraints, local corrections are made. Within the path segments that do not meet the constraints, a transition curve is constructed. The kinematic constraint matrix is ​​converted into the boundary constraints of the transition curve, for example, the joint angle limit values ​​are converted into the position and posture constraints of the transition curve endpoints. The position coordinates and posture angles of the transition nodes that meet the boundary constraints are solved. The solved transition nodes are substituted into the piecewise continuous polynomial function to obtain the corrected path segment.

[0149] Finally, the path is spliced ​​and output. An interpolation method that maintains geometric continuity, such as cubic spline interpolation, is used to splice the path segments that meet the constraints with the corrected path segments to obtain a complete operation path. The final operation path is substituted into the kinematic model, the joint angle sequence is solved again, and it is determined whether the kinematic constraint matrix is ​​satisfied. If the constraints are satisfied, the operation path is output; if not, the local correction step is returned and corrected again.

[0150] In this embodiment, by using a piecewise continuous polynomial function for path fitting, a smooth initial path can be quickly generated, reducing the time cost of path planning. By incorporating kinematic constraints into the path planning process, it can be ensured that the generated path meets the motion constraints of the manual arm, avoiding the situation where the path is not executable. By locally correcting the path segments that do not meet the constraints, the smoothness and stability of the path can be optimized, improving the execution efficiency and safety of the path.

[0151] Figure 2FIG. 1 is a schematic diagram of a structure of a manual arm operation path planning system for a truck-mounted crane in a narrow space according to an embodiment of the present invention. Figure 2 As shown, the system comprises:

[0152] The first unit is used to collect an environmental image of the working space through binocular stereo vision cameras installed at both ends of the vehicle-mounted crane, extract the corresponding relationship of feature points of the environmental image through a feature point matching algorithm and calculate the depth information to obtain first depth information, scan the working space through a laser ranging sensor installed on the top of the vehicle-mounted crane to obtain an obstacle contour point cloud, obtain second depth information based on the obstacle contour point cloud calculation, perform multi-source information fusion on the first depth information and the second depth information to obtain fused depth information, establish a three-dimensional model of the working space in combination with the posture information obtained by the vehicle-mounted posture sensor of the vehicle-mounted crane, perform spatial indexing and data compression on the three-dimensional model through a voxel partitioning method, and output a real-time environmental map;

[0153] The second unit is used to identify and classify obstacles in the working space based on the real-time environment map by using a convolutional neural network to obtain obstacle attribute information, classify the obstacles into rigid obstacles and deformable obstacles according to the obstacle attribute information, set a first safety distance threshold and a second safety distance threshold for the rigid obstacles and the deformable obstacles respectively, establish an obstacle avoidance cost function in combination with the kinematic constraint parameters of the manual arm of the truck crane, generate path nodes by using a bidirectional probability sampling algorithm, optimize and connect the path nodes based on the obstacle avoidance cost function to obtain multiple groups of initial paths, input the load weight information of the manual arm of the truck crane into the dynamics model, calculate the joint rotation torque of the multiple groups of initial paths, and select the path that meets the dynamic constraints as the target optimization path;

[0154] The third unit is used to establish a comprehensive evaluation function including path length evaluation items, motion smoothness evaluation items and obstacle avoidance safety evaluation items for the target optimization path, establish a virtual spring damping model in the coordinate system of the end effector of the manual arm of the truck crane, adjust the spring stiffness coefficient and the damping coefficient of the virtual spring damping model according to the evaluation score calculated by the comprehensive evaluation function, perform real-time dynamic adjustment on the target optimization path to obtain an adjusted path, use a piecewise continuous polynomial function to fit the adjusted path to obtain a fitting path, calculate the joint angles of the fitting path based on the kinematic model of the manual arm of the truck crane, perform local correction on the path segments that do not meet the kinematic constraints, and finally output an operation path that meets the constraints.

[0155] According to a third aspect of the embodiments of the present invention,

[0156] An electronic device is provided, comprising:

[0157] processor;

[0158] a memory for storing processor-executable instructions;

[0159] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0160] A fourth aspect of the embodiments of the present invention is:

[0161] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0162] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0163] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for planning the operation path of a truck-mounted crane manual arm in a narrow space, characterized in that: include: The environment image of the working space is collected by binocular stereo vision cameras installed at both ends of the vehicle-mounted crane, the corresponding relationship of the feature points of the environment image is extracted by a feature point matching algorithm, and the depth information is calculated to obtain the first depth information, the working space is scanned by a laser ranging sensor installed on the top of the vehicle-mounted crane to obtain the obstacle contour point cloud, the second depth information is obtained based on the obstacle contour point cloud calculation, the first depth information and the second depth information are multi-source information fused to obtain fused depth information, and a three-dimensional model of the working space is established in combination with the posture information obtained by the vehicle-mounted posture sensor of the vehicle-mounted crane, the three-dimensional model is spatially indexed and data compressed by a voxel partitioning method, and a real-time environment map is output; Based on the real-time environment map, a convolutional neural network is used to identify and classify obstacles in the working space to obtain obstacle attribute information, and obstacles are divided into rigid obstacles and deformable obstacles according to the obstacle attribute information. A first safety distance threshold and a second safety distance threshold are set for the rigid obstacles and the deformable obstacles, respectively, and an obstacle avoidance cost function is established in combination with the kinematic constraint parameters of the manual arm of the truck crane, and a bidirectional probability sampling algorithm is used to generate path nodes. Based on the obstacle avoidance cost function, the path nodes are optimally connected to obtain multiple groups of initial paths, and the load weight information of the manual arm of the truck crane is input into the dynamics model, the joint rotation torque of the multiple groups of initial paths is calculated, and the path that meets the dynamic constraints is selected as the target optimization path; For the target optimized path, a comprehensive evaluation function including path length evaluation items, motion smoothness evaluation items and obstacle avoidance safety evaluation items is established, a virtual spring damping model is established in the coordinate system of the end effector of the manual arm of the truck crane, the spring stiffness coefficient and the damping coefficient of the virtual spring damping model are adjusted according to the evaluation score calculated by the comprehensive evaluation function, the target optimized path is dynamically adjusted in real time to obtain an adjusted path, a piecewise continuous polynomial function is used to fit the adjusted path to obtain a fitted path, the joint angles of the fitted path are calculated based on the kinematic model of the manual arm of the truck crane, the path segments that do not meet the kinematic constraints are locally corrected, and finally an operation path that meets the constraints is output.

2. The method according to claim 1, characterized in that: The binocular stereo vision cameras installed at both ends of the vehicle-mounted boom frame are used to collect the environment image of the working space, the feature point correspondence of the environment image is extracted by the feature point matching algorithm, and the depth information is calculated to obtain the first depth information, the laser ranging sensor installed at the top of the vehicle-mounted boom frame is used to scan the working space to obtain the obstacle contour point cloud, the second depth information is obtained according to the obstacle contour point cloud calculation, and the first depth information and the second depth information are multi-source information fused to obtain the fused depth information, including: The binocular stereo vision cameras installed at both ends of the vehicle-mounted boom frame are used to collect the working environment image and perform local histogram equalization processing to obtain a pre-processed image, an image autocorrelation matrix is ​​constructed according to the horizontal gradient and the vertical gradient of the pre-processed image, a ratio of the determinant value and the trace of the image autocorrelation matrix is ​​calculated to obtain a corner point response function, a mean value and a standard deviation of a local area of ​​the image are calculated based on the corner point response function to construct an adaptive threshold, and the corner point response function is screened by the adaptive threshold to obtain feature point data; Establishing a matching cost function based on spatial geometric constraints for the feature point data, wherein the matching cost function includes a basic matrix constraint term and a normalized cross-correlation term, determining a feature point correspondence relationship according to a minimum value of the matching cost function, projecting the feature point correspondence relationship into an image epipolar coordinate system to calculate a disparity value, and converting the disparity value into first depth information in combination with a camera calibration parameter; The laser ranging sensor installed on the top of the vehicle-mounted boom is used to scan the working environment to obtain point cloud data, calculate the local density distribution of the point cloud data, determine the cluster center point based on the local density distribution, calculate the minimum distance from each sampling point to the cluster center point, remove noise points whose distance exceeds the standard distance threshold, and perform regional growth on the remaining point cloud data based on the cluster center to obtain segmented point clouds, and use an iterative closest point algorithm to perform spatial registration and three-dimensional reconstruction on the segmented point clouds to obtain second depth information; A state vector including three-dimensional coordinate components and velocity components is established, a state space model is constructed using the first depth information and the second depth information as observation data, a state prediction value is obtained according to the state space model, and a Kalman gain matrix is ​​calculated, the observation data and the state prediction value are fused using the Kalman gain matrix to obtain a state estimation value, a state estimation error between the state estimation value and the observation data is calculated, a measurement confidence of the depth information is calculated according to the state estimation error, a fusion weight coefficient is determined based on the measurement confidence, and the first depth information and the second depth information are fused based on the fusion weight coefficient to obtain fused depth information.

3. The method according to claim 1, characterized in that The three-dimensional model of the working space is established by combining the posture information obtained by the posture sensor on the truck crane, and the three-dimensional model is spatially indexed and data compressed by the voxel partitioning method, and the real-time environment map is output, including: Acquire the position coordinates and attitude angle data of the truck crane from the vehicle-mounted attitude sensor, construct a transformation matrix from the vehicle coordinate system to the global coordinate system according to the position coordinate data and the attitude angle data, and multiply the transformation matrix with the spatial point set in the fused depth information to obtain the depth point cloud data in the global coordinate system; Calculating the spatial range of the depth point cloud data to obtain a spatial bounding box, dividing the spatial bounding box into an initial regular grid, calculating the Euclidean distance between the internal points of each initial regular grid to obtain a spatial distribution density, and adaptively adjusting the grid resolution according to the spatial distribution density to obtain a voxel grid; Calculating the spatial centroid and covariance matrix of the point cloud in each grid cell in the voxel grid, performing eigenvalue decomposition on the covariance matrix to obtain eigenvalues ​​and principal direction vectors, using the principal direction vectors as local surface normal vectors, calculating the local curvature according to the ratio of the eigenvalues, and calculating the voxel occupancy probability of the grid cell by counting the number of points in the grid cell; Establishing a sliding update mechanism of the time window, projecting the newly acquired depth point cloud data to the global coordinate system through the transformation matrix, spatially registering the projected point cloud data with the voxel grid, updating the state of the grid unit in the overlapping area using the voxel occupancy probability, and applying an attenuation coefficient to the occupancy probability according to the time window range to obtain the updated voxel occupancy probability; A spatial distance field function is constructed based on the updated voxel occupancy probability and the local surface normal vector, an isosurface of the spatial distance field function is extracted to obtain an environmental surface mesh model, the environmental surface mesh model is simplified and compressed according to the local curvature, and a real-time environmental map is output.

4. The method according to claim 1, characterized in that Based on the real-time environment map, a convolutional neural network is used to identify and classify obstacles in the working space to obtain obstacle attribute information. According to the obstacle attribute information, the obstacles are divided into rigid obstacles and deformable obstacles. A first safety distance threshold and a second safety distance threshold are set for the rigid obstacles and the deformable obstacles, respectively. An obstacle avoidance cost function is established in combination with the kinematic constraint parameters of the truck crane manual arm. A bidirectional probability sampling algorithm is used to generate path nodes. Based on the obstacle avoidance cost function, the path nodes are optimized and connected to obtain multiple groups of initial paths, including: A convolutional neural network is used to extract features from a real-time environment map. The convolutional neural network includes a feature extraction backbone network, a feature fusion neck network, and a prediction head network. The feature extraction backbone network extracts multi-scale features through a spatial pyramid pooling module and a channel attention mechanism. The feature fusion neck network uses an adaptive feature pyramid structure and skip connections to fuse features at different levels. The prediction head network outputs the location information, size information, and category information of obstacles through parallel branches. Classifying obstacles into rigid obstacles and deformable obstacles according to the category information, and calculating a first safety distance threshold of the rigid obstacles and a second safety distance threshold of the deformable obstacles based on a basic safety distance and an end effector speed, wherein the first safety distance threshold is greater than the second safety distance threshold; Obtaining the upper and lower limits of the joint position of the truck crane to obtain the joint position constraint, obtaining the maximum speed of the joint movement to obtain the speed constraint, obtaining the maximum acceleration of the joint movement to obtain the acceleration constraint, and combining the joint position constraint, the speed constraint and the acceleration constraint with the first safety distance threshold and the second safety distance threshold to construct an obstacle avoidance cost function; The current posture of the truck-mounted crane is obtained as the starting point of the path planning, and the target working position is obtained as the end point of the path planning. A bidirectional probabilistic path rapid expansion algorithm is used to simultaneously generate a search tree from the starting point and the end point. At each sampling, the target configuration is selected as a sampling node based on a set probability, and the distance from the sampling node to the nearest tree node is calculated. The satisfaction of the joint position constraint, velocity constraint and acceleration constraint of the sampling node is verified, and a candidate path node is generated; The candidate path nodes are optimized based on the obstacle avoidance cost function, and the optimized path nodes are connected. When the distance between two connected search tree nodes is less than a preset node distance threshold, multiple groups of initial paths are generated.

5. The method according to claim 1, characterized in that Inputting the load weight information of the manual arm of the truck crane into the dynamics model, calculating the joint rotation torques of the multiple groups of initial paths, and selecting the path that satisfies the dynamics constraints as the target optimization path includes: A dynamic model of the manual arm of the truck-mounted crane is established, wherein the dynamic model includes a connecting rod angular velocity transmission equation, an angular acceleration transmission equation, and a linear acceleration transmission equation. Based on the dynamic model, a dynamic relationship between adjacent connecting rods is constructed to obtain the connecting rod dynamic state parameters; Obtaining the load weight of the truck crane manual arm, substituting the load weight and the connecting rod dynamic state parameters into the dynamic equation of the end effector, and calculating the dynamic parameters of the end effector; The dynamic parameters of the end effector are transformed into coordinates through the Jacobian matrix, and the inverse dynamics recursive method is used to calculate the torque of each joint by combining the mass of the connecting rod and the connecting rod resultant force, and the maximum torque constraint of each joint is obtained; Node sampling is performed on multiple groups of initial paths, kinematic information of each sampled node is extracted, the kinematic information is substituted into the calculation formula of each joint torque, the torque curve of each initial path is obtained, and a path subset that meets the dynamic constraints is screened out by comparing the torque curve with the joint maximum torque constraint; The torque index value and trajectory characteristic value of each path in the path subset are calculated, the path subset is comprehensively evaluated based on the torque index value and trajectory characteristic value, and the path with the highest comprehensive evaluation score is selected as the target optimization path.

6. The method according to claim 1, characterized in that For the target optimization path, a comprehensive evaluation function including a path length evaluation item, a motion smoothness evaluation item, and an obstacle avoidance safety evaluation item is established, a virtual spring damping model is established in the coordinate system of the end effector of the manual arm of the truck crane, and the spring stiffness coefficient and the damping coefficient of the virtual spring damping model are adjusted according to the evaluation score calculated by the comprehensive evaluation function. The target optimization path is adjusted dynamically in real time to obtain an adjusted path including: An evaluation system is established for the path node position coordinate sequence and the joint angle sequence at the path nodes of the target optimization path. The path length evaluation item is obtained by calculating the weighted combination of the Manhattan distance and the Chebyshev distance between adjacent nodes. The motion smoothness evaluation item is obtained by calculating the linear combination of the first-order difference and the second-order difference of the joint angle at the node. The obstacle avoidance safety evaluation item is calculated by constructing a potential field function that considers the obstacle shape and motion trend. The path length evaluation item, the motion smoothness evaluation item and the obstacle avoidance safety evaluation item are combined using an adaptive weight coefficient to obtain a comprehensive evaluation function. A nonlinear virtual spring damping model is established in the coordinate system of the end effector of the manual arm of the truck crane, and a fuzzy control rule is used to adaptively adjust the spring stiffness coefficient and the damping coefficient of the virtual spring damping model according to the evaluation score of the comprehensive evaluation function; Substituting the spring stiffness coefficient and the damping coefficient into a virtual spring damping model, calculating the virtual force acting on the path node based on the nonlinear combination of the position deviation and the speed deviation of the path node, making real-time dynamic adjustments to the target optimization path according to the virtual force, and using the fourth-order Runge-Kutta method to solve the node motion equation to obtain the displacement adjustment of the path node; Applying the displacement adjustment amount to the path node position coordinates to obtain the adjusted path node position, and determining whether the joint angle, angular velocity and angular acceleration corresponding to the adjusted path node position meet the motion constraints by calculating the Jacobian matrix at the path node; When the motion constraint is satisfied, the adjusted path node position is substituted into the comprehensive evaluation function to recalculate the evaluation score. If the evaluation score is greater than the dynamic convergence threshold calculated based on the current number of iterations and has not reached the preset upper limit of the number of iterations, the process returns to the step of adjusting the spring stiffness coefficient and the damping coefficient of the virtual spring damping model until an adjusted path that satisfies the constraint condition is obtained.

7. The method according to claim 1, characterized in that The adjusted path is fitted by a piecewise continuous polynomial function to obtain a fitting path. The joint angles of the fitting path are calculated based on the kinematic model of the truck crane boom. The path segments that do not meet the kinematic constraints are locally corrected. The final output of the operation path that meets the constraints includes: Acquire mechanical limit values ​​and speed limit values ​​of each joint of the manual arm of the truck crane, wherein the mechanical limit values ​​include upper and lower limits of the joint rotation angle, and the speed limit values ​​include the maximum value of the joint angular velocity; and establish a kinematic constraint condition matrix based on the mechanical limit values ​​and speed limit values; Constructing a position component polynomial function that uses multiple control points to determine the shape of the curve, and constructing a spherical linear interpolation function based on a unit quaternion as an attitude component polynomial function; combining the position component polynomial function and the attitude component polynomial function to form a piecewise continuous polynomial function; The node position coordinate sequence and the node attitude angle sequence of the adjusted path are used as input data, the coefficients of the position component polynomial function and the attitude component polynomial function are solved by the least square method to obtain the polynomial coefficients, and the fitting path is calculated according to the polynomial coefficients to obtain the position coordinate sequence and the attitude angle sequence of the fitting path; A manual arm kinematic model is established based on a link parameter method, and the position coordinate sequence and posture angle sequence of the fitting path are substituted into the manual arm kinematic model to obtain a joint angle sequence, and the joint angle sequence is compared with a kinematic constraint matrix to calculate the angle excess and speed excess of each joint, and the path segment interval that does not meet the constraint condition is determined according to the angle excess and speed excess; Constructing a transition curve within the path segment interval, converting the kinematic constraint matrix into the boundary constraint of the transition curve, solving the position coordinates and attitude angle of the transition node that meets the boundary constraint; substituting the transition node into a piecewise continuous polynomial function to obtain a corrected path segment; An interpolation method that maintains geometric continuity is used to splice the path segment that meets the constraint conditions with the corrected path segment to obtain an operation path; the operation path is substituted into the kinematic model to solve the joint angle sequence, determine whether the kinematic constraint condition matrix is ​​met, and output the operation path when the constraint conditions are met.

8. A truck-mounted crane manual arm operation path planning system for narrow spaces, used to implement the method described in any one of claims 1 to 7, characterized in that: include: The first unit is used to collect an environmental image of the working space through binocular stereo vision cameras installed at both ends of the vehicle-mounted crane, extract the corresponding relationship of feature points of the environmental image through a feature point matching algorithm and calculate the depth information to obtain first depth information, scan the working space through a laser ranging sensor installed on the top of the vehicle-mounted crane to obtain an obstacle contour point cloud, obtain second depth information based on the obstacle contour point cloud calculation, perform multi-source information fusion on the first depth information and the second depth information to obtain fused depth information, establish a three-dimensional model of the working space in combination with the posture information obtained by the vehicle-mounted posture sensor of the vehicle-mounted crane, perform spatial indexing and data compression on the three-dimensional model through a voxel partitioning method, and output a real-time environmental map; The second unit is used to identify and classify obstacles in the working space based on the real-time environment map by using a convolutional neural network to obtain obstacle attribute information, classify the obstacles into rigid obstacles and deformable obstacles according to the obstacle attribute information, set a first safety distance threshold and a second safety distance threshold for the rigid obstacles and the deformable obstacles respectively, establish an obstacle avoidance cost function in combination with the kinematic constraint parameters of the manual arm of the truck crane, generate path nodes by using a bidirectional probability sampling algorithm, optimize and connect the path nodes based on the obstacle avoidance cost function to obtain multiple groups of initial paths, input the load weight information of the manual arm of the truck crane into the dynamics model, calculate the joint rotation torque of the multiple groups of initial paths, and select the path that meets the dynamic constraints as the target optimization path; The third unit is used to establish a comprehensive evaluation function including path length evaluation items, motion smoothness evaluation items and obstacle avoidance safety evaluation items for the target optimization path, establish a virtual spring damping model in the coordinate system of the end effector of the manual arm of the truck crane, adjust the spring stiffness coefficient and the damping coefficient of the virtual spring damping model according to the evaluation score calculated by the comprehensive evaluation function, perform real-time dynamic adjustment on the target optimization path to obtain an adjusted path, use a piecewise continuous polynomial function to fit the adjusted path to obtain a fitting path, calculate the joint angles of the fitting path based on the kinematic model of the manual arm of the truck crane, perform local correction on the path segments that do not meet the kinematic constraints, and finally output an operation path that meets the constraints.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images