A heavy load carrier cooperative handling control method

Abstract

This invention discloses a collaborative handling control method for heavy-duty transport vehicles, belonging to the technical field of intelligent heavy-duty transport equipment. This invention utilizes a terrain-aware positioning sphere position calculation network to fuse multiple sources, including laser point clouds, image cropping regions, the attitudes of the preceding vehicle and the gimbal, and the previous position of the positioning sphere. An attitude branch and a time-recursive layer are explicitly introduced within the network. By utilizing attitude features to remove perspective change components, extrapolation is performed based on historical positions and attitude increments when occlusion occurs, ensuring the spatial position sequence of the positioning sphere remains continuous in time and conforms to actual motion laws in space. This solves the problem of difficult-to-observe, physically realizable relative position and attitude of two vehicles when working on uneven ground such as potholes, undulations, or slopes, or when the target object such as the positioning sphere may experience drastic changes in perspective or short-term occlusion. This leads to the problem of safe collaborative handling control of two heavy-duty transport vehicles carrying the same oversized and overweight cargo.

Description

Technical Field

[0001] This invention relates to the field of intelligent heavy-duty handling equipment technology, specifically to a collaborative handling control method for heavy-duty handling vehicles. Background Technology

[0002] In the handling of oversized and heavy goods such as large components, wind turbine blades, and bridge steel beams within factories and on construction sites, two or more transport vehicles are often used to lift a single item to share the weight and improve maneuverability. To ensure that the goods do not undergo excessive bending, twisting, or drops during transport, it is necessary to know the relative positions and postures of each transport vehicle in real time and coordinate the control of vehicle speed and steering accordingly. This is especially true on uneven ground conditions such as potholes, undulations, or slopes, where changes in vehicle posture are more drastic, placing higher demands on collaborative control and safety monitoring.

[0003] In existing technologies, to achieve multi-vehicle collaborative handling, position and attitude sensors, such as GNSS / IMU units, lidar, or cameras, are typically installed on the front and rear transport vehicles. The relative pose is estimated by the observation of the rear vehicle or the end of the cargo by the front vehicle. Some solutions set feature targets such as reflectors or spherical targets on the rear vehicle or cargo. The front vehicle uses lidar or vision algorithms to detect and locate the target, and then calculates the relative distance and attitude quantities such as pitch, roll, and yaw by combining the installation geometry of the two vehicles. Other solutions mainly rely on the inertial measurement and wheel speed information of each vehicle. The pose of each vehicle is obtained through coordinate transformation, thereby calculating the spatial position of the two ends of the cargo. Based on the preset spacing and angular deviation, speed and steering control commands are generated to achieve basic collaborative driving and attitude monitoring.

[0004] The aforementioned existing technologies are mostly based on single-frame observation or simple geometric fusion, which do not adequately consider the rapid changes in vehicle attitude caused by uneven ground. They fail to explicitly distinguish between changes in sensor viewpoint and the actual spatial displacement of the following vehicle or the target being located. When the target is briefly obscured by cargo or structural components, reliable position information cannot be continuously provided, easily leading to jumps and discontinuities in the relative attitude estimation of the two vehicles. At the same time, existing solutions generally lack safety index modeling and constraint mechanisms for the overall attitude of the cargo. The control layer mainly tracks the spacing or angle, making it difficult to reflect and suppress changes in the height difference, pitch difference, and roll difference between the two ends of the cargo in a timely manner, thus providing insufficient support for the safe collaborative handling of heavy-duty cargo in complex terrain. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide a collaborative handling control method for heavy-duty transport vehicles, in order to solve the problem of the difficulty in obtaining the relative position and attitude of two vehicles in a continuous and physically achievable manner under working conditions with uneven ground such as potholes, undulations or slopes, and when the target object such as the positioning ball may experience drastic changes in perspective or be temporarily obscured, thus leading to the problem of safe collaborative handling control of two heavy-duty transport vehicles for the same oversized and overweight cargo.

[0006] To achieve the above objectives, the method of the present invention is aimed at a heavy-duty transport vehicle comprising a front transport vehicle and a rear transport vehicle that travel in front and behind each other. The front transport vehicle is equipped with a gimbal sensor, and the rear transport vehicle is equipped with a positioning ball. The cooperative transport control method includes the following steps:

[0007] S1. Obtain the installation positions and orientations of the front transport vehicle body, gimbal sensor, rear transport vehicle body, and positioning ball. Establish a spatial relationship calculation model from the coordinate system of the front transport vehicle body to the coordinate system of the gimbal sensor, to the coordinate system of the positioning ball installation, and to the coordinate system of the rear transport vehicle body. Obtain the spatial relationship model between the two vehicles and output the relative distance, relative pitch, relative roll, and relative yaw angle between the two vehicles.

[0008] S2. Based on the spatial relationship model between the two vehicles, control the gimbal sensor to point to the area where the positioning ball is located, collect the laser point cloud and image of the area, obtain the current attitude of the front transport vehicle and the gimbal attitude, and organize the laser point cloud, image, front transport vehicle attitude and gimbal attitude into continuous observation data of the positioning ball in chronological order.

[0009] S3. Input the continuous observation data of the positioning ball into the terrain-aware positioning ball position calculation network to obtain the spatial position sequence of the positioning ball at continuous time moments;

[0010] The positioning sphere position calculation network includes a branch layer, a fusion layer, a temporal regression layer, and an output layer arranged sequentially. The branch layer includes a point cloud branch for processing the laser point cloud, an image branch for processing the image, and a posture branch for processing the attitude of the front transport vehicle and the gimbal. The fusion layer is used to fuse the data processed by each branch and the positioning sphere position at the previous moment. The temporal regression layer is used to combine the fusion representation result with the previous state output by the temporal regression layer at the previous moment to update and output a new temporal state. The output layer is used to generate the spatial position of the positioning sphere at the current moment through three fully connected layers and aggregate them into a spatial position sequence of the positioning sphere in chronological order.

[0011] S4. Input the spatial position sequence of the positioning ball output by the terrain-aware positioning ball position calculation network into the spatial relationship model of the two vehicles, complete the continuous calculation and constraint correction of the relative position and attitude of the two vehicles in the coordinate system of the front transport vehicle, and output the direction data for gimbal control.

[0012] S5. Output the coordinated control parameters of the front and rear transport vehicles and the gimbal control commands based on the direction data.

[0013] Furthermore, in step S1, establishing the spatial relationship calculation model from the front transport vehicle body coordinate system to the gimbal sensor coordinate system, to the positioning ball mounting coordinate system, and to the rear transport vehicle body coordinate system includes:

[0014] Unify the installation positions and orientations of the front transport vehicle body, gimbal sensor, positioning ball, and rear transport vehicle body to the coordinate system of the front transport vehicle body, determine the installation offset and installation orientation parameters of each component, and give the origin and attitude reference direction of the coordinate system of the front transport vehicle body.

[0015] The pitch and yaw rotation axes of the gimbal sensor are calibrated to establish the mapping between the pitch angle and yaw angle and the line of sight of the gimbal sensor coordinate system.

[0016] Based on the fixed loading position of the positioning ball on the rear transport vehicle body, establish the spatial relationship between the positioning ball installation coordinate system and the rear transport vehicle body coordinate system, and determine the position and orientation of the positioning ball center relative to the rear transport vehicle body coordinate system;

[0017] By connecting the aforementioned spatial relationships, a spatial relationship calculation model is formed, from the coordinate system of the front transport vehicle to the coordinate system of the gimbal sensor, to the coordinate system of the positioning ball installation, and to the coordinate system of the rear transport vehicle.

[0018] The method of this invention calibrates the pitch and yaw rotation axes of the gimbal sensor, establishes a mapping between the pitch and yaw angles and the line of sight of the gimbal sensor coordinate system, and uses this to distinguish the changes in viewing angle in the subsequent terrain perception positioning ball position calculation network. It clarifies the definition of the zero position angle of the gimbal sensor and the direction of angle increase, thereby enabling accurate determination of the relative position and attitude of the two vehicles and safe collaborative transport control even when the positioning ball may undergo drastic changes in viewing angle.

[0019] Furthermore, in step S2, controlling the gimbal sensor to point towards the area where the positioning ball is located based on the spatial relationship model between the two vehicles includes:

[0020] Based on the direction data from the gimbal sensor to the positioning ball output by the spatial relationship model of the two vehicles, the gimbal sensor is controlled to rotate so that the area where the positioning ball is located falls into the center of the observation range of the gimbal sensor, and the difference between the direction data and the gimbal attitude does not exceed the preset threshold.

[0021] In the data acquisition phase, this invention uses the output of the spatial relationship model between the two vehicles as the control basis, and controls the gimbal sensor to rotate to keep the direction data (i.e., the direction data from the gimbal to the positioning ball) and the gimbal attitude difference from not exceeding a preset threshold. This ensures that the area where the positioning ball is located enters the center of the observation range of the gimbal sensor and maintains a stable direction. This provides a stable set of local points of the positioning ball for the subsequent terrain perception positioning ball position calculation network, so that even if the positioning ball may change drastically in terms of viewing angle, the relative position and attitude of the two vehicles can still be accurately determined, and safe collaborative transport control can be performed.

[0022] Further, in step S3, the fusion layer is used to fuse the data processed by each branch and the position of the positioning ball at the previous moment to generate spatial features that reflect the shape and distance of the positioning ball from the laser point cloud and the image. Based on the attitude of the front transport vehicle and the attitude of the gimbal, the network distinguishes the perspective changes caused by the movement of the vehicle body and the gimbal from the spatial displacement caused by the actual movement of the positioning ball.

[0023] The time-recursive layer combines the fusion representation result with the previous state output by the time-recursive layer at the previous moment to update the data, so as to model the continuous observation data of the positioning ball in the time dimension. When the positioning ball is obscured by the cargo and the ground height changes, the spatial position of the positioning ball at the current moment is predicted based on the spatial position of the positioning ball at the previous moment, combined with the current attitude of the front transport vehicle and the attitude of the gimbal, so as to obtain the spatial position sequence of the positioning ball at continuous moments.

[0024] Further, in step S3, the point cloud branch includes voxels and 3D convolutional layers, outputting 128-dimensional features; the image branch includes 4 layers of 2D convolutions, outputting 128-dimensional features; the pose branch includes a fully connected layer that connects the 128-dimensional feature input to the 64-dimensional feature output; the fusion layer includes a fully connected layer that concatenates the 128-dimensional features output from the point cloud branch, the 128-dimensional features output from the image branch, the 64-dimensional feature output from the pose branch, and the 3D coordinates of the previous positioning ball position into a 319-dimensional feature input, which is then connected to the 128-dimensional feature output; the time recursion layer includes a fully connected layer that connects the 128-dimensional temporal state output from the previous time recursion layer and the 128-dimensional feature output from the fusion layer to the 128-dimensional output of the new temporal state; and the output layer includes a fully connected layer that connects the 128-dimensional output of the new temporal state from the time recursion layer to the 64-dimensional output and then to the 3-dimensional output of the current positioning ball spatial position.

[0025] This invention performs voxel segmentation and 3D convolutional layer extraction on the laser point cloud branch, and combines global convergence to generate 128-dimensional point cloud features. These features reflect the circular shape, surface texture, and distance distribution of the positioning sphere, and are used for subsequent fusion. The image cropping region is downsampled stepwise by four layers of 2D convolution on the image branch and globally converged to obtain 128-dimensional image features, used to distinguish the positioning sphere from environmental objects when cargo corners, vehicle structural components, and ground protrusions appear simultaneously. The attitude of the front transport vehicle and the gimbal are input into the attitude branch and mapped to 64-dimensional attitude features through two fully connected layers. These features are used to calculate the viewpoint changes caused by the pitch, roll, gimbal pitch, and gimbal yaw in the attitude fusion part. In the fusion layer, the 128-dimensional point cloud features, 128-dimensional image features, 64-dimensional attitude features, and the previous time step of three numbers are combined. The spatial positions of the positioning sphere are stitched together to form a 319-dimensional fusion feature, which is then mapped to a 128-dimensional fusion representation. This representation is updated in the time-recursive layer by combining the 128-dimensional temporal state from the previous moment, maintaining the temporal state in chronological order. In the attitude fusion part, the perspective change and the actual displacement of the positioning sphere are separated based on the attitude features. When the continuous observation data of the positioning sphere only contains part of the point cloud or part of the image contour, the spatial position of the positioning sphere at the current moment is calculated based on the spatial position of the positioning sphere at the previous moment, combined with the current attitude of the forward transport vehicle and the gimbal. When the positioning sphere is briefly obscured by cargo and the ground height changes significantly, the position is extrapolated based on the trend of the spatial position change of the positioning sphere in recent moments, combined with the current attitude of the forward transport vehicle and the gimbal. In the output layer, three fully connected layers generate the three numbers of the spatial position of the positioning sphere at the current moment, which are then aggregated into a spatial position sequence of the positioning sphere in chronological order.

[0026] Further, in step S3, the distinction between the viewpoint changes caused by the movement of the vehicle body and the gimbal and the spatial displacement caused by the actual movement of the positioning ball within the network based on the attitude of the front transport vehicle and the attitude of the gimbal includes: setting a viewpoint change criterion threshold, combining the 64-dimensional feature output of the attitude branch with the 128-dimensional feature output of the fusion layer, determining whether the viewpoint change amplitude exceeds the set viewpoint change criterion threshold, and when the laser point cloud and the image have an effective contour or point cloud feature of the positioning ball at the current moment and the angle change amplitude exceeds the set viewpoint change criterion threshold, using the imaging change direction given by the attitude feature to correct the spatial features.

[0027] This invention sets a threshold for judging viewpoint changes. This threshold is used to identify the range of viewpoint changes triggered by the pitch and roll of the front transport vehicle and the pitch and yaw of the gimbal in the attitude fusion part. It separates the viewpoint changes from the actual displacement of the positioning ball, thus avoiding the spatial position jump of the positioning ball caused by sudden changes in viewpoint when crossing potholes and slopes.

[0028] Further, in step S4, the output of directional data for gimbal control includes: inputting the spatial position sequence of the positioning ball into the spatial relationship model between the two vehicles; calculating the initial relative position attitude sequence of the transport vehicle relative to the front transport vehicle based on the spatial position of the positioning ball at each time; calculating the predicted relative position attitude sequence based on the relative position attitude of the two vehicles at the previous time and the current attitude of the front transport vehicle; combining the initial relative position attitude sequence and the predicted relative position attitude sequence under preset attitude change constraints; when the attitude change exceeds the preset attitude change constraint at a certain time, suppressing the attitude change at that time based on the predicted relative position attitude sequence to obtain a time-continuous relative position attitude sequence of the two vehicles; and determining the directional data from the gimbal to the positioning ball at each time from the relative position attitude sequence of the two vehicles.

[0029] Furthermore, in step S5, the output of coordinated control parameters for the front and rear transport vehicles and gimbal control commands based on the direction data includes:

[0030] Based on the relative position and attitude sequence of the two vehicles and the fixed loading position of the cargo on the front and rear transport vehicles, the height difference and attitude difference between the two ends of the cargo at each moment are calculated to generate a cargo attitude safety index that characterizes whether the cargo meets the transportation safety requirements.

[0031] Based on the relative position and attitude sequence of the two vehicles, the direction data of the gimbal pointing to the positioning ball, and the cargo attitude safety indicators, the system generates two-vehicle collaborative control parameters for controlling the coordinated movement of the front and rear transport vehicles, as well as gimbal control commands for controlling the gimbal to follow the rotation of the positioning ball. The system then outputs the two-vehicle collaborative control parameters and the gimbal control commands.

[0032] Furthermore, in step S5, generating cargo attitude safety indicators that characterize whether the cargo meets transportation safety requirements includes:

[0033] The relative position and attitude sequence of the two vehicles is associated with the fixed loading position of the goods on the front and rear transport vehicles in the coordinate system of the front transport vehicle. The spatial position of the front and rear of the goods at each moment is calculated, and the endpoint of the fixed loading position is used as the reference point for the front and rear of the goods.

[0034] The height difference between the two ends of the cargo is determined based on the vertical coordinate difference between the front and rear spatial positions of the cargo at each moment, and the cargo attitude difference is calculated based on the combination of the relative pitch and relative roll of the two vehicles and the fixed loading position, including the end pitch difference and end roll difference.

[0035] Set preset height difference threshold and preset posture difference threshold, determine threshold parameters based on cargo length and weight and fixed loading position, determine threshold for height difference and posture difference at each time, and generate judgment results for compliance and non-compliance at each time.

[0036] The judgment results at each moment are merged into the cargo attitude safety index in chronological order, so that the cargo attitude safety index is consistent with the relative position attitude sequence of the two vehicles in chronological order.

[0037] Further, in step S5, generating the two-vehicle collaborative control parameters for controlling the coordinated movement of the front and rear transport vehicles, and the gimbal control commands for controlling the gimbal to follow the rotation of the positioning ball, includes:

[0038] The relative position and attitude sequence of the two vehicles is correlated with the cargo attitude safety index. Based on the relative distance and each relative angle, the initial values ​​of the two-vehicle cooperative control parameters required for the coordinated driving of the front and rear transport vehicles are calculated.

[0039] Based on the judgment results of the cargo posture safety index and the preset control threshold, the range of change of the collaborative control parameters of the two vehicles is constrained, so that the parameters can be progressively adjusted within the range of relative height and relative angle changes.

[0040] Based on the relative position and attitude sequence of the two vehicles and the direction data of the gimbal pointing to the positioning ball, a gimbal control command is generated to control the rotation of the gimbal sensor and ensure that the pointing deviation does not exceed the preset pointing threshold.

[0041] The constrained collaborative control parameters of the two vehicles and the gimbal control commands are output in chronological order to control the coordinated driving of the front and rear transport vehicles and keep the positioning ball within the observation area that the terrain-aware positioning ball position calculation network can receive.

[0042] The method of the present invention has the following advantages:

[0043] This invention uses a terrain-aware positioning ball position calculation network to perform multi-source fusion of laser point cloud, image cropping region, the attitude of the preceding vehicle and gimbal, and the previous position of the positioning ball. It also explicitly introduces attitude branches and time recursion layers into the network, uses attitude features to remove the perspective change component, and extrapolates based on historical position and attitude increment when occlusion occurs, so that the spatial position sequence of the positioning ball remains continuous in time and conforms to the actual motion law in space.

[0044] On the one hand, this invention uses a two-vehicle spatial relationship model to strictly connect the coordinate system of the front vehicle, the gimbal sensor, the positioning ball installation coordinate system, and the rear vehicle coordinate system. Combined with the positioning ball position output by the terrain perception network, it obtains the relative attitude sequence of the two vehicles after attitude change threshold constraint and predicted attitude fusion correction, thus avoiding unattainable attitudes caused by measurement anomalies. On the other hand, based on the loading points of the cargo at the front and rear vehicles and the length and weight of the cargo, it calculates the height difference between the two ends of the cargo and the pitch difference and roll difference at the ends in real time, constructs cargo attitude safety indicators, and applies amplitude and rate of change constraints to the longitudinal speed and steering angle commands of the front and rear vehicles.

[0045] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described in detail below with reference to the accompanying drawings. Attached Figure Description

[0046] Figure 1 This is a schematic diagram of the collaborative handling control method for heavy-duty transport vehicles of the present invention;

[0047] Figure 2 This is a flowchart illustrating the process of obtaining the spatial relationship model between the two vehicles in step S1 of the present invention.

[0048] Figure 3 This is a schematic diagram of the process for obtaining continuous observation data of the positioning ball in step S2 of the present invention;

[0049] Figure 4 This is a flowchart illustrating the process of obtaining the spatial position sequence of the positioning ball in step S3 of the present invention.

[0050] Figure 5 This is a flowchart illustrating the process of obtaining the relative position and attitude sequence of the two vehicles in step S4 of the present invention.

[0051] Figure 6 This is a side view illustration of two moving vehicles working together to lift and transport oversized and overweight goods on uneven ground.

[0052] Figure 7 This is a top-down view showing the geometric relationship between the front and rear transport vehicles working together to lift and transport oversized and overweight goods.

[0053] Figure 8 This is a magnified view of the spatial geometric relationship between the gimbal sensor of the front transport vehicle and the positioning ball of the rear transport vehicle;

[0054] Figure 9 This is a schematic diagram of the terrain-aware positioning sphere location calculation network used in this embodiment. Detailed Implementation

[0055] The technical solution of the present invention will be clearly and completely described below with reference to specific embodiments. However, those skilled in the art should understand that the embodiments described below are only for illustrating the present invention and should not be regarded as limiting the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0056] Example of a collaborative handling control method for heavy-duty transport vehicles

[0057] The innovation of this invention lies in:

[0058] An improved terrain-aware positioning sphere location calculation method is adopted. Specifically, by introducing point cloud branches, image branches, attitude branches, and temporal state recursion into the neural network structure, compared with existing algorithms that only detect the positioning sphere based on single-frame point clouds or images, this method can explicitly utilize the pitch and roll of the transport vehicle, the pitch and yaw of the gimbal, and the spatial position of the positioning sphere at the previous moment to distinguish between viewpoint changes and the actual displacement of the sphere. In the fusion layer, the network concatenates multimodal features with the position at the previous moment. In the temporal recursion layer, a continuous temporal state is maintained. In the attitude fusion part, the interference of abrupt viewpoint changes caused by potholes and slopes on the positioning results is suppressed based on attitude features and viewpoint change thresholds. At the same time, when the positioning sphere is briefly obscured by cargo or structural components, extrapolation with attitude compensation is performed using the position and attitude increment of the two most recent moments. Through the above improvements, the network can still output a sequence of positioning ball positions that is continuous in time and conforms to the real motion law in space, even under uneven ground and short-term occlusion conditions. This provides a stable input for the subsequent determination of the relative attitude of the two vehicles and avoids the pose jump problem caused by unreliable single-frame observation in traditional methods.

[0059] A novel method for modeling and fusing spatial relationships between the two vehicles with attitude constraints is employed. Specifically, the coordinate systems of the front transport vehicle, the gimbal sensor, the positioning ball, and the rear transport vehicle are concatenated using installation offset and orientation parameters to construct a unified spatial relationship model from the front to the rear vehicle. Based on this, the spatial position of the positioning ball output by the terrain perception network is converted into the initial relative distance, pitch, roll, and yaw angles of the rear transport vehicle relative to the front transport vehicle. Simultaneously, the relative attitudes of the two vehicles at the previous moment and the current attitude change of the front vehicle are used as inputs to calculate and predict the relative attitude range. Relative height and relative angle change thresholds are set based on cargo length, weight, and fixed loading position. The initial and predicted relative attitudes are continuously combined and corrected under these constraints. When the relative height or relative angle exceeds the achievable range at a certain moment, the predicted attitude is used to suppress the excessive components, ensuring that the output relative attitude sequence of the two vehicles remains smooth in time and meets the spatially acceptable attitude change boundaries of the cargo. Compared with existing methods that rely solely on geometric inverse calculations, the method of this invention can maintain the physical reachability of relative attitude estimation even when measurement noise is high or terrain undulations are severe, and simultaneously output gimbal pointing data consistent with the attitude sequence, ensuring the stability of gimbal closed-loop tracking.

[0060] A holistic control method for heavy-duty collaborative transport on uneven ground is adopted. Based on the aforementioned positioning and attitude determination, the relative position and attitude sequence of the two vehicles is correlated with the coordinates of the loading points of the cargo on the front and rear transport vehicles. The spatial position of the cargo's front and rear ends, the height difference between the two ends, and the pitch and roll differences at the ends are calculated in real time to construct a cargo attitude safety index. This index is then used to constrain the variation range of the collaborative control parameters of the two vehicles. Specifically, the control layer first generates initial values ​​for the longitudinal speed and steering angle of the front and rear transport vehicles based on the relative distance error and relative attitude error. Then, based on the cargo attitude safety index and preset control thresholds, the speed and steering increments are pruned and smoothed over time to ensure that while adjusting the relative position and attitude of the two vehicles, the height difference or attitude difference between the two ends of the cargo does not exceed the safe range. Furthermore, the gimbal control commands are generated jointly from the constrained relative attitude sequence and direction data, ensuring that the positioning ball remains within the observation area that the terrain perception network can receive during the collaborative movement of the vehicles. Through the above-mentioned overall approach, an integrated design was achieved in complex terrain, from positioning ball observation and relative attitude modeling of the two vehicles to cargo attitude safety constraints and collaborative control. This approach specifically addresses the problem of obtaining temporally continuous and physically achievable relative attitudes and implementing safe collaborative control based on them when two heavy-duty transport vehicles are lifting the same ultra-long and ultra-heavy cargo on uneven roads.

[0061] like Figure 1 The method flow of the present invention includes:

[0062] S1. When two transport vehicles are carrying oversized and overweight goods and traveling on ground with potholes, undulations or slopes, obtain the installation position and orientation of the front transport vehicle body, the gimbal sensor installed on the front transport vehicle, the rear transport vehicle body and the positioning ball fixed on the rear transport vehicle, establish a spatial relationship calculation model between the front transport vehicle, the gimbal sensor, the positioning ball and the rear transport vehicle, and obtain the spatial relationship model between the two vehicles.

[0063] S2. Based on the spatial relationship model between the two vehicles, control the gimbal sensor to point to the area where the positioning ball is located, collect the laser point cloud and image of the area, obtain the current attitude of the front transport vehicle and the gimbal attitude, and organize the laser point cloud, image, front transport vehicle attitude and gimbal attitude into continuous observation data of the positioning ball in chronological order.

[0064] S3. Input the continuous observation data of the positioning ball, the attitude of the front transport vehicle, and the attitude of the gimbal into the terrain-aware positioning ball position calculation network. Generate spatial features reflecting the shape and distance of the positioning ball from the laser point cloud and the image. Based on the attitude of the front transport vehicle and the attitude of the gimbal, distinguish the perspective changes caused by the movement of the vehicle and the gimbal from the spatial displacement caused by the actual movement of the positioning ball within the network. Model the continuous observation data of the positioning ball in the time dimension. When the positioning ball is briefly obscured by the cargo and the ground height changes significantly, infer the spatial position of the positioning ball at the current moment based on the spatial position of the positioning ball at the previous moment combined with the current attitude of the front transport vehicle and the attitude of the gimbal, and obtain the spatial position sequence of the positioning ball at continuous moments.

[0065] S4. Input the spatial position sequence of the positioning ball into the spatial relationship model between the two vehicles. Calculate the initial relative position attitude sequence of the transport vehicle relative to the front transport vehicle based on the spatial position of the positioning ball at each moment. Then, calculate the predicted relative position attitude sequence based on the relative position attitude of the two vehicles at the previous moment and the current attitude of the front transport vehicle. Combine the initial relative position attitude sequence and the predicted relative position attitude sequence under the preset attitude change constraint. When the attitude change exceeds the preset attitude change constraint at a certain moment, the attitude change at that moment is suppressed based on the predicted relative position attitude sequence to obtain a time-continuous relative position attitude sequence of the two vehicles. Determine the direction data from the gimbal to the positioning ball at each moment from the relative position attitude sequence of the two vehicles.

[0066] S5. Based on the relative position and attitude sequence of the two vehicles and the fixed loading position of the goods on the front and rear transport vehicles, calculate the height difference and attitude difference between the two ends of the goods at each moment, and generate a cargo attitude safety index that characterizes whether the goods meet the transportation safety requirements.

[0067] S6. Based on the relative position and attitude sequence of the two vehicles, the direction data of the gimbal pointing to the positioning ball, and the cargo attitude safety index, generate the two-vehicle collaborative control parameters for controlling the coordinated movement of the front and rear transport vehicles, as well as the gimbal control command for controlling the gimbal to follow the rotation of the positioning ball, and output the two-vehicle collaborative control parameters and the gimbal control command.

[0068] like Figure 2 Step S1 specifically includes:

[0069] Unify the installation positions and orientations of the front transport vehicle body, gimbal sensor, positioning ball, and rear transport vehicle body to the coordinate system of the front transport vehicle body, determine the installation offset and installation orientation parameters of each component, and give the origin and attitude reference direction of the coordinate system of the front transport vehicle body.

[0070] The pitch and yaw rotation axes of the gimbal sensor are calibrated, and the mapping of the pitch and yaw angles to the line of sight of the gimbal sensor coordinate system is established. This is used by the subsequent terrain perception positioning ball position calculation network to distinguish the changes in viewing angle and to clarify the definition of the zero position angle of the gimbal sensor and the direction of angle increase.

[0071] Based on the fixed loading position of the positioning ball on the rear transport vehicle body, establish the spatial relationship between the positioning ball installation coordinate system and the rear transport vehicle body coordinate system, determine the position and orientation of the positioning ball center relative to the rear transport vehicle body coordinate system, and give the geometric boundary of the observable area on the outer surface of the positioning ball.

[0072] By connecting the aforementioned spatial relationships, a spatial relationship calculation model is formed, from the coordinate system of the front transport vehicle to the coordinate system of the gimbal sensor, to the coordinate system of the positioning ball installation, and to the coordinate system of the rear transport vehicle. When the input is the installation position and orientation, the output is the relative distance, relative pitch, relative roll, and relative yaw angle between the two vehicles, thus obtaining the spatial relationship model between the two vehicles.

[0073] like Figure 3 Step S2 specifically includes:

[0074] Based on the direction data from the gimbal sensor to the positioning ball output by the spatial relationship model of the two vehicles, the gimbal sensor is controlled to rotate so that the area where the positioning ball is located falls into the center of the observation range of the gimbal sensor, and the difference between the direction data and the gimbal attitude does not exceed the preset threshold.

[0075] The laser point cloud near the center of the observation range of the gimbal sensor is limited to a local set of points with a radius of three meters. Two thousand four hundred points are extracted at each moment, and the three-dimensional coordinates and intensity of each point are recorded to form the laser point cloud at the current moment.

[0076] Simultaneously acquire images corresponding to the laser point cloud, limit the image cropping area to a 400x400 color image with three channels, and form the image cropping area at the current moment with the center of the observation range of the gimbal sensor as the cropping center;

[0077] The current attitude of the front transport vehicle and the attitude of the gimbal are obtained. The attitude of the front transport vehicle includes the pitch and roll of the front transport vehicle, and the attitude of the gimbal includes the pitch and yaw of the gimbal. The laser point cloud, the image cropping area, the attitude of the front transport vehicle and the attitude of the gimbal are organized into continuous observation data of the positioning ball in time order according to a unified sampling period.

[0078] like Figure 4 Step S3 specifically includes:

[0079] The laser point cloud, image cropping region, front transport vehicle attitude, gimbal attitude, and the spatial position of the positioning ball in the continuous observation data of the positioning ball are input into the terrain perception positioning ball position calculation network to establish a unified input for the current moment and lock the time sequence.

[0080] Voxel division and 3D convolutional layer extraction are performed on the laser point cloud branches. Combined with global convergence, a 128-dimensional point cloud feature is generated. This point cloud feature reflects the circular shape, surface texture and distance distribution of the positioning sphere, which is used for subsequent fusion.

[0081] The image cropping region is downsampled stepwise by four layers of two-dimensional convolution in the image branch and then globally converged to obtain 128-dimensional image features, which are used to distinguish the positioning ball from environmental objects when cargo corners, vehicle body structural parts and ground protrusions appear simultaneously.

[0082] The attitude of the front transport vehicle and the attitude of the gimbal are input into the attitude branch and mapped to a 64-dimensional attitude feature through two fully connected layers. This feature is used to calculate the viewpoint changes caused by the pitch, roll, pitch and yaw of the front transport vehicle in the attitude fusion part.

[0083] In the fusion layer, the 128-dimensional point cloud features, 128-dimensional image features, 64-dimensional pose features and the spatial position of the positioning ball at the previous time step of the three numbers are spliced ​​together to form a 319-dimensional fusion feature, which is then mapped to a 128-dimensional fusion representation. In the time recursion layer, the 128-dimensional temporal state at the previous time step is combined and updated, and the temporal state is maintained in time sequence.

[0084] In the attitude fusion part, the viewpoint change and the actual displacement of the positioning ball are separated based on attitude characteristics. When the continuous observation data of the positioning ball only contains part of the point cloud or part of the image outline, the spatial position of the positioning ball at the current moment is calculated based on the spatial position of the positioning ball at the previous moment, combined with the current attitude of the front transport vehicle and the gimbal attitude. When the positioning ball is briefly occluded by the cargo and the ground height changes significantly, the position is extrapolated based on the trend of the spatial position change of the positioning ball in recent moments and the current attitude of the front transport vehicle and the gimbal attitude.

[0085] The output layer generates three numbers representing the spatial position of the positioning ball at the current moment using three fully connected layers, and then aggregates them into a spatial position sequence of the positioning ball in chronological order.

[0086] like Figure 5 Step S4 specifically includes:

[0087] The spatial position sequence of the positioning ball obtained by the terrain-aware positioning ball position calculation network is input into the spatial relationship model between the two vehicles. In the coordinate system of the front transport vehicle, the spatial position of the positioning ball at each moment is converted into the initial relative position attitude sequence of the rear transport vehicle, including relative distance, relative pitch, relative roll, and relative yaw angle.

[0088] Based on the relative position and attitude of the two vehicles at the previous moment and the current attitude of the front transport vehicle, calculate the predicted relative position and attitude sequence at the current moment, limit the range of attitude changes caused by ground potholes, undulations or slopes, and associate it with the attitude of the front transport vehicle.

[0089] Establish preset attitude change constraints, set relative height change threshold and relative angle change threshold, and determine constraint parameters based on the fixed loading position of the goods on the front and rear transport vehicles, and the length and weight of the goods.

[0090] Under preset attitude change constraints, the initial relative position attitude sequence and the predicted relative position attitude sequence are combined and calculated, and the combined relative position attitude is output in time sequence to maintain continuous update of the sequence.

[0091] When the relative height change or relative angle change of the relative position and attitude after a certain moment exceeds the preset attitude change constraint, the attitude change at that moment is suppressed and corrected to an achievable relative position and attitude based on the predicted relative position and attitude sequence.

[0092] The corrected relative position attitudes are collected in time sequence to form a sequence of relative position attitudes between the two vehicles, maintaining the continuity of relative distance and relative angles in the coordinate system of the front transport vehicle.

[0093] The direction data from the gimbal to the positioning ball at each moment is determined from the relative position and attitude sequence of the two vehicles, so that the direction data is consistent with the relative position and attitude sequence of the two vehicles. This is used for subsequent gimbal control to keep the positioning ball in the observation area that the terrain-aware positioning ball position calculation network can receive.

[0094] Step S5 specifically includes:

[0095] The relative position and attitude sequence of the two vehicles is associated with the fixed loading position of the goods on the front and rear transport vehicles in the coordinate system of the front transport vehicle. The spatial position of the front and rear of the goods at each moment is calculated, and the endpoint of the fixed loading position is used as the reference point for the front and rear of the goods.

[0096] The height difference between the two ends of the cargo is determined based on the vertical coordinate difference between the front and rear spatial positions of the cargo at each moment, and the cargo attitude difference is calculated based on the combination of the relative pitch and relative roll of the two vehicles and the fixed loading position, including the end pitch difference and end roll difference.

[0097] Set preset height difference threshold and preset posture difference threshold, determine threshold parameters based on cargo length and weight and fixed loading position, determine threshold for height difference and posture difference at each time, and generate judgment results for compliance and non-compliance at each time.

[0098] The judgment results at each moment are merged into the cargo attitude safety index in chronological order, so that the cargo attitude safety index is consistent with the relative position attitude sequence of the two vehicles in chronological order.

[0099] Step S6 specifically includes:

[0100] The relative position and attitude sequence of the two vehicles is correlated with the cargo attitude safety index. Based on the relative distance and each relative angle, the initial values ​​of the two-vehicle cooperative control parameters required for the coordinated driving of the front and rear transport vehicles are calculated.

[0101] Based on the judgment results of the cargo posture safety index and the preset control threshold, the range of change of the collaborative control parameters of the two vehicles is constrained, so that the parameters can be progressively adjusted within the range of relative height and relative angle changes.

[0102] Based on the relative position and attitude sequence of the two vehicles and the direction data of the gimbal pointing to the positioning ball, a gimbal control command is generated to control the rotation of the gimbal sensor and ensure that the pointing deviation does not exceed the preset pointing threshold.

[0103] The constrained collaborative control parameters of the two vehicles and the gimbal control commands are output in chronological order to control the coordinated driving of the front and rear transport vehicles and keep the positioning ball within the observation area that the terrain-aware positioning ball position calculation network can receive.

[0104] like Figure 6 This illustration depicts a side view of a front and rear transport vehicle lifting an oversized and overweight cargo on uneven, undulating, or sloping terrain. The cargo straddles the front and rear transport vehicles. While the two vehicles are at different ground heights and have different pitches, the cargo remains largely horizontal, with minimal height difference between its two ends, demonstrating that its posture meets transportation safety requirements. A gimbal sensor mounted on the front transport vehicle is located above the front of its body, while a positioning ball fixed on the rear transport vehicle is positioned above its body. The gimbal sensor's line of sight points towards the positioning ball, indicating that the spatial relationship model between the two vehicles controls the gimbal sensor's orientation towards the positioning ball's location, providing stable and continuous observation data for the terrain-aware positioning ball position calculation network. This invention obtains a temporally continuous spatial position sequence of the positioning ball under uneven ground conditions through a terrain-aware positioning ball position calculation network. Combined with the spatial relationship model of the two vehicles and the cargo posture safety index, it generates collaborative control parameters for the two vehicles and gimbal control commands, so that the relative position posture sequence of the two vehicles is continuous and stable, and the height difference and end posture difference of the cargo are controlled. This significantly improves the safety and reliability of heavy-duty collaborative handling under conditions of potholes, undulations or slopes.

[0105] like Figure 7This diagram uses a top-down view to illustrate the geometric relationship between the front and rear transport vehicles as they collaboratively lift and transport oversized and overweight cargo. The front and rear transport vehicles are connected by the oversized and overweight cargo, and their relative deflection and distance within a plane demonstrate the changes in their relative positions and attitudes during collaborative transport. A gimbal sensor mounted on the front transport vehicle is located at the front of its body, while a positioning ball fixed on the rear transport vehicle is located above its body. The fan-shaped area in front of the gimbal sensor represents its observation range, and the arrow between the gimbal sensor and the positioning ball indicates the directional data from the gimbal sensor to the positioning ball. The dashed line and double-headed arrow between the two vehicles represent their relative distance, and the curved arrows above the front and rear transport vehicles indicate their relative deflection. By using a two-vehicle spatial relationship model, the installation positions and orientations of the front transport vehicle body, rear transport vehicle body, gimbal sensor, and positioning ball are unified into the coordinate system of the front transport vehicle body. When the input is the installation position and orientation, the output is the relative distance and relative deflection angle between the two vehicles. Combined with the directional data from the gimbal sensor to the positioning ball, this is used to control the center of the gimbal sensor's observation range to point towards the area where the positioning ball is located. This provides stable and continuous positioning ball observation data for the terrain-aware positioning ball position calculation network, enabling the precise execution of the heavy-duty transport vehicle collaborative transport control method even when the relative position and attitude sequence of the two vehicles changes in the plane. This improves the safety and reliability of the two-vehicle collaborative transport control method under complex driving conditions.

[0106] like Figure 8 The diagram shows a magnified view of the spatial geometric relationship between the gimbal sensor of the front transport vehicle and the positioning ball of the rear transport vehicle. Figure 8 The gimbal sensor coordinate system is drawn with its geometric center as the origin, and its X, Y, and Z axes are marked to clarify the definition of gimbal attitude and line of sight. On the right is a positioning sphere, with its center marked. Dashed ellipses and arcs on the front of the sphere represent the geometric boundary of the observable area on its outer surface. A straight line connecting the origin of the gimbal sensor coordinate system to a point on the observable area of ​​the positioning sphere represents the directional data from the gimbal sensor to the positioning sphere, and also indicates the working state where the center of the gimbal sensor's observation range falls within the observable area of ​​the positioning sphere. This invention explicitly constrains the geometric boundary of the observable area on the outer surface of the positioning sphere and accurately calculates the directional data from the gimbal sensor to the positioning sphere based on the gimbal sensor coordinate system in the two-vehicle spatial relationship model. This ensures that the gimbal's line of sight stably passes through the observable area under vehicle movement, attitude changes, and terrain undulations, thereby obtaining continuous and reliable positioning sphere observation data. This provides high-quality input for the terrain-aware positioning sphere position calculation network, significantly improving the positioning accuracy and robustness of the two-vehicle cooperative transport control.

[0107] The specific implementation of the method of the present invention is as follows:

[0108] The specific implementation method of S1 is as follows:

[0109] The coordinate system of the front transport vehicle is unified and calibrated to establish a spatial relationship calculation model from the coordinate system of the front transport vehicle to the coordinate system of the gimbal sensor, to the coordinate system of the positioning ball, and to the coordinate system of the rear transport vehicle. The relative distance, relative pitch, relative roll, and relative yaw angles of the two vehicles are output, resulting in a spatial relationship model for the two vehicles. This model is used for the input organization of the subsequent gimbal sensor pointing control and terrain-aware positioning ball position calculation network. The coordinate system of the front transport vehicle is set as C... f Its origin is o f Attitude reference direction along x f Axis, perpendicular to the z-axis f Axis, lateral along y f The axis is set to C, and the coordinate system of the gimbal sensor is set to C. g The coordinate system for the positioning ball installation is set to C. b The coordinate system of the rear transport vehicle is set as C. r The installation offset and installation orientation parameters of the gimbal sensor are denoted as p. g With q g The installation offset and installation orientation parameters of the positioning ball are denoted as p. b With q b The installation offset indicates that the origin of the coordinate system is at C. f The position in the middle, the installation orientation parameter indicates the coordinate system axis in C f Orientation in;

[0110] The pitch and yaw rotation axes of the gimbal sensor are calibrated at C. g Define coordinate axes in the middle: x g The axis is set to the zero-position direction of the gimbal's line of sight, y g The axis is set as the positive direction of the left and right sides, z g The axis is set to the positive vertical direction, and the gimbal pitch angle is denoted as... The gimbal deflection angle is denoted as The line of sight of the gimbal is denoted as l. g The rotation sequence is specified as pitch first, then yaw, and the zero azimuth angle is defined as... and time l g With C f The attitude reference direction is consistent, and the direction of pitch angle increase is defined as circumferential around y. g Forward rotation, the deflection angle increases around the z-axis. g Positive rotation, establishing from pitch angle With deflection angle to l g The mapping is used in the attitude branch to distinguish the viewpoint changes caused by the attitude of the front transport vehicle and the gimbal, ensuring consistency between subsequent inputs and time modeling. The pitch of the front transport vehicle is denoted as... The front transport vehicle rolls sideways as... In C f The attitude reference direction is given in the middle, so that the attitude branches are aligned. and The encoding is consistent with the gimbal attitude encoding;

[0111] Based on the fixed loading position of the positioning ball on the rear transport vehicle body, establish the positioning ball installation coordinate system C. b With the rear transport vehicle body coordinate system C r The spatial relationship places the center of the positioning ball at C. r The position in is denoted as c. b The radius of the positioning sphere is denoted as r. b Center the positioning ball at C b The position in the middle is set as the origin, and the geometric boundary of the observable area on the outer surface of the positioning sphere is given, denoted by a preset included angle threshold. , with l g The angle between the positioning ball and the normal to the outer surface of the positioning ball is less than The spherical curve forms the boundary, used to limit the observation range that the positioning ball can receive in the terrain-aware positioning ball position calculation network during gimbal pointing control. The occlusion relationship between the rear transport vehicle and the positioning ball is determined according to C. r Structural components in C f The projection in the image is constrained, limiting the line of sight between the gimbal and C. r The intersection of structural components does not enter the observable area, thus preventing the observable area from crossing the boundary when two vehicles are traveling together;

[0112] By connecting the above spatial relationships in series, a spatial relationship model between the two vehicles is formed. This model is denoted as M, and the installation offset p is used. g p b With installation orientation parameter q g q b In C f The unified representation below, according to C f →C g C again f →C r →C b The coordinate relationships are combined in sequence so that when the input is the installation position and installation orientation, M outputs the relative distance, relative pitch, relative roll, and relative yaw angle between the two vehicles. The relative distance between the two vehicles is denoted as d. rel Relative elevation is denoted as Relative roll is recorded as The relative deflection angle is denoted as In C f The timing sequence of the output is maintained to be consistent with the subsequent spatial position sequence of the positioning ball. M is used to represent the line-of-sight direction of the gimbal sensor. gConstrain the relative relationship with the positioning ball's installation coordinate system, so that the pitch angle... With deflection angle The determined line-of-sight direction of the gimbal sensor is consistent with the geometric boundary of the observable area of ​​the positioning sphere, ensuring that the line of sight stably falls within the area where the positioning sphere is located when acquiring laser point clouds and images in step S2. This also provides the terrain-aware positioning sphere position calculation network with consistent attitude and coordinates. f The installation offset and installation orientation parameters are unified and connected in series with C. g C b C r Based on the coordinate relationship, establish and output the spatial relationship model M between the two vehicles, so that the relative distance d between the two vehicles is... rel Relative pitch Relative roll Relative deflection angle The subsequent calculation of the spatial relationship between the two vehicles and the gimbal pointing control maintain linear, progressive, and usable numerical input.

[0113] The specific implementation method of S2 is as follows:

[0114] During the data acquisition phase, the output of the two-vehicle spatial relationship model was used as the control basis, and the coordinate system of the front transport vehicle was denoted as C. f Let the coordinate system of the gimbal sensor be denoted as C. g The direction data from the gimbal to the positioning ball will be denoted as d. dir This directional data is in C g In this context, pitch and azimuth are used to represent the attitude of the gimbal, with the pitch angle denoted as... The gimbal attitude deflection angle is denoted as The pointing deviation is denoted as The preset pointing threshold is denoted as d at each moment dir The elevation and azimuth in the figure are respectively denoted as and , with the current and calculate The maximum value of the pitch absolute difference and the azimuth absolute difference is used as the deviation criterion to control the rotation of the gimbal sensor until the condition is met. This ensures that the area where the positioning ball is located enters the center of the gimbal sensor's observation range and maintains a stable pointing direction.

[0115] In laser point cloud acquisition, the space near the center of the observation range is limited to a local area with a radius of three meters, denoted as R. p =3m, and denot N as the number of points selected at each time step. p =2400, center the observation range at C f The position in is denoted as p. cenThis position is determined by the installation geometry of the gimbal sensor and the current line of sight in the spatial relationship model between the two vehicles. The current set of points scanned by the laser is then calculated using p... cen Sort by Euclidean distance and select the top N. p Let P be a point in the laser point cloud at the current moment. t Each point contains three-dimensional coordinates x and y. i y i , z i With intensity s i In this way, P is output at each time step. t And maintain spatial coverage consistent with the center direction of the observation range of the gimbal sensor, so as to provide a stable set of local points of the positioning sphere for the subsequent terrain-aware positioning sphere position calculation network;

[0116] During image acquisition, image frames corresponding to the laser point cloud are triggered synchronously. The image cropping region size is denoted as H=W=400, the number of channels is denoted as C=3, and the pan-tilt sensor imaging intrinsic parameter matrix is ​​denoted as K. g This matrix is ​​obtained by calibration of the gimbal sensor imaging system, and the directional data d dir In C g The vector is down-converted to a unit direction vector, and then multiplied by K using the imaging model. g Then, a normalized coordinate transformation is performed to obtain the image coordinate center, denoted as (u0, v0). Using (u0, v0) as the cropping center, the image cropping region at the current time is generated, denoted as I. t The size is H×W and the channel is C. During the cropping process, the center of the cropping window is kept consistent with the center of the observation range, so that the positioning ball appears continuously in the image cropping area or has inferable contour and background features when it is briefly occluded.

[0117] A uniform sampling period, denoted as T, is set in both attitude acquisition and time organization. s The pitch angle of the current forward transport vehicle is denoted as... The current roll angle of the transport vehicle is denoted as... And simultaneously record the current gimbal attitude and pitch angle at every moment. With deflection angle , with T s For time reference, the laser point cloud P t Image cropping area I t Front transport vehicle posture With gimbal posture Organize the data packets in chronological order, and denote each data packet at any given time as D. t The set of observations in chronological order is denoted as the continuous observation data D of the positioning sphere. seq Using the gimbal sensor trigger signal as the time reference, P is maintained. t with It The acquisition synchronization enables D seq It maintains consistency with the spatial relationship model between the two vehicles in both the time dimension and the spatial center direction, providing a unified input for the terrain-aware positioning sphere position calculation network, including laser point cloud, image cropping region, front transport vehicle attitude and gimbal attitude, and enabling the network to perform temporal modeling and position extrapolation under conditions of viewpoint change and short-term occlusion.

[0118] The specific implementation method of S3 is as follows:

[0119] During the inference phase, the continuous observation data of the positioning sphere is organized by time and input as follows: Figure 9 The terrain-aware positioning sphere location calculation network shown denotes the continuous observation data of the positioning sphere as D. seq The data packet at each time step is denoted as D. t Including laser point cloud P t Image cropping area I t The pitch angle of the front transport vehicle , Front transport vehicle attitude and roll angle Gimbal attitude and pitch angle Gimbal attitude deflection angle And the spatial position p of the set piece at the previous moment t-1 Set a uniform sampling period denoted as T. s D in the network input layer t As a unified input at the current moment and locking the time order, it ensures that all input branches are processed in alignment at the same time, so that subsequent fusion and temporal modeling have stable temporal constraints. For position extrapolation under short-term occlusion, the spatial position of the positioning ball at the previous moment is retained, denoted as p. t-2 ;

[0120] Process the laser point cloud at the branch of the point cloud, and set the laser point cloud P at the current time. t As input, the point set is divided into voxels and extracted using a 3D convolutional layer, followed by global aggregation to obtain 128-dimensional point cloud features, denoted as […]. This feature takes the circular shape, surface texture, and distance distribution of the positioning sphere as input to generate a spatial representation that distinguishes the positioning sphere from cargo corners, vehicle body structural components, and ground protrusions, providing a stable source of information about the shape and spatial distribution of the positioning sphere for subsequent fusion layers;

[0121] The image cropping region is processed in the image branch, and I t As input, a four-layer two-dimensional convolution is used for step-by-step downsampling and global convergence to obtain 128-dimensional image features, denoted as . This feature takes the circular outline of the image, local texture, and brightness distribution as input, and forms a separable visual feature when the positioning ball and environmental objects appear together. It is used to indicate the observable spatial position of the positioning ball at the current moment in the fusion stage together with the point cloud features.

[0122] The attitude of the forward transport vehicle and the attitude of the gimbal are processed in the attitude branch. , , , As input, it is mapped to a 64-dimensional pose feature through two fully connected layers, denoted as... The threshold for judging changes in viewing angle is set as follows: This threshold is used to identify the range of view changes triggered by the pitch and roll of the front transport vehicle and the pitch and yaw of the gimbal in the attitude fusion part, and to separate the view changes from the actual displacement of the positioning ball, so as to avoid the spatial position jump of the positioning ball caused by sudden changes in view when crossing potholes and slopes.

[0123] The 128-dimensional point cloud features are integrated into the fusion layer. 128-dimensional image features 64-dimensional posture features Compared to the previous moment's spatial position p t-1 The features are spliced ​​together to form a 319-dimensional fusion feature, denoted as... ,Will This is mapped to a 128-dimensional fused representation through a fully connected layer, denoted as... This fusion representation simultaneously carries spatial geometry, visual texture, pose encoding, and spatial location from the previous time step, providing the temporal recursion layer with temporal inputs usable under occlusion and abrupt viewpoint changes.

[0124] The time recursion layer maintains a 128-dimensional temporal state, denoted as h. t At every moment Compared with the previous time sequence state h t-1 A combined update is performed, using two fully connected layers to output the data and then adding each element to form a new temporal state h. t This enables the network to model continuous observation data of the positioning ball in the time dimension, and sets a threshold for the duration of occlusion, denoted as T. occ Set the valid observation marker at the current time as m. t When the laser point cloud and the image have a valid contour of the positioning sphere or point cloud features at the current moment, let m t =1, otherwise let m t =0, set the consecutive missing frame count as n miss When m t When =0, press T. s Perform counting and match with T occComparison is used to trigger position extrapolation within a short period of occlusion;

[0125] In the attitude fusion section, the viewpoint change and the actual displacement of the positioning ball are separated based on attitude characteristics. and Combine the data and determine whether the change in perspective exceeds [the specified range]. When m t =1 and the change in perspective exceeds At that time, the spatial features are corrected by using the imaging change direction given by the attitude features, preserving the spatial components corresponding to the true displacement of the positioning ball, and suppressing the imaging changes caused by the attitude of the front transport vehicle and the gimbal. When m t =0 and At that time, based on the spatial position p of the positioning ball at the previous moment... t-1 The previous position of the set piece in space, p t-2 Extrapolate the position based on the current attitude change to obtain the extrapolated position. This extrapolation achieves the separation of perspective change from actual displacement on a low-level physical quantity consisting only of the recent displacement trend and the magnitude of perspective change.

[0126] To perform single-step extrapolation with attitude compensation under short-term occlusion conditions, the following formula is used for calculation. :

[0127] ;

[0128] Where: p t-1 p is the three-dimensional vector of the spatial position of the positioning sphere at the previous moment, in meters. t-2 This is the three-dimensional vector of the sphere's spatial position at the previous moment, in meters. This represents the pitch angle increment of the front transport vehicle, in radians. This represents the increment of the roll angle of the front transport vehicle, in radians. This represents the increment of the gimbal's pitch angle, in radians. This represents the increment of the gimbal attitude deflection angle, in radians. The threshold for judging viewpoint change is given in radians, t is the index of the current time, t-1 and t-2 are the previous time and the time before that time, respectively, and the time step is the sampling period T. s , Indicates in and Take the smaller value from the middle, and extrapolate it to p. t-1 -p t-2It represents the relative displacement of the most recent cycle, and suppresses the pseudo displacement component caused by the sudden change in viewpoint by the weight obtained by attitude increment normalization, so that the extrapolation only dominates when the change in viewpoint does not exceed the threshold, ensuring that the spatial position input remains continuous in time under the rapid attitude changes caused by potholes, undulations or slopes.

[0129] The output layer generates three numbers representing the current spatial position of the positioning ball, denoted as p, from three fully connected layers. t , which contains x t y t z t The unit is meters. When m t When =1, directly output p obtained from fusion and time modeling. t When m t When =0 and the short-term occlusion condition is met, the output is obtained from the above formula. As the current spatial position of the positioning sphere, the spatial positions of the positioning sphere at each moment are compiled in chronological order into a spatial position sequence of the positioning sphere, denoted as S. pos and maintain the same as the sampling period T s Consistent timing intervals make S pos In the scenario of two-vehicle collaborative handling, the positioning ball is consistent with the overall movement law of the goods, providing a stable spatial position input for the positioning ball under uneven ground conditions for the spatial relationship model of the two vehicles in subsequent calculations of the relative position and attitude of the two vehicles. Through the linear process of the above-mentioned input organization, branch extraction, attitude fusion, time recursion and output aggregation, the positioning ball position calculation network completes the generation of positioning ball spatial position sequence under the conditions of potholes, undulations or slopes, so that the network can still perform calculations based on real physical displacement and maintain temporal continuity when the viewpoint changes drastically and there are short-term occlusions.

[0130] The specific implementation method of S4 is as follows:

[0131] The spatial position sequence of the positioning ball output by the terrain-aware positioning ball position calculation network is input into the spatial relationship model between the two vehicles. The relative position and attitude of the two vehicles are continuously calculated and constrained in the coordinate system of the front transport vehicle, and directional data for gimbal control is output. The coordinate system of the front transport vehicle is denoted as C. f Let M denote the spatial relationship model between the two vehicles, and S denote the spatial position sequence of the positioning spheres. pos The spatial position of the positioning ball at each moment in the sequence is denoted as p. t The unit is meters. The initial relative position and attitude sequence of the rear transport vehicle is denoted as S. init This includes the relative distance d rel (t) Unit is meters, relative pitch Units are in radians, relative roll. The units are radians and relative deflection angles. The unit is radians, and the change in relative height is denoted as h. rel (t) is in meters, and the predicted relative position and attitude sequence is denoted as S. pred Let the preset attitude change constraint be denoted as A. con The relative height change threshold is denoted as The unit is meters, and the threshold for relative angle change is denoted as... The unit is radians, and the relative position and attitude sequence of the two vehicles is denoted as S. rel The direction data from the gimbal to the positioning ball will be denoted as d. dir (t), and with pitch command angle The units are radians and azimuth command angles. The unit is radians, and the coordinate system of the gimbal sensor is denoted as C. g ;

[0132] In the initial relative position and attitude determination, S pos Enter M in chronological order. Based on the installation offset and orientation parameters of the front transport vehicle body, gimbal sensor, positioning ball, and rear transport vehicle, enter C. f The coordinate transformation is performed according to the preset attitude transformation chain, and the coordinates at each time point p are transformed. t Converted to a rear transport vehicle in C f The geometric relation quantities are transformed and output as d at each time step. rel (t), , , And calculate the corresponding h rel (t), constituting S init This process uses the time-continuous S-axis from the terrain-aware positioning sphere position calculation network. pos As input, ensure that the initial relative position and attitude have a continuous reference on the time axis;

[0133] In the calculation of predicted relative position and attitude, the relative position and attitude of the two vehicles at the previous moment and the current attitude of the forward transport vehicle are used as inputs. , , , Using the current pitch and roll increment mapping of the transport vehicle as the attitude change driver, and under the installation geometry of M, the range of relative position attitude changes achievable at the current moment is derived, forming S. pred This range describes the amplitude boundary of the propagation of the attitude change of the front transport vehicle caused by ground potholes, undulations or slopes in the two-vehicle system, and is used to limit the physical reachability of the attitude at the current moment.

[0134] When establishing preset attitude change constraints, the fixed loading positions of the goods on the front and rear transport vehicles, the length and weight of the goods are used as inputs to generate A. conThe length of the cargo is denoted as L. cargo The unit is meters, and the weight of the goods is denoted as W. cargo The unit is cow, and the fixed loading position is C. f The location of the feature point below is denoted as and The unit is meters, based on L cargo W cargo and , Determined during the system calibration phase and The specific value is set so that the threshold is consistent with the actual loading configuration of the cargo, and is used as an upper bound constraint on the changes in relative height and relative angle.

[0135] In combinatorial calculations, S init With S pred Input constraint module at each time step and A con Correlation, first for h at each time step rel (t) and Compare, and then , , and If the initial relative position and attitude do not exceed the constraints in both height and angle, then S is directly adopted. init The corresponding value is used as the relative position and attitude after combination at the current moment. If any component exceeds the constraint, then S is used. pred The given corresponding components are used as the basis for correction. The excess is suppressed, while the components that do not exceed the limit remain unchanged, thus forming a combined result that satisfies the constraints. To achieve unified limiting and correction of height and angle under a single calculation rule, the constraint correction result at the current moment is defined by the following formula:

[0136] ;

[0137] in: This is the initial four-dimensional vector of relative position and attitude. To predict the four-dimensional vector of relative position and attitude, u corr (t) is the corrected four-dimensional vector of relative position and attitude, and w(t) = min(1, max(0, r(t))) is the combined weight, which has dimensionless values. This is a normalized transfinite measure, dimensionless, where max(a,b) represents the larger of a and b, and min(a,b) represents the smaller of a and b. The formula uses u as an example. init Using r(t) as the baseline, when any component exceeds the threshold, the weight w(t) is calculated from r(t), and the result is then applied to u according to the degree of exceeding the threshold. pred(t) convergence, achieving unified suppression and correction of relative height and relative angle changes within a single step, maintaining the temporal continuity and physical accessibility of the numerical values;

[0138] In the attitude correction and sequence aggregation stage, the corrected relative position and attitude are output in time sequence to form S. rel In C f Maintain drel(t) and , , The continuity ensures that the difference between adjacent time points does not exceed the value determined by A. con The derived variation limit, and in the presence of short-time measurement bias, is expressed as S. pred For priority reference, ensure that the relative position and attitude sequence of the two vehicles is consistent with the achievable attitude of the heavy-duty cargo;

[0139] In directional data calculation, S rel The installation geometry is combined in M, based on S. rel In C f Determine the position vector of the positioning ball relative to the gimbal sensor at the current moment, and map this vector to C. g And project it as pitch and azimuth angles to obtain d dir (t), where the pitch command angle is The azimuth command angle is , will d dir (t) in chronological order with S rel The aligned output is used to keep the positioning ball within the observation area that the terrain-aware positioning ball position calculation network can receive in subsequent gimbal control. Through the linear process of input conversion, prediction range construction, constraint establishment, combined calculation, correction and direction data generation, the spatial relationship model of the two vehicles is adapted to the uneven ground condition, so that the relative position attitude sequence of the two vehicles is consistent with the spatial position sequence of the positioning ball and is connected to the gimbal control closed loop.

[0140] The specific implementation method of S5 is as follows:

[0141] The relative position and attitude sequence of the two vehicles is associated with a fixed loading position. The spatial position, height difference, and attitude difference of the front and rear ends of the cargo are calculated at each time step. Under threshold constraints, a cargo attitude safety index is formed. The coordinate system of the front transport vehicle is denoted as C. f The vertical direction is the z-axis. The spatial relationship model between the two vehicles is denoted as M, and the relative position and attitude sequence of the two vehicles is denoted as S. rel This includes the relative distance d rel (t) Unit is meters, relative pitch Units are in radians, relative roll. The units are radians and relative deflection angles. The unit is radians, and the pitch and roll of the front transport vehicle are denoted as . , The unit is radians, and the fixed loading position is at C. f The calibration points below are respectively denoted as and The unit is meters, and the length and weight of the goods are denoted as L respectively. cargo The units are meters and W. cargo The unit is cattle;

[0142] S rel Combined with M, in C f The loading point position at each time step is obtained, and the time-varying positions of the previous loading point and the subsequent loading point are denoted as follows: and The unit is meters, using relative distance d. rel (t) and relative deflection angle Determine the relative orientation of the two loading points in the horizontal plane, using relative pitch. With relative roll Determine the relative orientation of the two loading points in the vertical and horizontal directions. and The time-varying transformation is performed based on the installation geometry of M, and the output is... , And in C f Let u be the unit vector of the line connecting the two points. cargo (t) is used to indicate the longitudinal orientation of the goods;

[0143] The reference points for the front and rear ends of the cargo are determined using the endpoints of a fixed loading position, and the front end position is denoted as p. front (t), let p be the position of the back end. rear (t), in meters, let l be the distance from the loading point to the front of the cargo. f The unit is meters, and the distance from the rear loading point to the rear of the cargo is denoted as l. r The unit is meters, and it satisfies l f +l r =L cargo At each moment along u cargo (t) Direction pair and Apply a length offset to obtain p front (t) and p rear (t), where the coordinates of the front and rear ends of the cargo on the z-axis are denoted as z and z respectively. front (t) and z rear (t), the unit is meters;

[0144] The height difference and attitude difference are calculated, and the height difference between the two ends of the cargo is denoted as... Press zfront (t) and z rear The coordinate difference of (t) is calculated, and the sign is retained to describe the state of front high and back low or front low and back high. The pitch difference at the end is denoted as The end roll difference is denoted as The unit is radians, based on the relative pitch of the two vehicles. With relative roll In conjunction with the loading length allocation, a rule for allocating end attitude differences is constructed: Let the length ratio coefficient k be... f With k r For dimensionless values ​​between zero and one, according to l f With l r Calculation of proportion k f With k r It is used to distribute relative pitch and relative roll at both ends, and then... , Using the front end as a reference, the difference between the pitch and roll values ​​at both ends is obtained and output. and The allocation rule depends only on l at each time step. f l r , , , , The underlying values ​​are guaranteed to be directly calculable;

[0145] Set a safety threshold and make a judgment; record the preset height difference threshold as... The unit is meters, and the preset attitude difference threshold is denoted as... The unit is radians, based on L. cargo W cargo and , The system calibration is given and The specific values ​​are compared at each moment. and ,Compare and The result of the time determination is recorded as b. t When neither comparison exceeds the threshold, let b t =1, otherwise let b t =0.

[0146] The judgment results at each moment are merged in chronological order to form a cargo attitude safety index sequence, denoted as S. safe In C f Keep S below safe With S rel Same time index, and with p front (t), p rear (t), , , By storing the relative position and attitude sequence of the two vehicles together and associating it with the fixed loading position, quantifying the spatial position and attitude difference between the two ends of the cargo and judging it with a threshold, the cargo attitude safety index is continuously output over time, providing a constraint reference for upper-level scheduling and gimbal control, so that the monitoring and control decisions are consistent with the actual force boundary of the two vehicles in the case of uneven ground conditions.

[0147] The specific implementation method of S6 is as follows:

[0148] The relative position and attitude sequence of the two vehicles is correlated with the cargo attitude safety index to generate collaborative control parameters for the two vehicles and gimbal control commands, which are then output sequentially under constraints. The coordinate system of the front transport vehicle is denoted as C. f The relative position and attitude sequence of the two vehicles is denoted as S. rel The relative distance is denoted as d. rel (t) is in meters, relative elevation is denoted as The unit is radians, and the relative roll is denoted as . The unit is radians, and the relative deflection angle is denoted as . The unit is radians, and the sequence of cargo attitude safety indicators is denoted as S. safe The result of each time step is recorded as b. t When the standard is met, b t =1, b if not meeting the standard t =0, fix the loading position at C f The location of the calibration point below is denoted as and The unit is meters, and the relative distance to the target is denoted as d. tar Defined as and The calibration distance is in meters, and the sampling period is denoted as T. s The cooperative control parameter vector of the two vehicles is denoted as , where v f (t) represents the longitudinal speed command of the front transport vehicle, in meters per second. This is the steering angle command for the front transport vehicle, in radians, v r (t) represents the longitudinal speed command of the rear transport vehicle, in meters per second. This is the steering angle command for the rear transport vehicle, in radians, and the proportional coefficient is denoted as k. d , , , All are dimensionless, and the initial value vector of the coordinated control parameters is denoted as u. init (t), where the preset control threshold is denoted as The unit is meters per second. The unit is radians per second, and the gimbal direction data is denoted as d. dir (t), denoted as the pitch command angle. The unit is radians and the azimuth command angle is denoted as . The unit is expressed in radians, and the current pitch angle of the gimbal is denoted as... The unit is radians, and the deflection angle is denoted as . The unit is radians, and the pointing deviation is denoted as... The dimensionless criterion is to take the maximum value of the absolute difference between pitch and azimuth, and denote the preset pointing threshold as... Dimensionless;

[0149] In the initial value calculation of the two-vehicle cooperative control parameters, S rel With fixed loading position at C f To perform the association, first calculate the distance error, denoted as... The unit is meters. Calculate the angular error, denoted as... , , The unit is radians, and a temporary increment is set, denoted as . , The unit is meters per second. , The unit is radians, and the calculation at each moment is performed according to the following procedure: Through the proportionality coefficient k d Mapped to the same-direction increments of longitudinal velocity, Through proportional coefficient The opposing increments of the steering angle are allocated to adjust both vehicles in the direction that eliminates relative deflection. The attitude compensation amount is denoted as... ,Depend on and through , The linear combination is obtained, and half of the temporary increments in the longitudinal and longitudinal velocities are allocated to form u. init (t), this process only depends on , , , The underlying values ​​of the proportional coefficients ensure that they can be directly calculated at the current moment;

[0150] In the parameter variation constraint, S safe Associated with a preset control threshold, the upper bound of each increment is calculated and denoted as . The unit is meters per second multiplied by seconds. The unit is radians per second multiplied by seconds, when b t When =1, use and As the upper and lower bounds of the amplitude clipping, when b t When =0, the shrinkage coefficient is set as . Dimensionless, with values ​​less than one, reducing the upper and lower bounds to . and Amplitude clipping is defined as: limiting the temporary increment of each component to the corresponding lower and upper bound interval, and then comparing it with the increment of the previous time step u. ctrl (t-1) component-wise addition yields u ctrl (t), ensuring that the control parameters can be progressively adjusted within the range of relative height and relative angle changes, while maintaining time continuity;

[0151] In the generation of PTZ control commands, S rel With d dir (t) are combined, so that the pitch and azimuth of the gimbal target are respectively and With the current gimbal posture , Calculate pointing deviation Using the maximum absolute difference as the criterion, at each time step... and The comparison results set the gimbal rotation angular velocity, and the pitch and azimuth angular velocities, under the premise of not exceeding the mechanical upper limit, are directed towards... , Convergence ensures that the pointing deviation does not exceed and with T s Alignment complete, command refresh;

[0152] In the time output, the constrained collaborative control parameters of the two vehicles and the gimbal control commands are output in time sequence, and the control parameter sequence is denoted as U. ctrl The sequence of gimbal control commands is denoted as U. gimbal In C f Keep U ctrl with U gimbal With S rel S safe Indexed at the same time, and recording u at each time step. ctrl (t), , , By associating the relative position and attitude sequence of the two vehicles with the cargo attitude safety index, pruning the amplitude of the collaborative control parameters of the two vehicles and maintaining them continuously over time, and generating gimbal control commands with directional data, the collaborative driving control of the front and rear transport vehicles under uneven ground conditions is realized, while keeping the positioning ball within the observation area that the terrain-aware positioning ball position calculation network can receive.

[0153] Compared to the closest existing technology, the method of this invention has significant technical advantages in the relative attitude acquisition stage between the two vehicles. Existing solutions mostly detect targets on the following vehicle based on single-frame point clouds or images, or only estimate the pose using independent IMU / GNSS of each vehicle. This makes it difficult to distinguish between viewpoint changes and the actual displacement of the target under conditions such as potholes and slopes, and also lacks processing for short-term occlusion. This invention uses a terrain-aware positioning sphere position calculation network to fuse multiple sources, including laser point clouds, image cropping regions, the attitude of the preceding vehicle and the gimbal, and the position of the positioning sphere at the previous moment. It also explicitly introduces attitude branches and time recursion layers within the network, using attitude features to remove viewpoint change components. When occlusion occurs, it extrapolates based on historical positions and attitude increments, ensuring that the spatial position sequence of the positioning sphere remains continuous in time and conforms to actual motion laws in space. Taking a typical potholed road surface as an example, even when the positioning ball is temporarily obscured by cargo for 3 to 5 frames, the method of this invention can still control the position jump of the positioning ball in two consecutive frames to the order of 0.05m, while the traditional single-frame geometric inverse calculation scheme often results in a position jump of more than 0.2m. At the same time, the instantaneous fluctuations of relative pitch and roll estimation can be compressed to within 0.5°, which significantly reduces the sensitivity of subsequent control links to measurement noise.

[0154] At the level of collaborative control and safety constraints, the method of this invention also demonstrates comprehensive technical advantages that distinguish it from existing technologies. Existing multi-vehicle collaborative handling methods typically only perform simple closed-loop control on the distance or relative angle between the two vehicles, lacking a unified spatial relationship model and cargo attitude safety indicators. This leads to problems such as the physical unreachability of relative attitude estimation and the difficulty in timely control of the height difference between the two ends of the cargo when the terrain is undulating. On the one hand, this invention uses a two-vehicle spatial relationship model to strictly connect the coordinate system of the front vehicle, the gimbal sensor, the positioning ball mounting coordinate system, and the rear vehicle coordinate system. Combined with the positioning ball position output by the terrain perception network, it obtains the relative attitude sequence of the two vehicles after attitude change threshold constraints and predicted attitude fusion correction, avoiding unattainable attitudes caused by measurement anomalies. On the other hand, based on the loading points of the cargo at the front and rear vehicles and the length and weight of the cargo, it calculates the height difference between the two ends of the cargo and the pitch and roll differences at the ends in real time, constructs cargo attitude safety indicators, and applies amplitude and rate of change constraints to the longitudinal speed and steering angle commands of the front and rear vehicles. In practical applications, the height difference between the two ends of the cargo can be steadily controlled within a preset threshold (such as 50mm). Compared with the traditional scheme based solely on spacing control, the number of times the cargo posture crosses the boundary can be significantly reduced, and the force distribution of the cargo during collaborative handling is more balanced. Thus, safe and controllable collaborative handling of oversized and overweight cargo can be achieved under uneven ground conditions.

[0155] This invention addresses the in-plant and on-site handling needs of ultra-long and ultra-heavy components, applicable to industries such as wind turbine blades, tower sections, bridge steel beams, large molds, and heavy equipment parts. In large manufacturing bases, ports, and wind power and infrastructure construction sites, as manually driven vehicles evolve into automated, multi-vehicle collaborative handling systems, the demand for attitude perception and safety control capabilities in complex terrains is increasing. This invention has significant application potential and expansion prospects in this specific scenario.

[0156] This invention's method can be embedded into existing heavy-duty transport vehicles, tractor units, or unmanned transport vehicle platforms as a basic module for multi-vehicle collaborative transport. Utilizing general-purpose gimbal sensors and positioning ball hardware, it achieves upgrades through a terrain perception network and a spatial relationship model between the two vehicles at the software and algorithm layers. This eliminates the need for large-scale vehicle structure replacements and allows for integration with existing vehicle control and scheduling systems. By imposing real-time constraints on the relative attitude of the two vehicles and the safety indicators of the cargo attitude, it can reduce the risk of cargo damage and the probability of safety accidents during transport operations under complex road conditions, thereby improving equipment availability and the effective workload per unit time.

[0157] The core algorithm of this invention is based on standardized sensor inputs and a clear geometric calibration process, making it portable to different types of heavy-duty transport vehicles or modified vehicles. By adapting to their respective control buses and safety systems, it can be used to build its own multi-vehicle collaborative transport solution. For scenarios requiring the transfer of large components within factory areas or construction sites, this invention helps to increase automated collaborative capabilities on the basis of existing vehicles, reducing reliance on highly experienced drivers and external hoisting resources, thereby achieving a more controllable safety and maintenance cost structure in long-term operation.

[0158] Although the present invention has been described in detail above with general descriptions and specific embodiments, modifications or improvements can be made to it, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of the present invention fall within the scope of protection claimed by the present invention.

Claims

1. A method of coordinated handling control of a heavy load carrier, characterized in that The heavy-duty transport vehicle includes a front transport vehicle and a rear transport vehicle that travel in a forward-backward position. The front transport vehicle is equipped with a gimbal sensor, and the rear transport vehicle is equipped with a positioning ball. The cooperative transport control method includes the following steps: S1. Obtain the installation positions and orientations of the front transport vehicle body, gimbal sensor, rear transport vehicle body, and positioning ball. Establish a spatial relationship calculation model from the coordinate system of the front transport vehicle body to the coordinate system of the gimbal sensor, to the coordinate system of the positioning ball installation, and to the coordinate system of the rear transport vehicle body. Obtain the spatial relationship model between the two vehicles and output the relative distance, relative pitch, relative roll, and relative yaw angle between the two vehicles. S2. Based on the spatial relationship model between the two vehicles, control the gimbal sensor to point to the area where the positioning ball is located, collect the laser point cloud and image of the area, obtain the current attitude of the front transport vehicle and the gimbal attitude, and organize the laser point cloud, image, front transport vehicle attitude and gimbal attitude into continuous observation data of the positioning ball in chronological order. S3. Input the continuous observation data of the positioning ball into the terrain-aware positioning ball position calculation network to obtain the spatial position sequence of the positioning ball at continuous time moments; The positioning ball position calculation network includes a branch layer, a fusion layer, a time recursion layer, and an output layer arranged sequentially. The branch layer includes a point cloud branch for processing the laser point cloud, an image branch for processing the image, and a posture branch for processing the attitude of the front transport vehicle and the gimbal. The fusion layer is used to fuse the data processed by each branch and the positioning ball position at the previous moment. The time recursion layer is used to combine the fusion representation result with the previous state output by the time recursion layer at the previous moment to update and output a new temporal state. The output layer is used to generate the spatial position of the positioning ball at the current moment through three fully connected layers and aggregate them into a spatial position sequence of the positioning ball in chronological order. S4. Input the spatial position sequence of the positioning ball output by the terrain-aware positioning ball position calculation network into the spatial relationship model of the two vehicles, complete the continuous calculation and constraint correction of the relative position and attitude of the two vehicles in the coordinate system of the front transport vehicle, and output the direction data for gimbal control. S5. Output the coordinated control parameters of the front and rear transport vehicles and the gimbal control commands based on the direction data.

2. The heavy load carrier coordinated handling control method according to claim 1, characterized in that, Step S1 involves establishing a spatial relationship calculation model from the coordinate system of the front transport vehicle to the coordinate system of the gimbal sensor, to the coordinate system of the positioning ball installation, and to the coordinate system of the rear transport vehicle. Unify the installation positions and orientations of the front transport vehicle body, gimbal sensor, positioning ball, and rear transport vehicle body to the coordinate system of the front transport vehicle body, determine the installation offset and installation orientation parameters of each component, and give the origin and attitude reference direction of the coordinate system of the front transport vehicle body. The pitch and yaw rotation axes of the gimbal sensor are calibrated to establish the mapping between the pitch angle and yaw angle and the line of sight of the gimbal sensor coordinate system. Based on the fixed loading position of the positioning ball on the rear transport vehicle body, establish the spatial relationship between the positioning ball installation coordinate system and the rear transport vehicle body coordinate system, and determine the position and orientation of the positioning ball center relative to the rear transport vehicle body coordinate system; By connecting the aforementioned spatial relationships, a spatial relationship calculation model is formed, from the coordinate system of the front transport vehicle to the coordinate system of the gimbal sensor, to the coordinate system of the positioning ball installation, and to the coordinate system of the rear transport vehicle.

3. The heavy hauler coordinated haul control method of claim 1, wherein, In step S2, controlling the gimbal sensor to point to the area where the positioning ball is located based on the spatial relationship model between the two vehicles includes: Based on the direction data from the gimbal sensor to the positioning ball output by the spatial relationship model of the two vehicles, the gimbal sensor is controlled to rotate so that the area where the positioning ball is located falls into the center of the observation range of the gimbal sensor, and the difference between the direction data and the gimbal attitude does not exceed the preset threshold.

4. The heavy hauler coordinated haul control method of claim 1, wherein, In step S3, the fusion layer is used to fuse the data processed by each branch and the position of the positioning ball at the previous moment to generate spatial features that reflect the shape and distance of the positioning ball from the laser point cloud and the image. Based on the attitude of the front transport vehicle and the attitude of the gimbal, the network distinguishes the perspective changes caused by the movement of the vehicle and the gimbal from the spatial displacement caused by the actual movement of the positioning ball. The time-recursive layer combines the fusion representation result with the previous state output by the time-recursive layer at the previous moment to update the data, so as to model the continuous observation data of the positioning ball in the time dimension. When the positioning ball is obscured by the cargo and the ground height changes, the spatial position of the positioning ball at the current moment is predicted based on the spatial position of the positioning ball at the previous moment, combined with the current attitude of the front transport vehicle and the attitude of the gimbal, so as to obtain the spatial position sequence of the positioning ball at continuous moments.

5. The heavy load carrier coordinated handling control method according to claim 4, characterized in that, In step S3, the point cloud branch includes voxels and 3D convolutional layers, outputting 128-dimensional features; the image branch includes four 2D convolutional layers, outputting 128-dimensional features; the pose branch includes a fully connected layer that connects the 128-dimensional feature input to the 64-dimensional feature output; the fusion layer includes a fully connected layer that concatenates the 128-dimensional features output from the point cloud branch, the 128-dimensional features output from the image branch, the 64-dimensional feature output from the pose branch, and the 3D coordinates of the previous positioning ball position into a 319-dimensional feature input, which is then connected to the 128-dimensional feature output; the time recursion layer includes a fully connected layer that connects the 128-dimensional temporal state output from the previous time recursion layer and the 128-dimensional feature output from the fusion layer to the 128-dimensional output of the new temporal state; and the output layer includes a fully connected layer that connects the 128-dimensional output of the new temporal state from the time recursion layer to the 64-dimensional output and then to the 3-dimensional output of the current positioning ball spatial position.

6. The heavy load carrier coordinated handling control method according to claim 5, characterized in that, In step S3, the distinction between the perspective changes caused by the movement of the vehicle body and the gimbal and the spatial displacement caused by the actual movement of the positioning ball within the network is based on the attitude of the front transport vehicle and the attitude of the gimbal. This includes: setting a perspective change criterion threshold, combining the 64-dimensional feature output of the attitude branch with the 128-dimensional feature output of the fusion layer, determining whether the perspective change amplitude exceeds the set perspective change criterion threshold, and when the laser point cloud and the image have an effective contour or point cloud feature of the positioning ball at the current moment and the angle change amplitude exceeds the set perspective change criterion threshold, the spatial features are corrected by using the imaging change direction given by the attitude features.

7. The heavy hauler coordinated haul control method of claim 1, wherein, In step S4, the output of directional data for gimbal control includes: inputting the spatial position sequence of the positioning ball into the spatial relationship model between the two vehicles; calculating the initial relative position attitude sequence of the transport vehicle relative to the front transport vehicle based on the spatial position of the positioning ball at each time; calculating the predicted relative position attitude sequence based on the relative position attitude of the two vehicles at the previous time and the current attitude of the front transport vehicle; combining the initial relative position attitude sequence and the predicted relative position attitude sequence under preset attitude change constraints; when the attitude change exceeds the preset attitude change constraint at a certain time, suppressing the attitude change at that time based on the predicted relative position attitude sequence to obtain a time-continuous relative position attitude sequence of the two vehicles; and determining the directional data from the gimbal to the positioning ball at each time from the relative position attitude sequence of the two vehicles.

8. The heavy-duty transport vehicle collaborative transport control method according to claim 1, characterized in that, In step S5, the coordinated control parameters for the front and rear transport vehicles, as well as the gimbal control commands, are output based on the direction data, including: Based on the relative position and attitude sequence of the two vehicles and the fixed loading position of the cargo on the front and rear transport vehicles, the height difference and attitude difference between the two ends of the cargo at each moment are calculated to generate a cargo attitude safety index that characterizes whether the cargo meets the transportation safety requirements. Based on the relative position and attitude sequence of the two vehicles, the direction data of the gimbal pointing to the positioning ball, and the cargo attitude safety indicators, the system generates two-vehicle collaborative control parameters for controlling the coordinated movement of the front and rear transport vehicles, as well as gimbal control commands for controlling the gimbal to follow the rotation of the positioning ball. The system then outputs the two-vehicle collaborative control parameters and the gimbal control commands.

9. The heavy-duty transport vehicle collaborative transport control method according to claim 8, characterized in that, In step S5, generating cargo attitude safety indicators that characterize whether the cargo meets transportation safety requirements includes: The relative position and attitude sequence of the two vehicles is associated with the fixed loading position of the goods on the front and rear transport vehicles in the coordinate system of the front transport vehicle. The spatial position of the front and rear of the goods at each moment is calculated, and the endpoint of the fixed loading position is used as the reference point for the front and rear of the goods. The height difference between the two ends of the cargo is determined based on the vertical coordinate difference between the front and rear spatial positions of the cargo at each moment, and the cargo attitude difference is calculated based on the combination of the relative pitch and relative roll of the two vehicles and the fixed loading position, including the end pitch difference and end roll difference. Set preset height difference threshold and preset posture difference threshold, determine threshold parameters based on cargo length and weight and fixed loading position, determine threshold for height difference and posture difference at each time, and generate judgment results for compliance and non-compliance at each time. The judgment results at each moment are merged into the cargo attitude safety index in chronological order, so that the cargo attitude safety index is consistent with the relative position attitude sequence of the two vehicles in chronological order.

10. The heavy-duty transport vehicle collaborative transport control method according to claim 9, characterized in that, In step S5, generating the two-vehicle collaborative control parameters for controlling the coordinated movement of the front and rear transport vehicles, and the gimbal control commands for controlling the gimbal to follow the rotation of the positioning ball, includes: The relative position and attitude sequence of the two vehicles is correlated with the cargo attitude safety index. Based on the relative distance and each relative angle, the initial values ​​of the two-vehicle cooperative control parameters required for the coordinated driving of the front and rear transport vehicles are calculated. Based on the judgment results of the cargo posture safety index and the preset control threshold, the range of change of the collaborative control parameters of the two vehicles is constrained, so that the parameters can be progressively adjusted within the range of relative height and relative angle changes. Based on the relative position and attitude sequence of the two vehicles and the direction data of the gimbal pointing to the positioning ball, a gimbal control command is generated to control the rotation of the gimbal sensor and ensure that the pointing deviation does not exceed the preset pointing threshold. The constrained collaborative control parameters of the two vehicles and the gimbal control commands are output in chronological order to control the coordinated driving of the front and rear transport vehicles and keep the positioning ball within the observation area that the terrain-aware positioning ball position calculation network can receive.

Images