A F-PBVS-based automatic adjustment system and method for a harvester unloading cylinder pose

Abstract

The application provides a kind of based on F-PBVS's harvester grain unloading cylinder pose automatic adjustment system and method, system includes harvester-grain transport vehicle communication system, based on PBVS's grain unloading cylinder motion planning system, based on F-PBVS's hydraulic grain unloading cylinder dynamic control system;Harvester-grain transport vehicle communication system is used for real-time information interaction between harvester and grain transport vehicle;Based on PBVS's grain unloading cylinder motion planning system accurately senses unloading area environmental information, obtains unloading cylinder pose information and best unloading point set, finally matches best pose function model, carries out the motion trajectory planning of grain unloading cylinder;Based on F-PBVS's hydraulic grain unloading cylinder dynamic control system combines fuzzy logic control and position vision servo technology, through the collaborative work of hydraulic drive control module and error correction module, realizes the high-precision trajectory tracking of grain unloading cylinder and the accurate alignment of unloading port in complex environment.

Description

Technical Field

[0001] This invention belongs to the field of agricultural machinery unmanned technology, and in particular relates to an automatic adjustment system and method for the unloading hopper posture of a harvester based on fuzzy control and position-based visual servoing (F-PBVS). Background Technology

[0002] With the deepening development of agricultural mechanization, intelligent and unmanned operations have become important directions for the development of combined harvesting operations. However, traditional harvesters rely on manual operation to adjust the position of the unloading hopper during grain unloading, significantly increasing the complexity of the operation and affecting its efficiency and accuracy. In complex operating environments, the accuracy of manually adjusting the position of the unloading hopper is difficult to guarantee, easily leading to problems such as low grain unloading efficiency and grain loss. Therefore, realizing the automation and intelligent adjustment of the unloading hopper position has become an important research topic in the current intelligentization of agricultural machinery and unmanned farms.

[0003] In collaborative grain unloading operations, the system consisting of a harvester and a grain transport vehicle dynamically adjusts the unloading equipment by monitoring the operational status during the unloading process in real time, ensuring the smooth and accurate delivery of grain to the transport vehicle. Among these adjustments, the unloading hopper's posture adjustment is a core component of the unloading process, and its accuracy and response speed directly affect the efficiency and uniformity of grain unloading. For complex operating environments, intelligent unloading hopper posture adjustment technology can effectively avoid problems such as grain scattering and uneven accumulation, significantly improving unloading efficiency and operational quality. In recent years, with the widespread application of visual sensor technology, the accuracy of unloading hopper posture adjustment has been further improved. By installing depth cameras on the harvester or grain transport vehicle, the system can collect grain flow information and position change data in real time, and combine this with advanced image processing algorithms to dynamically optimize and adjust the unloading hopper's posture. This technological development has not only significantly improved the automation level of collaborative grain unloading but also provided important technical support and research pathways for the intelligent and efficient operation of agricultural machinery.

[0004] However, the current collaborative unloading process has the following drawbacks: the unloading accuracy is insufficient when the harvester and unloading vehicle are unloading together, and the unloading hopper often cannot unload the grain in the predetermined area; the position adjustment method of the harvester's unloading hopper is simple and cannot adjust the position of the unloading hopper according to the size and shape of its own grain bin, often resulting in fixed-point unloading, which makes it difficult to achieve uniform and efficient unloading.

[0005] Therefore, how to enhance the intelligence level of the collaborative grain unloading system, solve the high dependence on the control precision and visual recognition capability of the unloading system in the existing collaborative grain unloading operation, and improve the adaptability and stability of unloading operations in complex environments are the key issues that urgently need to be addressed in the automation of the collaborative operation of harvesters and grain transport vehicles. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides an automatic adjustment system and method for the unloading hopper posture of a harvester based on F-PBVS. After the harvester and the grain transport vehicle enter the coupling position, the unloading signal is received through the harvester-grain transport vehicle communication system, initiating the unloading process. The controller controls the hydraulic drive control module to extend the unloading hopper, bringing it to the initial unloading position. Subsequently, a depth camera on the unloading hopper acquires environmental image data of the grain silo on the grain transport vehicle and transmits it to the image processor of the environmental perception module for processing. The image processor performs feature matching and 3D reconstruction on the image data, analyzing basic information such as the size and shape of the grain silo. Simultaneously, the visual servo module combines the real-time posture information of the unloading hopper with the unloading signal provided by the solid flow meter. Parameters such as grain unloading speed are used to calculate the optimal set of unloading points by calling the optimal grain unloading point prediction model based on fully connected neural networks (FCNN). The motion planning module generates the matching posture function of the unloading cylinder, thereby realizing the motion trajectory planning of the unloading cylinder. During the unloading process, the hydraulic unloading cylinder dynamic control system based on F-PBVS adjusts the position and posture of the unloading cylinder in real time according to the trajectory planning results. The hydraulic drive control module is responsible for tracking the motion trajectory of the unloading cylinder in real time. At the same time, the fuzzy control algorithm accurately corrects the angle of each joint of the unloading cylinder based on the dynamic change rules during the unloading process, ensuring the precise alignment of the unloading cylinder and achieving uniform unloading of grain into the grain transport vehicle, which significantly improves the unloading efficiency and operation quality.

[0007] The present invention achieves the above-mentioned technical objectives through the following technical means.

[0008] An automatic position adjustment system for a harvester unloading hopper based on F-PBVS includes a harvester-transporter communication system, a PBVS-based unloading hopper motion planning system, and an F-PBVS-based hydraulic unloading hopper dynamic control system. The harvester-transporter communication system, through communication modules deployed on both the harvester and the transporter, enables scheduling and real-time information exchange between them. The PBVS-based unloading hopper motion planning system accurately perceives environmental information in the unloading area through an environmental perception module, obtains the unloading hopper's position information and the optimal unloading point set using a visual servo module, and plans the unloading hopper's motion trajectory by matching the optimal position function model using a motion planning module. The F-PBVS-based hydraulic unloading hopper dynamic control system combines fuzzy logic control and position visual servo technology, achieving high-precision trajectory tracking and accurate alignment of the unloading hopper in complex environments through the coordinated operation of a hydraulic drive control module and an error correction module. The environmental perception module includes a depth camera and a solid flow meter.

[0009] The automatic adjustment method for the unloading hopper position of a harvester based on the F-PBVS-based automatic adjustment system includes the following steps:

[0010] Step 1: The harvester and the grain transport vehicle enter the coupling position and enter the unloading stage. The harvester receives the unloading signal through the harvester-grain transport vehicle communication system. The hydraulic drive control module dynamically adjusts the hydraulic system to drive the unloading cylinder to extend and enter the initial unloading position.

[0011] Step 2: The depth camera on the unloading hopper of the harvester collects image data of the grain silo environment of the grain transport vehicle and transmits it to the image processor of the environment perception module. The image processor performs feature matching and 3D reconstruction, analyzes the basic feature information of the size and shape of the grain silo, and then the visual servo module obtains the real-time pose information of the unloading hopper. It also comprehensively analyzes the size and shape of the grain silo of the grain transport vehicle and the unloading speed parameters of the unloading port obtained by the solid flow meter. It calls the optimal unloading point prediction model based on FCNN to calculate the optimal set of unloading points. Finally, the motion planning module matches the optimal unloading hopper pose function model to plan the motion trajectory of the unloading hopper.

[0012] Step 3: Based on the trajectory planning results, the dynamic control system of the hydraulic unloading drum uses visual servoing combined with fuzzy control algorithm and hydraulic drive control module to control the hydraulic system to adjust the position and attitude of the unloading drum. The system tracks the motion trajectory of the unloading drum throughout the unloading process. At the same time, it corrects the angle of each joint of the unloading drum based on fuzzy rules to achieve precise alignment and uniform unloading.

[0013] Step 4: When the grain bin is full, the harvester-grain transport vehicle communication system sends a signal to end unloading. The hydraulic drive control module controls the hydraulic system to retract the unloading hopper, thus ending the unloading process.

[0014] Furthermore, the specific process of step 2 is as follows:

[0015] Step 2.1: Acquire color and depth images of the grain truck and grain warehouse environment using a depth camera, and then use an image processor to perform scene segmentation and feature extraction on the color images to obtain the feature information of the grain truck and grain warehouse.

[0016] Step 2.2: Based on the image processing results and the DH convention, establish the kinematic model of the unloading hopper:

[0017] The homogeneous transformation matrix associated with the coordinates of every two adjacent joints is obtained based on the structural parameters of the n-DoF hydraulic unloading hopper.

[0018] The base of the unloading hopper is regarded as the first joint, and the transformation matrices of each joint are multiplied in sequence to obtain the transformation matrix between the base of the unloading hopper and the unloading port.

[0019] If the installation position of the depth camera is adjustable and known, then the homogeneous transformation matrix from the grain unloading port to the depth camera is:

[0020]

[0021] in, n-1 T camera Let x represent the homogeneous transformation matrix from the unloading port to the depth camera, (x camera ,y camera ,z camera Let represent the relative position of the depth camera and the unloading port, then the transformation matrix between the unloading hopper base and the depth camera is... base T camera for:

[0022]

[0023] Before unloading, the visual servo module analyzes the grain silo feature data of the grain transport vehicle, and, considering parameters such as unloading speed and the dynamic position of the unloading vehicle, calls the optimal unloading point prediction model to calculate the best set of unloading points for the unloading hopper: setting the ideal position of the unloading port at a certain time point as... W, L, and H represent the width, length, and height of the grain silo, respectively. unload Let (X0, Y0, Z0) be the unloading speed, (X0, Y0, Z0) be the initial unloading point position, and θ0 be the angle between the unloading port and the unloading cylinder. Then, the spatial distribution function of the unloading point is:

[0024]

[0025] Wherein, function f t () is obtained from the learning model embedded in the system;

[0026] Based on the target unloading point set By determining the position of the depth camera, the positional relationship between the depth camera and the unloading point is obtained, and thus the transformation matrix between the depth camera and the optimal unloading point is obtained:

[0027]

[0028] in, camera T target This represents the transformation matrix between the depth camera and the optimal unloading point.

[0029] This represents the coordinates of the optimal unloading point in the camera coordinate system;

[0030] The transformation matrix between the unloading hopper base and the unloading point. base T target for:

[0031] base T target = base T camera · camera T target

[0032] When the depth camera continuously acquires image data at certain time intervals, the kinematic model of the unloading hopper-depth camera system can be established through the above steps, thereby obtaining the real-time pose information of the unloading hopper and the dynamic positional relationship between the unloading point and the unloading hopper base.

[0033] Step 2.3: The motion planning module, based on the optimal unloading point set calculated by the visual servoing module and the spatial distribution of unloading points at various time points throughout the unloading process, uses an embedded deep learning model to solve for the unloading hopper pose information at each time point during the entire unloading process, i.e., the unloading hopper pose function.

[0034]

[0035] in, Let g be the angle between the two joints of the unloading hopper at a certain time point; g() represents the optimal pose function of the unloading hopper.

[0036] Furthermore, the specific process of step 3 is as follows:

[0037] Step 3.1: A hydraulic system control method based on fuzzy control, combined with a position-based visual servoing algorithm, dynamically adjusts the required angle changes of each joint to achieve real-time control of the hydraulic system.

[0038] Step 3.1.1: First, the visual servo module obtains the position P of the unloading point in the world coordinate system through PBVS motion planning. target =(X target ,Y target Z target The location of the unloading port is (X). current ,Y current Z current Therefore, the ideal unloading position is reached when the unloading port position coincides with the unloading point position, i.e., X. target =X current Y target =Y current Z target =Z current ;

[0039] Step 3.1.2: Based on the motion planning results, the F-PBVS-based dynamic control system for the hydraulic unloading hopper obtains multiple unloading points, i.e., dynamic unloading points. The hydraulic drive control module adjusts the rotation angle of each joint. The flow rate and speed of the hydraulic motors at each joint are calculated, and the grain unloading drum is driven accordingly.

[0040]

[0041] in, The flow rate of the hydraulic motors at the two joints of the grain unloading hopper, Let k be the rotational speed of the hydraulic motors at the two joints of the unloading hopper, and k() be the function for solving the flow rate and rotational speed of the hydraulic motors.

[0042] The actual location of the unloading point will not coincide with the dynamic unloading point, resulting in an error. The error is:

[0043]

[0044] Step 3.2: The fuzzy rule base outputs the adjustment amount of the joint angle based on the magnitude and rate of change of the error, i.e., according to... Adjust the angles of each joint and output the angle increment. Subsequently, the hydraulic drive control module outputs according to the angle increment. To control the hydraulic system:

[0045]

[0046] Where h() represents the error correction function of the hydraulic drive control module; These represent the flow rate increments of the two hydraulic components; These represent the increments in rotational speed of the two hydraulic components at the joints;

[0047] In each control cycle, the system adjusts the joint angle through a fuzzy controller based on the current error value until the error disappears, ensuring that the unloading port is aligned with the unloading point.

[0048] The present invention has the following beneficial effects:

[0049] (1) A method for automatic adjustment of the position of the unloading hopper of a harvester based on the combination of fuzzy control and position visual servoing (F-PBVS) is proposed for the first time. This method can effectively cope with the uncertainty and disturbance in the complex working environment and ensure that the unloading hopper can be quickly and accurately adjusted to the designated position.

[0050] (2) The automatic adjustment method of unloading hopper posture based on F-PBVS uses a vision sensor installed at the end of the unloading hopper to obtain information about the grain bin of the grain transport vehicle. At the same time, it can comprehensively analyze parameters such as the size and shape of the grain bin of the grain transport vehicle and the unloading speed of the unloading port. Through machine learning, it calculates the spatial position of the unloading point during the entire unloading process and performs motion planning of the unloading hopper.

[0051] (3) The F-PBVS-based control method combines the advantages of fuzzy control, further enhancing the accuracy and robustness of the system. It has good fault tolerance and nonlinear processing capabilities, and can effectively cope with various uncertainties in operation, realize accurate tracking of the unloading hopper's trajectory, thereby improving unloading accuracy.

[0052] (4) This method reduces manual intervention, thereby improving overall work efficiency. In addition, this method can precisely adjust the movement of the unloading hopper according to the size of the error, preventing vibration and shaking caused by excessive adjustment, which can further reduce grain loss. At the same time, it can achieve more precise agricultural operations, improve the overall efficiency of grain harvesting, transportation and storage, and provide technical support for the intelligent upgrading of agricultural machinery.

[0053] This invention demonstrates its great potential in improving unloading efficiency and reducing human intervention. It can maintain a high level of operational precision and automation in variable working environments, enhance the intelligence level of the collaborative operation system of harvesters and grain transport vehicles, and promote the further development of intelligent agricultural mechanization. Attached Figure Description

[0054] Figure 1 This is a flowchart of an automatic adjustment method for the unloading hopper position of a harvester based on F-PBVS;

[0055] Figure 2 This is a schematic diagram of the collaborative operation between a harvester and a grain transport vehicle from a planar perspective;

[0056] Figure 3 This is a schematic diagram of the harvester and grain transport vehicle working together from a frontal view. Detailed Implementation

[0057] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but the scope of protection of the present invention is not limited thereto.

[0058] The automatic position adjustment system for the unloading hopper of a harvester based on F-PBVS described in this invention includes a harvester-grain transport vehicle communication system, a PBVS-based unloading hopper motion planning system, and an F-PBVS-based hydraulic unloading hopper dynamic control system.

[0059] The harvester-grain transport vehicle communication system aims to achieve efficient scheduling and real-time information exchange between the harvester and the grain transport vehicle, ensuring their coordinated operation and successful task completion. This system utilizes communication modules deployed on both the harvester and the grain transport vehicle, and leverages a 4G data transmission unit (DTU) to achieve stable long-range wireless communication. Each communication module is equipped with core components such as a DTU, processor, and antenna for data transmission, reception, parsing, and processing.

[0060] Before the unloading operation begins, the harvester uses its communication module to transmit location information, operating status, and other data in real time to the grain transport vehicle's communication module, while simultaneously receiving location information and status data from the grain transport vehicle. Upon initial communication establishment, the harvester's unloading hopper automatically extends to the initial unloading position, ready for unloading. Through 4G network data transmission, the grain transport vehicle's communication module can accurately receive dispatch instructions and real-time location information from the harvester, enabling the grain transport vehicle to perform precise path planning and attitude adjustments based on the harvester's location. This ensures accurate docking at the designated unloading position, achieving a high degree of automation in the unloading process.

[0061] During the unloading process, the harvester continuously sends real-time dispatch instructions and dynamic location information to the grain transport vehicle to support the vehicle's automatic driving system in making continuous attitude adjustments, maintaining an accurate parking position, reducing human intervention, and improving operational efficiency. Simultaneously, the grain transport vehicle's communication module transmits its own location information, speed, remaining capacity, and other status data back to the harvester via a 4G DTU. This allows the harvester to optimize the unloading process based on the grain transport vehicle's status data, avoiding grain loss and operational interruptions caused by attitude errors or other factors.

[0062] Once the unloading process is complete, the harvester uses its communication module to send a signal indicating the end of unloading to the grain transport vehicle. Simultaneously, the unloading hopper automatically retracts, completing the unloading operation. Upon receiving the end signal, the grain transport vehicle stops adjusting and returns to its original state, preparing for subsequent operations. Through the harvester-grain transport vehicle communication system, the harvester and the grain transport vehicle achieve highly coordinated real-time data interaction, ensuring the accuracy and real-time nature of data transmission during the unloading process. This significantly improves the efficiency and precision of unloading operations, promotes the automation and intelligence of agricultural operations, reduces human intervention, and increases production efficiency.

[0063] The PBVS-based grain unloading hopper motion planning system aims to achieve high-precision area identification and accurate positioning in automated grain unloading operations. It includes an environmental perception module, a visual servo module, and a motion planning module. These modules are organically combined to achieve efficient and accurate grain unloading operations.

[0064] The environmental perception module uses sensors, including depth cameras and solid flow meters, to perceive the grain unloading process. The depth camera, mounted on the harvester, comprehensively scans the unloading area, capturing key feature information of the grain silos on the transport vehicles, including two-dimensional images and depth data. The environmental perception module uses this data to construct a spatial model of the unloading area, accurately extracting the distance and relative position of target points, providing detailed input for subsequent system calculations. The solid flow meter, installed at the unloading port, collects unloading speed data in real time. Through real-time environmental perception using these two types of sensors, the environmental perception module ensures high-precision acquisition and processing of unloading process data.

[0065] Based on the high-precision data provided by the environmental perception module, the visual servoing module employs a position-based visual servoing (PBVS) algorithm to identify and locate the unloading point. The visual servoing module combines image processing technology to extract feature information of the unloading area, including the shape, size, and relative position of the grain silo on the grain transport vehicle. It then uses a coordinate transformation algorithm to convert the position information of the unloading point from the camera coordinate system to the world coordinate system, thereby obtaining the pose information of the unloading hopper. Based on this data, the visual servoing module integrates parameters such as unloading speed and the dynamic position of the unloading vehicle, and calls an optimal unloading point prediction model based on a fully connected neural network (FCNN) to calculate the optimal set of unloading points for the unloading hopper.

[0066] The motion planning module matches the optimal pose function model with the set of unloading point coordinates generated by the visual servo module to plan the motion trajectory of the unloading hopper. The motion trajectory planning module aims to ensure that the unloading hopper can complete the unloading operation at the optimal angle and height, while ensuring the stability and efficiency of the operation process.

[0067] In summary, the PBVS-based grain unloading hopper motion planning system achieves accurate perception of the unloading area environment through the environmental perception module, obtains the pose information of the unloading hopper and the optimal set of unloading points using the visual servoing module, and finally, the motion planning module matches the optimal pose function model to plan the motion trajectory of the unloading hopper. Through the synergistic effect of these three modules, the PBVS-based grain unloading hopper motion planning system can dynamically adjust the unloading strategy in complex environments, ensuring high precision and stability of the unloading operation and providing a reliable guarantee for the application of automated grain unloading technology.

[0068] The F-PBVS-based dynamic control system for hydraulic unloading hoppers combines fuzzy logic control and position vision servo technology. Through the coordinated operation of the hydraulic drive control module and the error correction module, it achieves high-precision trajectory tracking of the unloading hopper and accurate alignment of the unloading port in complex environments, providing an efficient solution for nonlinear control problems.

[0069] The hydraulic drive control module is primarily responsible for the precise driving of the unloading hopper joints. Based on the optimal spatial distribution of the unloading points and the corresponding joint angles obtained from the PBVS unloading hopper motion planning system, the hydraulic drive control module dynamically adjusts the hydraulic system to drive the unloading hopper along a preset trajectory and evaluates the unloading hopper's motion status in real time to ensure the high-efficiency response capability of the hydraulic system. The hydraulic drive control module comprehensively considers the dynamic nonlinear characteristics of the hydraulic system to achieve smooth and efficient movement of the unloading hopper.

[0070] The error correction module, based on a fuzzy control and position visual servo (F-PBVS) strategy, performs real-time compensation and dynamic adjustment for errors during the unloading process. The module compares the current unloading position with the optimal unloading point position obtained by the visual servo module. Based on the error value and its rate of change, it generates joint angle adjustment amounts using the rule base of the fuzzy controller, thereby dynamically adjusting the control parameters of the hydraulic system. By gradually reducing the error value, the hydraulic unloading hopper dynamic control system can optimize the joint angles of the unloading hopper in each control cycle, ensuring the alignment accuracy of the unloading port. To address the hysteresis of hydraulic response and environmental changes, the error correction module enhances the robustness of the F-PBVS-based hydraulic unloading hopper dynamic control system through a real-time feedback mechanism. This allows it to dynamically respond to complex conditions such as unloading vehicle position deviations or changes in unloading area characteristics, ensuring the unloading hopper moves smoothly along the optimal trajectory.

[0071] Reference Figures 1 to 3 The automatic adjustment method for the unloading hopper position of a harvester based on F-PBVS, utilizing the aforementioned automatic adjustment system for the unloading hopper position of a harvester, includes the following steps:

[0072] Step 1: The harvester and the grain transport vehicle enter the coupling position and enter the unloading stage. The harvester receives the unloading signal through the harvester-grain transport vehicle communication system. The controller controls the hydraulic drive control module to drive the unloading cylinder to extend and enter the initial unloading position.

[0073] Step 2: The depth camera on the unloading hopper of the harvester collects environmental image data of the grain silo of the grain transport vehicle and transmits it to the image processor of the environmental perception module. The image processor performs feature matching and 3D reconstruction based on the images, analyzes the basic feature information such as the size and shape of the grain silo, and then the visual servo module obtains the real-time pose information of the unloading hopper. It also comprehensively analyzes the grain silo size and shape of the grain transport vehicle, as well as parameters such as the unloading speed of the unloading port obtained by the solid flow meter. The optimal unloading point prediction model based on a fully connected neural network (FCNN) is called to calculate the optimal set of unloading points. Finally, the motion planning module matches the optimal unloading hopper pose function model to plan the motion trajectory of the unloading hopper. The specific process is as follows:

[0074] Step 2.1: Using a depth camera mounted on the unloading hopper, acquire color and depth images of the grain storage environment of the grain truck. Then, the image processor performs scene segmentation and feature extraction on the color images to obtain the feature information of the grain storage of the grain truck.

[0075] Step 2.2: Based on the image processing results and the DH convention, establish the kinematic model of the unloading hopper:

[0076] The Denavit-Hartenberg Convention (DH Convention) is a standardized method for describing the kinematics of robotic arms or multi-degree-of-freedom systems. It defines a set of regularized parameters to represent the geometric relationships between links and joints in a mechanical system. DH parameters include link length, link torsion angle, link offset, and joint variables, which describe the length of the link, the rotation angle between links, the distance between links, and the rotation or movement of the joint, respectively.

[0077] Based on the structural parameters of the n-DoF hydraulic unloading hopper, the homogeneous transformation matrix associated with every two adjacent joint coordinates is:

[0078]

[0079] in, i T i+1 θ represents the transformation matrix between the coordinates of the rigidly attached i-th joint and the (i+1)-th joint; i+1 It is the rotation angle of joint i+1; L i+1 α i+1 d i+1 Let represent the length, torque, and offset of the i-th link, respectively. Considering the unloading hopper base as the first joint, and multiplying the transformation matrices of each joint sequentially, we can obtain the transformation matrix between the unloading hopper base and the unloading port:

[0080]

[0081] in, base T outlet This represents the transformation matrix between the unloading hopper base and the unloading port. 0 T n-1 This represents the transformation matrix from the unloading drum base to the unloading drum end actuator, i.e., the unloading port, where n represents the number of degrees of freedom;

[0082] If the installation position of the depth camera is adjustable and known, then the homogeneous transformation matrix from the grain unloading port to the depth camera is:

[0083]

[0084] in, n-1 T camera Let x represent the homogeneous transformation matrix from the unloading port to the depth camera, (x camera ,y camera ,z camera Let be the coordinates of the depth camera's installation position, i.e., its installation position in coordinate system 2, representing the relative position of the depth camera and the unloading port. Then, the transformation matrix between the unloading hopper base and the depth camera is... base T camera for:

[0085]

[0086] Before unloading, the visual servo module analyzes the grain bin feature data of the grain transport vehicle, and, taking into account parameters such as unloading speed and the dynamic position of the unloading vehicle, calls the optimal unloading point prediction model to calculate the set of optimal unloading points for the unloading hopper.

[0087] The ideal location for unloading grain at a certain time point is set as follows: W, L, and H represent the width, length, and height of the grain silo, respectively. unload Let (X0, Y0, Z0) be the unloading speed, (X0, Y0, Z0) be the initial unloading point position, and θ0 be the angle between the unloading port and the unloading cylinder. Then, the spatial distribution function of the unloading point is:

[0088]

[0089] Wherein, function f t () can be obtained by the learning model embedded in the system. The specific method is as follows: Based on the particle-volume flow model, analyze the velocity field and density field of the volume flow, and calculate the height field and accumulation angle of the accumulation area. According to the requirements of accumulation uniformity, stability and avoidance of overflow, set the evaluation criteria for ideal accumulation, establish the nonlinear correspondence between parameters such as grain warehouse characteristics and unloading speed and unloading point, select the fully connected neural network (FCNN) suitable for processing numerical features and continuous output problems as the learning model, train the optimal unloading point prediction model suitable for different operating vehicles and environments, and then calculate the point with the best accumulation effect as the ideal unloading point based on the grain warehouse characteristic parameters of the grain transport vehicle and the unloading speed of the unloading drum collected by the environmental perception module.

[0090] Based on the target unloading point set By determining the position of the depth camera, the positional relationship between the depth camera and the unloading point can be obtained. This allows us to obtain the transformation matrix between the depth camera and the optimal unloading point:

[0091]

[0092] in, camera T target This represents the transformation matrix between the depth camera and the optimal unloading point; camera x target , camera y target , camera z target () represents the coordinates of the optimal unloading point in the camera coordinate system;

[0093] The transformation matrix between the unloading hopper base and the unloading point. base T targetfor:

[0094] base T target = base T camera · camera T target (7)

[0095] When the depth camera continuously acquires image data at certain time intervals, the kinematic model of the unloading hopper-depth camera system can be established through the above steps, thereby obtaining the real-time pose information of the unloading hopper and the dynamic positional relationship between the unloading point and the unloading hopper base.

[0096] Step 2.3: After solving for the unloading points, each joint of the unloading drum needs to move to meet the requirements of precise unloading. The motion planning module, based on the optimal unloading point set calculated by the visual servo module and the spatial distribution of the unloading points at each time point throughout the unloading process, solves for the unloading drum pose information at each time point during the entire unloading process by calling the embedded deep learning model, i.e., the unloading drum pose function.

[0097]

[0098] in, Let g be the angle between the two joints of the unloading hopper at a certain time point; g() represents the optimal pose function of the unloading hopper.

[0099] g() utilizes deep learning to solve the inverse kinematics problem of a grain unloading silo. Its core lies in constructing a nonlinear mapping relationship between the target unloading point and joint angles, avoiding the complexity caused by redundant degrees of freedom or multiple solutions in traditional methods. First, a large amount of data is collected through simulation models or experiments, including the optimal unloading point and its corresponding joint angle combination. A fully connected neural network (FCNN) is selected as the learning model. Training, validation, and test sets are constructed using this data. After training, the model can quickly predict the optimal joint angles, meeting the real-time control requirements of the grain unloading silo. In practical applications, the model is further fine-tuned by combining experimental feedback data to improve accuracy and robustness. Compared to traditional solution methods, deep learning methods do not require explicit solution of complex equations, avoiding the problem of multiple solution selection, while simultaneously ensuring real-time performance and accuracy, providing reliable support for the efficient operation of the grain unloading silo.

[0100] Step 3: The F-PBVS-based dynamic control system for the hydraulic unloading hopper, based on trajectory planning results, uses visual servoing combined with fuzzy control algorithms to control the hydraulic system, adjust the position and attitude of the unloading hopper, track the motion trajectory of the entire unloading process, and simultaneously correct the angles of each joint of the unloading hopper based on fuzzy rules to achieve precise alignment and uniform unloading; the specific process is as follows:

[0101] Step 3.1: Based on the fuzzy control method for hydraulic systems, and combined with the aforementioned position-based visual servoing algorithm (PBVS), a control strategy (F-PBVS) is proposed that combines fuzzy control with the position-based visual servoing algorithm to dynamically adjust the required angle changes of each joint, thereby achieving real-time control of the hydraulic system. The specific process is as follows:

[0102] Step 3.1.1: First, the visual servo module in the Position-Based Visual Servo (PBVS) unloading hopper motion planning system obtains the position P of the unloading point in the world coordinate system through PBVS motion planning. target =(X target ,Y target Z target The location of the unloading port is (X). current ,Y current Z current Therefore, the ideal unloading position is reached when the unloading port position coincides with the unloading point position, i.e., X. target =X current Y target =Y current Z target =Z current ;

[0103] Step 3.1.2: Subsequently, based on the motion planning results, the system obtains multiple unloading points, i.e., dynamic unloading points. The hydraulic drive control module adjusts the rotation angle of each joint. Calculate the flow rate and speed of the hydraulic motors for each joint:

[0104]

[0105] in, The flow rate of the hydraulic motors at the two joints of the grain unloading hopper, Let k be the rotational speed of the hydraulic motors at the two joints of the unloading hopper, and k() be the function for solving the flow rate and rotational speed of the hydraulic motors.

[0106] Due to factors such as the lag of the control valve, the actual position of the unloading port will not coincide with the dynamic unloading point, resulting in an error.

[0107]

[0108] Step 3.2: Next, the fuzzy rule base outputs the adjustment amount of the joint angle based on the magnitude and rate of change of the error, i.e., according to... Adjust the angles of each joint and output the angle increment. Subsequently, the hydraulic drive control module outputs according to the angle increment. To control the hydraulic system:

[0109]

[0110] Where h() represents the error correction function of the hydraulic drive control module; These represent the flow rate increments of the two hydraulic components; These represent the increments in rotational speed of the two hydraulic components at the joints;

[0111] In each control cycle, the system adjusts the joint angle through a fuzzy controller based on the current error value until the error disappears, ensuring that the unloading port can be stably aligned with the unloading point.

[0112] Step 4: When the grain bin is full, the harvester-grain transport vehicle communication system sends a signal to end unloading, and the hydraulic drive control module controls the unloading cylinder to retract, ending the unloading process.

[0113] Figure 2 , 3 In the diagram, L1 represents the length of the first unloading hopper, L2 represents the length of the second unloading hopper, θ1 represents the rotation angle of joint 1, θ2 represents the rotation angle of joint 2, and x... camera z camera These represent the camera's mounting positions on the X' and Z' axes in coordinate system 2, respectively.

[0114] The embodiments described above are preferred embodiments of the present invention, but the present invention is not limited to the above embodiments. Any obvious improvements, substitutions or modifications that can be made by those skilled in the art without departing from the essence of the present invention shall fall within the protection scope of the present invention.

Claims

1. A method for automatic adjustment of the position and posture of the unloading hopper of a harvester based on F-PBVS, characterized in that, The process includes the following: Step 1: The harvester and the grain transport vehicle enter the coupling position and enter the unloading stage. The harvester receives the unloading signal through the harvester-grain transport vehicle communication system. The hydraulic drive control module dynamically adjusts the hydraulic system to drive the unloading cylinder to extend and enter the initial unloading position. Step 2: The depth camera on the unloading hopper of the harvester collects image data of the grain silo environment of the grain transport vehicle and transmits it to the image processor of the environment perception module. The image processor performs feature matching and 3D reconstruction, analyzes the basic feature information of the size and shape of the grain silo, and then the vision servo module obtains the real-time pose information of the unloading hopper. It also comprehensively analyzes the size and shape of the grain silo of the grain transport vehicle and the unloading speed parameters of the unloading port obtained by the solid flow meter. It calls the optimal unloading point prediction model based on FCNN to calculate the optimal set of unloading points. Finally, the motion planning module matches the optimal unloading hopper pose function model to plan the motion trajectory of the unloading hopper. Step 3: The F-PBVS-based hydraulic unloading hopper dynamic control system, based on trajectory planning results, uses visual servoing combined with fuzzy control algorithms. The hydraulic drive control module controls the hydraulic system to adjust the position and attitude of the unloading hopper, tracking its motion trajectory throughout the entire unloading process. Simultaneously, the error correction module corrects the angles of each joint of the unloading hopper based on fuzzy rules, achieving precise alignment and uniform unloading. The specific process is as follows: Step 3.1: A hydraulic system control method based on fuzzy control, combined with a position-based visual servoing algorithm, dynamically adjusts the required angle changes of each joint to achieve real-time control of the hydraulic system. Based on the motion planning results of the PBVS-based unloading hopper motion planning system, the F-PBVS-based hydraulic unloading hopper dynamic control system obtains multiple unloading points, i.e., dynamic unloading points. The hydraulic drive control module adjusts the rotation angle of each joint. The flow rate and speed of the hydraulic motors at each joint are calculated, and the grain unloading drum is driven accordingly. ； in, The flow rate of the hydraulic motors at the two joints of the grain unloading hopper, The rotational speed of the hydraulic motors at the two joints of the grain unloading hopper. A function for calculating the flow rate and speed of a hydraulic motor; Step 3.2: The actual location of the unloading point will not coincide with the dynamic unloading point, resulting in an error. The error is: ; The error correction module's fuzzy rule base outputs the adjustment amount of the joint angle based on the magnitude and rate of change of the error. , , Adjust the angles of each joint and output the angle increment. and Subsequently, the hydraulic drive control module outputs according to the angle increment. To control the hydraulic system: ； in, This represents the error correction function for the hydraulic drive control module. , These represent the flow rate increments of the two hydraulic components; , These represent the increments in rotational speed of the two hydraulic components at the joints; In each control cycle, the joint angle is adjusted by the fuzzy controller according to the current error value until the error disappears, ensuring that the unloading port is aligned with the unloading point. Step 4: When the grain bin is full, the harvester-grain transport vehicle communication system sends a signal to end unloading. The hydraulic drive control module controls the hydraulic system to retract the unloading hopper, thus ending the unloading process.

2. The automatic adjustment method for the position of the unloading hopper of a harvester based on F-PBVS according to claim 1, characterized in that, The specific process of step 2 is as follows: Step 2.1: Acquire color and depth images of the grain truck and grain warehouse environment using a depth camera, and then use an image processor to perform scene segmentation and feature extraction on the color images to obtain the feature information of the grain truck and grain warehouse. Step 2.2: Based on the image processing results and the DH convention, establish the kinematic model of the unloading hopper: The homogeneous transformation matrix associated with the coordinates of every two adjacent joints is obtained based on the structural parameters of the n-DoF hydraulic unloading hopper. The base of the unloading hopper is regarded as the first joint, and the transformation matrices of each joint are multiplied in sequence to obtain the transformation matrix between the base of the unloading hopper and the unloading port. If the installation position of the depth camera is adjustable and known, then the homogeneous transformation matrix from the grain unloading port to the depth camera is: ； in, This represents the homogeneous transformation matrix from the grain unloading port to the depth camera. Let represent the relative position of the depth camera and the unloading port, then the transformation matrix between the unloading hopper base and the depth camera is... for: ； Before unloading, the visual servo module analyzes the grain silo feature data of the grain transport vehicle, integrating parameters including unloading speed and the dynamic position of the unloading vehicle, and calls the optimal unloading point prediction model to calculate the best set of unloading points for the unloading hopper: setting the ideal position of the unloading port at a certain time point as... , These are the width, length, and height of the grain warehouse. To speed up grain unloading, This is the initial unloading point location. Let be the angle between the unloading port and the unloading hopper, then the spatial distribution function of the unloading point is: ； Among them, the function Obtained by the learning model embedded in the system; Based on the target unloading point set By determining the position of the depth camera, the positional relationship between the depth camera and the unloading point is obtained, and thus the transformation matrix between the depth camera and the optimal unloading point is obtained: ； in, This represents the transformation matrix between the depth camera and the optimal unloading point. This represents the coordinates of the optimal unloading point in the camera coordinate system; The transformation matrix between the unloading hopper base and the unloading point. for: ； When the depth camera continuously acquires image data at certain time intervals, the kinematic model of the unloading hopper-depth camera system can be established through the above steps, thereby obtaining the real-time pose information of the unloading hopper and the dynamic positional relationship between the unloading point and the unloading hopper base. Step 2.3: The motion planning module, based on the optimal unloading point set calculated by the visual servoing module and the spatial distribution of unloading points at various time points throughout the unloading process, uses an embedded deep learning model to solve for the unloading hopper pose information at each time point during the entire unloading process, i.e., the unloading hopper pose function. ； in, The angles of the two joints of the grain canister at a specific point in time; This represents the optimal pose function for the grain unloading hopper.

3. An F-PBVS-based automatic adjustment system for the position of a harvester unloading hopper, used to implement the F-PBVS-based automatic adjustment method for the position of the unloading hopper of a harvester as described in claim 1, characterized in that, The system includes a harvester-grain transport vehicle communication system, a PBVS-based unloading hopper motion planning system, and an F-PBVS-based hydraulic unloading hopper dynamic control system. The harvester-grain transport vehicle communication system, through communication modules deployed on both the harvester and the grain transport vehicle, enables scheduling and real-time information exchange between them. The PBVS-based unloading hopper motion planning system accurately perceives environmental information in the unloading area through an environmental perception module, obtains the unloading hopper's pose information and the optimal unloading point set using a visual servo module, and uses a motion planning module to match the optimal pose function model for unloading hopper trajectory planning. The F-PBVS-based hydraulic unloading hopper dynamic control system combines fuzzy logic control and position visual servo technology, achieving high-precision trajectory tracking and accurate alignment of the unloading hopper in complex environments through the coordinated work of a hydraulic drive control module and an error correction module. The environmental perception module includes a depth camera and a solid flow meter.

Images