F-PBVS-based harvester unloading cylinder pose automatic adjusting system and method

Abstract

The invention provides an F-PBVS-based harvester unloading cylinder pose automatic adjusting system and method. The system comprises a harvester-grain transport vehicle communication system, a PBVS-based unloading cylinder motion planning system and an F-PBVS-based hydraulic unloading cylinder dynamic control system. The harvester-grain transport vehicle communication system is used for real-time information interaction between the harvester and the grain transport vehicle; the PBVS-based grain unloading cylinder motion planning system accurately senses the environmental information of a grain unloading area, obtains the pose information of the grain unloading cylinder and an optimal grain unloading point set, and finally carries out the matching of an optimal pose function model, and carries out the motion track planning of the grain unloading cylinder. The F-PBVS-based dynamic control system for the hydraulic grain unloading cylinder is combined with fuzzy logic control and a position visual servo technology, and high-precision trajectory tracking of the grain unloading cylinder and accurate alignment of a grain unloading port in a complex environment are realized through cooperative work of a hydraulic drive control module and an error correction module.

Description

Technical Field

[0001] The present invention belongs to the technical field of unmanned agricultural machinery, and in particular relates to a system and method for automatically adjusting the posture of a grain unloading drum of a harvester based on fuzzy control and position visual servoing (Fuzzy Position-Based Visual Servoing, F-PBVS). Background Art

[0002] With the in-depth development of agricultural mechanization, intelligence and unmanned operation have become important directions for the development of combined harvesting operations. However, the traditional harvester relies on manual operation to adjust the position of the grain unloading barrel during grain unloading, which significantly increases the complexity of the operation and affects the efficiency and accuracy of the operation. In a complex operating environment, the accuracy of manual adjustment of the grain unloading barrel is difficult to guarantee, which can easily lead to problems such as low grain unloading efficiency and grain loss. Therefore, realizing the automation and intelligence of the grain unloading barrel position adjustment has become an important topic in the current research on agricultural machinery intelligence and unmanned farms.

[0003] In the coordinated grain unloading operation, the system composed of the harvester and the grain transporter dynamically adjusts the grain unloading equipment by real-time monitoring the operating status of the grain unloading process to ensure that the grain is smoothly and accurately transported to the grain transporter. Among them, the grain unloading barrel posture adjustment is the core link of the grain unloading process, and its accuracy and response speed directly affect the efficiency and uniformity of grain unloading. For complex operating environments, the intelligent grain unloading barrel posture adjustment technology can effectively avoid problems such as grain scattering and uneven accumulation, and significantly improve the grain unloading efficiency and operation quality. In recent years, with the widespread application of visual sensor technology, the accuracy of grain unloading barrel posture adjustment has been further improved. By installing a depth camera on the harvester or grain transporter, the system can collect grain flow information and position change data in real time, and combine advanced image processing algorithms to dynamically optimize and adjust the grain unloading barrel posture. The development of this technology not only significantly improves the automation level of coordinated grain unloading, but also provides important technical support and research paths for the intelligent and efficient operation of agricultural machinery.

[0004] However, the current coordinated grain unloading process has the following shortcomings: the unloading accuracy is insufficient when the harvester and the grain unloading vehicle coordinate unloading, and the grain unloading drum often cannot achieve unloading in the predetermined area; the posture adjustment method of the harvester's grain unloading drum is simple, and the posture of the grain unloading drum cannot be adjusted according to the size and shape of its own granary, and the grain is often unloaded at a fixed point, which makes it difficult to achieve uniform and efficient unloading.

[0005] Therefore, how to enhance the intelligence level of the collaborative grain unloading system, solve the high dependence of the existing collaborative grain unloading operation on the control accuracy and visual recognition ability of the grain unloading system, and improve the adaptability and stability of the grain unloading operation in complex environments are the key issues that need to be urgently solved in the automation of harvester-grain transporter collaborative operations. Summary of the invention

[0006] In view of the deficiencies in the prior art, the present invention provides a system and method for automatic adjustment of the posture of a grain unloading drum of a harvester based on F-PBVS. After the harvester and the grain transport vehicle enter the coupling position, the grain unloading signal is received through the harvester-grain transport vehicle communication system to start the grain unloading process, and the controller controls the hydraulic drive control module to drive the grain unloading drum to extend so that it enters the initial position for unloading. Subsequently, the depth camera on the grain unloading drum collects environmental image data of the grain bin of the grain transport vehicle and transmits it to the image processor of the environmental perception module for processing. The image processor performs feature matching and three-dimensional reconstruction on the image data, and analyzes basic information such as the size and shape of the grain bin. At the same time, the visual servo module combines the real-time posture information of the grain unloading drum and the unloading information provided by the solid flow meter to automatically adjust the posture of the grain unloading drum. The optimal unloading point prediction model based on the fully connected neural network (FCNN) is called based on the grain port unloading speed and other parameters to calculate the optimal unloading point set, and the matching grain unloading drum posture function is generated through the motion planning module to realize the motion trajectory planning of the grain unloading drum. During the unloading process, the hydraulic grain unloading drum dynamic control system based on F-PBVS adjusts the position and posture of the grain unloading drum in real time according to the trajectory planning results. The hydraulic drive control module is responsible for real-time tracking of the motion trajectory of the grain unloading drum. At the same time, the fuzzy control algorithm accurately corrects the angles of each joint of the grain unloading drum based on the dynamic change rules during the unloading process, ensuring the precise positioning of the grain unloading drum and realizing the uniform unloading of grain into the grain transport vehicle, which significantly improves the unloading efficiency and operation quality.

[0007] The present invention achieves the above technical objectives through the following technical means.

[0008] The invention discloses an automatic grain unloading drum posture adjustment system for a harvester based on F-PBVS, comprising a harvester-grain transport vehicle communication system, a grain unloading drum motion planning system based on PBVS, and a hydraulic grain unloading drum dynamic control system based on F-PBVS; the harvester-grain transport vehicle communication system realizes the scheduling and real-time information interaction between the harvester and the grain transport vehicle through communication modules deployed on the harvester and the grain transport vehicle; the grain unloading drum motion planning system based on PBVS accurately perceives the environmental information of the grain unloading area through an environmental perception module, obtains the grain unloading drum posture information and the optimal grain unloading point set through a visual servo module, and matches the optimal posture function model through a motion planning module to plan the motion trajectory of the grain unloading drum; the hydraulic grain unloading drum dynamic control system based on F-PBVS combines fuzzy logic control and position visual servo technology, realizes high-precision trajectory tracking of the grain unloading drum and precise alignment of the grain unloading port in a complex environment through the coordinated work of a hydraulic drive control module and an error correction module; the environmental perception module includes a depth camera and a solid flow meter.

[0009] The method for automatically adjusting the position of a grain unloading drum of a harvester based on F-PBVS using the automatic adjustment system for the position of a grain unloading drum of a harvester based on F-PBVS comprises the following steps:

[0010] Step 1: The harvester and the grain transporter enter the coupling position and enter the grain unloading phase. The harvester receives the grain unloading signal through the harvester-grain transporter communication system, and the hydraulic drive control module dynamically adjusts the hydraulic system to drive the grain unloading barrel to extend and enter the initial position for grain unloading;

[0011] Step 2: The depth camera on the grain unloading drum of the harvester collects the environmental image data of the grain bin of the grain transport vehicle and transmits it to the image processor of the environmental perception module. The image processor performs feature matching and 3D reconstruction, analyzes the basic feature information of the size and shape of the grain bin, and then obtains the real-time posture information of the grain unloading drum by the visual servo module. It also comprehensively analyzes the size and shape of the grain bin of the grain transport vehicle and the unloading speed parameters of the grain unloading port obtained by the solid flow meter, calls the optimal grain unloading point prediction model based on FCNN, calculates the optimal grain unloading point set, and finally matches the optimal grain unloading drum posture function model by the motion planning module to plan the motion trajectory of the grain unloading drum;

[0012] Step 3: Based on the trajectory planning results, the dynamic control system of the hydraulic grain unloading drum based on F-PBVS uses visual servoing combined with fuzzy control algorithm, and uses the hydraulic drive control module to control the hydraulic system to adjust the position and posture of the grain unloading drum, and track the motion trajectory of the entire grain unloading process of the grain unloading drum. At the same time, the angles of each joint of the grain unloading drum are corrected based on fuzzy rules to achieve accurate alignment and uniform grain unloading;

[0013] Step 4: When the grain bin is full, the harvester-grain transport vehicle communication system sends a signal to end grain unloading, and the hydraulic drive control module controls the hydraulic system to retract the grain unloading drum, thus ending the grain unloading process.

[0014] Furthermore, the specific process of step 2 is as follows:

[0015] Step 2.1: The depth camera is used to collect the color image and depth image of the grain truck granary environment, and then the image processor performs scene segmentation and feature extraction on the color image to obtain the feature information of the grain truck granary;

[0016] Step 2.2: According to the image processing results and based on the DH convention, the kinematic model of the grain unloading drum is established:

[0017] According to the structural parameters of the n-DoF hydraulic grain unloading drum, the homogeneous transformation matrix associated with the coordinates of every two adjacent joints is obtained;

[0018] The grain unloading barrel base is regarded as the first joint, and the transformation matrix of each joint is multiplied in sequence to obtain the transformation matrix between the grain unloading barrel base and the grain unloading port;

[0019] The installation position of the depth camera is adjustable and known, so the homogeneous transformation matrix from the grain unloading port to the depth camera is:

[0020]

[0021] in, n-1 T camera Represents the homogeneous transformation matrix from the unloading port to the depth camera, (x camera ,y camera ,z camera ) represents the relative position of the depth camera and the grain unloading port, then the transformation matrix between the grain unloading barrel base and the depth camera is base T camera for:

[0022]

[0023] Before unloading grain, the visual servo module analyzes the characteristic data of the grain truck's grain bin, comprehensively considers parameters such as the unloading speed and the dynamic position of the unloading truck, calls the optimal unloading point prediction model, and calculates the optimal unloading point set for the grain unloading barrel: the ideal position of the unloading port at a certain time point is set as W, L, H are the width, length and height of the granary respectively, v unload is the grain unloading speed, (X0, Y0, Z0) is the initial grain unloading point position, θ0 is the angle between the grain unloading port and the grain unloading barrel, then the grain unloading point spatial distribution function is:

[0024]

[0025] Among them, the function f t () obtained by the learning model embedded in the system;

[0026] Gather according to target unloading point The position of the depth camera is used to obtain the positional relationship between the depth camera and the unloading point, and then the transformation matrix between the depth camera and the optimal unloading point is obtained:

[0027]

[0028] in, camera T target Represents the transformation matrix between the depth camera and the optimal unloading point;

[0029] represents the coordinates of the optimal unloading point in the camera coordinate system;

[0030] Then the transformation matrix between the grain unloading barrel base and the grain unloading point is base T target for:

[0031] base T target = base T camera · camera T target

[0032] When the depth camera continuously collects image data at a certain time interval, the kinematic model of the grain unloading barrel-depth camera system is established through the above steps, and the real-time posture information of the grain unloading barrel and the dynamic position relationship between the grain unloading point and the grain unloading barrel base can be obtained;

[0033] Step 2.3: The motion planning module uses the optimal unloading point set calculated by the visual servo module and the spatial distribution of the unloading points at each time point in the entire unloading process to solve the unloading barrel posture information at each time point in the entire unloading process by calling the embedded deep learning model, that is, the unloading barrel posture function:

[0034]

[0035] in, is the angle of the two joints of the grain unloading barrel at a certain point in time; g() represents the optimal posture function of the grain unloading barrel.

[0036] Furthermore, the specific process of step 3 is as follows:

[0037] Step 3.1: The hydraulic system control method based on fuzzy control is combined with the position-based visual servo algorithm to dynamically adjust the required angle of each joint to achieve real-time control of the hydraulic system:

[0038] Step 3.1.1: First, the visual servo module obtains the position P of the unloading point in the world coordinate system through PBVS motion planning target =(X target ,Y target ,Z target ), the position of the grain unloading port is (X current ,Y current ,Z current ), therefore, when the unloading port position coincides with the unloading point position, the ideal unloading position is reached, that is, X target =X current , Y target =Y current , Z target =Z current ;

[0039] Step 3.1.2: According to the motion planning results, the dynamic control system of the hydraulic unloading drum based on F-PBVS obtains multiple unloading points, namely the dynamic unloading points The hydraulic drive control module adjusts the rotation angle of each joint Calculate the flow and speed of the hydraulic motors of each joint, and drive the grain unloading drum to move accordingly:

[0040]

[0041] in, The flow rate of the hydraulic motors of the two joints of the grain unloading drum. is the speed of the hydraulic motors at the two joints of the grain unloading drum, k() is the function for solving the flow and speed of the hydraulic motor;

[0042] The actual unloading port position will not coincide with the dynamic unloading point, and there is an error, the error is:

[0043]

[0044] Step 3.2: The fuzzy rule base outputs the adjustment amount of the joint angle according to the size and change rate of the error, that is, according to Adjust the angle of each joint and output the angle increment The hydraulic drive control module then outputs the angle increment To control the hydraulic system:

[0045]

[0046] Wherein, h() represents the error correction function of the hydraulic drive control module; They represent the flow increments of the hydraulic components of the two joints respectively; They represent the rotation speed increments of the hydraulic components of the two joints respectively;

[0047] In each control cycle, the system will adjust the joint angle through the fuzzy controller according to the current error value until the error disappears, ensuring that the grain unloading port is aligned with the grain unloading point.

[0048] The present invention has the following beneficial effects:

[0049] (1) For the first time, a method for automatic adjustment of the position of a harvester unloading drum based on fuzzy control and position visual servoing (F-PBVS) was proposed. This method can effectively cope with the uncertainties and disturbances in complex working environments and ensure that the unloading drum can be adjusted to the specified position quickly and accurately.

[0050] (2) The automatic adjustment method of the grain unloading drum posture based on F-PBVS uses the visual sensor installed at the end of the grain unloading drum to obtain the information of the grain bin of the grain transport vehicle. At the same time, it can comprehensively analyze the parameters such as the size and shape of the grain bin of the grain transport vehicle and the unloading speed of the grain unloading port. Through the machine learning method, it calculates the spatial position of the unloading point in the entire unloading process and performs the motion planning of the grain unloading drum.

[0051] (3) The control method based on F-PBVS combines the advantages of fuzzy control, further enhancing the accuracy and robustness of the system. It has good fault tolerance and nonlinear processing capabilities, can effectively deal with various uncertainties in the operation, and achieve accurate tracking of the motion trajectory of the grain unloading drum, thereby improving the grain unloading accuracy.

[0052] (4) This method reduces manual intervention, thereby improving overall work efficiency. In addition, this method can accurately adjust the movement of the grain unloading drum according to the size of the error, preventing vibration and shaking caused by excessive adjustment, which can further reduce grain losses. At the same time, it can achieve more precise agricultural operations and improve the overall efficiency of grain harvesting, transportation and storage, providing technical support for the intelligent upgrading of agricultural machinery.

[0053] The present invention demonstrates its great potential in improving grain unloading efficiency and reducing human intervention. It can maintain a high level of operating accuracy and automation capabilities in a changing working environment, improve the intelligence level of the collaborative operation system of harvesters and grain transport vehicles, and promote the further development of intelligent agricultural mechanization. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is a flow chart of the automatic adjustment method of the harvester grain unloading drum position based on F-PBVS;

[0055] Figure 2 It is a schematic diagram of the collaborative operation of the harvester and the grain transporter from a planar perspective;

[0056] Figure 3 This is a schematic diagram of the collaborative operation of a harvester and a grain transporter from the front view. DETAILED DESCRIPTION

[0057] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments, but the protection scope of the present invention is not limited thereto.

[0058] The F-PBVS-based automatic adjustment system for the harvester grain unloading drum position of the present invention comprises a harvester-grain transport vehicle communication system, a grain unloading drum motion planning system based on PBVS, and a hydraulic grain unloading drum dynamic control system based on F-PBVS.

[0059] The harvester-grain truck communication system is designed to achieve efficient scheduling and real-time information exchange between harvesters and grain trucks to ensure that the two can work together and complete tasks smoothly during the operation. The system uses communication modules deployed on harvesters and grain trucks and 4G data transmission units (DTUs) to achieve stable long-distance wireless communication. Each communication module is equipped with core components such as data transmission units (DTUs), processors, and antennas for sending, receiving, parsing, and processing data.

[0060] Before the grain unloading operation begins, the harvester uses its communication module to transmit location information, operation status and other data to the grain transport vehicle's communication module in real time, and receives the location information and status data fed back by the grain transport vehicle. When the initial communication is established, the harvester's grain unloading barrel will automatically extend to the initial grain unloading position to prepare for grain unloading. Through 4G network data transmission, the grain transport vehicle's communication module can accurately receive dispatch instructions and real-time location information from the harvester, allowing the grain transport vehicle to perform precise path planning and posture adjustment based on the harvester's position, thereby ensuring accurate docking at the designated grain unloading position and achieving a high degree of automation in grain unloading docking.

[0061] During the grain unloading process, the harvester continuously sends real-time dispatch instructions and dynamic position information to the grain truck to support the grain truck's automatic driving system to continuously adjust its posture, maintain an accurate docking position, reduce human intervention and improve operating efficiency. At the same time, the grain truck's communication module transmits its own position information, driving speed, remaining capacity and other status data back to the harvester through 4GDTU, so that the harvester can optimize the grain unloading process based on the status data of the grain truck, avoiding grain loss and operation interruption caused by posture errors or other factors.

[0062] When the grain unloading process is completed, the harvester uses its communication module to send a grain unloading completion signal to the grain transport vehicle, and the grain unloading cylinder is automatically retracted to complete the grain unloading operation. After receiving the completion signal, the grain transport vehicle stops adjusting and returns to its original state to prepare for the subsequent operation stage. Through the harvester-grain transport vehicle communication system, the harvester and the grain transport vehicle achieve highly coordinated real-time data interaction, ensuring the accuracy and real-time nature of data transmission during the grain unloading operation, significantly improving the efficiency and accuracy of the grain unloading operation, promoting the automation and intelligence of the agricultural operation process, reducing human intervention and improving production efficiency.

[0063] The grain unloading drum motion planning system based on PBVS is designed to achieve high-precision area recognition and precise positioning in automated grain unloading operations. It includes an environmental perception module, a visual servo module, and a motion planning module. The modules are organically combined to achieve efficient and accurate grain unloading operations.

[0064] The environmental perception module completes the perception of the grain unloading process through sensors including depth cameras and solid flow meters. The depth camera is installed on the harvester and can fully scan the grain unloading area and capture the key feature information of the grain bin of the grain transporter, including two-dimensional images and depth data. The environmental perception module uses this data to build a spatial model of the grain unloading area, accurately extract the distance and relative position of the target point, and provide detailed input for the subsequent calculation of the system; the solid flow meter is installed at the grain unloading port to collect the grain unloading speed in real time; through the real-time environmental perception of the two sensors, the environmental perception module ensures the high-precision collection and processing of the grain unloading process data.

[0065] Based on the high-precision data provided by the environmental perception module, the visual servo module uses the position-based visual servo (PBVS) algorithm to identify and locate the grain unloading point. The visual servo module combines image processing technology to extract the characteristic information of the grain unloading area, including the shape, size and relative position of the grain bin of the grain transporter, and uses the coordinate transformation algorithm to convert the position information of the grain unloading point from the camera coordinate system to the world coordinate system, thereby obtaining the posture information of the grain unloading barrel. Based on these data, the visual servo module integrates the parameters such as the grain unloading speed and the dynamic position of the grain unloading vehicle, and calls the optimal grain unloading point prediction model based on the fully connected neural network (FCNN) to calculate the optimal grain unloading point set of the grain unloading barrel.

[0066] The motion planning module matches the optimal posture function model according to the unloading point coordinate set generated by the visual servo module to plan the motion trajectory of the grain unloading drum; the motion trajectory planning module is designed to ensure that the grain unloading drum can complete the unloading operation at the optimal angle and height, while ensuring the smoothness and efficiency of the operation process.

[0067] In general, the grain unloading barrel motion planning system based on PBVS uses the environmental perception module to accurately perceive the environmental information of the grain unloading area, uses the visual servo module to obtain the posture information of the grain unloading barrel and the optimal grain unloading point set, and finally the motion planning module matches the optimal posture function model to plan the motion trajectory of the grain unloading barrel. With the synergistic effect of the three modules, the grain unloading barrel motion planning system based on PBVS can dynamically adjust the grain unloading strategy in a complex environment, ensure the high precision and high stability of the grain unloading operation, and provide reliable guarantee for the application of automated grain unloading technology.

[0068] The dynamic control system of the hydraulic grain unloading drum based on F-PBVS combines fuzzy logic control and position visual servo technology. Through the collaborative work of the hydraulic drive control module and the error correction module, it realizes high-precision trajectory tracking of the grain unloading drum and precise alignment of the grain unloading port in complex environments, providing an efficient solution to nonlinear control problems.

[0069] The hydraulic drive control module is mainly responsible for the precise driving of the grain unloading drum joints. According to the optimal spatial distribution of grain unloading points and the corresponding joint angles of the grain unloading drum completed by the PBVS grain unloading drum motion planning system, the hydraulic drive control module dynamically adjusts the hydraulic system to drive the grain unloading drum to run according to the preset trajectory, and evaluates the movement state of the grain unloading drum in real time to ensure the efficient response capability of the hydraulic system. The hydraulic drive control module comprehensively considers the dynamic nonlinear characteristics of the hydraulic system to achieve smooth and efficient movement of the grain unloading drum.

[0070] The error correction module is based on the fuzzy control and position visual servo (F-PBVS) strategy to compensate and dynamically adjust the errors in the grain unloading process in real time. The error correction module compares the current grain unloading position with the optimal grain unloading point position obtained by the visual servo module, and generates the joint angle adjustment amount based on the error value and its rate of change using the rule base of the fuzzy controller, thereby dynamically adjusting the control parameters of the hydraulic system; by gradually reducing the error value, the dynamic control system of the hydraulic grain unloading barrel can optimize the joint angle of the grain unloading barrel in each control cycle to ensure the alignment accuracy of the grain unloading port. In view of the hysteresis of the hydraulic response and environmental changes, the error correction module enhances the robustness of the dynamic control system of the hydraulic grain unloading barrel based on F-PBVS through a real-time feedback mechanism, and can dynamically respond to complex working conditions such as the position offset of the grain unloading vehicle or the change of the characteristics of the grain unloading area, ensuring that the grain unloading barrel moves smoothly along the optimal trajectory.

[0071] Reference Figures 1 to 3 The method for automatically adjusting the position of a grain unloading drum of a harvester based on F-PBVS using the automatic adjustment system for the position of a grain unloading drum of a harvester based on F-PBVS comprises the following steps:

[0072] Step 1: The harvester and the grain transporter enter the coupling position and enter the grain unloading phase. The harvester receives the grain unloading signal through the harvester-grain transporter communication system. The controller controls the hydraulic drive control module to drive the grain unloading barrel to extend and enter the initial position for grain unloading.

[0073] Step 2: The depth camera on the grain unloading drum of the harvester collects the grain bin environment image data of the grain transport vehicle and transmits it to the image processor of the environmental perception module. The image processor performs feature matching and 3D reconstruction based on the image, analyzes the basic feature information such as the size and shape of the grain bin, and then obtains the real-time posture information of the grain unloading drum by the visual servo module. It also comprehensively analyzes the parameters such as the grain bin size and shape of the grain transport vehicle and the unloading speed of the grain unloading port obtained by the solid flow meter, calls the optimal grain unloading point prediction model based on the fully connected neural network (FCNN), calculates the optimal grain unloading point set, and finally matches the optimal grain unloading drum posture function model by the motion planning module to plan the motion trajectory of the grain unloading drum. The specific process is as follows:

[0074] Step 2.1: The depth camera mounted on the grain unloading drum is used to collect the color image and depth image of the grain bin environment of the grain transporter. The image processor then performs scene segmentation and feature extraction on the color image to obtain the feature information of the grain bin of the grain transporter.

[0075] Step 2.2: According to the image processing results and based on the DH convention, the kinematic model of the grain unloading drum is established:

[0076] The Denavit-Hartenberg Convention (DH Convention) is a standardized method for describing the kinematics of a robot or multi-DOF system. It represents the geometric relationship between the links and joints of a mechanical system by defining a set of regularized parameters. DH parameters include link length, link torsion angle, link offset, and joint variables, which respectively describe the length of the link, the rotation angle between the links, the distance between the links, and the rotation or movement of the joints.

[0077] According to the structural parameters of the n-DoF hydraulic unloading drum, the homogeneous transformation matrix associated with every two adjacent joint coordinates is:

[0078]

[0079] in, i T i+1 represents the transformation matrix between the coordinates of the rigid attachment to the i-th joint and the i+1 joint; θ i+1 is the rotation angle of the i+1 joint; L i+1 , α i+1 d i+1 Respectively represent the length of the i-th connecting rod, the connecting rod torque, and the connecting rod offset. The base of the grain unloading barrel is regarded as the first joint. At the same time, the transformation matrix between the grain unloading barrel base and the grain unloading port can be obtained by multiplying the transformation matrices of each joint in sequence:

[0080]

[0081] in, base T outlet Represents the transformation matrix between the grain unloading barrel base and the grain unloading port, 0 T n-1 It represents the transformation matrix from the base of the grain unloading barrel to the end effector of the grain unloading barrel, i.e., the grain unloading port, and n represents the number of degrees of freedom;

[0082] The installation position of the depth camera is adjustable and known, so the homogeneous transformation matrix from the grain unloading port to the depth camera is:

[0083]

[0084] in, n-1 T camera Represents the homogeneous transformation matrix from the unloading port to the depth camera, (x camera ,y camera ,z camera ) is the installation position coordinate of the depth camera, that is, the installation position under coordinate system 2, which represents the relative position of the depth camera and the grain unloading port. The transformation matrix between the grain unloading barrel base and the depth camera is base T camera for:

[0085]

[0086] Before unloading grain, the visual servo module analyzes the characteristic data of the grain tank of the grain transporter, comprehensively considers the unloading speed, the dynamic position of the grain unloading vehicle and other parameters, calls the optimal unloading point prediction model, and calculates the optimal unloading point set for the grain unloading barrel:

[0087] The ideal position of the unloading port at a certain time point is set as W, L, H are the width, length and height of the granary respectively, v unload is the grain unloading speed, (X0, Y0, Z0) is the initial grain unloading point position, θ0 is the angle between the grain unloading port and the grain unloading barrel, then the grain unloading point spatial distribution function is:

[0088]

[0089] Among them, the function f t () can be obtained by the learning model embedded in the system. The specific method is as follows: based on the particle-volume flow model, the velocity field and density field of the volume flow are analyzed, and the height field and the stacking angle of the stacking area are calculated at the same time. According to the requirements of stacking uniformity, stability and avoiding overflow, the evaluation criteria of ideal stacking are set, and the nonlinear correspondence between the parameters such as grain bin characteristics and grain unloading speed and the grain unloading point is established. The fully connected neural network (Fully Connected Neural Network, FCNN) suitable for processing numerical features and continuous output problems is selected as the learning model, and the optimal grain unloading point prediction model suitable for different operating vehicles and environments is trained. Then, according to the grain bin characteristic parameters of the grain transport vehicle collected by the environmental perception module and the grain unloading speed of the grain unloading barrel, the point with the best stacking effect is calculated as the ideal grain unloading point;

[0090] Gather according to target unloading point The position relationship between the depth camera and the unloading point can be obtained by Then, the transformation matrix between the depth camera and the optimal unloading point can be obtained:

[0091]

[0092] in, camera T target Represents the transformation matrix between the depth camera and the optimal unloading point; ( camera x target , camera y target , camera z target ) represents the coordinates of the optimal unloading point in the camera coordinate system;

[0093] Then the transformation matrix between the grain unloading barrel base and the grain unloading point is base T targetfor:

[0094] base T target = base T camera · camera T target (7)

[0095] When the depth camera continuously collects image data at a certain time interval, the kinematic model of the grain unloading barrel-depth camera system can be established through the above steps to obtain the real-time posture information of the grain unloading barrel and the dynamic position relationship between the grain unloading point and the grain unloading barrel base.

[0096] Step 2.3: After the unloading point is solved, the joints of the unloading barrel need to move to meet the requirements of accurate unloading. The motion planning module uses the optimal unloading point set calculated by the visual servo module and the spatial distribution of the unloading points at each time point in the entire unloading process to solve the unloading barrel posture information at each time point in the entire unloading process by calling the embedded deep learning model, that is, the unloading barrel posture function:

[0097]

[0098] in, is the angle of the two joints of the grain unloading barrel at a certain time point; g() represents the optimal posture function of the grain unloading barrel;

[0099] g() uses deep learning methods to solve the inverse kinematics problem of the grain unloading barrel. The core is to construct a nonlinear mapping relationship between the target unloading point and the joint angle to avoid the complexity caused by redundant degrees of freedom or multiple solutions in traditional methods. First, a large amount of data is collected through simulation models or experiments, including the optimal unloading point and the corresponding joint angle combination. A fully connected neural network (FCNN) is selected as the learning model, and these data are used to construct training sets, validation sets, and test sets. After training, the model can quickly predict the optimal joint angle to meet the real-time control requirements of the grain unloading barrel. In practical applications, the model is further fine-tuned to improve accuracy and robustness by combining experimental feedback data. Compared with traditional solution methods, deep learning methods do not need to explicitly solve complex equations, avoid the problem of multiple solution screening, and take into account both real-time and accuracy, providing reliable support for the efficient operation of the grain unloading barrel.

[0100] Step 3: Based on the trajectory planning results, the dynamic control system of the hydraulic grain unloading drum based on F-PBVS controls the hydraulic system through visual servo combined with fuzzy control algorithm, adjusts the position and posture of the grain unloading drum, tracks the motion trajectory of the entire grain unloading process of the grain unloading drum, and corrects the angles of each joint of the grain unloading drum based on fuzzy rules to achieve accurate alignment and uniform grain unloading; the specific process is as follows:

[0101] Step 3.1: Based on the fuzzy control method of the hydraulic system, and combined with the above-mentioned position-based visual servoing algorithm (PBVS), a control strategy (F-PBVS) combining fuzzy control with position-based visual servoing control algorithm is proposed to dynamically adjust the angles required to change for each joint to achieve real-time control of the hydraulic system. The specific process is as follows:

[0102] Step 3.1.1: First, the visual servo module in the position-based visual servoing (PBVS) grain unloading barrel motion planning system obtains the position P of the grain unloading point in the world coordinate system through PBVS motion planning. target =(X target ,Y target ,Z target ), the position of the grain unloading port is (X current ,Y current ,Z current ), therefore, when the unloading port position coincides with the unloading point position, the ideal unloading position is reached, that is, X target =X current , Y target =Y current , Z target =Z current ;

[0103] Step 3.1.2: Then, according to the motion planning results, the system obtains multiple unloading points, namely dynamic unloading points The hydraulic drive control module adjusts the rotation angle of each joint Calculate the flow and speed of each joint hydraulic motor:

[0104]

[0105] in, The flow rate of the hydraulic motors of the two joints of the grain unloading drum. is the speed of the hydraulic motors at the two joints of the grain unloading drum, k() is the function for solving the flow and speed of the hydraulic motor;

[0106] Due to the hysteresis of the control valve and other reasons, the actual unloading port position will not coincide with the dynamic unloading point, and there is an error:

[0107]

[0108] Step 3.2: Then, the fuzzy rule base outputs the adjustment amount of the joint angle according to the size and change rate of the error, that is, according to Adjust the angle of each joint and output the angle increment The hydraulic drive control module then outputs the angle increment To control the hydraulic system:

[0109]

[0110] Wherein, h() represents the error correction function of the hydraulic drive control module; They represent the flow increments of the hydraulic components of the two joints respectively; They represent the rotation speed increments of the hydraulic components of the two joints respectively;

[0111] In each control cycle, the system will adjust the joint angle through the fuzzy controller according to the current error value until the error disappears, ensuring that the unloading port can be stably aligned with the unloading point.

[0112] Step 4: When the grain bin is full, the harvester-grain transporter communication system sends a signal to end grain unloading, and the hydraulic drive control module controls the grain unloading barrel to retract, ending the grain unloading process.

[0113] Figure 2 , 3 In the equation, L1 represents the length of the first grain unloading barrel, L2 represents the length of the second grain unloading barrel, θ1 represents the rotation angle of joint 1, θ2 represents the rotation angle of joint 2, and x camera 、z camera They respectively represent the installation positions of the camera on the X' and Z' axes in coordinate system 2.

[0114] The embodiments are preferred implementations of the present invention, but the present invention is not limited to the above-mentioned implementations. Any obvious improvements, substitutions or modifications that can be made by those skilled in the art without departing from the essential content of the present invention belong to the protection scope of the present invention.

Claims

1. An automatic adjustment system for the position of a grain unloading drum of a harvester based on F-PBVS, characterized in that: It includes a harvester-grain transport vehicle communication system, a grain unloading drum motion planning system based on PBVS, and a hydraulic grain unloading drum dynamic control system based on F-PBVS; the harvester-grain transport vehicle communication system realizes the scheduling and real-time information interaction between the harvester and the grain transport vehicle through the communication modules deployed on the harvester and the grain transport vehicle; the grain unloading drum motion planning system based on PBVS accurately perceives the environmental information of the grain unloading area through the environmental perception module, obtains the grain unloading drum posture information and the optimal grain unloading point set using the visual servo module, and uses the motion planning module to match the optimal posture function model to plan the motion trajectory of the grain unloading drum; the hydraulic grain unloading drum dynamic control system based on F-PBVS combines fuzzy logic control and position visual servo technology, and realizes high-precision trajectory tracking of the grain unloading drum and precise alignment of the grain unloading port in complex environments through the collaborative work of the hydraulic drive control module and the error correction module; the environmental perception module includes a depth camera and a solid flow meter.

2. The method for automatically adjusting the position of a grain unloading drum of a harvester based on F-PBVS using the automatic adjustment system for the position of a grain unloading drum of a harvester based on F-PBVS according to claim 1 is characterized in that: The process includes the following: Step 1: The harvester and the grain transporter enter the coupling position and enter the grain unloading phase. The harvester receives the grain unloading signal through the harvester-grain transporter communication system, and the hydraulic drive control module dynamically adjusts the hydraulic system to drive the grain unloading barrel to extend and enter the initial position for grain unloading; Step 2: The depth camera on the grain unloading drum of the harvester collects the environmental image data of the grain bin of the grain transport vehicle and transmits it to the image processor of the environmental perception module. The image processor performs feature matching and 3D reconstruction, analyzes the basic feature information of the size and shape of the grain bin, and then obtains the real-time posture information of the grain unloading drum by the visual servo module. It also comprehensively analyzes the size and shape of the grain bin of the grain transport vehicle and the unloading speed parameters of the grain unloading port obtained by the solid flow meter, calls the optimal grain unloading point prediction model based on FCNN, calculates the optimal grain unloading point set, and finally matches the optimal grain unloading drum posture function model by the motion planning module to plan the motion trajectory of the grain unloading drum; Step 3: Based on the trajectory planning results, the dynamic control system of the hydraulic grain unloading drum based on F-PBVS uses visual servoing combined with fuzzy control algorithm to control the hydraulic system using the hydraulic drive control module to adjust the position and posture of the grain unloading drum, and tracks the motion trajectory of the entire grain unloading process of the grain unloading drum. At the same time, the error correction module corrects the angles of each joint of the grain unloading drum based on fuzzy rules to achieve accurate alignment and uniform grain unloading; Step 4: When the grain bin is full, the harvester-grain transport vehicle communication system sends a signal to end grain unloading, and the hydraulic drive control module controls the hydraulic system to retract the grain unloading drum, thus ending the grain unloading process.

3. The method for automatically adjusting the position of a grain unloading drum of a harvester based on F-PBVS according to claim 2 is characterized in that: The specific process of step 2 is as follows: Step 2.1: The depth camera is used to collect the color image and depth image of the grain truck granary environment, and then the image processor performs scene segmentation and feature extraction on the color image to obtain the feature information of the grain truck granary; Step 2.2: According to the image processing results and based on the DH convention, the kinematic model of the grain unloading drum is established: According to the structural parameters of the n-DoF hydraulic grain unloading drum, the homogeneous transformation matrix associated with the coordinates of every two adjacent joints is obtained; The grain unloading barrel base is regarded as the first joint, and the transformation matrix of each joint is multiplied in sequence to obtain the transformation matrix between the grain unloading barrel base and the grain unloading port; The installation position of the depth camera is adjustable and known, so the homogeneous transformation matrix from the grain unloading port to the depth camera is: in, n-1 T camera Represents the homogeneous transformation matrix from the unloading port to the depth camera, (x camera ,y camera ,z camera ) represents the relative position of the depth camera and the grain unloading port, then the transformation matrix between the grain unloading barrel base and the depth camera is base T camera for: Before unloading grain, the visual servo module analyzes the characteristic data of the grain truck's grain bin, comprehensively considers parameters such as the unloading speed and the dynamic position of the unloading truck, calls the optimal unloading point prediction model, and calculates the optimal unloading point set for the grain unloading barrel: the ideal position of the unloading port at a certain time point is set as W, L, H are the width, length and height of the granary respectively, v unload is the grain unloading speed, (X0, Y0, Z0) is the initial grain unloading point position, θ0 is the angle between the grain unloading port and the grain unloading barrel, then the grain unloading point spatial distribution function is: Among them, the function f t () obtained by the learning model embedded in the system; Gather according to target unloading point The position of the depth camera is used to obtain the positional relationship between the depth camera and the unloading point, and then the transformation matrix between the depth camera and the optimal unloading point is obtained: in, camera T target Represents the transformation matrix between the depth camera and the optimal unloading point; ( camera x target , camera y target , camera z target ) represents the coordinates of the optimal unloading point in the camera coordinate system; Then the transformation matrix between the grain unloading barrel base and the grain unloading point is base T target for: base T target = base T camera · camera T target When the depth camera continuously collects image data at a certain time interval, the kinematic model of the grain unloading barrel-depth camera system is established through the above steps, and the real-time posture information of the grain unloading barrel and the dynamic position relationship between the grain unloading point and the grain unloading barrel base can be obtained; Step 2.3: The motion planning module uses the optimal unloading point set calculated by the visual servo module and the spatial distribution of the unloading points at each time point in the entire unloading process to solve the unloading barrel posture information at each time point in the entire unloading process by calling the embedded deep learning model, that is, the unloading barrel posture function: in, is the angle of the two joints of the grain unloading barrel at a certain point in time; g() represents the optimal posture function of the grain unloading barrel.

4. The method for automatically adjusting the position of a grain unloading drum of a harvester based on F-PBVS according to claim 2 is characterized in that: The specific process of step 3 is as follows: Step 3.1: The hydraulic system control method based on fuzzy control is combined with the position-based visual servo algorithm to dynamically adjust the required angle of each joint to achieve real-time control of the hydraulic system: According to the motion planning results of the grain unloading barrel motion planning system based on PBVS, the hydraulic grain unloading barrel dynamic control system based on F-PBVS obtains multiple grain unloading points, namely dynamic grain unloading points The hydraulic drive control module adjusts the rotation angle of each joint Calculate the flow and speed of the hydraulic motors of each joint, and drive the grain unloading drum to move accordingly: in, The flow rate of the hydraulic motors of the two joints of the grain unloading drum. is the speed of the hydraulic motors at the two joints of the grain unloading drum, k() is the function for solving the flow and speed of the hydraulic motor; Step 3.2: The actual unloading port position will not coincide with the dynamic unloading point, and there is an error. The error is: The fuzzy rule base of the error correction module outputs the adjustment amount of the joint angle according to the size and change rate of the error, that is, according to Adjust the angle of each joint and output the angle increment and The hydraulic drive control module then outputs the angle increment To control the hydraulic system: Wherein, h() represents the error correction function of the hydraulic drive control module; They represent the flow increments of the hydraulic components of the two joints respectively; They represent the rotation speed increments of the hydraulic components of the two joints respectively; In each control cycle, the joint angle is adjusted through the fuzzy controller according to the current error value until the error disappears, ensuring that the grain unloading port is aligned with the grain unloading point.

Images