Automatic throwing and filling control method of silage corn harvester

Abstract

The present application relates to a kind of silage corn harvester automatic throwing filling control method, comprising, S1, according to the expected drop point coordinate of the identified carriage pose acquisition;S2, adjust the height of throwing cylinder to the first expected drop point of discharge port alignment, start loading operation;S3, actual drop point coordinate is obtained, and the distance in the direction of expected drop point and actual drop point u, v is calculated;S4, realize the serial communication of controller, and transmit data to controller;S5, repeat the above steps, sequentially fill the rest of each point.The present application aims at a series of problems, such as low efficiency of manually adjusting throwing cylinder pose when silage corn harvester is operated, and poor throwing precision, and a kind of silage corn harvester automatic throwing filling control method is invented, which is beneficial to continuous and accurate delivery to expected drop point in the process of spraying material, improves filling efficiency, saves manpower and material resources.

Description

TECHNICAL FIELD

[0001] The application relates to an automatic throwing and filling control method of a silage corn harvester, and belongs to the technical field of agricultural machinery automation. BACKGROUND

[0002] Silage has high nutritional value, is easy to digest, is soft and fragrant, has good palatability, has a long storage period, and has low cost, and has great development potential in the future development of the animal husbandry in China. At present, the throwing cylinder of a silage machine can complete basic functions such as throwing and unloading during operation, but the change of the throwing direction of the throwing cylinder must be realized under the control of a person. In this process, the operator needs to adjust the amplitude of the overall throwing cylinder and the discharge plate in real time according to the actual discharge amount and the relative speed of the vehicle body, and controls the angle and height of the throwing cylinder nozzle in a manual mode through a handle. In most cases, the operator needs to turn his head to observe the filling state at the same time, which increases the attention to the throwing cylinder throwing device and reduces the control of the direction and the observation of the terrain factors, and further increases the risk degree and the probability of vehicle body damage. The application relates to an automatic filling control method of a silage corn harvester, which uses a YOLOv5 network to complete the identification and calibration of a material car carriage and the detection of a material falling point position, sets a filling rule based on a pixel coordinate system, and completes automatic throwing work under the given filling rule, solves a series of problems such as low throwing efficiency and poor throwing precision, saves manpower and material resources, improves the loading efficiency, and finally achieves the purpose of accurate throwing.

[0003] The published automatic filling device throwing cylinder control method of a silage corn harvester has achieved good results in the automatic throwing task, and can accurately identify the position of the carriage, but the identification box error is large when the throwing cylinder mechanical arm rotates and the camera position changes. There is no clear identification method for the position of the preset filling point, and the position of the material falling point is only in the estimation stage of the falling point position under the condition that the throwing cylinder does not rotate. SUMMARY

[0004] The application solves the problems of low efficiency, low automation, low accuracy of carriage identification, and low reliability of throwing motion control in the current silage unloading process, and provides an automatic throwing and filling control method of a silage corn harvester to accurately identify the positions of the carriage, the expected falling point in the carriage and the actual falling point, comprehensively considers the stacking rule of the material in the carriage, sets an efficient filling rule to achieve the purpose of accurate throwing.

[0005] In order to overcome the above-mentioned existing technical problems, the application provides an automatic throwing and filling control method of a silage corn harvester, which completes the automatic throwing process of silage according to the relative position of the vehicle body and the throwing cylinder and the given filling rule, and comprises the following steps.

[0006] S1, obtain the expected falling point coordinate:

[0007] The industrial camera located above the discharge plate of the silage machine throwing cylinder obtains the carriage position and material filling condition in real time, the rotation-yolov5 target detection algorithm is used to detect the rotating target under the lens, the calibration and identification of the carriage are realized, and then the real-time coordinate of the expected falling point in the pixel coordinate system in the silage machine harvesting operation is obtained;

[0008] S2, adjust the height of the throwing cylinder, and start the loading operation:

[0009] According to the identified carriage pose and the expected falling point position, the position of the throwing cylinder is adjusted to align the discharge port with the first expected falling point, the silage machine starts harvesting, and the loading operation starts;

[0010] S3, obtain the actual falling point coordinate:

[0011] The industrial camera located above the discharge plate of the silage machine throwing cylinder obtains the material flow image of the material segment from the discharge port of the throwing cylinder to the material falling point in the vehicle; the camera uses a domestic Lian Tu 4G network camera with a focal length of 2.8mm, a 170° wide-angle view, and a ground clearance of 6m.

[0012] The YOLOv5 algorithm is used to identify and calculate the material flow falling point, the expected falling point coordinate and the actual falling point coordinate are defined in the pixel coordinate system parallel to the ground, the pixel coordinate system is defined as the coordinate system with the top vertex of the left corner of the camera imaging plane as the origin, the right is defined as the positive direction of u axis, and the downward is defined as the positive direction of v axis, wherein u and v represent the row and column of the image respectively, and the unit is pixel, the expected falling position p n and the actual falling position p m are calculated in real time in u and v directions d u , d v ;

[0013] S4, realize serial communication in the controller:

[0014] The pyserial library is used to realize the serial communication between python and arduino, and the real-time distance d u , d v is sent to the arduino controller, when the controller detects that the distance d u , d v of the two points in u and v directions is greater than the allowed maximum distance d ex , the throwing cylinder mechanical arm joint is controlled to adjust the discharge and falling position;

[0015] S5, fill the rest of the points in turn:

[0016] Continue to fill the car according to the preset filling rule, obtain the material height at the expected dropping point position in the car carriage in real time, and repeat the steps S1, S2, S3 and S4 to fill the next expected dropping point position when reaching the preset height.

[0017] Further, the step S1 includes the following contents:

[0018] The YOLO network model is a real-time target detection method, and the model can achieve a good balance between speed and accuracy. The YOLO network model uses a single convolutional neural network (CNN) to process images, and can directly calculate the position coordinates of objects in the image and the classification results. The YOLOv5 algorithm is the fifth version of YOLO, and has the characteristics of high detection accuracy and fast detection speed, and meets the demand of real-time monitoring. The YOLOv5 network is used to realize the calibration of the carriages under different angles and the identification of the material flow, so as to realize the positioning of the expected dropping point and the actual dropping point in the pixel coordinate system.

[0019] Further, the silage machine throwing cylinder comprises a discharge plate 1, a discharge plate front section hydraulic oil cylinder 2, a throwing cylinder deflection elbow 3, a throwing cylinder main section lower mouth missing plate 4, a throwing cylinder main section pitch hydraulic oil cylinder 5, a throwing cylinder base 6, a throwing cylinder rotating bottom shaft 7, a throwing cylinder main section inner side plate 8, a throwing cylinder mechanical arm side plate 9, a throwing cylinder outer wall 10, a throwing cylinder front section 11; the throwing cylinder mechanical arm side plate 9 is symmetrically distributed on both sides of the throwing cylinder main section inner side plate 8; the tail of the throwing cylinder main section is the throwing cylinder deflection elbow 3, the material obtains centrifugal force here, collides with the throwing cylinder outer wall 10, and finally slides away along the outer wall; the throwing cylinder rotating bottom shaft 7 is located between the throwing cylinder base 6 and the throwing cylinder main section, and together constitutes a throwing cylinder rotating mechanism, realizes the rotating motion of the throwing cylinder ±95°, and changes the material falling position in the direction of the vehicle during the material spraying process; the throwing cylinder main section pitch hydraulic oil cylinder 5 is installed between the rotating mechanism and the throwing cylinder main section lower mouth missing plate 4, and is used for realizing the lifting of the whole throwing cylinder; before the loading operation, the overall height needs to be adjusted so that the camera reaches the specified identification position and the throwing cylinder reaches the optimal discharge height; the throwing cylinder main section lower mouth missing plate 4 and the throwing cylinder main section inner side plate 8 are detachably connected by screws to solve the problem of material blockage at the deflection elbow of the throwing cylinder main section tail; the material leaves the throwing cylinder at the discharge plate 1 at the front end of the throwing cylinder front section 11; the discharge plate 1 at the front end of the throwing cylinder front section 11 comprises a two-section structure, and is connected to each other in a hinged manner; the discharge plate main section 101 is hingedly connected to the throwing cylinder front section 11, the discharge plate main section 101 is hingedly connected to the front section tongue plate 103, and the front section tongue plate 103 is the final material outflow position; after the whole material flows through the front section tongue plate 103 and is straightened, it is distributed in a rectangular shape under the plan view, the fluid characteristics are strengthened, and the identification accuracy of the material flow and the actual material falling position is improved; one end of the tension spring 102 is fixed on the discharge plate main section 101, and the other end is fixed on the front section tongue plate 103; one end of the discharge plate front section hydraulic oil cylinder 2 is fixed on the throwing cylinder front section 11, and the other end is installed on the discharge plate main section 101; under the action of the tension spring 102, the front section tongue plate 103 and the discharge plate main section 101 are deflected in direction; the discharge plate front section hydraulic oil cylinder 2 extends and retracts to drive the discharge plate main section 101 to swing, and the discharge plate tension spring 102 extends and retracts to control the pitch angle of the discharge plate, so that the throwing cylinder changes the material falling position perpendicular to the direction of the vehicle during the material spraying process, and then controls the final discharge direction; the silage machine throwing cylinder comprises three movable degrees of freedom, which are the whole rotation of the throwing cylinder, the whole height adjustment of the throwing cylinder under the action of the throwing cylinder main section pitch hydraulic oil cylinder 5, and the angle change of the discharge plate 1 under the action of the discharge plate front section hydraulic oil cylinder 3, thereby controlling the material discharge direction.

[0020] Further, the above step S3 comprises the following contents:

[0021] S3.1, obtain the image dataset of the material flow at the top of the discharge plate of the throwing cylinder by network crawling and self-shooting, wherein 60% of the image dataset is obtained from the network and 40% of the image dataset is obtained by self-shooting, obtain 1400 pictures of the material flow in the working environment, and 800 pictures of the material flow in the working environment are obtained along the direction of the vehicle frame, and the pictures are saved in jpg format after preprocessing to enhance the acquisition of feature points; wherein the overhead image dataset of the throwing cylinder during the working of the silo is obtained, the image dataset includes the material flow segment from the discharge port to the dropping position, and the image dataset is used for the training of the YOLOv5 network, and a total of 1400 pictures are obtained and preprocessed;

[0022] S3.2, mark the obtained pictures by using the rolabelimg tool, save the annotation information in the format of the PASCAL VOC (PASCAL Visual Object Classes) dataset, convert the label format to txt format by using, and divide the training set and the test set, wherein the training set accounts for 80%, and the test set accounts for 20%;

[0023] S3.3, introduce the divided dataset into the yolo network for iterative training, set the batch size to 16, and set the total iteration number to 120, evaluate the trained model, and the evaluation index is: average precision mean (mAP), recall rate (Recall), and accuracy (Precision). When the iteration number is close to 40, the average precision mean is close to 0.996, and the recall rate and the accuracy are close to 1.

[0024] S3.4, after completing the iterative training, save the optimal weight model and test it on the test set, detect the material flow under different attention conditions and different shooting angles, output the midpoint coordinates of the rectangular detection frame close to the actual dropping point, and mark the actual dropping point coordinates;

[0025] The pyserial library is a module for realizing serial communication under python, and the command pip install pyserial is used in cmd to install the library. The coordinate information obtained by the YOLOv5 network target detection process and the real-time distance calculation results of the u and v directions are transmitted to the arduino controller through serial communication to realize the on-off of the electromagnetic relay and the control of the joints of the throwing cylinder mechanical arm. The expected dropping point and the actual dropping point detection experiment environment uses Window 10 operating system, adopts Pytorch framework, and uses Geforce GTX 1060 graphics card for operation.

[0026] Further, the filling rule in the above step S5 includes the following contents:

[0027] S5.1 Before discharge, adjust the pitch hydraulic cylinder 5 of the main section of the silage throwing cylinder to make the identification device reach a suitable height, and the discharge port of the throwing cylinder is vertical and aligned with the preset filling point P1; the silage throwing cylinder includes three movable degrees of freedom: the throwing cylinder rotates around the rotating bottom shaft; the pitch extension hydraulic cylinder of the main section of the throwing cylinder realizes the height adjustment of the entire throwing cylinder; the discharge plate of the front section of the throwing cylinder realizes the discharge direction adjustment under the action of the discharge plate front section hydraulic cylinder 3;

[0028] S5.2 and YOLOv5 network detection simultaneously output the actual material landing point p in the pixel coordinate system of the material car compartment. m Coordinates, compare the expected drop point position p at time t. n (u n v n ) and the actual material drop point position p m (u m v m ), calculate p in the u and v directions respectively. n p m Real-time coordinate difference: d u =u n -u m ;d v =v n -v m Calculate the real-time distance and the maximum allowable distance d in the u and v directions, respectively. ex Difference s = d ex -|d u |,z=d ex -|d v |;

[0029] If s < 0, the discharge cylinder rotates; if z < 0, the discharge plate angle changes; d u At time >0, the throwing tube rotates clockwise. u When the distance is less than 0, the throwing tube rotates counterclockwise at an angular velocity ω = 0.052 rad / s. v When the value is greater than 0, the discharge plate swings along the positive direction of the v-axis, d v When the value is less than 0, the discharge plate swings along the negative direction of the v-axis.

[0030] S5.3 Desired material drop point location p n Filling is carried out in a specific order, and the desired material drop point position p is... n Several imaginary points are equidistantly distributed along the center line of the driving direction inside the rectangular carriage in the pixel coordinate system. During filling, the imaginary points at the desired material drop points are filled one at a time, and when the last imaginary point of the interval is reached, the filling continues forward in the driving direction.

[0031] The blanking height and the carriage height at the desired blanking position are measured in real time by a distance measuring sensor above the discharge plate;

[0032] Seven desired blanking point positions P are set n From the front to the rear, they are P1, P2, P3, P4, P5, P6 and P7, and the filling sequence is as follows: P2, P4 and P6 are filled in turn, if the material height of one of them reaches 80% of the overall carriage height, the other one is replaced for filling, P7, P5, P3 and P1 are filled in turn when P6 reaches the specified height, if the material height of one of them reaches 110% of the overall carriage height, the other one is replaced for filling, until the material at P1 reaches the specified height, and the filling is completed.

[0033] In the filling rule, the movement of the throwing cylinder mechanical arm joint is involved in the replacement of the desired blanking point operation, the throwing cylinder base 6 realizes the overall rotational movement of the throwing cylinder in the form of a worm drive and under the drive of a hydraulic motor; the discharge plate 1 realizes the angle change under the action of the front section hydraulic oil cylinder 2 of the discharge plate, the throwing cylinder mechanical arm moves in the replacement of the desired falling point operation and when the distance between the desired falling point and the actual falling point exceeds the threshold, the real-time distance is calculated according to the real-time feedback information of the visual detection, the throwing cylinder mechanical arm rotates to correct the u-direction distance, and the discharge plate swings to correct the v-direction distance.

[0034] In S1, the rotation-yolov5 target detection algorithm comprises:

[0035] Based on the YOLOv5 target detection algorithm, the data loading part, the Head part and the loss function part are modified respectively to obtain the rotation-yolov5 target detection algorithm for realizing the identification of a rotating target, the long side representation method is used to define the rotating frame in the data loading part, the roLabelImg software is used for labeling, the constructor of the Detect class is modified in the Head part, the loss function includes four parts of confidence loss, class classification loss, theta angle classification loss and bbox bounding box regression loss, the weight coefficient of the confidence branch is selected between the IOU / GIOU / CIOU / DIOU of the horizontal frame to accelerate the training speed and the gpu utilization rate; since the added theta is a classification task, a classification loss is added, in the bbox bounding box regression loss, the bounding box loss function in the YOLOv5 source code adopts IOU / GIOU / CIOU / DIOU, which is suitable for calculating the IOU between horizontal rectangular frames, the loss calculation part of the rotating frame is divided into angle loss and horizontal frame loss two parts, and the bounding box regression loss part still adopts the above loss function.

[0036] The beneficial effects of the present application are as follows:

[0037] 1.The application relates to an automatic throwing and filling control method of a silage corn harvester, which is based on a YOLOv5 neural network model structure and detects and identifies a material car carriage and an output material flow, and simultaneously outputs expected and actual material falling point coordinates in a carriage identification frame.

[0038] 2.The application adopts a point identification and throwing and filling mode, can realize orderly throwing of materials based on a YOLOv5 identification mechanism, simplifies a control process, and solves the problems of material waste and poor throwing efficiency in the research of an automatic throwing system of a silage machine.

[0039] 3.The application sets a filling rule in a certain order, solves the problems of uneven material and insufficient material in the automatic throwing process of materials, and provides a prerequisite for the stability of the automatic throwing and filling process. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1 Fig. 1 is a throwing and control flowchart of a throwing cylinder mechanical arm in the application.

[0041] Figure 2 Fig. 2 is a schematic structural diagram of a throwing cylinder mechanical arm of a silage machine used in the application.

[0042] Figure 3 Fig. 3 is a schematic structural diagram of a discharging plate of a throwing cylinder mechanical arm of a silage machine used in the application.

[0043] Figure 4 Fig. 4 is a working environment simulation schematic diagram of an automatic throwing device of a silage machine in the example.

[0044] Fig. 5 is a filling state flowchart according to a preset filling rule in the example, wherein Fig. 5(a) is a schematic diagram of an expected falling point of a carriage, Fig. 5(b) is a schematic diagram of a material state in a carriage in a first filling stage, and Fig. 5(c) is a schematic diagram of a material state in a carriage after filling is completed.

[0045] Figure 6 Fig. 6 is a schematic diagram of a step S3 substep in the application.

[0046] Figure 7 Fig. 7 is a schematic diagram of a data set part sample in the application and a detection result diagram thereof.

[0047] Figure 8 Fig. 8 is a sample detection comparison diagram of a carriage frame in a top view when a silage machine works.

[0048] The figure shows the following: 1 - discharge plate, 2 - front section hydraulic cylinder of discharge plate, 3 - deflection elbow of throwing cylinder, 4 - lower opening missing plate of main section of throwing cylinder, 5 - pitch hydraulic cylinder of main section of throwing cylinder, 6 - base of throwing cylinder, 7 - rotating bottom shaft of throwing cylinder, 8 - inner side plate of main section of throwing cylinder, 9 - side plate of throwing cylinder mechanical arm, 10 - outer wall of throwing cylinder, 11 - front section of throwing cylinder, 101 - main section of discharge plate, 102 - tension spring, 103 - tongue plate of front section. DETAILED DESCRIPTION

[0049] In order to more clearly show the specific technical and advantages of the present application, the specific embodiments of the present application are further described below, but it should be understood that the present application is not limited to the scope of the specific embodiments, and all the inventions made by using the concept of the present application are included.

[0050] As Figure 1 shown is the mechanical arm control flow chart of the throwing cylinder in the present application, before loading starts, the operator needs to manually adjust the spatial position of the throwing cylinder mechanical arm to make the industrial camera moving with the mechanical arm accurately recognize and calibrate the output material flow and the car compartment, and make the discharge port of the throwing cylinder align with the first expected drop point position P1. During the operation, the overall height of the throwing cylinder does not change, and after the target is recognized by the recognition system, the expected drop point position coordinates and the actual drop point position coordinates are calculated, which are based on the camera pixel coordinate system, and the movement of the mechanical arm needs to be calculated in pixel proportion, and the maximum distance d allowed in the u, v direction of the expected drop point position and the actual drop point position is set ex , and the actual distance is calculated, and the movement direction of each joint of the mechanical arm is controlled in turn, and the actual drop position is changed in real time.

[0051] As Figure 2As shown, the silage machine throwing cylinder includes a discharge plate 1, a discharge plate front section hydraulic oil cylinder 2, a throwing cylinder deflection elbow 3, a throwing cylinder main section lower mouth missing plate 4, a throwing cylinder main section pitch hydraulic oil cylinder 5, a throwing cylinder base 6, a throwing cylinder rotating bottom shaft 7, a throwing cylinder main section inner side plate 8, a throwing cylinder mechanical arm side plate 9, a throwing cylinder outer wall 10, a throwing cylinder front section 11; the throwing cylinder mechanical arm side plate 9 is symmetrically distributed on both sides of the throwing cylinder main section inner side plate 8; the tail of the throwing cylinder main section is the throwing cylinder deflection elbow 3, the material obtains centrifugal force here, collides with the throwing cylinder outer wall 10, and finally slides along the outer wall to leave; the throwing cylinder rotating bottom shaft 7 is located between the throwing cylinder base 6 and the throwing cylinder main section, and together constitutes a throwing cylinder rotating mechanism, realizing ±95° rotation movement of the throwing cylinder; the throwing cylinder main section pitch hydraulic oil cylinder 5 is installed between the rotating mechanism and the throwing cylinder main section lower mouth missing plate 4, and is used for realizing lifting of the whole throwing cylinder; the throwing cylinder main section lower mouth missing plate 4 and the throwing cylinder main section inner side plate 8 belong to detachable screw connection, so as to solve the material blocking problem of the material at the deflection elbow of the tail of the throwing cylinder main section; the material leaves the throwing cylinder at the discharge plate 1 at the front end of the throwing cylinder front section 11; the discharge plate front end of the throwing cylinder front section 11 includes two section structures, which are connected in a hinged form, the discharge plate main section 101 is hingedly connected with the throwing cylinder front section 11, the discharge plate main section 101 is hingedly connected with the front section tongue plate 103, the front section tongue plate 103 is the final outflow position of the material, one end of the tension spring 102 is fixed on the discharge plate main section 101, and the other end is fixed on the front section tongue plate 103, one end of the discharge plate front section hydraulic oil cylinder 2 is fixed on the throwing cylinder front section 11, and the other end is installed on the discharge plate main section 101, and the direction deflection of the front section tongue plate 103 and the discharge plate main section 101 is realized under the action of the tension spring 102; the discharge plate front section hydraulic oil cylinder 2 extends and retracts to drive the discharge plate main section 101 to swing, the discharge plate tension spring 102 extends and retracts to control the pitch angle of the discharge plate, and then the final discharge direction is controlled; the silage machine throwing cylinder includes three movable degrees of freedom, which are respectively the whole rotation of the throwing cylinder, the whole height adjustment of the throwing cylinder under the action of the throwing cylinder main section pitch hydraulic oil cylinder 5, and the angle change of the discharge plate 1 under the action of the discharge plate front section hydraulic oil cylinder 3, and then the material outflow direction is controlled.

[0052] As Figure 4 shown is a silage machine automatic throwing device working environment simulation schematic diagram in the example of the application, the material obtains acceleration at the acceleration fan at the bottom of the throwing cylinder when the silage corn harvester is working, enters the car compartment at a certain speed according to a fixed track, the p points evenly distributed on the center line in the direction of the car frame in the figure are preset filling points, the preset filling points are obtained through the center point algorithm after identifying the car compartment, and the filling rules are used to fill P2, P4 and P6 to a height of 80% of the height of the car compartment, as shown in Figure 5(b), and fill P1, P3, P5 and P7 to a height of 110% of the height of the car compartment, as shown in Figure 5(c);

[0053] As shown in Figure 6 The step S3 needs to collect the images of the material flow under different discharge states and different shooting angles under the working conditions, and after a certain image processing process, the material flow segment at the position of the silo throwing cylinder outlet to the car compartment dropping point is retained as the data set for the YOLOv5 network training this time. After training, the optimal weight is used for testing and live detection after connecting the camera. In the present application, the midpoint coordinates of the edge line of the material flow dropping point position in the car compartment are considered as the actual dropping point position, and the coordinate values in the pixel coordinate system are output. The pyserial serial communication module is installed, and the coordinate difference of the calculated expected dropping point and the actual dropping point u, v direction is transmitted to the arduino controller.

[0054] As shown in Figure 7 The silo is working, and the material flow part samples under different visual angles and detection results are shown in the figure. The center coordinates of the recognition box in the figure are the midpoint coordinates of the bottom edge of the recognition box. In the present application, this point is defined as the actual dropping point, and the point coordinates are defined as the actual dropping point coordinates based on the image coordinate system.

[0055] As shown in Figure 8 The simulation silo working sample detection comparison chart under the overhead visual angle is shown. The conventional YOLOv5 algorithm and the rotation-yolov5 target detection algorithm are used for detection comparison, wherein, Figure 8 The first row is the detection result of the improved YOLOv5 target detection algorithm, and the second row is the detection result of the improved rotation-yolov5 target detection algorithm. The detection result shows that the rotation-yolov5 target detection algorithm can greatly improve the detection accuracy of the car frame, and adapt to and meet the detection accuracy of the expected dropping point in the car compartment when the throwing cylinder spraying angle changes.

[0056] The following is a detailed description of the working process of the present application:

[0057] The specific steps are as follows:

[0058] S1, obtaining the expected dropping point coordinates: Before discharging, the operator needs to manually adjust the height of the whole throwing cylinder to make the car compartment and the material flow enter the recognition range of the camera, so as to facilitate the accurate recognition of the car compartment by the recognition device and the accurate output of the expected dropping position coordinates;

[0059] The industrial camera located above the silo throwing cylinder discharge plate obtains the car compartment position and material filling condition in real time, uses the rotation-yolov5 target detection algorithm to detect the rotating target under the lens, realizes the calibration and recognition of the car compartment, and further obtains the real-time coordinates of the expected dropping point in the pixel coordinate system in the silo harvesting operation.

[0060] The S1, the rotation-yolov5 target detection algorithm includes:

[0061] Based on the YOLOv5 target detection algorithm, the data loading part, the Head part and the loss function part are modified respectively, and the rotation-yolov5 target detection algorithm is obtained for realizing the recognition of the rotating target. In the data loading part, the long side representation method is used to define the rotating frame, and the roLabelImg software is used for labeling. In the Head part, the constructor of the Detect class is modified. The loss function includes confidence loss, class classification loss, θ angle classification loss and bbox bounding box regression loss. The weight coefficient of the confidence branch selects the IOU / GIOU / CIOU / DIOU between the horizontal frames, which accelerates the training speed and the gpu utilization rate. Since the added θ is a classification task, a classification loss is added. In the bbox bounding box regression loss, the IOU / GIOU / CIOU / DIOU is used in the bounding box loss function in the YOLOv5 source code, which is suitable for calculating the IOU between the horizontal rectangular frames. The loss calculation part of the rotating frame is divided into angle loss and horizontal frame loss two parts, and the above loss functions are still used in the bounding box regression loss part.

[0062] S2, adjust the height of the throwing cylinder, and start the loading operation: the recognition system calibrates and recognizes the car body, and outputs the expected falling point position coordinates in the car body. After obtaining the expected falling point position, the discharge plate position is adjusted. According to the recognized car body pose and the expected falling point position, the throwing cylinder is adjusted to align with the first expected falling point P1. The silage machine starts harvesting, the throwing cylinder discharges, and the loading operation starts;

[0063] S3, obtaining the actual falling point coordinates: through the industrial camera located above the discharge plate of the silage machine throwing cylinder, the material flow image of the material segment from the throwing cylinder discharge port to the car is obtained in real time;

[0064] The YOLOv5 algorithm is used to identify and calculate the coordinates of the material flow falling point. The expected falling point coordinates and the actual falling point coordinates are defined in a pixel coordinate system parallel to the ground. The pixel coordinate system is defined as a coordinate system with the top vertex of the left corner of the camera imaging plane as the origin. The right direction is defined as the positive direction of the u-axis, and the downward direction is defined as the positive direction of the v-axis. Wherein u and v represent the row and column of the image respectively, and the unit is pixel. The expected falling position p n and the actual falling position p m are calculated in real time in the u and v directions distance d u , d v ;

[0065] S3.1, an image dataset of the top camera view of the discharge plate of the delivery cylinder is made by web crawling and self-shooting, 60% of which is obtained from the network and 40% is self-shot, and 1400 pictures of the material flow in the working environment are obtained; preprocessing is performed to enhance the acquisition of feature points and save them in jpg format;

[0066] S3.2, the obtained pictures are labeled using the rolabelimg tool, the annotation information is saved in the format of the PASCAL VOC dataset, the label format is converted to txt format, and the training set and the test set are divided, wherein the training set accounts for 80% and the test set accounts for 20%;

[0067] S3.3, the divided dataset is introduced into the YOLO network for iterative training, the batch size is set to 16, the total number of iterations is 120, the trained model is evaluated, and the evaluation index is: average precision mean, recall rate, and accuracy; when the number of iterations is close to 40, the average precision mean is close to 0.996, and the recall rate and the accuracy are close to 1;

[0068] S3.4, after completing the iterative training, the optimal weight model is saved and tested on the test set, the material flow is detected under different light conditions and different shooting angles, and the midpoint coordinates of the edge line of the falling position of the rectangular detection frame are output, which are recorded as the actual falling point coordinates.

[0069] The expected falling point and the actual falling point detection experiment environment uses Window 10 operating system, adopts Pytorch framework, and uses Geforce GTX 1060 graphics card for operation.

[0070] S4, serial communication with the controller: realize the serial communication of python and arduino by using pyserial library, and send the real-time distance d u , d v to the arduino controller, when the controller detects that the distance d u , d v in the two directions u and v is greater than the maximum allowed distance d ex , the discharge and falling position of the delivery cylinder mechanical arm joint is adjusted; the pyserial library is a module used to realize serial communication under python, and is installed by using the pip install pyserial command in cmd, the coordinate information obtained by the YOLOv5 network target detection process and the real-time distance calculation results in the u and v directions are transmitted to the arduino controller through serial communication, the on-off of the electromagnetic relay is realized, and the control of each joint of the delivery cylinder mechanical arm is realized.

[0071] S5, sequentially filling the rest of the points: continue to fill the car according to the preset filling rule, real-time acquisition of the material height at the expected dropping point position in the car compartment, and repetition of S1, S2, S3 and S4 steps for filling the next expected dropping point position when reaching the preset height.

[0072] The preset filling rule in the step S5 includes the following steps:

[0073] S5.1 Adjust the main section pitch hydraulic cylinder 5 of the silage throwing cylinder before discharging to make the recognition device reach the appropriate height, and the discharge port of the throwing cylinder is vertical and aligned above the preset filling point P1; the silage throwing cylinder includes three movable degrees of freedom: the whole throwing cylinder rotates around the rotating bottom shaft; the main section pitch hydraulic cylinder realizes the height adjustment of the whole throwing cylinder; the discharge plate of the front section of the throwing cylinder realizes the adjustable discharge direction under the action of the discharge plate front section hydraulic cylinder 3;

[0074] S5.2, the YOLOv5 network detects the material dropping point p m in the pixel coordinate system at the same time n (u n , v n ) and the actual dropping point position p m (u m , v m ), respectively calculate the real-time coordinate difference of p n , p m in u, v direction: d u =u n -u m ; d v =v n -v m ; respectively calculate the real-time distance and the maximum allowed distance d ex difference s=d ex -|d u |, z=d ex -|d v |;

[0075] If s<0, the throwing cylinder rotates, z<0, the discharge plate angle changes, d u >0, the throwing cylinder rotates clockwise, d u <0, the throwing cylinder rotates counterclockwise, the rotation angle speed ω=0.052 rad / s, d v >0, the discharge plate swings along the positive direction of v axis, d v <0, the discharge plate swings along the negative direction of v axis;

[0076] S5.3 fill the expected dropping point position p n in a certain order, the expected dropping point position p nA number of imaginary points are equidistantly distributed along the center line of the vehicle in the pixel coordinate system, and when filling, the imaginary points are filled from back to front, and when the last imaginary point is filled, the filling is performed from front to back.

[0077] Seven expected drop points are set, and the expected drop points are P1, P2, P3, P4, P5, P6 and P7 from the front to the back of the vehicle. n The filling sequence is as follows: P2, P4 and P6 are filled in sequence, and when the material height of one of them reaches 80% of the overall vehicle height, another point is replaced for filling; when the material height of P6 reaches the specified height, P7, P5, P3 and P1 are filled in sequence, and when the material height of one of them reaches 110% of the overall vehicle height, another point is replaced for filling; and the filling is completed when the material height of P1 reaches the specified height.

[0078] The drop height at the expected drop point and the vehicle height are measured by a distance measuring sensor above the discharge plate.

[0079] In the filling rule, the movement of the throwing cylinder mechanical arm joint is involved in the replacement of the expected drop point, the throwing cylinder base 6 is driven by a hydraulic motor to realize the overall rotation of the throwing cylinder in the form of a worm drive; the discharge plate 1 realizes the change of the angle under the action of the front section hydraulic oil cylinder 2 of the discharge plate, the movement of the throwing cylinder mechanical arm is involved in the replacement of the expected drop point and the distance between the expected drop point and the actual drop point exceeds the threshold, the real-time distance is calculated according to the real-time feedback information of the visual detection, the throwing cylinder mechanical arm rotates to correct the u-direction distance, and the discharge plate swings to correct the v-direction distance.

Claims

1. A method for automatic throw-and-fill control of a silage corn harvester, characterized by, Specifically comprising the following steps: S1, obtaining the expected falling point coordinates: Through the industrial camera located above the discharge plate of the silage machine throwing cylinder, the position of the carriage and the filling of the material are obtained in real time, and the rotation-yolov5 target detection algorithm is used to detect the rotating target under the lens, so as to realize the calibration and identification of the carriage, and then obtain the real-time coordinates of the expected falling point in the pixel coordinate system in the silage machine harvesting operation; S2, adjusting the height of the throwing cylinder, and starting the loading operation: Adjust the throw cylinder to align the discharge opening to a first desired drop point based on the identified car pose and desired drop point location The silo begins harvesting and the loading operation begins. S3, obtaining the actual falling point coordinates: Through the industrial camera located above the discharge plate of the silage machine throwing cylinder, the material flow image of the material segment from the discharge port of the throwing cylinder to the material falling point in the carriage is obtained in real time; The YOLOv5 algorithm is used for identifying and calculating the coordinates of the material flow drop point. It is expected that the expected drop point coordinates and the actual drop point coordinates are defined in a pixel coordinate system parallel to the ground. The pixel coordinate system is defined as a coordinate system with the top vertex of the left corner of the camera imaging plane as the origin. The right direction is defined as the positive direction of the u-axis, and the downward direction is defined as the positive direction of the v-axis. Where u and v represent the row and column of the image, respectively, and the unit is pixel. The expected drop position and the actual drop position are calculated respectively , ; S4, realizing serial communication with the controller: This section describes how to implement serial communication between Python and Arduino using the pyserial library to transmit real-time distance data. , The data is sent to the Arduino controller. When the controller detects the distance between two points in the u and v directions... , Greater than the maximum allowed distance At the same time, control the joints of the throwing cylinder robotic arm to adjust the discharge and dropping positions; S5, filling the remaining points in turn: Continue to fill the carriage according to the preset filling rule, and obtain the material height at the expected falling point position in the carriage in real time. When the preset height is reached, repeat steps S1, S2, S3 and S4 to fill the next expected falling point position.

2. The method of claim 1, wherein the method comprises: The preset filling rule in step S5 includes the following steps: S5.1 Adjust the main section pitch hydraulic cylinder (5) of the silage throwing cylinder before discharging to make the identification device reach the appropriate height, and the discharge port of the throwing cylinder is vertical and aligned above the preset filling point P1. The silage throwing cylinder includes three movable degrees of freedom: the throwing cylinder rotates around the rotating bottom shaft; the main section pitch hydraulic cylinder of the throwing cylinder adjusts the height of the whole throwing cylinder; the discharge plate of the front section of the throwing cylinder adjusts the discharge direction under the action of the discharge plate front section hydraulic cylinder (2); S5.2, YOLOv5 network detects the material car carriage and outputs the actual material falling point in the pixel coordinate system Coordinate, compare the expected falling point position at time t , ) with the actual falling point position , , respectively calculate the real-time coordinate difference in the u and v directions ; ; respectively calculate the real-time distance and the maximum allowed distance difference in the u and v directions , ;​​​ If , the throwing cylinder rotates clockwise, , the discharge plate changes angle, , the throwing cylinder rotates clockwise, , the throwing cylinder rotates counterclockwise, the rotation angle speed 0.052 , , the discharge plate swings along the positive direction of the v-axis, , the discharge plate swings along the negative direction of the v-axis; S5.3 the desired drop point positions are filled in a certain order, the desired drop point positions a number of imaginary points in the pixel coordinate system that are equidistantly distributed along the center line in the driving direction within the rectangular vehicle interior, when filling, the imaginary points of the desired drop point positions that are equidistantly distributed along the center line in the driving direction are filled in intervals one behind the other, and when the last imaginary point of an interval is filled, the filling is continued in the driving direction in the order of the next intervals. The falling height at the expected falling position and the carriage height are obtained by real-time measurement with the ranging sensor above the discharge plate.

3. The method of claim 1, wherein the method further comprises: The silage machine's throwing cylinder includes a discharge plate (1), a hydraulic cylinder (2) in front of the discharge plate, a throwing cylinder deflection pipe (3), a notch (4) at the bottom of the main section of the throwing cylinder, a hydraulic cylinder (5) for pitching the main section of the throwing cylinder, a throwing cylinder base (6), a throwing cylinder rotating shaft (7), an inner side plate (8) of the main section of the throwing cylinder, a side plate (9) of the throwing cylinder robotic arm, an outer wall (10) of the throwing cylinder, and a front section (11) of the throwing cylinder. The side plate (9) of the throwing cylinder robotic arm is symmetrically distributed on both sides of the inner side plate (8) of the main section of the throwing cylinder. The throwing cylinder deflection pipe (3) is located at the tail of the main section of the throwing cylinder, where the material gains centrifugal force and collides with the outer wall (10) of the throwing cylinder, eventually sliding away along the outer wall. The rotating shaft (7) of the throwing cylinder is located between the throwing cylinder base (6) and the main section of the throwing cylinder, together forming a throwing cylinder rotation mechanism to realize the throwing cylinder rotation. The rotational motion causes the throwing cylinder to change its dropping position in the direction of travel during the spraying process; the pitching hydraulic cylinder (5) of the throwing cylinder main section is installed between the rotating mechanism and the lower notch plate (4) of the throwing cylinder main section, and is used to realize the overall lifting of the throwing cylinder. Before loading, its overall height needs to be adjusted so that the camera reaches the designated recognition position and the throwing cylinder reaches the optimal discharge height; the lower notch plate (4) of the throwing cylinder main section and the inner side plate (8) of the throwing cylinder main section are detachable screw connections to solve the problem of material deflection at the tail of the throwing cylinder main section. To address material blockage issues; the material exits the throwing cylinder at the discharge plate (1) at the front end of the front section (11) of the throwing cylinder; the discharge plate (1) at the front end of the front section of the throwing cylinder comprises two sections connected to each other by hinges, the main section (101) of the discharge plate is hinged to the front section (11) of the throwing cylinder, and the main section (101) of the discharge plate is hinged to the front tongue plate (103), the front tongue plate (103) is the final outlet position of the material, and the overall material flow is straightened after passing through the front tongue plate (103) and is distributed in a rectangular shape from a top view, enhancing the fluid characteristics. The system improves the accuracy of material flow and actual material drop position identification; one end of the tension spring (102) is fixed on the main section (101) of the discharge plate, and the other end is fixed on the front tongue plate (103). One end of the hydraulic cylinder (2) of the front section of the discharge plate is fixed on the front section (11) of the throwing cylinder, and the other end is installed on the main section (101) of the discharge plate. Under the action of the tension spring (102), the front tongue plate (103) and the main section (101) of the discharge plate are deflected in opposite directions; the hydraulic cylinder (2) of the front section of the discharge plate extends and retracts to push the material out. The main section (101) of the plate swings, and the tension spring (102) of the discharge plate extends and retracts, thereby controlling the pitch angle of the discharge plate, so that the throwing cylinder changes the dropping position perpendicular to the traveling direction during the spraying process, thereby controlling the final discharge direction; the silage machine throwing cylinder has three movable degrees of freedom, namely the overall rotation of the throwing cylinder; the overall height adjustment of the throwing cylinder is realized under the action of the pitch hydraulic cylinder (5) of the main section of the throwing cylinder; the discharge plate (1) changes its angle under the action of the hydraulic cylinder (2) of the front section of the discharge plate, thereby controlling the discharge direction of the material flow.

4. The method of claim 1, wherein the method further comprises: Step S3 specifically includes: S3.1 Make the material flow image data set of the camera view angle at the top of the discharge plate of the throwing cylinder, obtain 1400 material flow pictures under the view angle of the working environment, and 800 carriage frames along the center line of the carriage in different angles, preprocess them to enhance the acquisition of feature points, and save them as jpg format; S3.2 Label the obtained pictures using the rolabelimg tool, save the annotation information in the format of PASCAL VOC data set, convert the label format to txt format, and divide the training set and test set, wherein the training set accounts for 80% and the test set accounts for 20%; S3.3 Introduce the divided data set into the YOLO network for iterative training, set the batch size to 16, and the total iteration number to 120. Evaluate the trained model, and the evaluation index is: average precision mean, recall rate, and accuracy. When the iteration number approaches 40 times, the average precision mean approaches 0.996, and the recall rate and accuracy both approach 1; S3.4 After completing the iterative training, save the optimal weight model and test it on the test set. Detect the material flow under different light conditions and different shooting angles, output the midpoint coordinates of the falling position edge line of the rectangular detection frame, and mark it as the actual falling point coordinates.

5. The method of claim 1, wherein the method further comprises: The pyserial library is a module for realizing serial communication under python, installed in cmd using the pip install pyserial command, and the coordinate information obtained by the YOLOv5 network target detection process and the real-time distance calculation results in the u and v directions are transmitted to the arduino controller through serial communication to realize the on-off of the electromagnetic relay and the control of the joints of the throwing cylinder mechanical arm.

6. The method of claim 1, wherein the method further comprises: The expected and actual drop point detection experiment environment uses Window 10 operating system, adopts Pytorch framework, and uses Geforce GTX 1060 graphics card for operation.

7. The method of claim 2, wherein the method further comprises: In the filling rule, the expected drop point operation involves the movement of the throwing cylinder mechanical arm joints, the throwing cylinder base (6) realizes the overall rotational movement of the throwing cylinder in the form of a worm drive and under the drive of a hydraulic motor; the discharge plate (1) realizes angle change under the action of the discharge plate front section hydraulic cylinder (2), the throwing cylinder mechanical arm moves when the expected drop point operation and the distance between the expected drop point and the actual drop point exceeds the threshold, the real-time distance is calculated according to the real-time feedback information of visual detection, the throwing cylinder mechanical arm rotates to correct the u direction distance, and the discharge plate swings to correct the v direction distance.

8. The method of claim 2, wherein the method further comprises: Seven desired material drop locations are set. At that time, from the front to the rear of the vehicle, in order: , , , , , , The filling sequence is as follows: sequentially filling the blanks with ... Filling is performed at three points. If the material height at any one point reaches 80% of the overall car body height, the next point is moved for filling. Upon reaching the designated height, proceed sequentially... , , During the filling process, when the material height at one point reaches 110% of the overall car body height, the process moves to the next point for filling, continuing until the material reaches 110%. Once the material reaches the designated height, the filling process is complete.

9. The method of automatic throw-and-fill control for a silage corn harvester according to claim 1, wherein, In S1, the rotation-yolov5 target detection algorithm includes: Based on the YOLOv5 target detection algorithm, the data loading part, the Head part and the loss function part are modified respectively to obtain the rotation-yolov5 target detection algorithm for realizing the recognition of rotating targets, the long side representation method is used to define the rotating frame in the data loading part, the roLabelImg software is used for labeling, the constructor of the Detect class is modified in the Head part, the loss function includes confidence loss, class classification loss, θ angle classification loss and bbox bounding box regression loss, the weight coefficient of the confidence branch is selected between the horizontal bounding box IOU / GIOU / CIOU / DIOU to speed up the training speed and gpu utilization; since the added θ is a classification task, a classification loss is added, in the bbox bounding box regression loss, the bounding box loss function in the YOLOv5 source code uses IOU / GIOU / CIOU / DIOU, which is suitable for calculating IOU between horizontal rectangular bounding boxes, the loss calculation part of the rotating frame is divided into angle loss and horizontal bounding box loss two parts, and the bounding box regression loss part still uses the above loss function.

Images