A method for detecting the cutting width, cutting height, and crop density of a rice-wheat combine harvester.

Abstract

A method for detecting cutting width, cutting height, and crop density in a rice-wheat combine harvester is proposed. This method utilizes a trained semantic segmentation model, UANet, to obtain unharvested areas. The world coordinates of pixels in the unharvested areas are obtained through coordinate transformation between pixel coordinates and the world coordinate system. The actual cutting width is then calculated, and the plant height data for the selected plant height measurement area is obtained. After random downsampling and K-means clustering of the plant height data, the maximum average value within the maximum value interval is calculated as the actual plant height. Combined with the header height, the actual cutting height is obtained. By inputting rice and wheat ear images and density detection boxes into a trained improved YOLOv5 model, the density of rice and wheat ears can be obtained from the number of ears in the density detection boxes. Ultimately, this method ensures the accuracy of cutting width, cutting height, and crop density detection.

Description

Technical Field

[0001] This invention relates to the field of detection and control technology for rice and wheat combine harvesters, and in particular to a method for detecting the cutting width, cutting height, and crop density of rice and wheat combine harvesters. Background Technology

[0002] Automatic navigation for rice and wheat combine harvesters has developed rapidly as a key technology for the construction of unmanned and smart farms. However, the actual navigation speed of the harvester is still a fixed empirical value or a low value, resulting in poor adaptability to working conditions, low operating efficiency, and the risk of machine blockage.

[0003] To predict and control the travel speed of the automatic navigation system of rice and wheat combine harvesters, it is necessary to estimate the harvester's feed rate. This estimation is typically based on various harvester detection data, such as harvester load, rotational speed and torque of key rotating components, including harvester width, cut height, and crop density. However, current technologies can only obtain the cut width, cut height, and crop density through rough estimation methods. This makes it difficult to establish a mapping relationship between crop image information and feed rate when estimating feed rate based on crop status, resulting in large feed rate estimation errors. This adversely affects the prediction of the optimal operating travel speed of the combine harvester, leading to inaccurate speed prediction. Therefore, the accurate detection of harvester width, cut height, and crop density has become a factor restricting the prediction of harvester travel speed. Summary of the Invention

[0004] The purpose of this invention is to provide a method for detecting the cutting width, cutting height, and crop density of a rice and wheat combine harvester, which can accurately detect the cutting width, cutting height, and crop density.

[0005] The technical solution adopted by this invention to solve the above-mentioned technical problems is: a method for detecting the cutting width, cutting height, and crop density of a rice-wheat combine harvester, comprising the following steps:

[0006] Step 1: Install a depth camera and an industrial camera on the rice and wheat combine harvester and calibrate them;

[0007] Step 2: The rice and wheat combine harvester operates under various working conditions. RGB and depth images of the rice and wheat crops in front of the combine harvester are acquired using a depth camera, and images of the rice and wheat ears are acquired using an industrial camera.

[0008] Step 3: Input the image acquired in Step 2 into the semantic segmentation model for image segmentation to obtain the unharvested crop area;

[0009] The semantic segmentation model is a lightweight semantic segmentation model UANet trained based on the ShuffleNet v2 model, which consists of multiple ShuffleNet v2 basic units and downsampling modules;

[0010] Step 4: Discard closed regions in the unharvested crop area whose pixel area is less than 1 / 5 of the image area as incorrect segments, and obtain the center point of the largest inscribed rectangle of each remaining closed region;

[0011] When the rice and wheat combine harvester operates counterclockwise, the closed area whose center point of the largest inscribed rectangle is to the left of the image centerline and is closest to the lower boundary of the image is selected as the first unharvested area;

[0012] When the rice and wheat combine harvester operates clockwise, the closed area whose center point of the largest inscribed rectangle is to the right of the image centerline and is closest to the lower boundary of the image is selected as the first unharvested area.

[0013] Step 5: Use the Sobel operator to detect the region edges. During the detection process, extract the region boundary points by skipping lines and then use the least squares method to fit the edge lines of the undetected regions.

[0014] When the rice and wheat combine harvester operates counterclockwise, every n rows, it traverses from the left side of the image to the right, and takes the rightmost boundary point as the valid boundary point.

[0015] When the rice and wheat combine harvester operates clockwise, every n rows, it traverses from right to left in the image and takes the leftmost boundary point as the valid boundary point.

[0016] Step 6: Use the depth camera as the origin O c Establish pixel coordinate system O c -X c Y c Z c The coordinate axis X c The cross-section parallel to the rice and wheat combine harvester, coordinate axis Z c The optical axis direction of the depth camera;

[0017] Using the projection of the depth camera onto the ground as the origin O w Establish the world coordinate system O w -X w Y w Z w The coordinate axis X w With coordinate axis X c Parallel, coordinate axis Y w The Z-axis represents the forward direction of the rice and wheat combine harvester. w The vertical direction is represented by the origin O. c Located on the Z-axis w Above and at the origin O w The spacing is the installation height h of the depth camera;

[0018] Based on the unharvested region obtained after edge fitting in step 5, the pixel coordinates of the pixels in the unharvested region in the pixel coordinate system are obtained. Then, through dynamic coordinate system transformation, the world coordinates of the pixels in the unharvested region are obtained. The coordinate transformation formula is:

[0019] (1);

[0020] In equation (1), φ is the sum of the angle β between the optical axis of the depth camera and the horizontal plane and the pitch angle θ of the rice-wheat combine harvester; u and v represent the horizontal and vertical coordinates of any point in the pixel coordinate system, respectively, and u0 and v0 are the center coordinates of the image; Z c The Z-axis value of the camera coordinates represents the distance from the target to the camera; (X) w Y w Z w f represents the coordinates of any point in the world coordinate system; x f y These are the focal lengths of the camera on the x and y axes, respectively;

[0021] Step 7: Calculate the actual harvesting width of the rice-wheat combine harvester based on the world coordinates of the pixels in the unharvested area. The formula is:

[0022] (2);

[0023] (3);

[0024] In equations (2) and (3), w1 is the distance between the depth camera and the right side of the cutter table, w2 is the distance between the depth camera and the left side of the cutter table, and x i The x-coordinate of the boundary point of the unharvested area in the world coordinate system;

[0025] Step 8: Select the plant height measurement area, obtain the plant height data of the plant height measurement area based on the world coordinates of the pixels in the unharvested area, randomly downsample the plant height data, and then calculate the maximum point and maximum range through the data histogram distribution.

[0026] The maximum value interval is divided into multiple classes by K-means clustering. The average value of the plant height data corresponding to the maximum value interval of each class is calculated, and the maximum average value is taken as the actual plant height.

[0027] Subtract the actual plant height from the cutting platform height obtained in step 2 to obtain the actual cutting height;

[0028] Step 9: Set a density detection box to match the rice and wheat ear images acquired in Step 2. The density detection box is a rectangular area enclosed by four vertices. Determine the world coordinates of the four vertices based on the actual harvest width and actual plant height. Then, transform the world coordinates of the four vertices of the density detection box into corresponding pixel coordinates using a dynamic coordinate system. The transformation formula is as follows:

[0029] (4);

[0030] The symbols in equation (4) have the same meaning as those in equation (2);

[0031] Step 10: Obtain the improved YOLOV5 model through training. Input the rice and wheat ear images and density detection boxes into the improved YOLOV5 model to obtain the number of rice and wheat ears in the density detection boxes. Then, obtain the rice and wheat ear density based on the ratio of the number of rice and wheat ears to the area of ​​the density detection boxes.

[0032] Preferably, in step 1, both the depth camera and the industrial camera are mounted on a bracket in front of the cab; the depth camera is a D456 depth camera.

[0033] Preferably, in step 2, the depth camera uses automatic mode, with an image acquisition frequency of 0.5 Hz and an image resolution of 1280×720, while the industrial camera has an image resolution of 2560×1680, and both the depth camera and the industrial camera acquire more than 1500 images.

[0034] Preferably, a left-side imager is introduced into the stereo algorithm of the D456 depth camera. When there are depth outliers and hole points in the depth image, five depth data points are selected from the left and top and bottom of the outlier point. The maximum value of the five depth data points is taken as the depth value of the outlier point, thus completing the preprocessing of depth outliers and hole points.

[0035] Preferably, in step 3, the training process of the UANet model is as follows: manually label and create a training dataset containing no less than 1500 images, with the images labeled as two categories: unharvested regions and harvested regions. Then, the UANet model is trained and tested using the training dataset.

[0036] Preferably, in step 4, the preferred unharvested areas are binarized and opened to achieve smoothing of the area edges while reducing noise interference.

[0037] Preferably, in step 5, the n rows are no less than 10 rows; after extracting the region boundary points, the K-nearest neighbor algorithm is used to remove abnormal boundary points, and the algorithm K value is set to 3.

[0038] Preferably, in step 8, the method for selecting the plant height measurement area is as follows: in the forward direction of the rice and wheat combine harvester, 200 pixels are selected forward from the boundary point of the unharvested area; in the lateral direction of the rice and wheat combine harvester, when the actual harvesting width is greater than or equal to 1.75 m, 400 pixels are selected from the boundary point towards the unharvested area, and when the actual harvesting width is less than 1.75 m, 200 pixels are selected from the boundary point towards the unharvested area; the resulting quadrilateral area is the plant height measurement area.

[0039] Preferably, in step 9, the improved YOLOv5 model introduces FasterNet modules into the YOLOv5 backbone and neck network respectively to optimize the CSPDarknet53 modules in the backbone and neck networks, constructing new modules C3_FasterNet and C3_FasterNet_Neck; the strided convolutions in the YOLOv5 backbone network are modified by introducing downsampling modules and spatial depth transformation convolutions to reduce feature loss during downsampling; channel and spatial attention modules with fused residual structures are added to both the backbone and neck networks to improve the model's feature extraction capability and adaptability to the wheat ear dataset; the loss functions used are EIoU loss function and CIoU loss function;

[0040] The training process for the improved YOLOv5 model involves manually labeling and creating a training dataset of rice and wheat ears containing no fewer than 1,500 images. The dataset is then subjected to HSV enhancement, translation, scaling, horizontal flipping, and stitching to simulate complex lighting conditions in the field, as well as the vibration and shaking of harvesters. The improved YOLOv5 model is then trained and tested using the rice and wheat ear training dataset.

[0041] According to the above technical solution, the beneficial effects of the present invention are:

[0042] This invention utilizes a trained semantic segmentation model, UANet, to obtain unharvested areas. Through coordinate transformation between pixel coordinates and the world coordinate system, the world coordinates of pixels in the unharvested areas are obtained, leading to the calculation of the actual harvesting width. Plant height data for the selected plant height measurement area is also obtained. After random downsampling and K-means clustering of the plant height data, the maximum average value within the maximum value interval is calculated as the actual plant height. Combined with the header height, the actual cutting height can be obtained. By inputting rice and wheat ear images and density detection boxes into a trained improved YOLOv5 model, the density of rice and wheat ears can be obtained from the number of ears in the density detection boxes. Therefore, this invention can accurately detect cutting width, cutting height, and crop density, thereby improving the prediction accuracy of the optimal operating speed of the combine harvester through accurate detection data. Attached Figure Description

[0043] Figure 1 A schematic diagram illustrating the spatial relationship between the depth camera and the ground;

[0044] Figure 2 This is a schematic diagram illustrating the positional relationship between the pixel coordinate system and the world coordinate system.

[0045] Figure 3 This is a schematic diagram for measuring the cutting width;

[0046] Figure 4 This is a schematic diagram of the spatial coordinates for detecting the density of rice and wheat ears. Detailed Implementation

[0047] This embodiment provides a method for detecting the cutting width, cutting height, and crop density of a rice and wheat combine harvester, including the following steps:

[0048] Step 1: Install a depth camera and an industrial camera on the rice and wheat combine harvester and calibrate them.

[0049] Step 2: The rice and wheat combine harvester operates under various working conditions. RGB and depth images of the rice and wheat crops in front of the combine harvester are acquired using a depth camera, and images of the rice and wheat ears are acquired using an industrial camera.

[0050] The depth camera is a D456 depth camera, using automatic mode, with an image acquisition frequency of 0.5 Hz and an image resolution of 1280×720. The industrial camera has an image resolution of 2560×1680. Both the depth camera and the industrial camera acquire more than 1500 images.

[0051] In the stereo algorithm of the D456 depth camera, a left-side imager is introduced. When there are depth outliers and hole points in the depth image, five depth data points are selected from the left and top and bottom of the outlier point. The maximum value of the five depth data points is taken as the depth value of the outlier point, thus completing the preprocessing of depth outliers and hole points.

[0052] Step 3: Input the image acquired in Step 2 into the semantic segmentation model for image segmentation to obtain the unharvested crop area.

[0053] The semantic segmentation model is a lightweight semantic segmentation model UANet, which is trained based on the ShuffleNet v2 model and consists of multiple ShuffleNet v2 basic units and downsampling modules.

[0054] The training process for the UANet model involves manually labeling and creating a training dataset containing no fewer than 1,500 images. The images are labeled into two categories: unharvested regions and harvested regions. The UANet model is then trained and tested using the training dataset.

[0055] Step 4: Discard closed regions in the unharvested crop area whose pixel area is less than 1 / 5 of the image area as incorrect segments, and obtain the center point of the largest inscribed rectangle of each remaining closed region.

[0056] When the rice and wheat combine harvester operates counterclockwise, the closed area whose center point of the largest inscribed rectangle is to the left of the image centerline and is closest to the lower boundary of the image is selected as the first unharvested area.

[0057] When the rice and wheat combine harvester operates clockwise, the closed area whose center point of the largest inscribed rectangle is to the right of the image centerline and is closest to the lower boundary of the image is selected as the first unharvested area.

[0058] Binarization and opening operations are performed on the preferred unharvested areas to smooth the area edges and reduce noise interference.

[0059] Step 5: Use the Sobel operator to detect the region edges. During the detection process, extract the region boundary points by skipping rows, and then use the least squares method to fit the edge lines of the undetected regions.

[0060] When the rice and wheat combine harvester operates counterclockwise, it traverses from the left side of the image to the right every 10 rows, taking the rightmost boundary point as the valid boundary point.

[0061] When the rice and wheat combine harvester operates clockwise, it traverses from right to left every 10 rows, taking the leftmost boundary point as the valid boundary point.

[0062] To avoid lateral boundary points (field edges) and interference points (segmentation errors), after extracting the region boundary points, the K-nearest neighbor algorithm is used to remove abnormal boundary points, with the K value set to 3.

[0063] Step 6, as follows Figure 1 , 2 As shown, in this embodiment, the camera is installed at a height of 2.40 m, the angle β between the camera's optical axis and the horizontal plane is 30°, and the depth camera is used as the origin O. c Establish pixel coordinate system O c -X c Y c Z c The coordinate axis X c The cross-section parallel to the rice and wheat combine harvester, coordinate axis Z c This refers to the optical axis direction of the depth camera.

[0064] Using the projection of the depth camera onto the ground as the origin O w Establish the world coordinate system O w -X w Y w Z w The coordinate axis Xw With coordinate axis X c Parallel, coordinate axis Y w The Z-axis represents the forward direction of the rice and wheat combine harvester. w The vertical direction is represented by the origin O. c Located on the Z-axis w Above and at the origin O w The spacing is the installation height h of the depth camera.

[0065] Based on the unharvested region obtained after edge fitting in step 5, the pixel coordinates of the pixels in the unharvested region in the pixel coordinate system are obtained. Then, through dynamic coordinate system transformation, the world coordinates of the pixels in the unharvested region are obtained. The coordinate transformation formula is:

[0066] (1);

[0067] In equation (1), φ is the sum of the angle β between the optical axis of the depth camera and the horizontal plane and the pitch angle θ of the rice-wheat combine harvester; u and v represent the horizontal and vertical coordinates of any point in the pixel coordinate system, respectively, and u0 and v0 are the center coordinates of the image; Z c The Z-axis value of the camera coordinates represents the distance from the target to the camera; (X) w Y w Z w f represents the coordinates of any point in the world coordinate system; x f y These are the camera's focal lengths on the x and y axes, respectively.

[0068] Step 7: Calculate the actual harvesting width of the rice-wheat combine harvester based on the world coordinates of the pixels in the unharvested area. The formula is:

[0069] (2);

[0070] (3);

[0071] like Figure 3 As shown, in equations (2) and (3), w1 is the distance between the depth camera and the right side of the cutter table, w2 is the distance between the depth camera and the left side of the cutter table, and x i Here is the x-coordinate of the boundary point of the unharvested area in the world coordinate system. After obtaining the world coordinates of the harvest boundary point, the cutting width L is calculated as shown in Formula 5. To reduce the measurement error of the width, the average width of the 10 boundary points closest to the cutting platform is calculated (excluding the maximum and minimum values), and the value of i in Formula 5 is 8.

[0072] Step 8: Convert the pixel coordinates of the unharvested crop area to 3D world coordinates, select the plant height measurement area, perform random downsampling and clustering operations on the plant height data of the measurement area, remove the height data corresponding to the middle and lower parts of the rice and the bare ground, and take the average of the processed plant height data as the crop plant height. The specific method is as follows:

[0073] First, the plant height measurement area is selected. The method is as follows: in the forward direction of the rice and wheat combine harvester, 200 pixels are selected forward from the boundary point of the unharvested area; in the lateral direction of the rice and wheat combine harvester, when the actual harvesting width is greater than or equal to 1.75 m, 400 pixels are selected from the boundary point towards the unharvested area, and when the actual harvesting width is less than 1.75 m, 200 pixels are selected from the boundary point towards the unharvested area; the resulting quadrilateral area is the plant height measurement area.

[0074] Then, the plant height data of the plant height measurement area is obtained based on the world coordinates of the pixels in the unharvested area. The plant height data is randomly downsampled, and the maximum point and maximum range are calculated through the data histogram distribution.

[0075] The maximum value intervals are divided into multiple classes using K-means clustering. The average value of the plant height data corresponding to the maximum value intervals of each class is calculated, and the maximum average value is taken as the actual plant height.

[0076] Subtract the actual plant height from the cutting height obtained in step 2 to get the actual cutting height.

[0077] Step 9: Set a density detection box to match the rice and wheat ear images acquired in Step 2. The density detection box is a rectangular area enclosed by four vertices. Determine the world coordinates of the four vertices based on the actual harvest width and actual plant height. Then, transform the world coordinates of the four vertices of the density detection box into corresponding pixel coordinates using a dynamic coordinate system. The transformation formula is as follows:

[0078] (4);

[0079] The symbols in equation (4) have the same meaning as those in equation (2).

[0080] Step 10: Obtain the improved YOLOV5 model through training. Input the rice and wheat ear images and density detection boxes into the improved YOLOV5 model to obtain the number of rice and wheat ears in the density detection boxes. Then, obtain the rice and wheat ear density based on the ratio of the number of rice and wheat ears to the area of ​​the density detection boxes.

[0081] The YOLOv5 model is improved by introducing FasterNet modules into the YOLOv5 backbone and neck network, respectively, to optimize the CSPDarknet53 modules in the backbone and neck networks, and constructing new modules C3_FasterNet and C3_FasterNet_Neck. The strided convolutions in the YOLOv5 backbone are replaced with downsampling modules and spatial depth transformation convolutions to reduce feature loss during downsampling. Channel and spatial attention modules that fuse residual structures are added to both the backbone and neck networks to improve the model's feature extraction capabilities and adaptability to wheat ear datasets. The loss functions used are EIoU and CIoU.

[0082] The training process for the improved YOLOv5 model involves manually labeling and creating a training dataset of rice and wheat ears containing no fewer than 1,500 images. The dataset is then subjected to HSV enhancement, translation, scaling, horizontal flipping, and stitching to simulate complex lighting conditions in the field, as well as the vibration and shaking of harvesters. The improved YOLOv5 model is then trained and tested using the rice and wheat ear training dataset.

[0083] This implementation example Figure 4 As shown, two coordinate systems were constructed based on the installation location of the industrial camera: O c -X c Y c Z c The imaging plane of the camera coordinate system, X c Z is parallel to the cross-section of the harvester. c Indicates the direction along the camera's optical axis; O w -X w Y w Z w X represents a world coordinate system aligned with the Earth's horizontal plane. w Parallel to X c Axis, O w The origin corresponds to the camera's projection point on the ground, Y w This indicates the direction of travel for the combine harvester.

[0084] Since the actual harvesting width is not fixed, a density detection frame is selected 0.5 m in front of the harvester's header. The selection rules for the detection frame are as follows:

[0085] If the harvest width is greater than or equal to 1.75 m, the detection frame size is 1.4 m × 1.0 m;

[0086] If the harvest width is greater than or equal to 1.5 m and less than 1.75 m, the detection frame size is 1.2 m × 1.0 m;

[0087] If the harvest width is less than 1.5 m, the detection frame size is 1.0 m × 1.0 m;

[0088] The center of all detection frames is located on the center line of the harvester's header width.

[0089] By measuring, the bottom edge of the detection frame is aligned with O. w -X w Z w The distance L between the planes is 2.2 m. The camera is installed at a height of H = 2.4 m, and the original angle μ between the camera's optical axis and the horizontal plane is 30°.

[0090] Based on the dynamic coordinate system transformation model, the relationship between camera coordinates and world coordinates can be obtained, and the external reference matrix of the camera can be obtained.

[0091] The camera's internal reference matrix can be obtained using the "Zhang Zhengyou calibration method".

[0092] Then, using the cut width and plant height data, the three-dimensional world coordinates of the four vertices of the detection box can be transformed into the corresponding pixel coordinates.

[0093] To avoid irregular shapes in the detection box caused by coordinate transformation errors, which would make wheat ear detection difficult, after obtaining the pixel coordinates of the four vertices of the detection box, the minimum and maximum values ​​of the horizontal and vertical coordinates are selected respectively to optimize the detection box into a rectangle. The number of wheat ears in the detection box is obtained using an improved YOLOv5, which enables visual density calculation.

[0094] This embodiment utilizes the trained semantic segmentation model UANet to obtain unharvested areas. The world coordinates of pixels in the unharvested areas are obtained through coordinate transformation between the pixel coordinate system and the world coordinate system. The actual harvest width is then calculated, and the plant height data for the selected plant height measurement area is obtained. After random downsampling and K-means clustering of the plant height data, the maximum average value is calculated from the average value of the maximum value interval as the actual plant height. Combined with the header height, the actual cutting height can be obtained. By inputting rice and wheat ear images and density detection boxes into the trained improved YOLOv5 model, the density of rice and wheat ears can be obtained from the number of rice and wheat ears in the density detection box, ultimately ensuring the accuracy of cutting width, cutting height, and crop density detection.

Claims

1. A method of detecting the swath width, cutting height and crop density of a rice and wheat combine harvester, characterized by, The method comprises the following steps: Step 1, install a depth camera and an industrial camera on a combine harvester and calibrate them; Step 2, the combine harvester works under various working conditions, the RGB image and the depth image of the rice and wheat crops in front of the combine harvester are collected by the depth camera, and the ear image of the rice and wheat is collected by the industrial camera; Step 3, the images collected in step 2 are input into a semantic segmentation model for image segmentation to obtain an unharvested area of the crops; The semantic segmentation model is a lightweight semantic segmentation model UANet trained based on a ShuffleNet v2 model, which is composed of multiple ShuffleNet v2 basic units and down-sampling modules; Step 4, discard the closed area in the unharvested area of the crops whose pixel area is less than 1 / 5 of the image area as an error segmentation, and obtain the center point of the maximum inscribed rectangle of each remaining closed area; When the combine harvester works counterclockwise, the closed area with the center point of the maximum inscribed rectangle on the left side of the image center line and closest to the lower boundary of the image is taken as the preferred unharvested area; When the combine harvester works clockwise, the closed area with the center point of the maximum inscribed rectangle on the right side of the image center line and closest to the lower boundary of the image is taken as the preferred unharvested area; Step 5, the Sobel operator is used for region edge detection, and the region boundary points are extracted by skip-line traversal during the detection process, and then the least square method is used for edge line fitting of the unharvested area; When the combine harvester works counterclockwise, every n rows are traversed from left to right, and the rightmost boundary point is taken as the effective boundary point; When the combine harvester works clockwise, every n rows are traversed from right to left, and the leftmost boundary point is taken as the effective boundary point; Step 6, take the depth camera as the origin O c , establish the pixel coordinate system O c -X c Y c Z c , wherein the coordinate axis X c is parallel to the cross section of the rice-wheat combine harvester, and the coordinate axis Z c is the optical axis direction of the depth camera; Using the projection of the depth camera onto the ground as the origin O w Establish the world coordinate system O w -X w Y w Z w The coordinate axis X w With coordinate axis X c Parallel, coordinate axis Y w The Z-axis represents the forward direction of the rice and wheat combine harvester. w The vertical direction is represented by the origin O. c Located on the Z-axis w Above and at the origin O w The spacing is the installation height h of the depth camera; According to the unharvested area obtained after step 5 edge line fitting, the pixel coordinates of the unharvested area pixel points in the pixel coordinate system are obtained, and then the world coordinates of the unharvested area pixel points are obtained through dynamic coordinate system transformation, and the coordinate conversion formula is: （1）； In formula (1), φ is the sum of the angle β between the depth camera optical axis and the horizontal plane and the pitch angle θ of the combine harvester; u and v represent the horizontal and vertical coordinates of any coordinate point in the pixel coordinate system, respectively; u0 and v0 are the center coordinates of the image, respectively; Z c is the Z-axis value of the camera coordinate, representing the distance from the target to the camera; (X w , Y w , Z w ) are the coordinates of any point in the world coordinate system; f x , f y are the focal lengths of the camera in the x-axis and y-axis, respectively; Step 7, the actual harvesting width of the combine harvester is calculated according to the world coordinates of the unharvested area pixel points, and the formula is: （2）； （3）； In formula (2), (3), w1 is the distance between the depth camera and the right side of the header, w2 is the distance between the depth camera and the left side of the header, x i is the horizontal coordinate of the unharvested area boundary point in the world coordinate system; Step 8, select a plant height measurement area, obtain the plant height data of the plant height measurement area according to the world coordinates of the unharvested area pixel points, randomly down-sample the plant height data, and then calculate the maximum value point and the maximum value interval through data histogram distribution; The maximum value interval is divided into multiple classes through K-means clustering, the average values of the plant height data corresponding to the multiple maximum value intervals are calculated respectively, and the maximum average value is taken as the actual plant height; The actual plant height is subtracted from the header height obtained in step 2 to obtain the actual cutting height; Step 9, set a density detection frame matching the ear image collected in step 2, the density detection frame is a rectangular area surrounded by four vertices, the world coordinates of the four vertices are determined according to the actual harvesting width and the actual plant height respectively, and then the world coordinates of the four vertices of the density detection frame are converted into corresponding pixel coordinates through dynamic coordinate system transformation, and the conversion formula is: （4）； The symbols in formula (4) have the same meanings as those in formula (2); Step 10, the improved YOLOV5 model is obtained by training, the rice and wheat ear image and the density detection frame are input into the improved YOLOV5 model, the number of rice and wheat ears in the density detection frame is obtained, and then the density of the rice and wheat ears is obtained according to the ratio of the number of rice and wheat ears to the area of the density detection frame.

2. The method for detecting cutting width, cutting height and crop density of a rice-wheat combine harvester according to claim 1, characterized in that: In step 1, the depth camera and the industrial camera are both installed on the front support of the cab.

3. The method for detecting the cutting width, cutting height, and crop density of a rice-wheat combine harvester according to claim 2, characterized in that: In step 2, the depth camera adopts an automatic mode, the image acquisition frequency is 0.5 Hz, the image resolution is 1280*720, the image resolution of the industrial camera is 2560*1680, and the image acquisition quantity of the depth camera and the industrial camera is greater than 1500.

4. The method according to claim 2, wherein the method is characterized in that: In the stereo algorithm of the D456 depth camera, a left imager is introduced, when there are depth outliers and hollow point data in the depth image, 5 depth data on the left and below the abnormal point are selected, the maximum value of the 5 depth data is taken as the depth value of the abnormal point, and the preprocessing of the depth outliers and the hollow points is completed.

5. The method for detecting cutting width, cutting height and crop density of a rice-wheat combine harvester according to claim 1, characterized in that: In step 3, the training process of the UANet model is that a training data set containing not less than 1500 images is artificially labeled and prepared, the images are labeled as two categories of unharvested area and harvested area, and then the UANet model is trained and tested by using the training data set.

6. The method for detecting cutting width, cutting height and crop density of a rice-wheat combine harvester according to claim 1, characterized in that: In step 4, the preferred unharvested area is subjected to binarization and opening operation processing, so as to realize the smoothing of the region edge and reduce the noise interference.

7. The method for detecting cutting width, cutting height and crop density of a rice-wheat combine harvester according to claim 1, characterized in that: In step 5, the n rows are not less than 10 rows; after the region boundary points are extracted, the K nearest neighbor algorithm is used to remove abnormal boundary points, and the K value of the algorithm is set to 3.

8. The method for detecting cutting width, cutting height and crop density of a rice-wheat combine harvester according to claim 1, characterized in that: In step 8, the selection method of the plant height measurement area is that, in the forward direction of the rice and wheat combined harvester, 200 pixel points are selected from the boundary point of the unharvested area as the starting point; in the transverse direction of the rice and wheat combined harvester, when the actual harvesting width is greater than or equal to 1.75 m, 400 pixel points are selected from the boundary point to the unharvested area, and when the actual harvesting width is less than 1.75 m, 200 pixel points are selected from the boundary point to the unharvested area; and the finally formed quadrilateral area is the plant height measurement area.

9. The method for detecting cutting width, cutting height and crop density of a rice-wheat combine harvester according to claim 1, characterized in that: In step 9, the improved YOLOV5 model introduces FasterNet modules in the YOLOV5 backbone network and neck network respectively, which are used to optimize the CSPDarknet53 modules in the backbone network and the neck network, construct new modules C3_FasterNet and C3_FasterNet_Neck; the stride convolution in the YOLOV5 backbone network introduces a downsampling module and a spatial depth conversion convolution to reduce the feature loss in the downsampling process; the backbone network and the neck network both add channel and spatial attention modules with fusion residual structure to improve the feature extraction capability of the model and the adaptability to the wheat ear data set; the loss function adopts EIoU loss function and CIoU loss function; The improved YOLOV5 model training process is that artificial labeling and making rice and wheat ear training data set containing not less than 1500 images, carrying out HSV enhancement, translation, scaling, horizontal flip and splicing on the data set to simulate complex light in the field and harvesting machine vibration and shaking, and then training and testing the improved YOLOV5 model by using the rice and wheat ear training data set.

Images