A method and system for on-line detection of loading state of a grain tank of a grain cart

Abstract

The application provides a kind of grain wagon grain box loading state online detection method and system, obtain grain wagon grain box image by binocular camera;Image is converted into HSV color space, and grain area in grain box is extracted;Grain box is divided into multiple grain unloading areas, and different loading level value standard points are set for different areas with grain box mouth upper plane as reference;Grain box mouth upper plane is extracted in depth image, the distance from the upper surface of grain to grain box mouth upper plane of each area center point is obtained, and if the total gap volume of adjacent areas is greater than predetermined value, there is gap;From the center of grain box, gap is identified in turn according to distance, and grain box loading state information is obtained.The application can quickly and accurately detect the loading state of grain wagon grain box, solve the problem that the loading state of grain wagon grain box is unknown when the harvester unloads grain, which leads to time-consuming and laborious manual unloading, low utilization rate of grain box, etc., and can provide basic data support for the unloading control of the harvester when the grain wagon and the harvester work together.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of visual intelligent harvesting equipment, and particularly relates to a method and system for online detection of loading state of a grain tank of a grain transport vehicle. BACKGROUND

[0002] Harvesting operation is the last link of grain production, and needs to seize the favorable farming time to achieve fast harvesting. However, during the unloading process, the driver in the cab cannot observe the loading state of the grain tank of the grain transport vehicle, and has to stop the machine to check or listen to others to adjust the position of the unloading cylinder, which not only increases the labor intensity of the driver, but also seriously reduces the harvesting efficiency. In addition, with the development of intelligent agricultural machinery, the grain transport vehicle and the harvester are gradually working together on the historical stage, and the unloading is carried out at the same time as the harvester is working, which greatly improves the harvesting efficiency. Therefore, the detection of the loading state of the grain tank of the grain transport vehicle is particularly important to further meet the intelligent and efficient harvesting demand under the working mode of the grain transport vehicle and the harvester working together.

[0003] At present, there are some researches on the detection of the loading state of the grain tank of the grain transport vehicle. One prior art detects the loading state of the grain in the grain transport vehicle through a color camera installed on the side wall of the grain tank of the grain transport vehicle. This method can only collect planar color images, and is easily affected by light and dust. A large amount of dust is generated during the unloading process, which affects the accuracy of real-time detection of the loading state of the grain. Another prior art carries out three-dimensional reconstruction of the entire grain surface through a depth camera installed diagonally on the grain tank of the grain transport vehicle. This method has large point cloud calculation amount and poor real-time performance. SUMMARY

[0004] In view of the above technical problems, one of the purposes of one mode of the present application is to provide a method for online detection of the loading state of the grain tank of the grain transport vehicle. The image of the grain tank of the grain transport vehicle is acquired through a binocular camera, the grain area in the grain tank is extracted, and then the gaps are identified in order from the center of the grain tank according to the distance, so as to obtain the loading state information of the grain tank. When the loading level of each area reaches the loading level value standard point, the surface grain tank is full. The method has high detection precision and fast detection speed, and can solve the problems of time-consuming and labor-intensive manual unloading, low utilization rate of the grain tank and the like caused by the unknown loading state of the grain tank of the grain transport vehicle during the unloading of the harvester, and can provide reliable technical support for the collaborative work of the grain transport vehicle and the harvester.

[0005] One of the purposes of one mode of the present application is to provide a system for online detection of the loading state of the grain tank of the grain transport vehicle, which comprises an image acquisition module, an image processing module and a grain tank loading state analysis module. The image processing technology is used to accurately detect the loading state of the grain tank of the grain transport vehicle in real time, to provide basic data support for the unloading control of the harvester, and to realize more efficient unloading.

[0006] Note that the description of these objects does not preclude the existence of other objects. One embodiment of the present application does not necessarily achieve all the above objects. Other objects can be drawn from the description, the drawings, and the claims.

[0007] The present application achieves the above technical objects through the following technical means.

[0008] A method for on-line detection of loading state of grain tank of grain transport vehicle, comprising the following steps:

[0009] Step S1, acquiring grain tank image of the grain transport vehicle by binocular camera;

[0010] Step S2, converting the grain tank image of the grain transport vehicle acquired by the binocular camera into HSV color space, and extracting the grain region in the grain tank in the HSV color space;

[0011] Step S3, dividing the grain tank into multiple unloading regions according to the shape and size of the grain tank, and setting different loading level standard points for different regions based on the upper plane of the grain tank mouth;

[0012] Step S4, extracting the upper plane of the grain tank mouth in the depth image, obtaining the distance from the upper surface of the grain to the upper plane of the grain tank mouth of the center point of each region, and proving that there is a gap in the region if the total gap volume of adjacent regions is greater than a predetermined value;

[0013] Step S5, sequentially identifying the gaps from the center of the grain tank according to the distance, thereby obtaining the loading state information of the grain tank.

[0014] In the above scheme, the step S1 of acquiring the grain tank image by the binocular camera specifically comprises the following steps:

[0015] Step S1.1, the binocular camera comprises a first binocular camera and a second binocular camera, the first binocular camera is installed on the front side wall of the grain tank of the grain transport vehicle, and the second binocular camera is installed below the unloading cylinder of the harvester, the angle of the camera is adjusted, the first binocular camera acquires the RGB color image and the depth image above the front side of the grain tank of the grain transport vehicle, and the second binocular camera acquires the RGB color image and the depth image above the left side of the grain tank of the grain transport vehicle;

[0016] Step S1.2, taking the first binocular camera on the front side wall of the grain tank of the grain transport vehicle as the first image acquisition module, and taking the second binocular camera below the unloading cylinder as the second image acquisition module, if the first binocular camera fails or the collected image data is unqualified due to problems such as too strong light, too much dust, and mutual shielding of grain piles, the second image acquisition module below the unloading cylinder is enabled.

[0017] In the above scheme, the step S2 of extracting the grain region in the HSV color space specifically comprises the following steps:

[0018] Step S2.1: pre-process the image, obtain the grain tank region in the gray image by threshold segmentation method, and extract the grain tank contour by edge detection;

[0019] Step S2.2: for each pixel in the image, perform pixel value conversion in the HSV color space;

[0020] Step S2.3: the difference between the grain and the grain tank region is the largest in the H space of the HSV color space, and the grain region is obtained by threshold segmentation;

[0021] Step S2.4: convert the image pixel coordinate system to the world coordinate system for the depth image, reconstruct the elevation image, and obtain the depth value of the grain region.

[0022] Further, the pixel value conversion in the HSV color space in step S2.2 is as follows:

[0023]

[0024]

[0025] v = max

[0026] wherein r, g, b are pixel channel values in the RGB color space,

[0027] max is the maximum of r, g and b,

[0028] min is the minimum of r, g and b,

[0029] h, s, v are pixel channel values in the HSV color space.

[0030] In the above scheme, the step S3 of dividing the grain tank into multiple unloading regions and setting different loading level standard values specifically includes the following steps:

[0031] Step S3.1: divide the grain tank into multiple unloading regions according to the shape and size of the grain tank;

[0032] Step S3.2: different regions set different height standard values, the standard value is related to the grain repose angle and the distance from the center point of the region to the highest region center point, and the height standard value decreases from the center of the grain tank outward, and more grain is loaded under the premise that the grain tank does not overflow;

[0033] Step S3.3: taking the upper plane of the grain tank as the reference, convert the height standard values of different regions into loading level standard values;

[0034] C t = (H t -H1) × S n

[0035] wherein, C t is the height standard value of the highest region, n is the area of the region.

[0036] Further, the size of the height standard value in step S3.2 is calculated according to the following formula:

[0037]

[0038]

[0039] wherein, H max is the height standard value of the highest region,

[0040] H min is the height standard value of the lowest region,

[0041] H t is the height standard value of the different region,

[0042] W1 is the length of the short side of the grain tank,

[0043] AW t is the distance from the center point of the different region to the center point of the highest region,

[0044] a is the angle of repose of the grain.

[0045] In the above scheme, the step S4 of judging whether there is a gap in the loading of the grain tank specifically comprises the following steps:

[0046] Step S4.1: detecting other regions except the region being unloaded;

[0047] Step S4.2: taking the upper plane of the grain tank as the reference, the height of the grain above the upper plane of the grain tank is positive, and the height below the plane is negative, the height difference between the grain surface of each region and the upper plane of the grain tank is converted into the loading level, and the calculation formula of the loading level is as follows:

[0048] C n = AH n × S n

[0049] wherein, C n is the loading level of the region,

[0050] AH n is the height difference between the center grain surface of the region and the upper plane of the grain tank,

[0051] S n is the area of the region;

[0052] Step S4.3: Calculate the void volume of the region by the different region loading level value and the standard value, if the total void volume of the adjacent region is greater than the set threshold, it proves that the region has a void, the calculation formula of the void volume is as follows:

[0053] ΔV = ∑ΔC i = ∑(C t -C n )

[0054] Wherein, ΔC i is the void volume of the region,

[0055] ΔV is the total void volume of the adjacent region.

[0056] In the above scheme, the step S5 is to identify the void from the center of the grain tank according to the distance, until the loading level of each region reaches the loading level standard value, which indicates that the grain tank is full;

[0057] The grain tank loading state information includes the void volume ΔV r of each void, the center position (X r , Y r ) of the void and the grain tank loading rate La,

[0058]

[0059] Va = L × W × H + ∑C t

[0060] Wherein, ∑ΔV r is the total void volume of the grain tank, Va is the volume of the grain in the full load grain tank, L, W, H are the length, width and height of the grain tank, and ∑C t is the sum of the loading level standard values of all regions of the grain tank.

[0061] A system for implementing the online detection method of the grain tank loading state of the grain transport vehicle includes an image acquisition module, an image processing module and a grain tank loading state analysis module;

[0062] The image acquisition module is used to acquire the RGB color image and the depth image of the left upper side and the front upper side of the grain tank of the grain transport vehicle;

[0063] The image processing module is used to extract the grain region in the HSV color space in the images collected at the two positions and obtain the depth value of the grain region through the reconstruction of the elevation image;

[0064] The grain tank state analysis module is used for dividing the grain tank into multiple unloading areas, setting different loading level value standard points for different areas based on the upper plane of the grain tank opening, extracting the upper plane of the grain tank opening in the depth image, obtaining the distance from the upper surface of the grain in each area center point to the upper plane of the grain tank opening, finding the area with gaps through different area loading level values and standard values, and proving that the area has gaps if the total gap volume of adjacent areas is greater than a predetermined value; and sequentially identifying the gaps from the grain tank center according to the distance, so as to obtain the grain tank loading state information.

[0065] In the above scheme, the camera selection module is further included; the camera selection module is used for selecting to enable the second binocular camera when the image processing module cannot collect effective image data of the first binocular camera.

[0066] Compared with the prior art, the present application has the following beneficial effects:

[0067] According to one mode of the present application, the present application obtains the grain tank image of the grain cart through the binocular camera, sequentially identifies the gaps from the grain tank center according to the distance after extracting the grain area in the grain tank, so as to obtain the grain tank loading state information; and when the loading level of each area reaches the loading level value standard point, the surface grain tank is full. The method has high detection precision and fast detection speed, can solve the problems of time-consuming and labor-intensive manual unloading and low utilization rate of the grain tank caused by unknown loading state of the grain tank of the grain cart during the unloading of the current harvester, and can provide reliable technical support for the cooperative operation of the grain cart and the harvester.

[0068] According to one mode of the present application, a system for realizing the above-mentioned online detection method of the grain tank loading state of the grain cart is provided, which includes an image acquisition module, an image processing module and a grain tank loading state analysis module, and can accurately detect the grain tank loading state of the grain cart in real time by using image processing technology, provide basic data support for the unloading control of the harvester, and realize more efficient unloading.

[0069] Note that the description of these effects does not hinder the existence of other effects. One mode of the present application does not necessarily have all the above-mentioned effects. Effects other than the above-mentioned can be clearly seen and extracted from the description, drawings, claims, etc. BRIEF DESCRIPTION OF DRAWINGS

[0070] Figure 1 is a flowchart of one embodiment of the present application.

[0071] Figure 2 is a top view schematic diagram of the positions of the installation of two binocular cameras of one embodiment of the present application.

[0072] Figure 3 is a front view schematic diagram of the positions of the installation of two binocular cameras of one embodiment of the present application.

[0073] Figure 4 is a schematic diagram of the maximum minimum height standard value and the rest angle of an embodiment of the present application.

[0074] Figure 5 is a grain box area division diagram of an embodiment of the present application.

[0075] Figure 6 is a grain unloading point area detection diagram in different loading states of an embodiment of the present application, wherein Figure 6 (a) is a grain unloading point area detection diagram in a 10% loading state, (b) is a grain unloading point area detection diagram in a 40% loading state, (c) is a grain unloading point area detection diagram in a 60% loading state, and (d) is a grain unloading point area detection diagram in an 80% loading state.

[0076] In the figure, 1. first binocular camera; 2. second binocular camera; 3. harvester; 4. grain cart; 5. grain box; 6. grain unloading cylinder. DETAILED DESCRIPTION

[0077] Embodiments of the present application are described in detail below with reference to examples shown in the attached drawings, wherein the same or similar notations represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by reference to the attached drawings are exemplary and are intended to explain the present application, and cannot be understood as a limitation of the present application.

[0078] In the description of the present application, it should be understood that the orientation or positional relationship indicated by the terms "center", "longitudinal", "transverse", "length", "width", "thickness", "front", "back", "left", "right", "up", "down", "axial", "radial", "vertical", "horizontal", "inner", "outer", etc. are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation of the present application. In addition, the terms "first", "second" are only for the purpose of description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first", "second" can explicitly or implicitly include one or more of the features. In the description of the present application, the meaning of "a plurality of" is two or more, unless otherwise specifically limited.

[0079] In the present application, unless otherwise explicitly specified and limited, the terms "mounting", "connecting", "connecting", "fixing" and the like should be understood broadly, for example, it can be fixedly connected, or it can be detachably connected, or integrally connected; it can be mechanically connected, or it can be electrically connected; it can be directly connected, or it can be indirectly connected through an intermediate medium, or it can be the communication inside two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0080] Embodiment 1

[0081] Figure 1 The figure shows a preferred embodiment of the grain cart grain box loading state online detection method, the grain cart grain box loading state online detection method, comprising the following steps:

[0082] Step S1, acquiring the image of the grain cart grain box 5 by the binocular camera;

[0083] Step S2, converting the grain cart grain box image acquired by the binocular camera into HSV color space, and extracting the grain region in the grain box in the HSV color space;

[0084] Step S3, according to the shape and size of the grain box, the grain box is divided into multiple unloading regions, and different loading level value standard points are set for different regions based on the upper plane of the grain box opening;

[0085] Step S4, extracting the upper plane of the grain box opening in the depth image, obtaining the distance from the upper surface of the grain to the upper plane of the grain box opening of each region center point, and if the total gap volume of adjacent regions is greater than a predetermined value, it is proved that there is a gap in the region;

[0086] Step S5, identifying the gap in order of distance from the center of the grain box, thereby obtaining the grain box loading state information.

[0087] In combination Figure 2 And 3 As shown, the step S1 of acquiring the grain box image by the binocular camera specifically comprises the following steps:

[0088] Step S1.1: the binocular camera includes a first binocular camera 1 and a second binocular camera 2, the first binocular camera 1 is installed on the front side wall of the grain cart 4, the second binocular camera 2 is installed below the unloading cylinder 6 of the harvester 3, the angle of the camera is adjusted, the first binocular camera 1 acquires the RGB color image and the depth image above the front side of the grain cart, and the second binocular camera 2 acquires the RGB color image and the depth image above the left side of the grain cart;

[0089] Step S1.2: The first binocular camera 1 at the front side wall of the grain tank is the first image acquisition module, and the second binocular camera 2 below the unloading cylinder is the second image acquisition module. If the first binocular camera 1 fails or cannot obtain the depth information of the grain area in the depth image due to problems such as excessive light, large dust, and mutual shielding of grain piles, the second image acquisition module below the unloading cylinder is enabled.

[0090] The step S2 of extracting the grain area in the HSV color space specifically includes the following steps:

[0091] Step S2.1: Preprocess the image, obtain the grain tank area in the gray image by threshold segmentation, and extract the grain tank contour by edge detection;

[0092] Step S2.2: For each pixel in the image, perform pixel value conversion in the HSV color space;

[0093] Step S2.3: The difference between the grain and the grain tank area is the largest in the H space of the HSV color space, and the grain area is obtained by threshold segmentation;

[0094] Step S2.4: Convert the image pixel coordinate system to the world coordinate system for the depth image, reconstruct the elevation image, and obtain the depth value of the grain area.

[0095] The pixel value conversion in the HSV color space in step S2.2 is as follows:

[0096]

[0097]

[0098] v=max

[0099] Wherein, r, g, b are the pixel channel values in the RGB color space,

[0100] max is the maximum of r, g and b,

[0101] min is the minimum of r, g and b,

[0102] h, s, v are the pixel channel values in the HSV color space.

[0103] In combination with Figure 4 and 5 The step S3 of dividing the grain tank into multiple unloading areas and setting different loading level standard values specifically includes the following steps:

[0104] Step S3.1: Divide the grain tank into multiple unloading areas according to the shape and size of the grain tank;

[0105] Step S3.2: Different regions set different height standard values, the standard values are related to the grain repose angle and the distance from the center point of the region to the center point of the highest region, and the height standard values decrease from the center of the grain tank outward, and more grain is loaded under the premise that the grain tank does not overflow;

[0106] Step S3.3: The height standard values of different regions are converted into loading level standard values based on the upper plane of the grain tank mouth;

[0107] C t = (H t -H1) × S n

[0108] wherein C t is the loading level standard value of the region,

[0109] H1 is the height of the grain tank,

[0110] S n is the area of the region.

[0111] The size of the height standard value in step S3.2 is calculated according to the following formula:

[0112]

[0113]

[0114] wherein H max is the height standard value of the highest region,

[0115] H min is the height standard value of the lowest region,

[0116] H t is the height standard value of different regions,

[0117] W1 is the length of the short side of the grain tank,

[0118] ΔW t is the distance from the center point of the different region to the center point of the highest region,

[0119] α is the grain repose angle.

[0120] The step S4 of judging whether there is a gap in the loading of the grain tank specifically includes the following steps:

[0121] Step S4.1: Due to the shielding of the grain flow at the unloading port, the grain height detection in the region being unloaded is not accurate, and the region being unloaded is temporarily ignored, and other regions except the region being unloaded are detected;

[0122] Step S4.2: Taking the upper plane of the grain tank opening as the reference, the height of the grain above the upper plane of the grain tank opening is positive, and the height below the plane is negative. The height difference between the grain surface of each region and the upper plane of the grain tank opening is converted into the loading level, and the calculation formula of the loading level is as follows:

[0123] C n = ΔH n × S n

[0124] Wherein, C n is the loading level of the region,

[0125] ΔH n is the height difference between the center grain surface of the region and the upper plane of the grain tank opening,

[0126] S n is the area of the region;

[0127] Step S4.3: The void volume of the region is obtained by the loading level value and the standard value of different regions. If the total void volume of adjacent regions is greater than the set threshold value, it is proved that there is a void in the region, and the calculation formula of the void volume is as follows:

[0128] ΔV = ∑ΔC i = ∑(C t -C n )

[0129] Wherein, ΔC i is the void volume of the region,

[0130] ΔV is the total void volume of adjacent regions.

[0131] The step S5 identifies the voids from the center of the grain tank in order of distance, until the loading level of each region reaches the loading level standard value, which indicates that the grain tank is full. The grain tank loading state information includes the void volume ΔV r , the center position (X r , Y r ) of the void, and the grain tank loading rate La.

[0132]

[0133] Va = L × W × H + ∑C t

[0134] Wherein, ∑ΔV r is the total void volume of the grain tank, Va is the volume of the grain in the grain tank when full, L, W, H are the length, width and height of the grain tank, and ∑C t is the sum of the loading level standard values of all regions of the grain tank.

[0135] In combination Figure 6As shown in the drawings, wherein Figure 6 (a) is the unloading point area detection diagram of the loading amount 10%, 6(b) is the unloading point area detection diagram of the loading amount 40%, 6(c) is the unloading point area detection diagram of the loading amount 60%, and 6(d) is the unloading point area detection diagram of the loading amount 80%.

[0136] The present application obtains the grain tank image of the grain transport vehicle through the binocular camera; converts the image into the HSV color space, extracts the grain area in the grain tank; divides the grain tank into multiple unloading areas according to the shape and size of the grain tank, sets different loading level value standard points for different areas based on the upper plane of the grain tank mouth; extracts the upper plane of the grain tank mouth in the depth image, obtains the distance from the upper surface of the grain in each area center point to the upper plane of the grain tank mouth, and proves that there is a gap in the area if the total gap volume of the adjacent areas is greater than a predetermined value; identifies the gap in turn from the center of the grain tank according to the distance, and thus obtains the grain tank loading state information. The present application can accurately detect the grain tank loading state of the grain transport vehicle in real time, solves the problems of time-consuming and laborious manual unloading, low utilization rate of the grain tank and the like caused by the unknown grain tank loading state of the grain transport vehicle during the unloading of the harvester, and can provide basic data support for the unloading control of the harvester during the collaborative operation of the grain transport vehicle and the harvester.

[0137] Example 2

[0138] A system for realizing the online detection method of the grain tank loading state of the grain transport vehicle according to example 1, comprising an image acquisition module, an image processing module and a grain tank loading state analysis module;

[0139] The image acquisition module is used to acquire the RGB color image and the depth image of the left upper side and the front upper side of the grain tank of the grain transport vehicle;

[0140] The image processing module is used to extract the grain area in the HSV color space in the images acquired at the two positions and obtain the depth value of the grain area through the reconstruction of the elevation image;

[0141] The grain tank state analysis module is used for dividing the grain tank into multiple unloading areas and setting different loading level value set points, and then calculating the loading level through the depth information, and finding the area with gaps through the different area loading level values and areas, so as to detect the loading state of the grain tank of the grain transport vehicle. Specifically, the grain tank state analysis module is used for dividing the grain tank 5 into multiple unloading areas according to the shape and size of the grain tank 5, setting different loading level value standard points for different areas based on the upper plane of the grain tank opening, extracting the upper plane of the grain tank opening in the depth image, obtaining the distance from the upper surface of the grain in each area center point to the upper plane of the grain tank opening, finding the area with gaps through the different area loading level values and standard values, and proving that the area has gaps if the total gap volume of adjacent areas is greater than a predetermined value; and then identifying the gaps in turn from the grain tank center according to the distance, so as to obtain the grain tank loading state information, which can be sent to the controller through the can bus to provide basic data support for the unloading control of the harvester, and realize more efficient unloading.

[0142] The system of the online detection method for the loading state of the grain tank of the grain transport vehicle further comprises a camera selection module; the camera selection module is used for enabling the second binocular camera 2 when the image processing module cannot collect effective image data of the first binocular camera 1.

[0143] The application obtains the image of the grain tank 5 of the grain transport vehicle 4 through the binocular camera, identifies the gaps in turn from the grain tank 5 center according to the distance after extracting the grain area in the grain tank 5, so as to obtain the grain tank loading state information, and proves that the surface grain tank is full when the loading level of each area reaches the loading level value standard point.

[0144] The method has high detection precision and fast detection speed, can solve the problems of time-consuming and labor-intensive manual unloading and low utilization rate of the grain tank caused by the unknown loading state of the grain tank of the grain transport vehicle during the unloading of the harvester, and can provide basic data support for the collaborative operation of the grain transport vehicle and the harvester.

[0145] It should be understood that although the present specification is described in terms of various embodiments, not every embodiment contains only one independent technical solution, and the description manner of the specification is only for the sake of clarity, and those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be combined to form other implementation manners that can be understood by those skilled in the art.

[0146] The series of detailed descriptions listed above are only specific descriptions of the feasible embodiments of the present application, and are not used to limit the protection scope of the present application, and any equivalent embodiments or changes made without departing from the spirit of the present application should be included in the protection scope of the present application.

Claims

1. An online detection method for loading state of a grain box of a grain cart, characterized in that, The method comprises the following steps: Step S1, acquiring a grain tank image of a grain cart by using a binocular camera; Step S2, converting the grain tank image acquired by the binocular camera into an HSV color space, and extracting a grain region in the grain tank in the HSV color space; Step S3, dividing the grain tank into multiple unloading regions, and setting different loading level standard points for different regions based on the upper plane of the grain tank opening; Step S4, extracting the upper plane of the grain tank opening in the depth image, obtaining the distance from the upper surface of the grain in each region center point to the upper plane of the grain tank opening, and proving that there is a gap in the region if the total gap volume of adjacent regions is greater than a predetermined value; Step S5, identifying the gaps in order of distance from the center of the grain tank, and thus obtaining the loading state information of the grain tank; The step S3 of dividing the grain tank (5) into multiple unloading regions and setting different loading level standard values comprises the following steps: Step S3.1, dividing the grain tank (5) into multiple unloading regions; Step S3.2, setting different height standard values for different regions, the standard value being related to the grain rest angle and the distance from the center point of the region to the center point of the highest region, and the height standard value decreasing from the center of the grain tank to the outside; Step S3.3, converting the height standard values of different regions into loading level standard values based on the upper plane of the grain tank opening; ； wherein, is a loading level standard value for the area, is a grain bin height, is an area of the area; The size calculation formula of the height standard value in the step S3.2 is as follows: ； ； wherein is the height standard value of the highest region, is the height standard value of the lowest region, is the height standard value of the different region, is the length of the short side of the grain tank, is the distance of the center point of the different region to the center point of the highest region, is the angle of repose of the grain. The step S4 of judging whether there is a gap in the loading of the grain tank (5) comprises the following steps: Step S4.1, detecting other regions except the region being unloaded; Step S4.2, converting the height difference between the grain surface of each region and the upper plane of the grain tank opening into a loading level based on the upper plane of the grain tank opening, the grain height being higher than the upper plane of the grain tank opening being a positive value, and the grain height being lower than the upper plane being a negative value, and the calculation formula of the loading level being as follows: ； wherein, is the loading level of the area, is the height difference between the central grain surface of the area and the upper plane of the grain tank opening, is the area of the area; Step S4.3, obtaining the gap volume of the region by using the loading level value and the standard value of different regions, proving that there is a gap in the region if the total gap volume of adjacent regions is greater than a predetermined threshold value, and the calculation formula of the gap volume being as follows: ； wherein, is the void volume of the region, is the total void volume of the adjacent region.

2. The method according to claim 1, wherein The step S1 of acquiring the grain tank image by using the binocular camera comprises the following steps: Step S1.1, the binocular camera comprises a first binocular camera (1) and a second binocular camera (2), the first binocular camera (1) is installed on the front side wall of the grain tank (5) of the grain cart, the second binocular camera (2) is installed below the unloading cylinder (6) of the harvester, the angle of the camera is adjusted, the first binocular camera (1) acquires the RGB color image and the depth image above the front side of the grain tank (5) of the grain cart, and the second binocular camera (2) acquires the RGB color image and the depth image above the left side of the grain tank (5) of the grain cart; Step S1.2, taking the first binocular camera (1) on the front side wall of the grain tank (5) of the grain cart as a first image acquisition module, taking the second binocular camera (2) below the unloading cylinder (6) as a second image acquisition module, if the first binocular camera (1) fails to acquire the color image and the depth image, or the acquired image data is unqualified, and the depth information of the grain region cannot be acquired, then the second image acquisition module below the unloading cylinder (6) is enabled.

3. The method according to claim 1, wherein, The step S2 of extracting the grain region in the HSV color space specifically comprises the following steps: Step S2.1: pre-processing the image, obtaining the grain tank region in the gray image by threshold segmentation, and extracting the grain tank contour by edge detection; Step S2.2: performing pixel value conversion in the HSV color space for each pixel in the image; Step S2.3: the grain and the grain tank region are most different in the H space of the HSV color space, and the grain region is obtained by threshold segmentation; Step S2.4: converting the image pixel coordinate system to the world coordinate system for the depth image, reconstructing the elevation image, and obtaining the depth value of the grain region.

4. The method according to claim 3, wherein, The pixel value conversion in the HSV color space in the step S2.2 is as follows: ； Wherein, r, g, b are pixel channel values in the RGB color space, max is the maximum of r, g and b, min is the minimum of r, g and b, h, s, v are pixel channel values in the HSV color space.

5. The method according to claim 1, wherein, The step S5 identifies the gaps in order of distance from the center of the grain tank (5) until the loading level of each area reaches the loading level standard value, which indicates that the grain tank is full. The grain tank loading state information includes the gap volume of each gap , the center position of the gap , and the grain tank loading rate , ； ； wherein, Vtot is the total void volume of the grain tank, Vgrain is the volume of grain in the tank at full load, , , LWH is the length, width and height of the grain tank, Vsum is the sum of the load level standard values for all areas of the grain tank.

6. A system for implementing the online detection method of the loading state of the grain box of the grain cart according to any one of claims 1-5, characterized in that, The system comprises an image acquisition module, an image processing module and a grain tank loading state analysis module. The image acquisition module is used to acquire the RGB color image and the depth image of the left upper side and the front upper side of the grain tank of the grain cart. The image processing module is used to extract the grain region in the HSV color space in the images collected at the two positions and obtain the depth value of the grain region by reconstructing the elevation image. The grain tank state analysis module is used to divide the grain tank into multiple grain unloading regions, set different loading level value standard points for different regions based on the upper plane of the grain tank opening, extract the upper plane of the grain tank opening in the depth image, obtain the distance from the upper surface of the grain in the center point of each region to the upper plane of the grain tank opening, find the region with gaps by the loading level value and the standard value of different regions, and prove that the region has gaps if the total gap volume of adjacent regions is greater than a predetermined value; and identify the gaps in order from the center of the grain tank according to the distance, so as to obtain the grain tank loading state information.

7. The system for online detection of the loading state of the food box of the food cart according to claim 6, characterized in that, The system further comprises a camera selection module; the camera selection module is used to enable the second binocular camera (2) when the image processing module cannot acquire the color image and the depth image by the first binocular camera (1), or the image data collected is unqualified, and the depth information of the grain region cannot be acquired.

Images