Grain level grain monitoring system for a combine harvester

Abstract

Embodiments of a grain level grain monitoring system include a grain camera positioned to capture a bulk grain sample image of currently harvested grain being taken into and processed by a combine harvester, a moisture sensor, and a display device. A controller architecture is coupled to the grain camera, to the moisture sensor, and to the display device. The controller architecture is configured to: (i) analyze the bulk grain sample image received from the grain camera to determine an average per kernel (APK) volume representative of an estimated volume of a single average kernel of the currently harvested grain; (ii) repeatedly calculate one or more top line harvesting parameters based at least in part on the determined APK volume and the moisture sensor data; and (iii) selectively present the top line harvesting parameters on the display device for viewing by an operator of the combine harvester.

Description

[0001] Cross-references to related applications

[0002] not applicable.

[0003] Federally funded research or development statement

[0004] not applicable. Technical Field

[0005] This disclosure relates to grain-level grain monitoring systems that use images of stockpiled grain samples and other data to monitor average grain parameters associated with grain harvested and processed by combine harvesters. Background Technology

[0006] Combine harvesters (also known as “agricultural combine harvesters”) greatly improve the efficiency of harvesting, threshing, cleaning, and collecting corn, canola, soybeans, wheat, oats, sunflowers, and other crops for distribution to consumers. Generally, a combine harvester is a relatively complex, self-propelled machine capable of harvesting large stalks of crop plants while simultaneously separating uncrushed grain from crushed grain and material other than grain (MOG) as it travels through a crop field. After cleaning, the currently harvested stockpile of grain is delivered as a “grain stream” into a grain bin (usually by conveying it through a clean grain elevator). Modern combine harvesters typically include a relatively complex sensor system capable of providing real-time estimates of grain quality, moisture content, test weight, and other grain quality parameters associated with the grain harvested and processed by a given combine harvester. Additionally, at least one grain camera can be used to monitor grain quality; this at least one grain camera is positioned to capture images of the stockpile of grain transported via the clean grain elevator (referred to herein as “stockpile grain sample images”). Some combine harvester systems also provide visualization tools to help operators assess grain quality when viewing grain quality image samples on the display screen, such as visually distinguishing MOGs and broken grains from clean, unbroken grains within the grain quality image sample by color coding or other means. Summary of the Invention

[0007] A grain-level grain monitoring system for use on a combine harvester is disclosed. In an embodiment, the grain-level grain monitoring system includes: a grain camera positioned to capture an image of a stockpiled grain sample acquired and processed by the combine harvester; a moisture sensor configured to generate moisture sensor data indicating the moisture level of the currently harvested grain; and a display device having a display screen on which parameters related to the currently harvested grain are selectively presented. A controller architecture is coupled to the grain camera, the moisture sensor, and the display device. The controller architecture is configured to: (i) analyze the stockpiled grain sample image received from the grain camera to determine an average volume per grain (APK) representing the estimated volume of a single average grain of the currently harvested grain; (ii) repeatedly calculate one or more topline harvesting parameters based at least in part on the determined APK volume and the moisture sensor data; and (iii) selectively present the topline harvesting parameters on the display device for view by the combine harvester operator.

[0008] In another embodiment, the grain-level grain monitoring system includes: a grain camera positioned to capture an image of a stockpiled grain sample acquired and processed by a combine harvester; a moisture sensor configured to generate moisture sensor data indicating the moisture level of the currently harvested grain; and a display device having a display screen on which parameters related to the currently harvested grain are selectively presented. A controller architecture is coupled to the grain camera, the moisture sensor, and the display device. The controller architecture is configured to analyze the stockpiled grain sample image received from the grain camera to determine an average per grain (APK) parameter; and to determine, at least in part, a target setting adjustment for the actuated harvesting component of the combine harvester based on the APK parameter. Additionally, the controller architecture performs at least one of the following actions: (i) generating a notification prompting the operator to implement the target setting adjustment, such as a visual and / or auditory notification; and (ii) controlling the actuated harvesting component to automatically implement the target setting adjustment.

[0009] Details of one or more embodiments are set forth in the accompanying drawings and the following description. Other features and advantages will become apparent from the specification, drawings, and claims. Attached Figure Description

[0010] At least one example of this disclosure will be described below with reference to the following figures:

[0011] Figure 1This is a schematic diagram of a combine harvester equipped with a grain-level grain monitoring system, as shown in an example embodiment, which is configured to monitor a specific average per grain (APK) parameter of the grain stream processed by the combine harvester.

[0012] Figure 2 Additional components appropriately included in an embodiment of the example grain-level grain monitoring system are illustrated schematically;

[0013] Figure 3 It is a grain-level grain monitoring system ( Figure 1 and Figure 2 A flowchart illustrating an example process of appropriately implementing a controller architecture to monitor APK parameters related to the stacked grain being processed by a combine harvester and to perform other associated actions;

[0014] Figure 4 This schematically illustrates an example subprocess potentially performed by the controller architecture of a grain-level grain monitoring system to assess APK volume and / or APK morphology through dimensional image analysis of stacked grain sample images; and

[0015] Figure 5 This is a screenshot of an example graphical user interface (GUI) screen that is generated appropriately by the grain-level grain monitoring system on the display device of the combine harvester and visually transmits instantaneous values ​​of APK parameters, top-line harvesting measurements that consume APK parameters as input, and any combination of suggested combine harvester settings adjustments determined using APK parameters.

[0016] The same reference numerals in different figures indicate the same elements. For the sake of simplicity and clarity, descriptions and details of well-known features and techniques may be omitted to avoid unnecessarily obscuring the exemplary and non-limiting embodiments of the invention described in the following detailed description. It should also be understood that, unless otherwise stated, features or elements appearing in the figures are not necessarily drawn to scale. Detailed Implementation

[0017] Embodiments of this disclosure are illustrated in the accompanying drawings, which are briefly described above. As set forth in the appended claims, various modifications to the exemplary embodiments may be conceived by those skilled in the art without departing from the scope of the invention.

[0018] As appears throughout this document, the term "grain" is used in a broad or general sense to encompass all seeds and the edible portions of other grain crops harvested and processed by combine harvesters. By definition, the term "grain" includes the seeds of herbivorous crops (such as corn, oats, and wheat only); soybean and other legume seeds; and seeds of other plant types, including buckwheat, millet, quinoa, rapeseed, rice, and sunflower seeds. In contrast, the terms "cereal" and "staple grain" are used interchangeably to refer to large quantities of grain, particularly large quantities of grain processed and temporarily stored by combine harvesters or other harvesting machinery. Finally, the term "grain flow" refers to a mass of stacked grain (typically containing varying amounts of other crop material) in motion as it is transported within a combine harvester from one location to another.

[0019] Overview

[0020] As mentioned above, combine harvesters are typically equipped with relatively sophisticated sensor systems capable of real-time monitoring of grain quality during active harvesting; that is, as the combine harvester travels across large fields, it ingests harvested crop stalks, processes (threshes and cleans) the crop stalks to generate a stockpile of grain, and ultimately directs the stockpile of grain into a grain bin for temporary storage. To aid in grain quality assessment, the combine harvester may include at least one grain camera positioned adjacent to a clean grain elevator and capturing real-time images of the grain stream transported by the clean grain elevator to the grain storage bin. Furthermore, visualization tools have been developed that allow combine harvester operators not only to view the real-time camera images captured by such grain cameras but also to further enhance the real-time camera images using color coding (or similar visual enhancements) to help the operator visually distinguish between clean, unbroken grain and unwanted materials (e.g., foreign objects and broken grain) carried in the stockpile of grain. A similar method can also be used in conjunction with a stubble camera, which is positioned to capture images of stubble discharged from the rear of the combine harvester and can be effectively viewed by the operator when assessing grain loss.

[0021] By providing this visualization tool, combine harvester operators can gain a sense of increased grain quality at a given point in time as the combine harvester travels across different areas of the field and operates according to current settings. This, in turn, allows operators to better drive a given combine harvester by, for example, adjusting combine harvester settings to minimize grain loss, improve grain quality, or otherwise enhance the overall performance of combine harvester operation. In addition to providing this visualization tool, the combine harvester system also monitors certain fundamental or key harvesting parameters related to combine harvester efficiency and harvesting productivity levels. These parameters are referred to herein as “topline harvesting metrics” and include grain yield, grain loss, test weight, estimated grain mass flow rate, grain composition information, and other parameters. Generally, these topline harvesting metrics are calculated using data inputs from multiple sensors integrated into a given combine harvester and data received from other sources (such as operator input), or possibly from backend services (such as telematics gateway modules) via wireless data links. Relevant sensor data used to calculate top-line harvesting metrics can include data collected from one or more moisture sensors on the combine harvester, grain loss sensors (e.g., impact plate sensors located at the rear end of the cleaning chamber and / or at the outlet end of the combine harvester's clean grain lift), mass flow rate sensors, grain bin weighing sensors, and grain lift speed sensors (to name just a few). This top-line harvesting metric can be repeatedly calculated or evaluated in real-time or near real-time, and the instantaneous value of the top-line harvesting metric can be displayed for the combine harvester operator to consider. Additionally, this top-line harvesting metric can be compiled with other data (e.g., time and Global Positioning System (GPS) data) and stored as a historical dataset to, for example, generate yield maps and similar data-driven resources, thereby aiding in the optimization of seeding and planting operations. Finally, in some cases, the combine harvester system can utilize such data input to determine and automatically implement targeted adjustments to actuated harvesting components on a given combine harvester, such as the cleaning chamber, screens, fans, and chaff screens.

[0022] In this manner, when operating combine harvesters in dynamic environments, the grain monitoring system integrated into modern combine harvesters provides the operator with basic real-time information, thus aiding in decision-making. Additionally, some existing combine harvester automation systems further enhance combine harvester performance by automatically and instantly fine-tuning combine harvester settings. Nevertheless, topline harvesting metrics and other datasets calculated by current combine harvester display and automation systems can introduce a relatively high degree of inaccuracy. This may be partly due to the use of certain standardized (assumed) values ​​for some key input parameters or crop attributes used in calculating topline harvesting metrics. For example, existing combine harvester display and automation systems may utilize predefined assumptions about grain weight and size when calculating key crop attributes, such as a predefined (assumed) test weight of 1000 grains; that is, the weight of 1000 grains when filled into 1 volume bushel (equivalent to 1.244 cubic feet or 0.035 cubic meters). However, grain weight and size can vary significantly with changing crop conditions. Therefore, reliance on such hypothetical or generalized data inputs can introduce undesirable inaccuracies when calculating top-line harvesting measurements and other data parameters. Overall, this inaccuracy can reduce the overall quality of the information presented to combine harvester operators, and simultaneously decrease combine harvester performance when real-time adjustments to combine harvester settings are automatically implemented based on this information.

[0023] Therefore, there is a persistent industry demand for combine harvester systems that can improve the accuracy and consistency of relevant data parameters calculated during combine harvester operation, particularly top-line harvesting metrics. Ideally, such a combine harvester system would improve the quality and consistency of data calculations while minimizing implementation costs and simplifying customer adoption by leveraging existing sensor arrays on the combine harvester. This paper provides an embodiment of such a combine harvester system, referred to as a “grain-level grain monitoring system.” As the descriptor “grain-level” indicates, a grain-level grain monitoring system monitors certain unique data parameters considered on a per-grain average basis; that is, such that a given data parameter generally (averagely or moderately) describes the individual grains within the grain stream consisting of the current stack of grain harvested by the combine harvester. This per-grain parameter is called the per-grain average parameter or “APK parameter,” indicating that the APK parameter represents the average or estimated value of the grains contained in a given stack of grain generated when the combine harvester processes crop plants during active harvesting. In embodiments, APK parameters can be extrapolated or otherwise utilized to better determine topline harvesting metrics, such as grain loss, grain yield, test weight, grain composition, and mass flow rate estimates calculated during combine harvester operation. This, in turn, can improve the quality of information displayed to the combine harvester operator (or stored for subsequent analysis); and, where applicable, improve the accuracy of target adjustment settings for the actuated components of the combine harvester determined by the combine harvester system and potentially implemented automatically.

[0024] Generally, embodiments of grain-level grain monitoring systems utilize images captured from at least one grain camera, along with other relevant data inputs, to track one or more of the following APK parameters: (i) APK volume, (ii) APK density, (iii) APK weight or mass, and (iv) APK morphology. As appears herein, the terms “APK volume,” “APK density,” and “APK weight (or mass)” represent estimated per-grain averages of the volume, density, and weight (or mass) of grains contained in a grain stream generated during combine harvester operation. In contrast, the term “APK morphology” refers to the average grain shape (including angularity) and size within the accumulated grain stream being analyzed. For example, in embodiments, APK morphology can be expressed in terms of classification or categorization (such as large spherical classification or small rectangular classification) based on the generalized three-dimensional shape (and possible size) of the grains contained in the grain stream. Additional descriptions of this aspect are provided below. Figure 3 supply.

[0025] Regarding APK volume, the grain-level grain monitoring system can monitor this APK parameter through three-dimensional image analysis of images of stacked grain samples captured by a grain camera; and specifically, by estimating and averaging certain size parameters (e.g., average length and diameter) of the detected grains in the stacked grain sample images. In embodiments, additional data, such as image depth data (if available) and any available morphological information related to the currently harvested grain, may also be considered when calculating the estimated APK volume. In the latter aspect, the combine harvester operator can input data specifying the grain type of the currently harvested grain into the grain-level grain monitoring system, which can then be used to establish grain shape categories or classifications; for example, linear, rectangular, elliptical, oval, spherical, or other three-dimensional grain shape categories. This information can then be taken into account by the grain-level grain monitoring system when evaluating the APK volume of the grains depicted in the stacked grain sample images. Additionally or alternatively, whether performed on a combine harvester or by a remote data source (e.g., a back-end service) that wirelessly communicates with the grain-level grain monitoring system via a network outside the combine harvester, the grain-level grain monitoring system can establish the APK morphology of the stacked grains through visual analysis of the images of the stacked grain samples themselves.

[0026] Embodiments of grain-level grain monitoring systems also utilize APK volume as input to usefully (though not essentially) monitor the APK density and / or APK weight (or mass) of grains within a stockpiled grain stream. In the case of APK density, the grain-level grain monitoring system can determine this parameter based on the APK volume and a suitable stockpile density parameter associated with the stockpiled grain stream. In embodiments, the stockpile density parameter can be input by the operator or retrieved from memory as a standardized value associated with a specific crop type, based on historical (e.g., georeferenced) data or other input data. However, more usefully, stockpile density measurements can be obtained using data from onboard sensors (such as grain moisture sensors) typically integrated into modern combine harvesters. In the latter case, existing moisture sensors monitor data based on the stockpile density of a sampled grain volume or data determined from capacitance-based moisture level readings. For example, in the case of capacitive moisture sensors, a portion of the stockpiled grain stream can be guided into or through a space of known volume defined by electrodes. The capacitance of the sampled volume of the grain stream is measured and converted into the bulk gravimetric density of the grain stream (which is typically expressed in weight per volume (lb / bushel)). Traditionally, this bulk gravimetric density measurement has been used for purposes other than those described herein; for example, when generating grain yield maps, mass or weight yield estimates (typically expressed in mass or weight per unit area) are converted into volume yield estimates (typically expressed in volume bushels per acre (i.e., bu / ac)). In addition to the aforementioned APK volume parameter, embodiments of grain-level grain monitoring systems can utilize the availability of this sensor data (or other sensor data indicating the bulk density measurement of the currently harvested grain) to calculate and track the APK density of the currently harvested grain in real time. Similarly, the APK weight (or mass) can be readily calculated based on APK volume and bulk weight (or mass) parameters obtained from sensor data on the combine harvester (such as impact plate sensors positioned at the exit end of the clean grain elevator, weighing sensors located inside the grain bin, or other sensors from which a bulk weight (or mass) measurement related to the currently harvested grain can be derived).

[0027] Embodiments of grain-level grain monitoring systems can utilize any or all of the aforementioned APK parameters to better obtain top-line harvesting metrics (improving their accuracy). When calculating grain loss, for example, using impact plate sensors placed at specific locations within a combine harvester (typically in the combine harvester's cleaning chamber and separator sections), the grain loss estimate can be improved by considering the APK mass or weight parameter. Specifically, in at least some embodiments, the APK weight can be scaled up to 1000 grain weight values, which can be used instead of the standardized 1000 grain weight values ​​used by existing automated systems and known to cause errors approaching or exceeding 35% in grain loss estimation calculations. By using 1000 grain weight values ​​(or other weight or mass parameters based on APK parameters) that better reflect "ground conditions," grain loss parameters can be calculated with greater accuracy and consistency. This, in turn, can improve grain loss mapping and grain loss performance target calibration processes by reducing field-to-field variability.

[0028] Therefore, contrary to the traditional practice of selecting from a limited number of size categories using standard (assumed) values ​​or operator input, the aforementioned APK morphology, particularly grain size, can also be considered to improve the accuracy of grain size classification. Similarly, depending on relevant factors (e.g., whether the mass flow rate is measured by measuring the impact of a plate positioned at the outlet end of a grain cleaning elevator (among other factors such as elevator belt speed), by using a grain camera to estimate the rate at which grain is transported through the grain cleaning elevator, or by other means), the APK weight and / or APK density values ​​calculated by a grain-level grain monitoring system can be used to improve mass flow rate measurement. In some cases, APK morphology can also provide a better understanding of the angle of repose (indicating the height of the grain pile), to improve the calibration of mass flow rate sensors, for example, by using data provided by grain bin weighing sensors, as combined below. Figure 3 As discussed, dry yield estimates, typically calculated using mass flow rate and other factors (e.g., header width, combine harvester ground speed or rate, and moisture data), can also be improved with improvements in mass flow rate estimates. Additionally, APK morphology (particularly size) and APK weight can allow seed producers and growers to improve their products and services from an agronomic perspective. Finally, when APK parameters are extrapolated or scaled up to specific, commonly used stacked grain parameters (such as test weight), grain filling indicators or conversion factors can be identified and considered to compensate for the air void volume in the space filled with a given volume of stacked grain currently harvested by the combine harvester.

[0029] Any combination of the APK parameters and top-line harvesting metrics mentioned above (e.g., depending on operator navigation between different graphical user interface (GUI) screens or pages displaying this information) can be presented on a display screen in the combine harvester cab for operator viewing. This information can also be stored and recalled at a later date for post-harvest analysis, such as enabling operators and growers to view yield maps, grain loss maps, and similar agronomic analysis tools that can be used to improve procedures and practices related to the field preparation, sowing, maintenance (treatment), and harvesting phases of planting operations. Additionally, this information can be used to determine target adjustments to combine harvester settings and: (i) prompt the operator to implement the determined target adjustments (e.g., via visual cues generated on a display device in the combine harvester cab, via auditory cues, or a combination thereof), or (ii) automatically implement target adjustments to the actuated harvesting components or equipment on the combine harvester in an automatic, immediate manner. In the latter aspect, in at least some embodiments, the grain-level grain monitoring system can determine and automatically implement adjustments to the positioning of the cleaning chamber, screen positioning, husk screen positioning, and fan speed and / or similar combine harvester positioning. As a specific but non-limiting example, considering the decreasing trend of heavier grains being carried away by airflow and discharged from the combine harvester, relatively fine adjustments can be made to the fan speed, for example, increasing the fan speed in conjunction with increasing the weight of the APK. Furthermore, in some cases, corresponding adjustments can also be made to the positioning of the cleaning chamber, screen, and / or husk screen to promote airflow in conjunction with increasing fan speed.

[0030] An example of a grain-level grain monitoring system will now be described in the context of an example combine harvester, as follows: Figure 1 and Figure 2 As shown and discussed. Additionally, the following is combined... Figure 3 and Figure 4 This paper describes an example method or process for appropriately implementing a controller architecture of a grain-level grain monitoring system to monitor multiple APK parameters, display one or more topline harvest metrics calculated using the APK parameters, and perform other actions. Finally, the following section combines... Figure 5 Examples are discussed of ways to display APK parameters to the combine harvester operator, calculate top-line harvesting measurements using the APK parameters, and determine recommended combine harvester setting adjustments using the APK parameters. The following description is provided by way of non-limiting illustration only and should not be construed as unduly limiting the scope of the appended claims in any way.

[0031] Combine harvesters equipped with an example grain-level grain monitoring system

[0032] refer to Figure 1According to an exemplary embodiment of this disclosure, a combine harvester 10 equipped with a grain-level grain monitoring system 12 is schematically depicted. The combine harvester 10 is presented in an illustrated manner to establish a non-limiting exemplary scenario in which an embodiment of the grain-level grain monitoring system 12 can be better understood. In other embodiments, the combine harvester 10 may take other forms and may include different combinations of components suitable for handling crop plants ingested into the harvester 10 while traveling on the field 14 during active harvesting. Further, for clarity, Figure 1 Only selected components of the grain-level grain monitoring system 12, such as the controller architecture 16, are shown below. (The following is in conjunction with...) Figures 2 to 5 Further descriptions and discussions are provided for the example grain-level grain monitoring system 12.

[0033] The example combine harvester 10 includes a chassis body or main frame 18 supported by a plurality of ground-jointing wheels 20. The ground-jointing wheels 20 are powered by an engine and drivetrain (including, for example, an electro-hydraulic transmission) not shown. On top of the forward portion of the main frame 18, a cab 22 encloses an operator station, which includes an operator seat (not shown), at least one display device 24, and an operator interface 26. A feed chamber 28 is mounted to the forward portion of the main frame 18 of the combine harvester 10 at a height generally lower than the cab 22. Various harvesting heads, or more simply “cutting heads,” are attached to the feed chamber 28 in an interchangeable manner to, for example, allow for customization of the combine harvester 10 for harvesting specific crop types. Figure 1 An example of such a cutting platform is shown, here it is harvesting platform 30.

[0034] As the combine harvester 10 travels forward over the field 14, the harvesting platform 30 collects the chopped crop stalks into a feed chamber 28, which then combines the chopped crop stalks for conveying (e.g., via a conveyor belt, not shown, contained within the feed chamber 28) into the interior of the combine harvester 10. Inside the combine harvester 10, the crop stalks are engaged by a rotating roller conveyor or "threshing drum" 32, which guides the crop stalks in a generally upward direction into a rotary threshing and separating section 34. The rotary threshing and separating section 34 may include various components for performing the desired function of separating grain and chaff from other plant material. The illustrated rotary threshing and separating section 34 includes, for example, a rotor or drum 36 having threshing characteristics and rotatably mounted within a housing or rotor housing 38. The rotation of the threshing drum 36 within the rotor housing 38 causes both grain and chaff to fall through the separating grid of the concave plate 40 and enter the inlet of the lower grain cleaning section 42. Simultaneously, straw and similar MOGs are guided toward the outlet end 44 of the rotary threshing and separating section 34 and are ultimately delivered to another rotary drum or "discharge threshing drum" 46 for discharge from the rear end of the combine harvester 10.

[0035] Now discussing the grain cleaning section 42 in more detail, this section of the combine harvester 10 includes various components suitable for cleaning newly harvested grain while separating the chaff from it. These components may include a chaff screen 48, a screen 50, and any number of fans, such as Figure 2 The centrifugal blower fan or impeller 78 is shown. Through the action of the grain cleaning section 42, newly cleaned grain is guided into the clean grain elevator 52 for upward transport to the storage container or grain bin 54 of the combine harvester 10. The path of the clean grain from the grain cleaning section 42 to the grain bin 54 is referred to herein as the “clean grain flow path,” and the grain traveling along this flow path is generally referred to as the “clean grain stream.” Generally, the grain and MOG traveling through the combine harvester 10 are more broadly referred to herein as the “grain stream.” Thus, the “clean grain stream” is a segment of the larger grain stream after threshing, separating, and cleaning. At least one camera 56 is positioned to capture images of the grain transported along the grain stream, and specifically in this embodiment as the “stacked grain sample image”. In this respect, and as... Figure 1As shown, the grain camera 56 can be positioned close to the clean grain elevator 52 to capture images of the stacked grain flow transported via the elevator 52 into the grain bin 54. In other embodiments, one or more additional cameras can be positioned at different locations along the clean grain flow (e.g., at two or more locations to capture images of the clean stacked grain transported via the elevator 52), while the image data is processed and averaged or otherwise mixed to produce the APK parameters described below. In embodiments, the grain camera 56 or associated sensor system can also provide depth data of the grains within the stacked grain sample image that can be used to evaluate the size of the grains. When applicable, this depth data can take the form of direct transducer-based depth measurements (e.g., captured by acoustic sensors, radar-based sensors, or similar sensors that detect reflected energy or sonar pulses), as well as other image data from which depth can be inferred (e.g., captured by a stereo camera assembly).

[0036] As the clean grain elevator 52 transports the newly harvested grain into the grain bin 54, the ears of grain fall onto the return elevator 58, which extends through the lower portion of the clean grain elevator 52. The return elevator 58 then recirculates the ears of grain back to the inlet of the threshing drum 36 for further threshing, allowing the grain processing steps described above to be repeated and maximizing the grain yield of the combine harvester 10. In this way, the combine harvester 10 efficiently picks up chopped crop stalks from the field 14, extracts the grain from the crop stalks, cleans the newly extracted grain, and then stores the grain in the grain bin 54 for subsequent unloading using, for example, an unloading screw conveyor 60. Moreover, during the use of the combine harvester 10, certain components within the combine harvester 10 can be adjusted in position, or any number of actuators 62 (such as hydraulic actuators, electrically controlled linear actuators, or rotary actuators, one of which in) Figure 1 The operating parameters of these components are modified by means of a symbol (generally denoted by 62). In this respect, the operating speed of any number of fans or conveyors can be varied, and the positions of any number of deflectors, chaff screen components, screen components, grain cleaning chamber components, etc., not shown, can also be varied. Such actuator 62 can be controlled in response to operator input received via operator interface 26 located in the cab 22, via command signals issued by controller architecture 16 included in grain level monitoring system 12, or otherwise commanded by another controller or control unit on combine harvester 10.

[0037] The combine harvester 10 includes a variety of other sensors besides those mentioned above, which can supply data to the controller architecture 16 during operation of the grain-level grain monitoring system 12. A non-exhaustive list of such additional onboard sensors includes: at least one grain moisture sensor 64 for providing data indicating the moisture level within the currently harvested grain, and in some cases, for providing a capacitance measurement indicating the bulk density of the grain (e.g., expressed in bushels per pound (lbs)); one or more mass flow sensors, which may take the form of one or more impact plates 66 mounted near the outlet end of the clean grain elevator 52 and positioned to collide with grains sprayed from the elevator 52 into the grain bin 54. In other embodiments, the mass flow sensors(s) on the combine harvester 10 may be implemented using another technique or method, including, for example, image analysis of camera feeds provided by the grain camera 56. Additionally, the combine harvester 10 may include a plurality (e.g., three or more) weighing sensors 68 located in the grain bin 54 and used to calibrate, for example, the mass flow sensors 66 by weighing the stored grain. In at least some embodiments, the combine harvester 10 may also include various sensors for measuring grain loss, such as one or more cleaning chamber loss sensors 70 (e.g., impact plate sensors) and / or unseen (e.g., impact plate) sensors located at the separator. Various other sensors (not shown) may also be deployed on the combine harvester 10, as conventionally, such as sensors for monitoring the speed of the cleaning grain elevator 52 and / or the load placed on the cleaning grain elevator 52. Additionally, in embodiments, cameras such as the ear-lift camera 72 may also be integrated into the combine harvester 10, wherein video feeds supplied by these cameras are selectively presented on the display screen of the display device 24, as described below. Figure 2 As described more comprehensively.

[0038] Now for reference Figure 2 The grain-level grain monitoring system 12 and certain internal harvesting mechanisms of the combine harvester 10 are shown in more detail. Where appropriate, the reference numerals are changed from... Figure 1 Continuing from the previous illustration, note the schematic representation of controller architecture 16, display device 24, and operator interface 26 presented in this figure. Additionally, the grain-level grain monitoring system 12 is depicted as including an "actuated harvesting component 76," which generally encompasses those components involved in ingesting and processing crop stalks to produce a grain accumulation flow during combine harvester operation, as previously described. Therefore, in the illustrated schematic, the actuated harvesting component 76 may include components utilizing… Figure 1 The actuator 62 is generally shown in the diagram. Figure 2The lower right portion shows any combination of actuated screens, cleaning chambers, deflectors, or chaff screen components adjusted by the centrifugal fan or blower impeller 78. Similarly, the generally described onboard sensors 80 encompass those integrated into the combine harvester 10, excluding the grain camera 56, including the aforementioned grain moisture sensor 64, mass flow sensor (e.g., impact plate sensor 66), grain bin weighing sensor 68, and cleaning chamber loss sensor 70. In addition to the aforementioned components, the grain-level grain monitoring system 12 also includes a computer-readable storage device 82 storing a database 84, which may contain fill density factors, morphological classification data, and other stored data used by the controller architecture 16 in performing the processes described below. Various data connections between these components are... Figure 2 The symbol is represented by a number of signal lines terminating with arrows, which typically represent any combination of wired or wireless data connections.

[0039] The controller architecture 16 of the grain-level grain monitoring system 12 can take any form suitable for performing the functions described throughout this document. The term "controller architecture" as used herein is used in a non-limiting sense to refer to the processing architecture of the grain-level grain monitoring system 12. The controller architecture 16 can encompass any actual number of processors, control computers, computer-readable storage, power supplies, storage devices, interface cards, and other standardized components, or can be associated with any actual number of processors, control computers, computer-readable storage, power supplies, storage devices, interface cards, and other standardized components. The controller architecture 16 may also include any number of firmware and software programs or computer-readable instructions designed to perform the various process tasks, calculations, and control / display functions described herein, or cooperate with any number of firmware and software programs or computer-readable instructions designed to perform the various process tasks, calculations, and control / display functions described herein. Such computer-readable instructions may be stored in non-volatile sectors of memory 82 along with one or more databases 84 described below. Although in Figure 2 While typically shown as a single frame, memory 82 can encompass any amount and type of storage medium suitable for storing computer-readable code or instructions, as well as other data for supporting the operation of the grain-level grain monitoring system 12. In embodiments, memory 82 can be integrated into the controller architecture 16, for example as a system-on-a-chip, a system-on-chip, or another type of microelectronic package or module.

[0040] The operator interface 26, located within the cab 22 of the combine harvester 10, can be any device or group of devices used by the operator to input commands to or otherwise control the grain-level grain monitoring system 12. In various embodiments, the operator interface 26 can be integrated into or otherwise associated with the display device 24. In this regard, the operator interface 26 may include physical inputs located on or near the display device 24 (e.g., buttons, switches, dial pads, etc.), a touchscreen module integrated into the display device 24, or a cursor input device (e.g., joystick, trackball, or mouse) for positioning a cursor to interact with GUI elements generated on the display device 24. In contrast, the display device 24 can be any image-generating device configured for operation within the cab 22 of the combine harvester 10. In embodiments, the display device 24 can be attached to the static structure of the combine harvester cab 22 and implemented as a head-down display (HDD). Figure 2 An example of a GUI screen 74 is shown, presenting a real-time image of the grain stream captured by grain camera 56. As can be seen, various operator-selectable options can also be generated on the GUI screen 74 to, for example, enable the operator to switch between presentations of camera feeds obtained from grain camera 56 and miscellaneous grain camera 72, apply visually enhanced grain quality analysis tools (applying color coding to broken grains and MOGs detected in the real-time camera images), and perform other functions such as providing visual advisory alerts about excessive lens debris.

[0041] During the operation of the grain-level grain monitoring system 12, the controller architecture 16 performs certain processes to monitor (repeatedly calculated) one or more APK parameters and further combines the monitoring of APK parameters to perform certain actions. Such actions may include generating various graphical, numerical readings, symbolic elements, or GUI screens presented on the display screen of the display device 24, based on operator navigation, for example, to a specific GUI screen. The selected GUI screen generated on the display device 24 may present one or more of the calculated APK parameters, topline harvest metrics calculated using those calculated APK parameters (e.g., grain loss estimates, grain mass flow rate estimates, or grain yield estimates), and / or other such information directly or indirectly related to the APK parameters. Corresponding audible alarms or advisory notifications may also be generated in conjunction with or in lieu of such visual notifications presented on the display screen 24. Additionally or alternatively, embodiments of the grain-level grain monitoring system 12 may utilize (i.e., at least in part based on) instantaneous values ​​or trends of APK parameters, instantaneous values ​​or trends of top-line harvesting metrics, or any combination thereof, to determine target setting adjustments for the actuated harvesting components of the combine harvester 10. When such target setting adjustments are determined, the controller architecture 16 may prompt the operator of the combine harvester 10 to implement the target setting adjustments by generating appropriate graphics on the display device 24, by generating auditory notifications (e.g., auditory alerts, bells, or other auditory cues), and / or by generating any other operator-perceived cues (e.g., tactile notifications). In other cases, the controller architecture 16 may automatically implement target setting adjustments in an immediate or automatic manner, while generating graphics (and / or generating accompanying auditory notifications or advisory alarms) on a given GUI display, thereby notifying the combine harvester operator when such automatic adjustments are performed. Other actions may also be performed by the controller architecture 16 in conjunction with determining and applying the APK parameters, as follows: Figure 3 The example grain-level grain monitoring process described in the text is further described.

[0042] Now for reference Figure 3According to a non-limiting example embodiment of this disclosure, a grain-level grain monitoring process 86 is presented. In embodiments of this disclosure, the grain-level grain monitoring process 86 may be implemented by the controller architecture 16 of the grain-level grain monitoring system 12, and is primarily as described below; however, in alternative embodiments, at least some process steps of the grain-level grain monitoring process 86 may be implemented by a remote network connection source (e.g., a back-end service, such as a cloud-based support service) communicating with the grain-level grain monitoring system 12 via a suitable wireless transceiver or data link (e.g., a telematics gateway module) on the combine harvester 10. The grain-level grain monitoring process 86 includes a plurality of process steps 88, 90, 92, 94, 96, 98, 100, each of which is described below in sequence. Depending on the specific manner in which the grain-level grain monitoring process 86 is implemented, Figure 3 Each step typically shown may require a single process or multiple sub-processes. Furthermore, Figure 3 The steps shown and described below are provided as non-limiting examples only. In alternative embodiments of the grain-level grain monitoring process 86, additional process steps may be performed, certain steps may be omitted, and / or the process steps shown may be performed in an alternative order.

[0043] In response to the occurrence of a predetermined triggering event, the grain-level grain monitoring process 86 begins at step 88. In some cases, the triggering event may be the detection of a cut crop stalk entering the combine harvester 10. In other cases, the grain-level grain monitoring process 86 may begin in response to different triggering events, such as operator input received via operator interface 26 instructing the grain-level grain monitoring process 86 to be executed. After the start (step 88), the controller architecture 16 of the grain-level grain monitoring process 86 proceeds to step 90, during which the controller architecture 16 collects current data inputs used in subsequent steps of the execution process 86. Such data inputs may be retrieved from memory 82 (including recollection of information stored in database 84); input by the operator using operator interface 26; received from any number of sensors on the combine harvester 10; and may be received via a data link (not shown) from a cloud-based backend service or other remote data source (such as a modular telematics gateway on the combine harvester 10). Examples of such data inputs are identified in Figure 3On the left side, and including one or more images 102 of piled grain samples captured by the grain camera 56 during or immediately before the current grain-level grain monitoring process 86, piled grain density data 104 related to the currently harvested grain, data 106 indicating grain filling (also referred to herein as “grain filling factor”), and any number of additional data inputs 108, such as manually entered data received from the operator of the combine harvester 10.

[0044] As described above, the heaped grain sample image 102 can take the form of any image data of the currently harvested grain captured by the grain camera 56 during the operation of the combine harvester 10. In the illustrated example, this includes the heaped grain sample image 102 captured by the grain camera 56 as the heaped grain stream is transported upward through the clean grain elevator 52 and into the grain bin 54 of the combine harvester 10. In other cases, the heaped grain sample image can be captured by multiple cameras deployed at any number of locations on the combine harvester 10. In at least some embodiments, image analysis of the images captured by the ear camera 72 can also be used to supplement the image analysis performed using the images captured by the grain camera 56; however, in other embodiments, only the image from the grain camera 56 can be considered, as this image depicts a grain stream in a clean state that is substantially free of broken grain and MOG, in order to facilitate image analysis in distinguishing the undisturbed grains from other materials (e.g., broken grains and MOG) within the captured image. Finally, as previously stated, in at least some embodiments of this disclosure, the grain camera 56 may potentially capture image data beyond the visible portion of the electromagnetic spectrum and / or may collect depth data (e.g., when the two cameras are in a stereo relationship).

[0045] like Figure 3As generally illustrated, the bulk density data 104 can take the form of any data indicating the bulk density of the currently harvested grain when filled into a given volume of space. In embodiments, the bulk density data 104 can be a default value retrieved from memory based on the type of grain currently harvested by the combine harvester 10 (which can be determined by operator input or possibly by image analysis of the grain camera image by the controller architecture 16 or by using other onboard sensor inputs). In this case, the controller architecture 16 can retrieve a specific value from a database 84 stored in computer-readable storage 82 based on the type of grain currently harvested; and then use this value to calculate the APK density value in the manner described below. In other cases, the bulk density data 104 can be directly input by the operator to the grain-level grain monitoring system 12 via the operator interface 26. Despite this possibility, it is useful, where possible, to obtain the bulk density data 104 directly from one or more sensors on the combine harvester 10. For this purpose, in embodiments where the combine harvester is equipped with a suitable moisture sensor, the bulk density data (e.g., ...) is obtained from the moisture sensor. Figure 1 Data from the moisture sensor 64 (generally shown) can be used to determine the bulk density of the currently harvested grain stream. In addition to providing data indicating the moisture content of the currently harvested grain, the moisture sensor 64 is generally capable of measuring the sample capacitance of the bulk grain stream, which can then be converted into a bulk density measurement for performing the processes or calculations described below.

[0046] Data indicating grain filling 106 can also be collected during step 90 of the grain-level grain monitoring process 86. When collected, this grain filling data 106 can take the form of any information that can be used to assess the fraction of space occupied by solid mass (grains of the currently harvested grain stream) compared to the cumulative void space of the space volume when filled into a given volume of space. When used, grain filling indicators can be determined through empirical testing, in which case a series of filling indicator values ​​corresponding to a series of APK morphologies can be stored. Specifically, in embodiments, grain filling data 106 can be determined based on the morphological classification (e.g., size and shape) of the currently harvested grain, whether based on the type of grain currently harvested, determined based on operator data input by recollection from a two-dimensional lookup table or other data structure stored in memory 82, or determined in another manner. In other cases, controller architecture 16 can perform further image analysis on the image of the stacked grain sample to measure or estimate the cumulative air voids in the image (e.g., air voids identified in the visible area of ​​the grain pile carried by the blades of the clean grain elevator 52) when filling with grains from the sampled grain stream, and extrapolate the voids to a percentage or fraction of the three-dimensional volume of space occupied by air voids rather than solid matter (grains). In other embodiments, a grain filling indicator can be determined in another manner.

[0047] Finally, various other data inputs 108 can be collected during step 90 of the grain-level grain monitoring process 86. These data inputs 108 may include other types of operator input data, historical data, and other sensor data that can be used to calculate APK parameters and topline harvesting metrics described below in conjunction with step 94. For example, data indicating mass flow rate can be obtained from any suitable type of mass flow rate sensor (e.g., Figure 1 The impact plate sensor 66 shown in the figure collects data indicating grain loss, which can be collected from the grain loss sensor 70 (or another type of grain loss sensor). Data for calculating grain yield (e.g., data describing the header width of the combine harvester 10, current ground speed or rate, and grain moisture data) can be collected, and the current GPS position of the combine harvester 10 can be determined (e.g., for constructing yield maps, such as...). Figure 5 (See Figure 152 for yield data). Finally, as mentioned above, the additional data input 108 may include operator input data describing various crop attributes input through the operator interface 26, such as operator input data identifying the type of grain currently being harvested or operator input data directly specifying the grain morphology classification of the currently harvested grain.

[0048] Proceeding to step 92 of the grain-level grain monitoring process 86, the controller architecture 16 then calculates at least one APK parameter using the data collected during step 90. As described throughout this document, such APK parameters may include any combination of: (i) APK volume, (ii) APK density, (iii) APK weight or mass, and (iv) APK morphology (size, shape, and angularity). In the initial discussion of APK volume, this parameter can be tracked by the controller architecture 16 of the grain-level grain monitoring system 12 through three-dimensional image analysis of images of piled grain samples. Specifically, the controller architecture 16 can visually analyze images of piled grain samples to identify the outlines of grains within the piled grain sample images; and then estimate and average certain dimensions (average length and diameter) of the detected grains. Such average dimensions can then be used, potentially combined with the known morphology or three-dimensional shape of the grains, to calculate the average volume per grain of a single average grain within the piled grain stream. The following is in conjunction with... Figure 4 Additional description of suitable image analysis techniques for monitoring APK volume using images of stacked grain samples.

[0049] The grain-level grain monitoring system 12 can further monitor or track the APK density of the currently harvested grain stream. In this case, the controller architecture 16 can: (i) determine the sample bulk density of the currently generated grain stream; and (ii) calculate the APK density based at least in part on the sample bulk density and APK volume of the grain stream. The controller architecture 16 can establish the sample bulk density of the grain stream based at least in part on data provided by the grain moisture sensor 64, for example, indicating the weight of 1000 grains or another weight parameter as previously described. Further, in an embodiment, when the bulk density measurement is converted into a per-grain density parameter using the grain filling data 106, the controller architecture 16 can further correct or compensate for air voids. Specifically, while the APK volume can be used to estimate the number of grains that can fit into a given volume of space or envelope, strictly speaking, the APK volume does not convey how the grains will fill (commonly, the “tightness” of the grain filling); and therefore, does not directly represent the fraction of the space volume occupied by air voids when filled by grains from the grain stream. In contrast, bulk density measurements (and other “bulk” measurements described herein) inherently take into account or account for the unoccupied (air voids) space within the grain-filled volume. Therefore, to compensate for this difference between the APK parameter and the bulk measurement, the controller architecture 16 of the grain-level grain monitoring system 12 can establish and utilize the aforementioned grain filling data 106. As described above, grain filling data 106 can be quantified as a percentage or fraction of the volume of space occupied by grain mass or by air voids; for example, in one embodiment, grain filling data 106 can be a percentage (ranging from, for example, 0.1% to 10%) of the estimated cumulative volume of air voids when the grains of the currently harvested grain are filled into a given volume or envelope.

[0050] When used, the grain filling data 106 can be determined by the controller architecture 16 based on image analysis of the stacked grain sample image, as combined with the following... Figure 4As discussed, the grain filling data 106 can be determined, at least in part, based on a stored grain filling indicator retrieved from a database 84 stored in memory 82 and corresponding to the type of grain currently being harvested using the combine harvester 10. Thus, during step 92 of the grain-level grain monitoring process 86, the controller architecture 16 can calculate the APK density using the sample bulk density of the grain stream, the APK volume, and the grain filling data 106. For example, in one approach, controller architecture 16 can determine the bulk density measurement of currently harvested grain when filled into a given volume of space, subtract an estimated air void volume from the total volume of space containing the bulk density measurement, and then calculate the APK density value by estimating the number of grains that can be filled into the corrected volume of space corresponding to the bulk density measurement (e.g., in the case of 1000 grain weights, the number of grains that can be filled into 1 bushel (1.244 cubic feet or 0.035 cubic meters) minus the volume of estimated air void volume established using grain filling data 106). Furthermore, while the 1000 grain weight discussed herein is a common example, other grain density measurements may be used in alternative embodiments, in addition to or instead of 1000 grain weights.

[0051] Turning to APK weight (or mass), controller architecture 16 can calculate this APK parameter based on the previously established APK volume and stacked grain weight (or mass) parameters. As is readily understood, because these parameters are proportional—note that weight is the product of mass multiplied by a standard value of gravitational acceleration—APK mass can be readily calculated using APK weight (and conversely, APK mass can be readily calculated using APK mass). Similar to APK density, APK weight (or mass) can be readily determined using the stacked weight (or mass) measurement combined with the APK volume determined by controller architecture 16 as described above. Grain filling data 106 is also useful, although controller architecture 16 does not substantially consider this grain filling data 106 when determining the current value of APK weight or APK mass. In general, controller architecture 16 can initially estimate the number of grains that can fill a given volume of space based on the APK volume and possibly also considering the grain filling data 106. After determining the grain count within a given volume of space, the controller architecture 16 can estimate the APK weight or mass by dividing the pile weight or mass measurement by the number of grains contained within the volume of space corresponding to the pile weight or mass measurement. Furthermore, this pile weight or mass measurement can currently be calculated by a conventional combine harvester using, for example, an impact plate sensor 66 positioned near the outlet end of the clean grain elevator 52, a grain bin weighing sensor 68, or other sensors 80.

[0052] Finally, addressing APK morphology, controller architecture 16 can establish APK morphology classification of stacked grains through visual analysis of images of stacked grain samples, whether executed on the combine harvester or off-site by a remote data source (e.g., a backend service) wirelessly communicating with a grain-level grain monitoring system. Specifically, controller architecture 16 can assign an APK morphology classification representing the average grain shape of the grains contained in the currently harvested grain stream. Several examples of this morphology classification or type are as follows... Figure 3 As shown, it includes classifications of shapes such as linear, rectangular, oval, ovate, spherical, and kidney-shaped. In addition... Figure 3 In addition to those shape categories presented in the text, or instead Figure 3 The shape categories presented can also use various other three-dimensional grain shape categories or three-dimensional seed shape categories. Additionally, in embodiments, the controller architecture 16 can also assign size or scale to morphology classifications based at least in part on three-dimensional analysis of the sample grain flow image. As an arbitrary example, a particular harvest of grain can be assigned a large rectangular or small spherical APK morphology classification, while the controller architecture 16 can potentially select from any actual size gradient. Therefore, APK morphology determination is similar, although different from the APK calculation described above. This will now be combined with... Figure 4 Further description of an example process by which the controller architecture 16 can potentially be used to analyze images of stacked grain samples and determine APK morphology classification, APK volume, or both APK morphology classification and APK volume.

[0053] Figure 4 An example image analysis subprocess 118 is schematically illustrated, which is potentially implemented by a controller architecture 16 to analyze images from one or more cameras on the combine harvester 10 (e.g., ...). Figure 1 and Figure 2 The image analysis of the stacked grain sample image received by the grain camera 56 shown is used to evaluate APK volume, APK morphology, or both of these APK parameters. The image analysis subprocess 118 comprises four steps (steps 120, 122, 124, 126), which are performed sequentially in the illustrated example. In step 120, the controller architecture 16 receives a raw camera image 128 of the grain flow from the grain camera 56. Next, in step 122, the controller architecture 16 can then process the image (e.g., by increasing contrast and identifying areas of high contrast or other such visual "landmarks" to distinguish grain outlines or perimeters), thereby producing a processed image 132. Figure 4The image shown is a portion of the processed image 132, a magnified region 130 of the original camera image 132. By identifying the outline or perimeter of the grains within the image, the controller architecture 16 can then identify the relevant dimensions of the grains. The specific dimensions identified by the controller architecture 16 during step 122, and the way these dimensions are applied to determine the APK volume, can vary depending on the three-dimensional shape of the grains to be analyzed. For example, when the grains are determined to have an average approximately spherical shape (whether determined based on operator input data, using a lookup table that associates the current crop type with the average grain shape, or based on the aforementioned APK morphology), the controller architecture 16 can determine only the boundaries and outlines of the visible grains, evaluate individual dimensions such as the diameter (or perimeter) of the visually sampled grains, and then use well-known formulas to calculate the spherical volume from the known diameter or perimeter values ​​to calculate the volume of the spherical grains as the APK volume using the average diameter.

[0054] In other cases, such as when the grains have rectangular or irregular shapes, the controller architecture 16 can determine the APK volume using multiple estimated dimensions of the grains depicted in an image of the stacked grain sample. In the latter case, the controller architecture 16 can visually measure the length and maximum width of each visually analyzed grain, such as... Figure 4 As indicated by arrows 134 and 136. Furthermore, as... Figure 4 As shown in step 124, this information can then be combined with an appropriate shape category or classification (in this case, an elliptical category) to calculate the APK volume. It is noteworthy that, in this case, the overall orientation of the grains depicted in the image of the stacked sample grains may become increasingly relevant. With this in mind, the controller architecture 16 can apply a screening procedure or perform similar actions to limit the size analysis to those grains with a length-oriented orientation (i.e., extending primarily in the horizontal direction) in the image being analyzed, rather than those grains currently exhibiting an increasingly upright orientation. For example, in an embodiment, the image analysis can be limited or restricted by the controller architecture 16 to those grains identified as having a larger surface area exposed to the grain camera 56 and / or adequately matching the established grain morphology shape, as seen from one or more selected sides of the grain. This, in turn, can reduce the degree of size inaccuracy, for example, when a certain number of oval-shaped grains are positioned in an upright orientation to expose a generally circular surface area to the grain camera 56 (essentially, as seen when viewing the top or bottom side of a given grain). Therefore, by limiting the three-dimensional analysis to those grains in which the oval side of the grain is exposed, the APK volume and / or APK morphology are determined with precision. Similar methods can also be used for various other grain morphology types, such as... Figure 3Those shown in boxed areas 116 in step 92 of the example grain-level grain monitoring process 86 described in the text.

[0055] like Figure 4 As further shown in steps 124 and 126, when the combine harvester 10 is equipped with a sensor capable of collecting depth data related to the currently harvested grain sample, the aforementioned visual analysis of the piled grain sample image can also take such depth data into account. In this case, the depth data can be used to establish the three-dimensional geometry or topology of the grains depicted in the piled grain sample image to further improve the accuracy of the aforementioned average volume per grain (APK) calculation and / or grain morphology assessment. Historical data can also be retrieved from memory 82 and mixed with such two-dimensional and three-dimensional image data when available and so desired. For example, when utilized, such historical data can be stored in a database 84 stored in memory 82 and can be associated with the morphological data (size and shape) of grains previously harvested in various field areas of a stored geographic reference map, such as a grain yield or grain loss map. Finally, techniques similar to those described above can also be used to assess the APK morphology of the grains depicted in the piled grain sample image. For example, the outline of the grain can be determined, and the overall shape of the grain outline can be compared with pre-stored shapes or templates, each associated with a specific morphological classification, stored in database 84. If depth data is available, a three-dimensional mesh can be generated and similarly matched with the three-dimensional mesh templates in database 84. In either case, controller architecture 16 can readily determine the APK morphological classification of the currently harvested grain through image analysis of images of stacked grain samples. If this APK morphological classification has a size component, such as in the case of a large linear APK morphological classification or a small kidney-shaped APK morphological classification, controller architecture 16 can determine this size component by performing size analysis on the captured image using a process or technique similar to those just described.

[0056] Back Figure 3The grain-level grain monitoring process 86 shown in the diagram proceeds to step 94, where the controller architecture 16 utilizes any or all of the aforementioned APK parameters to better understand the top-line harvesting metric (improving its accuracy). For example, when calculating grain loss using the impact plate sensor 70 placed at a specific location within the combine harvester 10, the grain loss estimate can be improved by considering the APK mass parameter; for example, by better estimating the mass of the lost grain, or by improving the impact count in more accurately distinguishing grain collisions from other non-grain (e.g., MOG) or broken grain collisions. In this regard, during step 94, the controller architecture 16 can repeatedly estimate or track the APK mass of the currently harvested grain; and monitor the grain loss parameters of the currently harvested grain based at least in part on the estimated APK mass, the collision data provided by the impact plate sensor 70, and any other relevant data inputs. Furthermore, in at least some embodiments, the determined APK weight can be upscaled by the controller architecture 16 to 1000 grain weight values ​​(or another multi-grain weight parameter), wherein this APK-based 1000 grain weights are used instead of the standardized 1000 grain weight values ​​conventionally relied upon by existing combine harvester systems in automatic combine harvester setup adjustments. Therefore, the accuracy and consistency of calculated and mapped grain losses can be improved.

[0057] When monitored during the example grain-level grain monitoring process 86, APK morphology classification can also be considered by the controller architecture 16 to determine grain loss estimates with greater accuracy. Mass flow rate measurements can also be improved using APK parameters calculated by the grain-level grain monitoring system 10, depending on whether the mass flow rate is tracked by measuring the impact of a strike plate 66 positioned at the outlet end of the clean grain elevator 52, estimating the rate of grain transport through the clean grain elevator 52 using the grain camera 56 (or another camera), estimating the mass flow rate using sensors indicating the load and speed of the clean grain elevator, or using another suitable technique to track the mass flow rate. Grain yield estimates can also be appropriately improved using current or instantaneous values ​​of the APK parameters. For example, in the latter case, improvements in the accuracy of grain yield calculations can come from more precise mass flow rate estimates, in addition to combine harvester header width, combine harvester ground speed or rate, and moisture data from moisture sensor 64, as previously discussed, mass flow rate estimates are commonly used to calculate grain yield. Similarly, the test weight (the weight of the currently harvested grain when tightly packed in a 1-volume bushel) can be determined using the APK parameters and possible grain filling data in the manner discussed above. Generally, embodiments of the grain-level grain monitoring system 12 then utilize different data inputs from the grain camera 56 and other onboard sensors (e.g., moisture sensor 64) to improve the accuracy of key crop information calculated on the combine harvester 10.

[0058] As discussed earlier, embodiments of the grain-level grain monitoring system 12 may, but are not required to, perform further image analysis of the piled grain sample images to assign the currently harvested grain to a specific one of a plurality of predetermined APK morphology categories or classes stored in the database 84. When determined by the controller architecture 16, the APK morphology (size and shape) category or classification of the grains contained in the currently harvested grain stream can also be usefully used as input in calculations or algorithms to further enhance combine harvester performance and provide other value-added features, such as creating georeferenced data tools for subsequent analysis by stakeholders. In the latter aspect, the APK morphology (size and shape) tracked by the controller architecture 16, along with possible APK weight parameters (which can be scaled up to one thousand grain weights or another multi-grain weight parameter), can be recorded along with GPS location, enabling seed producers and growers to improve products and services from an agronomic perspective. Furthermore, as... Figure 3As shown in Figure 95, the APK morphology (size and shape) can be used to more accurately infer or assess the angle of repose of the currently harvested grain, which is also known to be useful in some onboard calculations performed by the combine harvester 10. Generally, as the average grain shape becomes more rounded, the flowability of the piled grain increases, and the pile of piled grain tends to flatten, thus reducing the angle of repose. Conversely, grains with shapes that are increasingly closer to rectangular or irregular tend to increase the pile height and therefore have a larger angle of repose (identified as angle α in Figure 95). Once determined or estimated, the angle of repose can be used to fine-tune various calculations; for example, the angle of repose can be used to more accurately assess the amount or volume of grain piled onto the grain bin weighing sensor 68, which in turn can improve the mass flow sensor calibration technique performed using the weighing sensor data. In other words, the controller architecture 16 can estimate the angle of repose of the currently harvested grain based at least in part on the APK morphology classification; and then selectively calibrate the moisture sensor using the data generated by the weighing sensor and the estimated angle of repose.

[0059] Following step 94, controller architecture 16 proceeds to step 96 of the grain-level grain monitoring process 86, and performs one or more actions using the APK(s)(s) determined in step 92, the topline(s)(s) calculated in step 94, or a combination thereof. Such actions may include generating a GUI screen on display device 24, allowing the operator to selectively view current or historical values ​​of the APK(s) as desired. This possibility is generally... Figure 5 As described in the text, Figure 5 (In summary) a GUI screen or page 138 is presented on the display screen 140 of the display device 24 located in the cab 22 of the combine harvester 10. As shown in the upper bounding box 142, the controller architecture 16 can generate any quantity and type of digital readings or other visual indications conveying the current or instantaneous values ​​of the APK parameters tracked by the grain-level grain monitoring system 12. Similarly, in step 96 of the grain-level grain monitoring process 86, the controller architecture 16 can visually represent any topline harvesting metric calculated by the controller architecture 16 using one or more APK parameters, such as... Figure 5 As generally indicated within box 144. As shown in box 148, topline harvesting metrics can be expressed as numerical readings; for example, numerical readings of test weights, estimates of grain yield, estimates of grain loss, estimates of grain composition or structure, and / or estimates of mass flow rate, to name just a few. Alternatively, as indicated within box 150, topline harvesting parameters can be expressed in other graphical or symbolic forms, such as yield plots, grain loss plots, and other agronomic information. An example of such a yield plot 152 is shown in... Figure 5As shown in Figure 152, different crosshairs represent different color-coded regions indicating different yields.

[0060] In an embodiment, controller architecture 16 can determine target setting adjustments for the actuated harvesting components based on tracked APK parameters, top-line harvesting metrics calculated using the tracked APK parameters, or a combination thereof. This information can be used to determine target adjustments for combine harvester settings that control the combine harvester settings described above. Figure 2 The operation or positioning of the actuated harvesting component 76 is described. Further, the controller architecture 16 can generate images (e.g., text or symbolic visual notifications) on the display device 24 to alert the operator to implement determined target adjustments via appropriate visual cues. Additionally or alternatively, the controller architecture 16 can utilize auditory cues (such as by generating notification messages or other auditory notifications) to convey proposed adjustments to the combine harvester settings. As another possibility, the controller architecture 16 can automatically implement target setting adjustments to selected components of the actuated harvesting component 76 on the combine harvester 10 when appropriate. In the latter aspect, as an example, in at least some cases, husk screen adjustments, grain cleaning chamber adjustments, screen adjustments, fan speed adjustments, etc., can be determined and automatically implemented (i.e., implemented without operator input) by the controller architecture 16 of the grain-level grain monitoring system 12. This is in Figure 5 The area is represented by the lower box region 146. As an example, relatively fine adjustments can be made to the fan speed, for example, increasing the fan speed in conjunction with increasing the weight or size of the APK. In embodiments, fan speed adjustments can also be combined with corresponding automatic adjustments to the positioning of the grain cleaning chamber, screen, and / or chaff screen components. When such automatic adjustments are performed by the controller architecture 16, supplementary graphics can be presented on the display screen 140 of the display device 24 and / or auditory notifications can be generated to make the operator aware that such automatic adjustments are being performed.

[0061] Following step 96, for example, if the operator stops or interrupts crop harvesting by the combine harvester 10, the controller architecture 16 determines whether the grain-level grain monitoring process 86 should be terminated (step 98). If it is determined that the grain-level grain monitoring process 86 should be terminated, the controller architecture 16 proceeds to step 100 and terminates process 86 accordingly. Otherwise, the controller architecture 16 returns to step 88 and performs further iterations of the grain-level grain monitoring process 86 as described above. These steps can be performed on a relatively rapid basis to allow the grain-level grain monitoring system 12 to track the APK parameters described above and perform other actions described above in a highly responsive, real-time manner (e.g., displaying the selected APK parameters, displaying the topline harvesting metric calculated using the APK parameters, and / or displaying or automatically implementing target setting adjustments determined using the APK parameters). This can further enhance the quality of information provided by the grain-level grain monitoring system 12 to the combine harvester operator and can otherwise support the optimization of combine harvester operation.

[0062] Examples of grain-level monitoring systems

[0063] For ease of reference, the following examples of grain-level grain monitoring systems are provided and numbered.

[0064] 1. In a first exemplary embodiment, the grain-level grain monitoring system includes: a grain camera positioned to capture an image of a stockpiled grain sample acquired and processed by a combine harvester of currently harvested grain; a moisture sensor configured to generate moisture sensor data indicating the moisture level of the currently harvested grain; and a display device having a display screen on which parameters related to the currently harvested grain are selectively presented. A controller architecture is coupled to the grain camera, coupled to the moisture sensor, and coupled to the display device. The controller architecture is configured to: (i) analyze the stockpiled grain sample image received from the grain camera to determine an average volume per grain (APK) representing an estimated volume of a single average grain of the currently harvested grain; (ii) repeatedly calculate one or more topline harvesting parameters based at least in part on the determined APK volume and the moisture sensor data; and (iii) selectively present the topline harvesting parameters on the display device for viewing by a combine harvester operator.

[0065] 2. The grain-level grain monitoring system according to Example 1, wherein the top-line harvesting parameters include grain weight parameters.

[0066] 3. The grain-level grain monitoring system according to Example 2, wherein the grain weight parameter includes a test weight, which specifies the weight of the currently harvested grain when it is filled into a space of a predetermined volume.

[0067] 4. The grain-level grain monitoring system according to Example 3, wherein the controller architecture is further configured to calculate the test weight using grain filling data of accumulated void space within a space indicating a predetermined volume.

[0068] 5. The grain-level grain monitoring system according to Example 4, wherein the controller architecture is configured to determine grain filling data based at least in part on an image analysis assessment of the cumulative void space within an image of a stacked grain sample.

[0069] 6. The grain-level grain monitoring system according to Example 4 also includes an operator interface coupled to the controller architecture. The controller architecture is configured to determine grain filling data based at least in part on operator input data received via the operator interface.

[0070] 7. The grain-level grain monitoring system according to Example 4 also includes a database storing multiple grain filling data associated with different grain attributes. The controller architecture is configured to determine the grain filling data by invoking a selected one of the multiple grain filling data corresponding to the currently harvested grain.

[0071] 8. The grain-level grain monitoring system according to Example 1 further includes at least one impact plate sensor, which is coupled to the controller architecture and is impacted by the grains of the currently harvested grain as they are transported into the grain bin of the combine harvester. The controller architecture is coupled to the impact plate sensor and configured to: (i) estimate the APK mass of the currently harvested grain using APK volume and bulk density parameters; and (ii) monitor the mass flow rate of the currently harvested grain based at least in part on the estimated APK mass and the collision data provided by the impact plate sensor.

[0072] 9. The grain-level grain monitoring system according to Example 8, wherein the topline harvesting parameters include grain yield parameters calculated using mass flow rate as input.

[0073] 10. The grain-level grain monitoring system according to Example 1, wherein the controller architecture is configured to further analyze images of stacked grain samples received from a grain camera to determine an APK morphology classification indicating the grain size and shape category of the currently harvested grain.

[0074] 11. The grain-level grain monitoring system according to Example 10 also includes a weighing sensor located in the grain bin of the combine harvester. The controller architecture is also configured to: (i) estimate the angle of repose of the currently harvested grain based at least in part on APK morphology classification; and (ii) selectively calibrate the moisture sensor using the data generated by the weighing sensor and the estimated angle of repose.

[0075] 12. The grain-level grain monitoring system of Example 1 further includes an impact plate sensor coupled to the controller architecture and impacted by the grains of the currently harvested grain as they are ejected from the combine harvester. The controller architecture is coupled to the impact plate sensor and configured to: (i) estimate the APK mass of the currently harvested grain using APK volume and bulk density parameters; and (ii) monitor grain loss parameters of the currently harvested grain based at least in part on the estimated APK mass and the collision data provided by the impact plate sensor.

[0076] 13. The grain-level grain monitoring system of Example 1 further includes an actuated harvesting component on a combine harvester and coupled to a controller architecture. The controller architecture is also configured to determine target setting adjustments to the actuated harvesting component based at least in part on parameters calculated using the APK volume. Additionally, the controller architecture performs at least one of the following actions: (i) generating a graphic on a display device that visually prompts the operator to perform target setting adjustments to the actuated harvesting component, or (ii) automatically performing target setting adjustments to the actuated harvesting component.

[0077] 14. The grain-level grain monitoring system according to Example 13, wherein the parameters calculated using the APK volume include at least one of APK weight, APK mass and APK density, wherein the target setting adjustment includes fan speed adjustment.

[0078] 15. In another embodiment, the grain-level grain monitoring system includes: a grain camera positioned to capture an image of a stockpiled grain sample acquired and processed by a combine harvester; a moisture sensor configured to generate moisture sensor data indicating the moisture level of the currently harvested grain; and a display device having a display screen on which parameters related to the currently harvested grain are selectively presented. A controller architecture is coupled to the grain camera, the moisture sensor, and the display device. The controller architecture is configured to analyze the stockpiled grain sample image received from the grain camera to determine an average per grain (APK) parameter; and to determine, at least in part, a target setting adjustment for the actuated harvesting component of the combine harvester based on the APK parameter. Additionally, the controller architecture performs at least one of the following actions: (i) generating a notification prompting the operator to implement the target setting adjustment, such as a visual and / or auditory notification; and (ii) controlling the actuated harvesting component to automatically implement the target setting adjustment.

[0079] in conclusion

[0080] Therefore, embodiments of a grain-level grain monitoring system for use on combine harvesters have been provided. Embodiments of the grain-level grain monitoring system utilize images of piled grain samples obtained from at least one grain camera to calculate one or more APK parameters associated with the piled grain flow generated by the combine harvester. In embodiments, instantaneous values ​​of one or more of the APK parameters (such as APK volume (size), APK weight or mass, APK density, and / or APK morphology) can be directly displayed to the combine harvester operator. Additionally or alternatively, the APK parameters can be used to calculate topline harvesting metrics with greater accuracy and consistency, including mass flow rate, grain yield, test weight, grain loss, and any combination of other parameters. The topline harvesting metrics can then be displayed to the operator for consideration in determining the optimal way to operate the combine harvester under a given set of conditions, for calibrating sensors on the combine harvester (e.g., mass flow rate sensors), or stored in memory for subsequent reference or for use in compiling agricultural data tools (such as yield maps and grain loss maps). In some embodiments, target adjustments to combine harvester settings can be determined using APK parameters and / or topline harvesting metrics calculated using APK parameters, whereby the target setting adjustments are then transmitted (e.g., visually presented or audibly communicated) to the combine harvester operator or implemented automatically by a grain-level grain monitoring system. Advantageously, such APK parameters can be calculated by combining different sensor inputs from sensors typically deployed on existing combine harvesters, including grain cameras and moisture sensors. Therefore, in many cases, grain-level grain monitoring systems can be implemented primarily through software to simplify user adoption and reduce implementation costs. Thus, in general, embodiments of grain-level grain monitoring systems provide operators and automation systems with novel and higher-quality information to enhance all aspects of combine harvester operation.

[0081] As used herein, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that, when used in this specification, the terms “comprising” and / or “including” specify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0082] The description in this disclosure has been presented for purposes of illustration and description, but is not intended to be exhaustive or limited to the disclosure presented in its form. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of this disclosure. The embodiments expressly referenced herein were chosen and described in order to best explain the principles of this disclosure and its practical application, and to enable others skilled in the art to understand the contents of this disclosure and recognize the many alternatives, modifications, and variations of the described examples(s). Therefore, various embodiments and implementations other than those expressly described are within the scope of the appended claims.

Claims

1. A grain-level grain monitoring system (12) used on a combine harvester (10), the grain-level grain monitoring system (12) comprising: A grain camera (56) is positioned to capture images of a stockpile of grain samples that are acquired in and processed by the combine harvester (10) and are currently being harvested. Moisture sensor (64), the moisture sensor being configured to generate moisture sensor (64) data indicating the moisture level of the currently harvested grain; Display device (24), the display device having a display screen on which parameters related to the currently harvested grain are selectively presented; as well as A controller architecture (16), coupled to the grain camera (56), coupled to the moisture sensor (64), and coupled to the display device (24), is configured to: Analyze the images of the stacked grain samples received from the grain camera (56) to determine the average volume per grain (APK) representing the estimated volume of a single average grain representing the currently harvested grain. One or more top-line harvesting parameters are repeatedly calculated, at least in part based on the determined APK volume and the moisture sensor (64) data; as well as The top-line harvesting parameters are selectively displayed on the display device (24) for viewing by the operator of the combine harvester (10); The top-line harvesting parameters include grain weight parameters, which include test weight, specifying the weight of the currently harvested grain when it is filled into a space of a predetermined volume. and The controller architecture (16) is also configured to calculate the test weight using grain filling data that indicates the cumulative void space within the space of the predetermined volume.

2. The grain-level grain monitoring system (12) according to claim 1, wherein the controller architecture (16) is configured to determine the grain filling data based at least in part on an image analysis assessment of the cumulative void space within the image of the stacked grain sample.

3. The grain-level grain monitoring system (12) according to claim 1 further includes an operator interface (26) coupled to the controller architecture (16); and The controller architecture (16) is configured to determine the grain filling data based at least in part on operator input data received via the operator interface (26).

4. The grain-level grain monitoring system (12) according to claim 1 further includes a database (84) storing multiple grain filling data associated with different grain properties; and The controller architecture (16) is configured to determine the grain filling data by invoking one of the plurality of grain filling data corresponding to the currently harvested grain.

5. The grain-level grain monitoring system (12) according to claim 1 further includes at least one impact plate sensor (66), the at least one impact plate sensor being coupled to the controller architecture (16) and being struck by the grain of the currently harvested grain when the grain of the currently harvested grain is transported to the grain bin of the combine harvester (10); The controller architecture (16) is coupled to the impact plate sensor (66) and is configured to: The APK mass of the currently harvested grain is estimated using the APK volume and bulk density parameters of the grain; and The mass flow rate of the currently harvested grain is monitored at least in part based on the estimated APK mass and the collision data provided by the impact plate sensor (66).

6. The grain-level grain monitoring system (12) according to claim 5, wherein the topline harvesting parameters include grain yield parameters calculated using the mass flow rate as input.

7. The grain-level grain monitoring system (12) according to claim 1, wherein the controller architecture (16) is configured to further analyze the images of the stacked grain samples received from the grain camera (56) to determine an APK morphology classification indicating the grain size and shape category of the currently harvested grain.

8. The grain-level grain monitoring system (12) according to claim 7 further includes a weighing sensor (68) located in the grain bin of the combine harvester (10); and The controller architecture (16) is further configured to: The angle of repose (95°) of the currently harvested grain is estimated, at least in part, based on the APK morphology classification; and The moisture sensor (64) is selectively calibrated using the data generated by the weighing sensor (68) and the estimated angle of repose (95).

9. The grain-level grain monitoring system (12) according to claim 1 further includes an impact plate sensor (66) coupled to the controller architecture (16) and impacted by the grains of the currently harvested grain when the grains of the currently harvested grain are ejected from the combine harvester (10); The controller architecture (16) is coupled to the impact plate sensor (66) and is configured to: The APK mass of the currently harvested grain is estimated using the APK volume and bulk density parameters of the grain; and The grain loss parameters of the currently harvested grain are monitored at least in part based on the estimated APK quality and the impact data provided by the impact plate sensor (66).

10. The grain-level grain monitoring system (12) according to claim 1 further includes an actuated harvesting component (76) on the combine harvester (10) and coupled to the controller architecture (16). The controller architecture (16) is further configured to: The target setting adjustment for the actuated harvesting component (76) is determined, at least in part, based on parameters calculated using the APK volume; and Perform at least one of the following: (i) generate a graphic on the display device (24) that visually prompts the operator to perform target setting adjustment of the actuated harvesting component (76), or (ii) automatically perform target setting adjustment of the actuated harvesting component (76).

11. The grain-level grain monitoring system (12) according to claim 10, wherein the parameters calculated using the APK volume include at least one of APK weight, APK mass, and APK density; and The target setting adjustment mentioned above includes fan speed adjustment.

12. A grain-level grain monitoring system (12) used on a combine harvester (10) having an actuated harvesting component (76), the grain-level grain monitoring system (12) comprising: A grain camera (56) is positioned to capture images of a stockpile of grain samples that are acquired in and processed by the combine harvester (10) and are currently being harvested. Moisture sensor (64), the moisture sensor being configured to generate moisture sensor (64) data indicating the moisture level of the currently harvested grain; Display device (24), the display device having a display screen on which parameters related to the currently harvested grain are selectively presented; as well as A controller architecture (16), coupled to the grain camera (56), coupled to the moisture sensor (64), and coupled to the display device (24), is configured to: Analyze the images of the stacked grain samples received from the grain camera (56) to determine the average per grain (APK) parameter, which includes a grain weight parameter, which includes a test weight, which specifies the weight of the currently harvested grain when it is filled into a space of a predetermined volume; The test weight is calculated using grain-filling data that indicates the cumulative void space within the predetermined volume. The target setting adjustment of the actuated harvesting component (76) is determined at least in part based on the APK parameters; as well as Perform at least one of the following: (i) generate a notification prompting the operator to implement the target setting adjustment, and (ii) control the actuated harvesting component (76) to automatically implement the target setting adjustment.

Images