Mechanically controlled image selection

Abstract

An image of vegetation at a work site is assigned a vegetation index signature. The vegetation index signature indicates how a vegetation index value varies over the corresponding image. The image is selected for use in predictive map generation based on the vegetation index signature. The predictive map is provided to a harvester control system, which generates control signals applied to controllable subsystems of the harvester based on the predictive map and a location of the harvester.

Description

TECHNICAL FIELD

[0001] This specification relates to harvesting machines. More specifically, this specification relates to image selection for controlling a harvesting machine. BACKGROUND

[0002] There are various different types of agricultural machines. Some such machines include harvesting machines, such as combine harvesters, forage harvesters, cotton harvesters, sugar cane harvesters, and the like. Such machines can collect data for use in machine control.

[0003] Some current systems attempt to predict the yield of a field of crops to be harvested (or being harvested) by a harvesting machine. For example, some current systems use aerial images taken of a field of crops in an attempt to predict the yield of the field of crops.

[0004] Furthermore, some current systems attempt to use the predicted yield in controlling the harvesting machine. There is therefore a relatively large amount of work that goes into accurately predicting the yield from an image of a field of crops.

[0005] The above discussion is provided for general background information and does not serve as an aid in determining the scope of the claimed subject matter. SUMMARY

[0006] A method of controlling a work machine includes receiving spectral responses of a plurality of images at a work site, identifying a set of vegetation index metric values based on the spectral responses, identifying a vegetation index signature corresponding to each image, the vegetation index signature indicating how the set of vegetation index metric values vary in the corresponding image, selecting an image from the plurality of images based on the vegetation index signature, generating a prediction map from the selected image, and controlling a controllable subsystem of the work machine based on a location of the work machine and the prediction map.

[0007] This summary is provided to introduce some concepts in a simplified form that are further described below in the detailed description. This summary is neither BRIEF DESCRIPTION OF DRAWINGS

[0008] Figure 1 is a partial pictorial illustration of a combine harvester.

[0009] Figure 2 is a block diagram illustrating a computing system architecture including Figure 1 is a block diagram illustrating one example of a computing system architecture of the combine harvester shown.

[0010] Figure 3A and 3B(collectively referred to herein as FIG. 3) illustrate a flowchart showing Figure 2 one example of the operation of the computing system architecture shown.

[0011] Figure 4 is a block diagram illustrating one example of a computing environment that can be used in the architecture shown in the previous figures. Figure 1 is a block diagram illustrating one example of the architecture shown.

[0012] Figures 5-7 is shown. The computing environment 1000 can be used in the architecture shown in the previous figures.

[0013] Figure 8 is a block diagram illustrating one example of a computing environment that can be used in the architecture shown in the previous figures. DETAILED DESCRIPTION

[0014] As noted above, there has been a relatively large amount of work devoted to trying to predict the yield of a field based on aerial images of the field. For example, some current systems attempt to predict yield by assigning a vegetation index value representative of the development of vegetation to different locations shown in an aerial photograph. Yield is then predicted based on the vegetation index metric values assigned to different portions of the field in the image. One example of a vegetation index that has been used is called the Normalized Difference Vegetation Index (NDVI). Another vegetation index that has been used is called the Leaf Area Index. Current systems assign index values (using one of the index systems described above, or using a different index system) based on analysis of land-based images or remote sensing images that tend to indicate the development of vegetation being analyzed in the image.

[0015] However, this presents a problem. For example, at the peak vegetation characteristics of a plant (i.e., at the time in the growing season when the crop vegetation (leaves, etc.) is most fully developed), the plant NDVI value typically increases from 0 to about 0.6-0.7. At the peak vegetation characteristics of a plant, the plant spectral response reaches saturation, and thus the spectral response is clipped at a location near the maximum. This saturation means that when attempting to control a combine harvester based on yield predicted using these images, the sensitivity of the control algorithm is limited based on the saturation and signal-to-noise ratio in the yield prediction process.

[0016] Accordingly, the present specification describes a mechanism by which spectral analysis is used to select more useful images when performing mechanical control. The mechanism identifies those images that show sufficient variability in yield on the image. For example, in the early part of the growing season, images can not be useful because there is too little plant growth captured in the image. But at various points in time during the growing season, images can be very useful because there is already sufficient plant growth and there is active plant growth, but the vegetation index values on the image are still well distributed. Then, once the plants are fully grown, images can not be useful because the peak vegetation growth causes saturated images and low yield prediction accuracy. Again, in the later part of the season where there is active plant senescence, images can again be more useful in predicting yield because the images show good vegetation index variability, variability, and distribution on the image.

[0017] Accordingly, the present specification provides a system that selects images where the vegetation index metric values are sufficiently variable so that the image is useful in generating a prediction map. The prediction map is generated based on the selected images and provided to a harvester. The harvester control system uses the prediction map along with the current location and route of the harvester to control the harvester.

[0018] Figure 1 Fig. 1 is a partial pictorial illustration of an agricultural machine 100 in an example where the machine 100 is a multi-purpose harvester (or combine harvester). In this example, the machine 100 is a combine harvester that is harvesting a crop of corn. The machine 100 includes a header 102 that cuts the corn and feeds the corn into a threshing unit 104. The threshing unit 104 separates the corn kernels from the corn stalks and the corn cobs. The corn kernels are collected in a grain tank 106. The corn stalks and cobs are discharged from the machine 100. Figure 1As can be seen, the combine 100 illustratively includes a cab 101, which can have various different operator interface mechanisms for controlling the combine 100, as will be discussed in greater detail below. The combine 100 can include a set of front end equipment, which can include a header 102, and a cutter 104 generally indicated at 104. The combine 100 can also include a feedhouse 106, a feed accelerator 108, and a threshing machine generally indicated at 110. The threshing machine 110 illustratively includes a threshing rotor 112 and a set of concaves 114. In addition, the combine 100 can include a separator 116, which includes a separator rotor. The combine 100 can include a cleaning subsystem (or cleaning house) 118, which can itself include a cleaning fan 120, a chaffer 122, and a sieve 124. The material handling subsystem in the combine 100 can include (in addition to the feedhouse 106 and the feed accelerator 108) a beater 126, a tailing elevator 128, a clean grain elevator 130 (which moves clean grain into a clean grain tank 132), and an unloading auger 134 and an outlet 136. The combine 100 can also include a residue subsystem 138, which can include a chopper 140 and a spreader 142. The combine 100 can also have a propulsion subsystem, which includes an engine (or other power source) that drives ground engaging wheels 144 or tracks, etc. It should be noted that any of the above-mentioned subsystems in the combine 100 can have multiple (e.g., left and right cleaning houses, left and right separators, etc.).

[0019] In operation, and as an overview, the combine 100 illustratively moves through a field of crops in the direction indicated by arrow 146. As it moves, the header 102 engages and collects crops to be harvested toward the cutter 104. After the crops are cut, they are moved by a conveyor in the feedhouse 106 toward the feed accelerator 108, which accelerates the crops into the threshing machine 110. The crops are threshed by the rotor 112 rotating against the concaves 114. The threshed crops are moved in the separator 116 by the separator rotor. Here, some residue is moved by the beater 126 toward the residue subsystem 138. The residue can be chopped by the residue chopper 140 and spread on the field by the spreader 142. In other embodiments, the residue is simply dropped into a pile, rather than being chopped and spread.

[0020] The grain falls onto the clean room (or cleaning sub-system) 118. The upper sieve 122 separates some of the larger material from the grain, while the lower sieve 124 separates some of the finer material from the clean grain. The clean grain falls into an auger in the clean grain elevator 130, which moves the clean grain upward and deposits it in the clean grain bin 132. Residue can be removed from the clean room 118 by the airflow generated by the clean fan 120. This residue can also be moved rearward in the combine 100 toward the residue handling sub-system 138.

[0021] The chaff can be moved back through the chaff elevator 128 to the threshing machine 110, where it can be re-threshed. Alternatively, the chaff can also be passed (also using the chaff elevator or another transport mechanism) to a separate re-threshing mechanism, where it can also be re-threshed.

[0022] Figure 1 It is also shown that, in one example, the combine 100 can include a ground speed sensor 147, one or more separator loss sensors 148, a clean grain camera 150, and one or more clean room loss sensors 152. The ground speed sensor 147 illustratively senses the speed of travel of the combine 100 over the ground. This can be done by sensing the rotational speed of the wheels, drive shafts, axles, or other components. The speed of travel and position of the combine 100 can also be sensed by a positioning system 157, such as a global positioning system (GPS), a dead reckoning system, a LORAN system, or a variety of other systems or sensors that provide an indication of the speed of travel.

[0023] The clean room loss sensors 152 illustratively provide output signals indicative of the amount of grain loss on the right and left sides of the clean room 118. In one example, the sensors 152 are impact sensors (or strike sensors) that count the number of grain impacts per unit of time (or per unit of travel distance) to provide an indication of the grain loss of the clean room. The impact sensors on the right and left sides of the clean room can provide separate signals or combined or aggregated signals. It should be noted that the sensors 152 can also include only a single sensor, rather than multiple separate sensors for each clean room.

[0024] The separator loss sensors 148 provide signals indicative of the grain loss in the left and right separators. The sensors associated with the left and right separators can provide separate grain loss signals or combined or aggregated signals. This can also be done using a variety of different types of sensors. It should be noted that the separator loss sensors 148 can also include only a single sensor, rather than separate left and right sensors.

[0025] It will also be appreciated that the sensors and measurement mechanisms (in addition to the sensors already described) can also include other sensors on the combine 100. For example, they can include a residue setting sensor configured to sense whether the machine 100 is configured to chop residue, discard it into a pile, etc. They can include a clean room fan speed sensor that can be configured to be proximate to the fan 120 to sense the speed of the fan. They can include a threshing gap sensor that senses the gap between the rotor 112 and the concave 114. They include a threshing rotor speed sensor that senses the rotor speed of the rotor 112. They can include an upper sieve gap sensor that senses the opening size in the upper sieve 122. They can include a lower sieve gap sensor that senses the opening size in the lower sieve 124. They can include a moisture sensor of materials other than grain (MOG) that can be configured to sense the moisture content of materials other than grain passing through the combine 100. They can include a machine settings sensor configured to sense various configurable settings on the combine 100. They can also include a machine direction sensor that can be any of a variety of different types of sensors that can detect the direction or attitude of the combine 100. Crop characteristics sensors can sense a variety of different types of crop characteristics, such as crop type, crop moisture, and other crop characteristics. They can also be configured to sense characteristics of the crop as it is being processed by the combine 100. For example, they can sense the grain feed rate as the grain travels through the clean grain elevator 130. They can sense yield as a function of the location from which the grain was harvested, as indicated by the location sensor 157, or they can provide other output signals indicative of other sensed variables. Some other examples of types of sensors that can be used are described below.

[0026] Again, it will be noted that, Figure 1 Only one example of a machine 100 is shown. Other machines can also be used, such as forage harvesters, cotton harvesters, sugar cane harvesters, etc.

[0027] Figure 2 FIG. 18 is a block diagram illustrating one example of a computing system architecture 180. In Figure 2 In the example shown, the architecture 180 shows an aerial image capture system 182 that captures a set of aerial images, and / or other image capture systems 183 can capture other images. The images 184 represent the spectral response captured by one or more of the image capture systems 182 and 183 and are provided to a predicted map generation system 186. The predicted map generation system 186 selects the images 184 to use in generating a predicted map 188. The predicted map 188 is then provided to the control system of the harvester 100, where it is used to control the harvester 100.

[0028] Note that in Figure 2 the predictive map generation system 186 is shown as being separate from the harvester 100. Thus, it can be placed on a remote computing system or elsewhere. However, in another example, the predictive map generation system 186 can also be provided on the harvester 100. These and other architectures are contemplated herein. Before describing the overall operation of the architecture 180 in more detail, a brief description of certain items in the architecture 180 and their operation will first be provided.

[0029] The aerial image capture system 182 can be any type of system that captures aerial images (or spectral responses) of a field that is being harvested by the harvester 100 or is to be harvested by the harvester 100. Thus, the system 182 can include a satellite-based system in which satellite images are generated as the images 184. It can be a system that uses unmanned aerial vehicles or manned aircraft to capture the images 184. It can also be another type of aerial image capture system. The other image capture system 183 can be another type of ground-based image capture system that also captures spectral responses of the field.

[0030] The predictive map generation system 186 illustratively includes one or more processors or servers 190, a communication system 192, a data store 194, a spectral analysis system 196, an image selection logic circuit 198, a map generator logic circuit 200, a map correction logic circuit 202, and can include various other items 204. The spectral analysis system 196 can include an image quality identifier logic circuit 205, a vegetation index metric value identifier logic circuit 206 (which itself can include a leaf area index logic circuit 208, a crop orientation detector logic circuit 209, an NDVI logic circuit 210, other remote sensing index logic circuits 211, and / or other logic circuits 212), an image analysis logic circuit 214 (which itself can include a sufficient plant growth identifier 216, an amplitude identifier 215, a distribution identifier 217, a variability identifier 218, and can include other items 220), and other items 222. The image selection logic circuit 198 itself can include a multi-factor optimization logic circuit 223, a threshold logic circuit 224, a ranking logic circuit 226, and / or other logic circuits 228. In operation, the communication system 192 receives the images 184 from the system 182 and / or 183 or from another system that can store the images. It can also receive other information about the field, such as topography, crop type, environmental and crop conditions, etc. Thus, the communication system 192 can be a system configured to communicate over a wide area network, a local area network, a near field communication network, a cellular communication network, or any other network or combination of networks in a variety of different networks.

[0031] Once the image 184 is received, the spectral analysis system 196 performs spectral analysis on the image, and the image selection logic circuit 198 selects the image for use in generating the map based on the spectral analysis. In one example, an image quality identifier logic circuit 205 performs an initial check to see if the image has sufficient quality. For example, it can look for the presence of clouds, shadows, obstructions, etc. A vegetation index metric value identifier logic circuit 206 identifies a set of vegetation index metric values corresponding to each image received. Those metric values will vary across the image based on the particular metric being calculated. For example, in the case of using leaf area index, a leaf area index logic circuit 208 will generate leaf area index values corresponding to different portions of the aerial image (and thus to different portions of the farm field). In the case of using NDVI, an NDVI logic circuit 210 will generate NDVI metric values for different portions of the image. A crop direction detector logic circuit 211 can detect crop direction (e.g., laid down crops, lodged crops, etc.). Other remote sensing index logic circuits 211 can generate other index metric values for different portions of the image.

[0032] The image analysis logic circuit 214 then determines whether the vegetation index metric values are sufficient for that particular image. It can do so by identifying the range of the amplitude of the vegetation index metric values, how the vegetation index metric values vary and are distributed across the image. These can be performed by an amplitude identifier 215, a variability identifier 218, and a distribution identifier 217. If a leaf area index metric is used, it identifies those features for those metric values across the image. If an NDVI metric is used, it will identify those features for those metric values across the image.

[0033] However, first, a proper plant growth identifier 216 determines whether the image was taken in the presence of proper plant growth. If the image was taken too early in the season (e.g., before the plants have emerged or shortly after the plants have emerged), there can not be sufficient plant growth to produce vegetation index values sufficiently. Thus, a sufficient plant growth identifier 216 analyzes the image to ensure that there is sufficient plant growth reflected in the image so that the vegetation index values are meaningful.

[0034] Assuming that the analyzed image reflects proper plant growth, an amplitude identifier 215 identifies the range of the amplitude of the vegetation index metric values. A distribution identifier 217 identifies their distribution, while a variability identifier 218 then identifies the distribution or variability range of the vegetation index metric values across the image to identify the level of variability. This can be done through distribution analysis, mean trend observation, time series analysis of the way the distribution is trending, standard deviation, or using other tools.

[0035] Once the indications of amplitude, distribution, and variability have been assigned to the images, the image selection logic circuit 198 determines, based on those values, whether the image should be selected for use in the prediction map generation.

[0036] The image selection logic circuit 198 can determine whether an image is suitable in a variety of different ways. For example, the multi-factor optimization logic circuit 223 can use the amplitude, distribution, and variability to perform a multi-factor optimization. The images can be ranked and selected based on the optimization. The threshold logic circuit 224 can compare the values assigned to the images to a threshold. If they satisfy the threshold, the image can be selected as an image to be used for the prediction map generation.

[0037] In another example, the ranking logic circuit 226 can rank the processed images based on one or more of the amplitude, distribution, and / or variability values corresponding to each image. This ranking logic circuit 226 can use the top N images (those with the top N values for amplitude, distribution, and / or variability) to generate the prediction map. However, it should be noted that other mechanisms can also be used that select images based on the amplitude, variability, distribution, mean, or other statistical metric values assigned to the images.

[0038] The map generator logic circuit 200 receives the selected images from the image selection logic circuit 198 and generates a prediction map based on those selected images. In one example, the prediction map is a predicted yield map that generates predicted yield values for different locations in the agricultural field based on the selected images. The communication system 192 then provides the prediction map 198 to the harvester 100 (in examples where the system 186 is separate from the harvester 100) so that it can be used to control the harvester 100.

[0039] At some point, the harvester 100 can sense (or derive) actual yields. In that case, the actual yield values, along with the locations corresponding to those yield values, can be provided back to the system 186, where the map correction logic circuit 202 corrects the prediction map 188 based on the in-situ actual yield values. The corrected prediction map 188 can then be provided to the harvester 100 for continued control.

[0040] Figure 2In one example, harvester 100 includes one or more processors 230, a communication system 232, a data store 234, a set of sensors 236, a control system 238, controllable subsystems 240, operator interface mechanisms 242, and it can include various other items 244. An operator 246 interacts with operator interface mechanisms 242 in order to control and manipulate harvester 100. Thus, operator interface mechanisms 242 can include control levers, steering wheels, joysticks, buttons, pedals, linkages, etc. In the case of operator interface mechanisms 242 including touch-sensitive display screens, they can also include operator-actuatable elements such as links, icons, buttons, etc. that can be actuated using pointing and clicking devices or touch gestures. In the case of operator interface mechanisms 242 including voice processing functionality, they can include microphones, speakers, and other items for receiving voice commands and generating synthetic voice output. They can include a wide variety of other visual, audio, and tactile mechanisms.

[0041] Sensors 236 can include the above-described position sensors 157, a heading sensor 248 that identifies a heading or course that harvester 100 is taking. It can be implemented by processing the outputs of multiple position sensors and inferring a course or otherwise. It can illustratively include a speed sensor 147, a mass flow sensor 247, and a moisture sensor 249 as described above. Mass flow sensor 247 can sense the mass flow of grain into a clean grain tank. This, along with the crop moisture sensed by sensor 249, can be used to generate a yield metric that indicates yield. Sensors 236 can also include a wide variety of other sensors 250.

[0042] Controllable subsystems 240 can include a propulsion subsystem 252, a steering subsystem 254, mechanical actuators 256, a power subsystem 258, a crop processing subsystem 259, and it can include a wide variety of other items 260. Propulsion subsystem 252 can include an engine or other power source that controls the propulsion of harvester 100. Steering subsystem 254 can include actuators that can be actuated to cause harvester 100 to steer. Mechanical actuators 256 can include any of a variety of different types of actuators that can be used to change mechanical settings, change the configuration of the machine, raise and lower a header, change the speed of different subsystems (e.g., fan speed), etc. Power subsystem 258 can be used to control the power utilization of harvester 100. Power subsystem 258 can be used to control how much power is allocated to different subsystems, etc. Crop processing subsystem 259 can include things such as a front end, a threshing subsystem, a cleaning subsystem, a material handling subsystem, and a residue subsystem, all of which are described in detail above with respect to harvester 100. Figure 1 Controllable subsystems 240 can include a propulsion subsystem 252, a steering subsystem 254, mechanical actuators 256, a power subsystem 258, a crop processing subsystem 259, and it can include a wide variety of other items 260. Propulsion subsystem 252 can include an engine or other power source that controls the propulsion of harvester 100. Steering subsystem 254 can include actuators that can be actuated to cause harvester 100 to steer. Mechanical actuators 256 can include any of a variety of different types of actuators that can be used to change mechanical settings, change the configuration of the machine, raise and lower a header, change the speed of different subsystems (e.g., fan speed), etc. Power subsystem 258 can be used to control the power utilization of harvester 100. Power subsystem 258 can be used to control how much power is allocated to different subsystems, etc. Crop processing subsystem 259 can include things such as a front end, a threshing subsystem, a cleaning subsystem, a material handling subsystem, and a residue subsystem, all of which are described in detail above with respect to harvester 100.

[0043] In operation, once harvester 100 receives prediction yield map 188, control system 238 receives the current position of harvester 100 from a position sensor (or other variable indicative of future position) and receives the heading or course of harvester 100, and generates control signals to control controllable subsystems 240 based on prediction map 188 and the current and future position of harvester 100. For example, where prediction map 188 indicates that harvester 100 is about to enter a very high yield portion of the field, then control system 238 can control propulsion subsystem 252 to slow the ground speed of harvester 100 to maintain a generally constant feed rate. Where it indicates that harvester 100 is about to enter a very low yield portion, it can use steering subsystem 254 to divert harvester 100 to a higher yield portion of the field, or can control propulsion subsystem 252 to increase the speed of harvester 100. Control signals can be generated to control mechanical actuators 256 to change mechanical settings, or to control power subsystem 258 to change power utilization or to reallocate power among the subsystems differently, etc. It can also control any crop treatment subsystems 259.

[0044] Figure 3A and 3B FIG. 3 (collectively referred to herein as FIG. 3) shows a flowchart illustrating one example of the operation of architecture 180 in selecting images 184, generating prediction map 188, and using that prediction map 188 to control harvester 100. It is first assumed that aerial image capture systems 182 and / or 183 are deployed to capture images 184 of the field under consideration, which is indicated by block 262 in the flowchart of FIG. 3. As shown by block 264, images 184 can be captured over the course of a day or another period of time, or at different times during the growing season. Images 184 can also be captured in a variety of other ways, which is indicated by block 266.

[0045] The vegetation index metric identifier logic circuit 206 then selects an image to process. This is indicated by block 268. The image quality identifier logic circuit 205 then checks the image quality to determine if it is sufficient for further processing. Image quality can be affected for a variety of reasons, such as the presence of clouds, shadows, obstructions, etc. The image quality is calculated as indicated by block 267. The checking of clouds is indicated by block 269 and the checking of shadows is indicated by block 271. Checking for other things that can affect image quality, such as dust or other obstructions, etc. is indicated in block 273. A determination of whether the image quality is sufficient is indicated in block 275. If not, processing returns to block 268 where another image is selected. If the quality is sufficient, the vegetation index metric identifier logic circuit 206 illustratively assigns vegetation index metrics to different portions of the image (and thus to different locations of the farm that the image represents). This is indicated by block 270. As noted above, the leaf area index logic circuit 208 can generate leaf area index metrics for the image. This is indicated by block 272. The NDVI logic circuit 210 can generate NDVI metrics for the image. This is indicated by block 274. Crop moisture can also be measured or established as indicated by block 276, as can crop orientation as indicated by block 277. Or alternatively, other vegetation index values can be assigned as indicated by block 279.

[0046] The sufficient plant growth identifier 216 then determines whether the aerial image was taken at a point in time that sufficient plant growth is reflected in the image. This is done by determining whether the vegetation index values corresponding to the image have values that show sufficient active plant growth. Block 278 in the flowchart of FIG. 3 indicates the determination of whether the image shows sufficient plant growth.

[0047] The image analysis logic circuit 214 then performs additional image analysis on the selected image to assign it a level of magnitude, distribution, and variability exhibited in the image. This is indicated by block 280 in the flowchart of FIG. 3. In one example, the magnitude identifier 215 identifies the magnitude of the range of vegetation index values. This is indicated by block 282. The distribution identifier 217 identifies the distribution of those values. This is indicated by block 283. The variability identifier 218 calculates the variation of the vegetation index values on the image (or the variability of those values). This is indicated by block 284. Image analysis can also be performed in other ways, and this is indicated by block 285.

[0048] The spectral analysis system 196 then determines whether there are more images 184 to process. This is indicated by block 286. If so, processing returns to block 268 where the next image is selected for processing.

[0049] If there are no more images to process, the image selection logic circuit 198 selects one or more images to generate the prediction map 188 based on the level of magnitude, distribution, and / or variability corresponding to each image. This is indicated by block 288 in the flowchart of FIG. 3.

[0050] In one example, the multi-factor optimization logic circuit 223 performs optimization based on the magnitude, variability, and distribution of the vegetation index metric values. This is indicated by block 289. In another example, the threshold logic circuit 224 compares the variability (or other characteristic) value of each image to a corresponding threshold to determine if the image is sufficient. This is indicated by block 290. In another example, the ranking logic circuit 226 ranks the processed images by the above-discussed characteristics to identify those images with the highest (e.g., best) values and selects those images. This is indicated by block 292. The images can also be selected in other ways based on the magnitude, distribution, and / or variability values. This is indicated by block 294.

[0051] The map generator logic circuit 200 then generates the prediction map 188 using the selected images. This is indicated by block 296 in the flowchart of FIG. 3. In one example, the map generator logic circuit 200 can also use other information to generate the map. This information can include such as topography, crop type, crop characteristics (e.g., crop moisture, etc.), environmental conditions (e.g., soil moisture, weather, etc.), or prediction models based on data modeling. Block 298 indicates that other variables are considered in generating the map. The map can be a predicted yield map that gives predicted yield values for different areas in the field. This is indicated by block 300. The prediction map can be generated in other ways and also represents other crop characteristics such as biomass, moisture, protein, starch, oil, etc., and is indicated by block 302.

[0052] The prediction map 188 is then output to control the harvester 100. This is indicated by block 304.

[0053] The control system 238 can then receive sensor signals from the sensors 238 that indicate the position and course of the harvester (or indicate the direction in which the harvester is traveling or other values). This is indicated by block 306. The sensor values can include a current position 308, a speed 310, a heading 312, and / or a variety of other values 314.

[0054] Based on the location of the harvester, the direction of travel of the harvester, and also based on the values in the predictive map, the control system 238 generates harvester control signals to control one or more controllable subsystems 240. The generation of the control signals is indicated by block 316. The control signals are then applied to the controllable subsystems to control the harvester 100. This is indicated by block 318. It can be done in a variety of different ways. For example, as indicated by block 320, it can control the propulsion / speed of the harvester 100 (e.g., to maintain a constant feed rate, etc.). It can control the route or direction of the harvester 100 by controlling the steering subsystem 254. This is indicated by block 322. It can control any of a variety of different types of mechanical actuators 256, as indicated by block 324. It can control the power subsystem 258 to control the power utilization or power distribution among the subsystems, as indicated by block 326. It can control any of the crop handling subsystems 259, as indicated by block 327. It can apply control signals to different controllable subsystems, thereby controlling the harvester 100 in different ways as well. This is indicated by block 328.

[0055] As noted above, the sensors 236 can include a mass flow sensor 247 and a moisture sensor 249, which can be used to derive the actual yield on the harvester 100. This is indicated by block 330. The actual yield, as well as the location at which it was achieved, can be fed back to the map correction logic circuit 202, which makes any needed corrections to the predictive map 188 based on the actual yield values derived from the information sensed from the sensors 247 and 249. This is indicated by block 332. For example, if the actual yield consistently differs from the predicted yield by a certain value or function, that value or function can be applied to the remaining yield values on the predictive map as a correction value. This is just one example.

[0056] When the harvester completes harvesting of the field, the operation is complete. Until then, processing can return to block 304, where the corrected map (if any corrections were made) is provided back to the harvester control system 238, which uses the corrected map to control the harvester. Block 334 indicates a determination of whether the operation is complete.

[0057] It is noted that the above discussion has described various different systems, components and / or logic circuits. It is to be understood that such systems, components and / or logic circuits can be made up of hardware items, such as processors and associated memory or other processing components, some of which are described below, which perform the functions associated with those systems, components and / or logic circuits. In addition, as described below, systems, components and / or logic circuits can be made up of software which is loaded into memory and subsequently executed by a processor or server or other computing component. Systems, components and / or logic circuits can also include different combinations of hardware, software, firmware, etc., some examples of which are described below. These are merely some examples of the different structures which can be used to form the systems, components and / or logic circuits described above. Other structures can also be used.

[0058] The present discussion has mentioned processors and servers. In one embodiment, the processors and servers include computer processors with associated memory and timing circuitry, not separately shown, which is collectively provided as processor 170 in the aforementioned system 100. The processors and servers are functional parts of the systems or devices to which they belong, and are activated by, and facilitate the functionality of, the other components or items in those systems.

[0059] In addition, a number of user interface displays have been discussed. They can take a variety of different forms, and can have a variety of different user-actuatable input mechanisms provided thereon. For example, the user-actuatable input mechanisms can be text boxes, check boxes, icons, links, drop-down menus, search boxes, etc. The user-actuatable input mechanisms can also be actuated in a variety of different ways. For example, a pointing device (e.g., a trackball or mouse) can be used to actuate the user-actuatable input mechanisms. The user-actuatable input mechanisms can be actuated using hardware buttons, switches, a joystick or keyboard, thumb switches or thumb pads, etc. The user-actuatable input mechanisms can also be actuated using a virtual keyboard or other virtual actuators. In addition, where the screen displaying the user-actuatable input mechanisms is a touch-sensitive screen, touch gestures can be used to actuate the user-actuatable input mechanisms. Also, where the device displaying the user-actuatable input mechanisms has speech recognition components, speech commands can be used to actuate the user-actuatable input mechanisms.

[0060] A number of data stores have also been discussed. It is noted that they can be divided into multiple data stores. All of the stores can be local to the system accessing them, all of the stores can be remote, or some stores can be local and some stores remote. All of these configurations are contemplated herein.

[0061] Furthermore, the drawings illustrate a plurality of blocks that are related to the functionality of each block. It should be noted that fewer blocks can be used to implement the functionality, and more blocks can be used to implement the functionality. Furthermore, the functionality of the blocks can be distributed among more blocks than are shown.

[0062] Figure 4 is Figure 2 the block diagram of the harvester 100 shown in FIG. 1, except that it is in communication with elements in a remote server architecture 500. In one example, the remote server architecture 500 can provide computing, software, data access and storage services that do not require an end user to know the physical location or configuration of the system providing the services. In various examples, the remote server can provide services over a wide area network such as the Internet using appropriate protocols. For example, the remote server can provide an application program over a wide area network, and the remote server can be accessed through a web browser or any other computing component. Figure 2 The software or components shown in FIG. 1 and the corresponding data can be stored on servers at a remote location. Computing resources in a remote server environment can be consolidated into a remote data center location or can be dispersed. The remote server infrastructure can provide services through a shared data center even though the remote server infrastructure appears as a single point of access to the user. Thus, the components and functionality described herein can be provided from a remote server at a remote location using a remote server architecture. Alternatively, the components and functionality described herein can be provided from a traditional server or can be installed directly or otherwise on a client device.

[0063] In Figure 4 the example shown in FIG. 1, some items are similar to the items shown in FIG. 1, and they are similarly numbered. Figure 2 Figure 4 It is specifically shown that the predictive map generation system 186 can be located at the remote server location 502. Thus, the harvester 100 accesses those systems through the remote server location 502.

[0064] Figure 4 Another example of a remote server architecture is also depicted. Figure 4 It is shown that some of the elements of Figure 2 the harvester 100 can be located at the remote server location 502, while other elements are not located at the remote server location 502. By way of example, the data store 194 can be located at a location separate from the location 502 and accessed through a remote server at the location 502. Regardless of where they are located, they can be accessed directly by the harvester 100 through a network (wide area network or local area network), they can be hosted through a service at a remote site, or they can be provided as a service or accessed through a connection service resident at a remote location.

[0065] ​Moreover, data can be stored substantially anywhere and can be accessed intermittently by interested parties or forwarded to interested parties. For example, a physical carrier can be used instead of or in addition to an electromagnetic wave carrier. In such an example, another mobile machine (e.g., a fuel truck) can have an automated information collection system in the event of poor or non-existent cell coverage. As the harvester approaches the fuel truck to add fuel, the system can automatically collect information from or deliver information to the harvester using any type of ad-hoc wireless connection. The collected information can then be forwarded to the main network when the fuel truck reaches a location where cellular coverage (or other wireless coverage) exists. For example, the fuel truck can enter a covered location when it travels to add fuel to other machines or to travel to a main fuel storage location. All of these architectures are contemplated herein. Moreover, information can be stored on the harvester until the harvester enters a covered location. The harvester itself can send and receive information to and from the main network.

[0066] It should also be noted that, Figure 2 Elements of or portions of them can be provided on a variety of different devices. Some of these devices include servers, desktop computers, laptop computers, tablet computers, or other mobile devices such as palmtop computers, cell phones, smart phones, multimedia players, personal digital assistants, and the like.

[0067] Figure 5 is a simplified block diagram of one illustrative example of a handheld or mobile computing device that can be used as a user handheld device 16 or a client handheld device 16 in which the present system (or portions of it) can be deployed. For example, a mobile device can be deployed in the cab of a harvester 100 for generating, processing, or displaying seat width and position data. Figures 6-7 is an example of a handheld or mobile device.

[0068] Figure 5 A general block diagram of components of a client device 16 is provided that can run some of the components shown in Figure 2 , interact with these components, or both. In the device 16, a communication link 13 is provided that allows the handheld device to communicate with other computing devices and, in some embodiments, provides a channel for automatically receiving information, for example, by scanning. Examples of the communication link 13 include allowing communication over one or more communication protocols (e.g., wireless services for providing cellular access to a network, and protocols for providing local wireless connections to a network).

[0069] In other examples, the application program can be received on a removable secure digital (SD) card connected to interface 15. Interface 15 and communication link 13 communicate with processor 17 (which can also incorporate the processor or server from the previous figures) along bus 19, which is also connected to memory 21 and input / output (I / O) components 23, as well as clock 25 and positioning system 27.

[0070] In one example, I / O components 23 are provided to facilitate input and output operations. I / O components 23 for various embodiments of device 16 can include input components, such as buttons, touch sensors, optical sensors, microphones, touch screens, proximity sensors, accelerometers, orientation sensors, and output components, such as display devices, speakers, and / or printer ports. Other I / O components 23 can also be used.

[0071] Clock 25 illustratively includes a real-time clock component that outputs time and date. It can also illustratively provide timing functions for processor 17.

[0072] Positioning system 27 illustratively includes a component that outputs a current geographic position of device 16. This can include, for example, a global positioning system (GPS) receiver, a LORAN system, a dead reckoning system, a cellular triangulation system, or other positioning. The positioning system can also include, for example, map software or navigation software that generates desired maps, navigational routes, and other geographic functions.

[0073] Memory 21 stores operating system 29, network settings 31, application programs 33, application configuration settings 35, data store 37, communication drivers 39, and communication configuration settings 41. Memory 21 can include all types of tangible volatile and non-volatile computer-readable memory devices. Memory 21 can also include computer storage media (as described below). Memory 21 stores computer readable instructions that, when executed by processor 17, cause the processor to perform computer-implemented steps or functions according to the instructions. Processor 17 can also be activated by other components to facilitate their functions.

[0074] Figure 6 One example is shown in which device 16 is a tablet computer 600. In Figure 6 In this example, computer 600 displays a user interface display 602. Display 602 can be a touch screen, or a pen-enabled interface that receives input from a pen or stylus. It can also use a virtual keyboard on the screen. Of course, it can also be attached to a keyboard or other user input device through a suitable attachment mechanism, such as a wireless link or a USB port. Computer 600 can also illustratively receive voice input.

[0075] Figure 7It is shown that the device can be a smartphone 71. The smartphone 71 has a touch-sensitive display 73 that displays icons or tiles or other user input mechanisms 75. A user can use the user input mechanisms 75 to run applications, make calls, perform data transfers, and the like. Typically, the smartphone 71 builds on a mobile operating system and provides more advanced computing capability and connectivity than a feature phone.

[0076] Note that other forms of the device 16 are possible.

[0077] Figure 8 is one example of a computing environment in which Figure 2 elements or portions of these elements (e.g.) can be deployed. With reference to Figure 8 , an example system for implementing some embodiments includes a computing device in the form of a computer 810. The components of computer 810 can include, but are not limited to, a processing unit 820 (which can include a processor or server from the previous figures), a system memory 830, and a system bus 821 that couples various system components including the system memory to the processing unit 820. The system bus 821 can be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. With reference to Figure 2 the memory and programs described can be deployed in respective portions of Figure 8 .

[0078] The computer 810 typically includes a variety of computer readable media. Computer readable media can be any available media that can be accessed by computer 810 and includes both volatile and nonvolatile media, removable and non-removable media. By way of example, and not limitation, computer readable media can comprise computer storage media and communication media. Computer storage media is different from, and does not include, a modulated data signal or carrier wave. It includes hardware storage media including both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by computer 810. Communication media can be embodied in a modulated data signal, specifically a carrier wave or other transport mechanism accessible via a wired, wireless, or other communication medium. The term "modulated data signal" means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal.

[0079] System memory 830 includes computer storage media in the form of volatile and / or non-volatile memory, such as read-only memory (ROM) 831 and random access memory (RAM) 832. A basic input / output system 833 (BIOS) is typically stored in ROM 831, containing basic routines that facilitate the transfer of information between components within computer 810, such as during startup. RAM 832 typically contains data and / or program modules that are readily accessible to and / or currently being operated by processing unit 820. This is by way of example and not limitation. Figure 8 The operating system 834, application program 835, other program modules 836, and program data 837 are shown.

[0080] Computer 810 may also include other removable / non-removable, volatile / non-volatile computer storage media. This is just one example. Figure 8 A hard disk drive 841 (which reads data from or writes data to a non-removable, non-volatile magnetic medium) and an optical disk drive 855 and a non-volatile optical disk 856 are shown. The hard disk drive 841 is typically connected to the system bus 821 via a non-removable memory interface (such as interface 840), and the optical disk drive 855 is typically connected to the system bus 821 via a removable memory interface (such as interface 850).

[0081] Alternatively or additionally, the functions described herein may be performed at least in part by one or more hardware logic circuit components. Examples, but not limited to, illustrative types of hardware logic circuit components that may be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (e.g., ASICs), application-specific standard products (e.g., ASSPs), system-on-a-chip (SoCs), and complex programmable logic devices (CPLDs).

[0082] Above Figure 8 The drives and associated computer storage media discussed and illustrated herein provide storage for computer-readable instructions, data structures, program modules, and other data for computer 810. Figure 8 For example, hard disk drive 841 is shown storing operating system 844, application program 845, other program modules 846, and program data 847. Note that these components may be the same as or different from operating system 834, application program 835, other program modules 836, and program data 837.

[0083] A user can enter commands and information into the computer 810 through input devices such as a keyboard 862, a microphone 863, and a pointing device 861, such as a mouse, trackball or touch pad, and a monitor or other type of display device. Other input devices (not shown) can include a joystick, game pad, satellite dish, scanner, or the like. These and other input devices are often connected to the processing unit 820 through a user input interface 860 that is coupled to the system bus, but can be connected by other interface and bus structures, as will be understood by those skilled in the art. A visual display 891 or other type of display device is also connected to the system bus 821 via an interface, such as a video interface 890. In addition to the monitor, computers can also include other peripheral output devices such as speakers 897 and a printer 896, which can be connected through an output peripheral interface 895.

[0084] The computer 810 is operated in a networked environment using logical connections to one or more remote computers, such as a remote computer 880. The remote computer 880 can be a personal computer, a hand-held device, a server, a router, a network PC, a peer device or other common network node, and the like, as

[0085] When used in a LAN networking environment, the computer 810 is connected to the LAN 871 through a network interface or adapter 870. When used in a WAN networking environment, the computer 810 typically includes a modem 872 or other means for establishing communications over the WAN 873, such as the Internet. In a networked environment, program modules can be stored in the remote memory storage device. Figure 8 It is shown that, for example, a remote application 885 can reside on the remote computer 880.

[0086] It should also be noted that the different examples described herein can be combined in different ways. That is, one or more parts of one or more examples can be combined with one or more parts of one or more other examples. All of these are considered to be within the scope of the present description.

[0087] Example 1 is a method of controlling a work machine, comprising:

[0088] receiving spectral responses of a plurality of images at a work site;

[0089] identifying a set of vegetation index metric values based on the spectral responses;

[0090] identifying a vegetation index signature corresponding to each image, the vegetation index signature indicating how the set of vegetation index metric values vary in the corresponding image;

[0091] selecting an image from the plurality of images based on the vegetation index signature;

[0092] generating a prediction map from the selected image; and

[0093] Controlling a controllable subsystem of the work machine based on the location of the work machine and the predicted map.

[0094] Example 2 is the method of any or all of the preceding examples, wherein identifying the vegetation index feature comprises:

[0095] determining a magnitude of a range of the vegetation index metric values, a distribution of the vegetation index metric values, and a variability of the vegetation index metric values.

[0096] Example 3 is the method of any or all of the preceding examples, wherein selecting the images comprises:

[0097] selecting the set of images from the plurality of images based on the vegetation index features corresponding to the images in the set of images.

[0098] Example 4 is the method of any or all of the preceding examples, wherein generating the predicted yield map further comprises:

[0099] generating the predicted yield map based on the selected set of images.

[0100] Example 5 is the method of any or all of the preceding examples, wherein identifying the vegetation index feature comprises:

[0101] calculating a set of image spectral values in a spectral response of the corresponding image; and

[0102] identifying a variability of the set of image spectral values over a set of vegetation index metric values.

[0103] Example 6 is the method of any or all of the preceding examples, wherein wherein selecting the set of images comprises:

[0104] selecting the set of images having vegetation index features that exhibit greater variability than vegetation index features of unselected images.

[0105] Example 7 is the method of any or all of the preceding examples, wherein selecting the set of images comprises:

[0106] selecting the set of images having vegetation index features that satisfy a vegetation index feature threshold.

[0107] Example 8 is the method of any or all of the preceding examples, wherein selecting the set of images comprises:

[0108] selecting the set of images based on a vegetation distribution represented in the images that inhibits spectral saturation and reflects a predetermined level of plant growth.

[0109] Example 9 is the method of any or all of the preceding examples, wherein controlling the controllable subsystem comprises controlling a mechanical actuator.

[0110] Example 10 is the method of any or all of the preceding examples, wherein controlling the controllable subsystem comprises controlling a propulsion subsystem.

[0111] Example 11 is the method of any or all of the preceding examples, wherein controlling the controllable subsystem comprises controlling a steering subsystem.

[0112] Example 12 is the method of any or all of the preceding examples, wherein controlling the controllable subsystem comprises controlling a crop treatment subsystem.

[0113] Example 13 is a work machine comprising:

[0114] a communication system configured to receive a plurality of images of vegetation at a work site;

[0115] a controllable subsystem;

[0116] an image selector configured to generate a vegetation index feature and select a set of images based on the vegetation index feature, the vegetation index feature comprising variability, distribution, and amplitude of vegetation index metric values corresponding to each image and indicative of how the vegetation index metric values vary across each image;

[0117] a processor configured to generate a predictive map based on the selected set of images; and

[0118] subsystem control logic configured to control the controllable subsystem of the work machine based on a location of the work machine and the predictive map.

[0119] Example 14 is the work machine of any or all of the preceding examples, wherein the image selector comprises:

[0120] a variability identifier logic configured to identify a set of vegetation index metric values corresponding to an image and determine variability of the vegetation index feature across the set of vegetation index metric values.

[0121] Example 15 is the work machine of any or all of the preceding examples, wherein the processor is configured to generate a predictive yield map based on the selected set of images.

[0122] Example 16 is the work machine of any or all of the preceding examples, wherein the variability identifier logic is configured to identify a set of leaf area index metric values corresponding to an image and identify variability of the set of leaf area index metric values across the set of vegetation index metric values.

[0123] Example 17 is the agricultural work vehicle of any preceding or all preceding examples, wherein the variability identifier logic circuit is configured to identify a set of normalized difference vegetation index metric values for the corresponding image and identify variability of the set of normalized difference vegetation index metric values over the set of vegetation index metric values.

[0124] Example 18 is the agricultural work vehicle of any preceding or all preceding examples, wherein the image selector is configured to select a set of images having vegetation index features that exhibit greater vegetation variability than vegetation index features of unselected images.

[0125] Example 19 is the agricultural work vehicle of any preceding or all preceding examples, wherein the image selector is configured to select a set of images having vegetation index features that satisfy a vegetation index feature threshold.

[0126] Example 20 is an image selection system comprising:

[0127] a communication system configured to receive a plurality of images of vegetation at a work site;

[0128] an image selector configured to generate a vegetation index variability metric and select an image based on the vegetation index variability metric, the vegetation index variability metric corresponding to each image and indicating how vegetation index values vary over each image; and

[0129] a processor configured to generate at least one predictive map based on the selected image.

[0130] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. A method of controlling a work machine, comprising: receiving spectral responses of a plurality of images at a work site; identifying a set of vegetation index metric values based on the spectral responses; identifying a vegetation index signature corresponding to each image, the vegetation index signature indicating how the set of vegetation index metric values vary in the corresponding image; selecting an image from the plurality of images based on the vegetation index signature; generating a predictive map from the selected image; and controlling a controllable subsystem of the work machine based on a location of the work machine and the predictive map. Identifying the vegetation index signature includes:

2. The method of claim 1, wherein, determining a magnitude of a range of the vegetation index metric values, a distribution of the vegetation index metric values, and a variability of the vegetation index metric values.

3. The method of claim 2, wherein selecting the image includes: selecting the set of images from the plurality of images based on the vegetation index signature corresponding to the images in the set of images. Generating the predictive map further includes:

4. The method of claim 3, wherein, generating a predictive yield map based on the selected set of images. Identifying the vegetation index signature includes:

5. The method of claim 1, wherein, calculating a set of image spectral values in the spectral response of the corresponding image; and identifying a variability of the set of image spectral values over the set of vegetation index metric values.

6. The method of claim 3, wherein selecting the set of images includes: selecting the set of images having vegetation index signatures exhibiting greater variation than vegetation index signatures of unselected images.

7. The method of claim 3, wherein selecting the set of images includes: selecting the set of images having vegetation index signatures satisfying a vegetation index signature threshold.

8. The method of claim 4, wherein selecting the set of images includes: selecting the set of images based on a vegetation distribution represented in the images that inhibits spectral saturation and reflects a predetermined level of plant growth.

9. The method of claim 1, wherein controlling the controllable subsystem includes controlling a machine actuator. Controlling the controllable subsystem includes controlling a propulsion subsystem.

10. The method of claim 1, wherein, Controlling the controllable subsystem includes controlling a steering subsystem.

11. The method of claim 1, wherein, Controlling the controllable subsystem includes controlling a crop treatment subsystem.

12. The method of claim 1, wherein, 13. A work machine, comprising: a communication system configured to receive a plurality of images of vegetation at a work site; a controllable subsystem; an image selector configured to generate a vegetation index signature and select a set of images based on the vegetation index signature, the vegetation index signature including a variability, a distribution, and a magnitude corresponding to each image and indicating how the vegetation index metric values vary over each image; a processor configured to generate a predictive map based on the selected set of images; and subsystem control logic configured to control the controllable subsystem of the work machine based on a location of the work machine and the predictive map. The image selector includes: a variability identifier logic configured to identify a set of vegetation index metric values of the corresponding image and determine a variability of the vegetation index signature over the set of vegetation index metric values.

14. The work machine of claim 13, wherein, The processor is configured to generate a predictive yield map based on the selected set of images. ​ 15. The work machine of claim 14, wherein, ​ 16. The work machine of claim 15, wherein, The variability identifier logic circuit is configured to identify a set of leaf area index metric values corresponding to the image and to identify variability of the set of leaf area index metric values over the set of vegetation index metric values.

17. The work machine of claim 15, wherein, The variability identifier logic circuit is configured to identify a set of normalized difference vegetation index metric values corresponding to the image and to identify variability of the set of normalized difference vegetation index metric values over the set of vegetation index metric values.

18. The work machine of claim 15, wherein, The image selector is configured to select a set of images having vegetation index features that exhibit greater vegetation variability than vegetation index features of non-selected images.

19. The work machine of claim 15, wherein, The image selector is configured to select a set of images having vegetation index features that satisfy a vegetation index feature threshold.

20. An image selection system comprising: a communication system configured to receive a plurality of images of vegetation at a worksite; an image selector configured to generate a vegetation index variability metric corresponding to each image and indicating how vegetation index values vary over each image and to select images based on the vegetation index variability metric; and a processor configured to generate at least one prediction map based on the selected images.

Images