Supervision and improvement systems for machine control

Abstract

The mobile machine includes a propulsion subsystem that propels the mobile machine across an operating environment. The mobile machine also includes: machine monitoring logic that receives sensor signals indicating values ​​of sensed machine variables; and operator monitoring logic that receives operator sensor signals indicating values ​​of sensed operator variables. The mobile machine also includes performance indicator generator logic that receives the sensed machine variables and the sensed operator values ​​and generates a performance indicator based on the sensed machine variables and the sensed operator values. The mobile machine also includes: an optimization system that accesses historical contextual data and receives the performance indicator and generates an optimization signal based on the performance indicator and the historical contextual data; and control signal generator logic that generates a control signal based on the optimization signal to perform a machine operation.

Description

Technical Field

[0001] This specification relates to controlling machines. More particularly, this specification relates to using machine learning and non-traditional data sources to improve overall machine control. Background Art

[0002] There are a variety of different types of mobile machines. These machines may include agricultural machines, forestry machines, construction machines, lawn care machines, etc. These machines can sometimes be difficult to control as desired.

[0003] However, even if this were not the case, it can be difficult to determine how best to operate the machine in different situations to achieve the desired performance in terms of efficiency or other aspects. This problem is exacerbated when considering the operation of a fleet of highly configurable machines (more than one machine), such as a combine harvester. It can be difficult to know how to control each of those machines in order to improve the overall performance of the individual machines (and thus the fleet).

[0004] The above discussion is provided for general background information only and is not intended to be used as an aid in determining the scope of the claimed subject matter. Summary of the Invention

[0005] The mobile machine includes a propulsion subsystem that propels the mobile machine across an operating environment. The mobile machine also includes: machine monitoring logic that receives sensor signals indicating values ​​of sensed machine variables; and operator monitoring logic that receives operator sensor signals indicating values ​​of sensed operator variables. The mobile machine also includes performance indicator generator logic that receives the sensed machine variables and the sensed operator values ​​and generates a performance indicator based on the sensed machine variables and the sensed operator values. The mobile machine also includes: an optimization system that accesses historical contextual data and receives the performance indicator and generates an optimization signal based on the performance indicator and the historical contextual data; and control signal generator logic that generates a control signal based on the optimization signal to perform a machine operation.

[0006] This Summary is provided to introduce a selection of concepts in a simplified form, which are further described in the Detailed Description below. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter. The claimed subject matter is not limited to implementations that solve any or all of the disadvantages identified in the Background. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 It is a partial diagram and partial schematic diagram of a combine harvester.

[0008] Figure 2 is a block diagram illustrating one example of a combine harvester in more detail.

[0009] Figure 3 is a block diagram illustrating one example of a supervision or optimization (or improvement) system in more detail.

[0010] Figure 4 is a flow chart illustrating one example of the operation of a supervisory or optimization (or improvement) system.

[0011] Figure 5 is a block diagram illustrating an example power allocation workflow.

[0012] Figure 6 is a block diagram illustrating one example of a mobile machine arranged in a remote server architecture.

[0013] Figures 7 to 9 An example of a mobile device that may be used in the architecture shown in the previous figures is shown.

[0014] Figure 10 is a block diagram illustrating one example of a computing environment that may be used in the architecture shown in the previous figures. DETAILED DESCRIPTION

[0015] As just one example, a mobile machine may include a combine harvester. The operation of these machines may be relatively complex, and they may be operated to attempt to enhance a performance value or index based on one of many different dimensions calculated based on machine sensor feedback. As an example, a combine harvester may be controlled in a manner to improve its performance in terms of yield (e.g., tons of grain harvested per hour). However, when resources are allocated to a subsystem to increase yield, this may reduce the resources allocated to another subsystem at the expense of another performance category (e.g., threshing quality or grain loss). Similarly, by increasing the rate (yield) of harvesting grain, performance may also deteriorate in other performance categories (e.g., grain loss and possibly even grain quality).

[0016] In some current systems, operators and even control systems attempt to allocate resources across various subsystems in order to improve a single performance category. However, this can reduce the overall efficiency of the machine (and, in this way, entire fleets of multiple machines) when performing operations, as the system is complex and a subtle adjustment in one subsystem can affect machine operation in unexpected ways. Instead of focusing on a single category (e.g., productivity), it is possible to provide a supervisory system and consider both productivity and fuel economy together when controlling a machine by modifying the operation of one subsystem (e.g., vehicle speed). When implemented on a larger scale, this can improve the overall efficiency of a machine and / or fleet.

[0017] This is difficult in current systems because performance category trade-offs and subsystem performance trade-offs are difficult for operators or simple control systems to predict in an ever-changing environment across many machines.

[0018] Therefore, this specification proposes a supervision and optimization (or improvement) system that uses a neural network or deep machine learning system to analyze situation data (e.g., machine settings, environmental factors, operator sensor feedback, site characteristics, etc.) and the performance / consumption indicators resulting from the situation data. In this way, the system can "learn" from a large amount of historical and / or runtime situation data and the indicators they result in when controlling the mobile machine to the optimal performance level. In addition, the supervision and optimization (or improvement) system can receive input from non-traditional sources (e.g., weather sources, yield maps, terrain maps, and human observation / interaction). Control can be performed in an automated closed-loop manner, or different control operations can be exposed to the operator and manually executed, or control can be performed as a combination of both automated and manual control operations.

[0019] Figure 1 is a partially pictorial, partially schematic illustration of a self-propelled agricultural harvester 100 in the example where the machine 100 is a combine harvester. Figure 1 As can be seen in the accompanying drawings, the combine harvester 100 illustratively includes an operator cab 101 which may have a variety of different operator interface mechanisms for controlling the combine harvester 100. The combine harvester 100 may include a set of front-end equipment which may include a header 102 and a cutter, generally indicated at 104. It may also include a feeder housing 106, a feed accelerator 108, and a thresher, generally indicated at 110. The header 102 is pivotally coupled to a frame 103 of the combine harvester 100 along a pivot axis 105. One or more actuators 107 drive the header 102 to move about the axis 105 in a direction generally indicated by arrow 109. Thus, the vertical position of the header 102 above the ground 111 on which it travels may be controlled by actuating the actuators 107. Although Figure 1 Not shown in FIG, it is possible that the pitch (or roll) angle of the header 102 or portions of the header 102 can be controlled by separate actuators. Pitch or roll refers to the orientation of the header 102 about the fore-aft longitudinal axis of the combine harvester 100.

[0020] The thresher 110 illustratively includes a threshing rotor 112 and a set of recesses 114. Furthermore, the combine 100 may include a separator 116, which includes a separator rotor. The combine 100 may include a grain cleaning subsystem (or grain cleaning chamber) 118, which itself may include a grain cleaning fan 120, a chaff screen 122, and a screen 124. The material handling subsystem in the combine 100 may include (in addition to the feeder housing 106 and the feed accelerator 108) a discharge agitator 126, a tailings elevator 128, a clean grain elevator 130 (which moves clean grain into a clean grain bin 132), and an unloading auger 134 and spout 136. The combine 100 may also include a residue subsystem 138, which may include a chopper 140 and a spreader 142. The combine 100 may also have a propulsion subsystem, which includes an engine that drives ground-engaging wheels 144 or tracks, etc. It will be noted that the combine harvester 100 may also have more than one of any of the subsystems described above (eg, left and right cleanrooms, separators, etc.).

[0021] In operation, as an overview, combine harvester 100 illustratively moves across a field in the direction indicated by arrow 146. As it moves, header 102 engages the crop to be harvested and gathers it toward cutterhead 104. An operator illustratively sets a height setting (and possibly a pitch or roll angle setting) for header 102, and a control system (described below) controls actuator 107 (and possibly a pitch or roll actuator (not shown)) to maintain header 102 at the set height above ground 111 (and at a desired roll angle). The control system responds to header errors (e.g., the difference between the set height and the measured height of header 104 above ground 111, and possibly roll angle errors) with a responsiveness determined based on a set sensitivity level. If the sensitivity level is set higher, the control system responds to smaller header position errors and attempts to reduce them more quickly than if the sensitivity level is set lower.

[0022] After the crop is cut by the cutter 104, it is moved via a conveyor in the feeder housing 106 toward the feed accelerator 108, which accelerates the crop into the thresher 110. The crop is threshed by the rotor 112, which rotates against the recess 114. The separator rotor moves the threshed crop in the separator 116, where the discharge agitator 126 moves some of the residue toward the residue subsystem 138. This residue can be chopped by the residue chopper 140 and spread across the field by the spreader 142. In other configurations, the residue is simply chopped and dropped into a pile, rather than being chopped and spread.

[0023] The grain falls into the clean grain chamber (or clean grain subsystem) 118. A chaff screen 122 separates some larger material from the grain, and a screen 124 separates some fine material from the clean grain. The clean grain falls onto an auger in the clean grain elevator 130, which moves the clean grain upward and deposits it in a clean grain bin 132. Residue can be removed from the clean grain chamber 118 by the airflow generated by the clean grain fan 120. The clean grain fan 120 directs air upward along an airflow path through the screen and chaff screen, and the airflow also carries residue back into the combine harvester 100 toward the residue handling subsystem 138.

[0024] The tailings elevator 128 can move the tailings back to the thresher 110 where they can be re-threshed. Alternatively, the tailings can be transferred to a separate re-threshing mechanism (also using a tailings elevator or another transport mechanism) where they can also be re-threshed.

[0025] Figure 1 Also shown in one example, the combine harvester 100 may include a ground speed sensor 147, one or more separator loss sensors 148, a clean grain camera 150, and one or more clean grain chamber loss sensors 152. The ground speed sensor 146 illustratively senses the speed at which the combine harvester 100 is traveling over the ground. This can be accomplished by sensing the speed of rotation of wheels, drive shafts, axles, or other components. The travel speed may also be sensed by a positioning system, such as a global positioning system (GPS), a dead reckoning system, a LORAN system, or a variety of other systems or sensors that provide an indication of travel speed.

[0026] The clean grain chamber loss sensor 152 illustratively provides output signals indicating the amount of grain loss on the right and left sides of the clean grain chamber 118. In one example, the sensor 152 is an impact sensor that counts grain impacts per unit time (or per unit distance traveled) to provide an indication of grain loss within the clean grain chamber. The impact sensors on the right and left sides of the clean grain chamber can provide separate signals or a combined or aggregated signal. It should be noted that the sensor 152 can also include a single sensor, rather than a separate sensor for each clean grain chamber.

[0027] The separator loss sensor 148 provides a signal indicating grain loss in the left and right separators. The sensors associated with the left and right separators can provide separate grain loss signals or a combined or aggregated signal. This can also be accomplished using a variety of different types of sensors. It will be noted that the separator loss sensor 148 can also include only a single sensor, rather than separate left and right sensors.

[0028] It will also be understood that the sensors and measurement mechanisms (in addition to the sensors already described) may also include other sensors on the combine harvester 100. For example, they may include a header height sensor that senses the height of the header 102 above the ground 111. They may include a stability sensor that senses the oscillation or jitter (and amplitude) of the combine harvester 100. They may include a residue setting sensor that is configured to sense whether the machine 100 is configured to chop residue, drop it into a pile, etc. They may include a cleaning chamber fan speed sensor that may be positioned near the fan 120 to sense the fan's speed. They may include a threshing gap sensor that senses the gap between the rotor 112 and the recess 114. They may include a threshing rotor speed sensor that senses the rotor speed of the rotor 112. They may include a chaff screen gap sensor that senses the size of the opening in the chaff screen 122. They may include a screen gap sensor that senses the size of the opening in the screen 124. They may include a material other than grain (MOG) moisture sensor that may be configured to sense the moisture level of material other than grain passing through the combine harvester 100. They may include machine setting sensors that are configured to sense various configurable settings on the combine harvester 100. They may also include machine orientation sensors, which may be any of a variety of different types of sensors that sense the orientation of the combine harvester 100. Crop property sensors may sense various different types of crop properties, such as crop type, crop size (e.g., stem width), crop moisture, and other crop properties. They may also be configured to sense characteristics of the crop as the combine harvester 100 processes the crop. For example, they may sense grain feed rate (e.g., mass flow) as the grain passes through the clean grain elevator 130, or provide other output signals indicative of other sensed variables. Environmental sensors may sense soil moisture, soil compaction, weather (which may be sensed or downloaded), temperature, standing water, and other properties of the soil, crop, machine, or environment. Some additional examples of the types of sensors that may be used are described below.

[0029] Figure 2 is a block diagram illustrating portions of the mobile device 100 in greater detail. Figure 2 The machine 100 is shown to include one or more processors 180, sensors 182 (which may be the sensors described above or different or additional sensors), a communication system 184, a control system 186, controllable subsystems 188, interface logic 190, an operator (user) interface mechanism 192, and which may include a variety of other items 194. Figure 2Also shown in one example, an operator 196 interacts with an operator interface mechanism 192 to control and operate the mobile machine 100. Thus, the operator interface mechanism 192 may include things such as levers, pedals, a steering wheel, a joystick, mechanical or other linkages, a user interface display, a display including user-actuable elements (e.g., links, icons, buttons, etc.), and the like. When the user interface mechanism 192 includes speech recognition functionality, the user interface mechanism 192 may include a microphone and a speaker for receiving user voice input and providing audible output. They may also include speech synthesis functionality. Additionally, the user interface mechanism 192 may include other audio, visual, tactile, or other mechanisms.

[0030] The interface logic 190 may illustratively generate outputs on the mechanism 192 and detect user interactions with the mechanism 192. It may provide indications of those interactions with other items in the mobile machine 100, or it may communicate those actions to one or more remote systems using the communication system 184. Thus, the communication system 184 may include a controller area network (CAN) communication system, a near field communication system, a wide area network, a local area network, a cellular communication system, or other communication system or combination of systems.

[0031] The control system 186 illustratively includes sensor signal processing logic 198, standard detection logic 200, a supervisory and optimization system 202, a control signal generator 204, and may include a variety of other items 206. The sensor signal processing logic 198 illustratively receives signals from the sensor 182 and / or other sensors and processes those sensor signals to identify sensed variables. The logic 198 may include conditioning logic that performs such things as filtering, normalization, amplification, linearization, and / or other conditioning operations. It may also perform other signal processing logic, such as aggregation, signal combining, or other processing functions.

[0032] Given the detected context (or in some cases manual input by the operator), the supervisory or optimization (or improvement) system 202 processes the sensor signals to identify different control actions to be performed and understand resource allocation in order to improve the performance of the mobile machine 100 across a number of different dimensions given the current context. Figures 4 and 5 System 202 is described in more detail.

[0033] Once system 202 has identified control operations, signals indicative of those operations are provided to control signal generator logic 204. Logic 204 generates control signals to control one or more of controllable subsystems 188 to perform the control operations identified by system 202, which in turn will improve or optimize machine performance of machine 100.

[0034] The controllable subsystem 188 may include the above Figure 1Any or all of the subsystems discussed (where mobile machine 100 is a combine harvester) may be controlled, or they may include other controllable subsystems (on a combine harvester or other machine). Some examples of controllable subsystems include: a propulsion subsystem 208, which drives the movement of machine 100; a steering subsystem 210, which steers machine 100; a machine setting actuator 212, which is actuated to change machine settings; a power subsystem 214, which can change the power utilization of machine 100; and an automation subsystem 216, which can change the amount of control of machine 100 performed using automated control components. For example, it is possible that steering can be automatically or manually controlled. Similarly, it is possible that the speed of the machine can be automatically or manually controlled. Furthermore, it is possible that the header height of a header on machine 100 can be automatically controlled. These and various other automation systems can be engaged and operated, or they can be disengaged or shut down. Accordingly, these automation subsystems 216 can be controlled (turned on or off, or otherwise configured) via control signals generated by control signal generator logic 204.

[0035] The controllable subsystem 188 may also include a variety of other controllable subsystems. This is indicated by block 218.

[0036] Sensors 182 include machine sensors 183, operator sensors 185, field sensors 187, environmental sensors 189, and may also include other sensors, as indicated by block 191. Machine sensors 183 monitor various components of mobile machine 100. For example, one or more machine sensors 183 may be present for each controllable subsystem 188. For example, machine direction and speed sensors may be present for steering subsystem 210 / propulsion subsystem 208, and / or pressure, temperature, and voltage sensors may be present to monitor various components of power subsystem 214. Of course, these are merely examples, and a variety of other machine sensors may be present.

[0037] Operator sensors 185 may include sensors that detect characteristics related to operator 196. For example, operator sensors 185 may include eye-tracking sensors that track operator 196's eye position and eye movement. This type of sensor can be used to detect where operator 196 is looking, which can indicate what is important to operator 196. In another example, operator sensors 185 include a heart rate monitor that detects operator 196's heart rate. An increase or decrease in heart rate can indicate how operator 196 perceives the performance of mobile machine 100. In another example, operator sensors 185 include body movement sensors. The body movement sensors may be body movement sensors. The body movement sensed by the body movement sensors can indicate machine performance based on conscious body movement (e.g., pointing at a component, hand gestures, etc.) or subconscious body movement (e.g., shaking, psychomotor agitation). In another example, operator sensors 185 may include a breathing sensor that detects operator 196's breathing rate. Holding breath or rapid breathing can indicate that operator 196 is anxious about machine performance. In another example, operator sensors 185 may include a facial expression sensor that detects facial expressions of the user.

[0038] The field sensor 187 may include various sensors that detect characteristics of the field or operating environment. For example, the field sensor 187 may include a crop / vegetation sensor, a moisture sensor, a soil type sensor, and the like.

[0039] Environmental sensors 189 may include sensors that detect various characteristics of the operating environment. For example, environmental sensors 189 may include weather radar, precipitation sensors, ambient temperature sensors, humidity sensors, geolocation sensors, etc. Sensors 182 may also include other types of sensors, as indicated by block 191.

[0040] Figure 3 2 is a block diagram illustrating in greater detail the monitoring and optimization (or improvement) system 202. The monitoring and optimization (or improvement) system 202 includes subsystem monitoring logic 204, external factor monitoring logic 214, performance indicator generator logic 224, user feedback logic 236, user feedback logic 236, cloud interaction logic 248, data storage 252, optimization (or improvement) logic 254, and may also include other items, as indicated by box 290.

[0041] Subsystem monitoring logic 204 includes propulsion / steering monitoring logic 207, power monitoring logic 209, and clean grain monitoring 210, and may also include other monitoring logic components, as indicated by block 213. Propulsion / steering monitoring logic 207 receives, processes, and records sensor signals (e.g., electrical pulses) from sensors monitoring the propulsion and steering subsystems. Power monitoring logic 208 receives, processes, and records sensor signals from sensors that sense characteristics of the power subsystem 214. For example, power monitoring logic 208 monitors available engine power and its distribution, arguably the most critical resource on the machine that requires proper allocation. Clean grain monitoring logic 210 receives, processes, and records sensor signals from sensors that sense characteristics of the clean grain subsystem. Other monitoring logic 212 may include logic components that receive, process, and record sensor signals corresponding to other subsystems (e.g., separation, threshing, storage, spreading, cutting, and transport / feeder subsystems).

[0042] The external factor monitoring logic 214 includes operator monitoring logic 216, field monitoring logic 218, environmental monitoring logic 220, and may also include other monitoring logic, as indicated by box 222. The operator monitoring logic 216 receives, processes, and records sensor signals from sensors that sense characteristics of the operator 196. For example, the operator monitoring logic 216 receives sensor signals from a heart rate sensor and converts the analog signals into digital heart rate values. The field monitoring logic 218 receives, processes, and records sensor signals from sensors that sense characteristics of the field. For example, the field monitoring logic 218 may receive sensor signals indicating soil moisture and convert these analog signals into digital moisture values ​​(e.g., percentages). Or, for example, the field monitoring logic 218 receives field images from sensors and determines the amount of standing versus fallen crops, stem height, etc.

[0043] The environmental monitoring logic 220 receives, processes, and records sensor signals from sensors that transmit characteristics of the operating environment. For example, the environmental monitoring logic 220 receives sensor signals and converts the received sensor signals from analog ambient temperature sensor signals into digital temperature values. Of course, the environmental monitoring logic 220 can also receive other sensor signals and output other available metrics, such as ambient temperature, sunlight, humidity, wind speed and direction, etc.

[0044] Performance metric generator logic 224 generates performance metrics for components of mobile machine 100. A performance metric is a value that quantifies how well the machine or a component of the machine performs (e.g., compared to operator expectations). For example, performance metric generator logic 224 generates performance metrics for the cutting, transporting, threshing, separating, cleaning, spreading, and propulsion / steering subsystems. To generate the performance metrics, performance metric generator logic 224 aggregates contributions from machine sensor component 226, user feedback component 228, and remote system component 230. Other components may also be used, as indicated by block 232. In some examples, performance metric generator logic 224 aggregates contributions from only a subset of the components listed above.

[0045] The machine sensor assembly 226 receives information from machine sensors that sense characteristics of machine subsystems, or receives pre-processed sensor data from the subsystem monitoring logic 204, and generates machine contribution values ​​for use in generating performance indicators. For example, the machine sensor assembly 226 receives data from seed impact sensors indicating grain count / accumulation in the grain cleaning system. As another example, the machine sensor assembly 226 receives data from hydraulic sensors on machine actuators indicating engine power consumption. In some current systems, performance indicators are generated solely based on feedback from machine sensors, without comparison to the operator's desired performance.

[0046] The user feedback component 228 receives data indicating user feedback and generates a user contribution value for generating a performance metric. For example, the user feedback component 228 receives information indicating user feedback from the user feedback logic 236, and this data modifies performance metrics generated from other sources. For example, if the machine appears to be operating well based on machine sensor feedback (e.g., from the machine sensor component 226), but the operator is dissatisfied with the performance, the overall performance metric may be lowered. Alternatively, if the machine appears to be operating poorly based on machine sensor feedback, but the operator is satisfied with the performance, the overall performance metric may be increased. Over time, the machine learning preference setting logic 259 may identify which performance levels satisfy which operators and adjust the overall performance metric accordingly.

[0047] The remote system component 230 receives data from one or more remote systems and generates additional contributions to the performance indicators generated by the performance indicator generator logic 224. For example, the remote system component 230 receives weather data from a remote weather source and modifies the performance indicators accordingly. For example, maximum performance in the rain (e.g., due to tire slippage in mud and wet crops being processed) is different from maximum performance on dry soil, so the performance indicators should be adjusted accordingly to account for this. In another example, the remote system component 230 receives yield maps or previous yield data and generates performance contributions. For example, if a single geographic portion of a field has historically had lower yields, this fact can be used to calculate the performance indicator in that area (e.g., by artificially boosting the indicator to compensate).

[0048] The user feedback logic 236 retrieves feedback from the operator 196. The user feedback logic 236 may automatically receive feedback from the operator 196, as indicated by block 238. For example, the user feedback logic 236 may receive monitoring data from the operator monitoring logic 216, which may include sensor data indicating feedback from the operator. In one example, this data may include eye tracking information of the operator, which may indicate that the operator is paying extra attention to a particular subsystem (e.g., an engine temperature gauge) or a performance category (e.g., tons of grain per hour). The user feedback logic 236 may manually receive user feedback, as indicated by block 240. For example, the user feedback logic 236 may generate an interface for the operator 196 to manually interact and provide feedback. The user feedback logic 236 may also receive feedback in other ways, as indicated by block 242.

[0049] The consumption index generator logic 234 generates a consumption index, which is a value that quantifies how much resource a component is using, for various components on the mobile machine 100. The consumption index can be compared to other systems, an absolute value (e.g., horsepower, watts), or as a fraction of the maximum resource that the machine can provide (e.g., 30% of maximum power utilization).

[0050] The cloud interaction logic 248 interacts with the cloud or other remote systems. For example, the cloud interaction logic 248 can retrieve weather data from an online weather source.

[0051] Improvement logic 254 operates to improve (e.g., optimize) overall machine performance. Improvement logic 254 includes neural network / deep learning processor(s) / server(s) 256, preference setting logic 258, resource allocation logic 260, algorithm selection logic 262, learning information storage 264, control system interface logic 272, initial training logic 274, and may also include other items, as indicated by block 276. Neural network / deep learning processor / server 256 provides processing power for the other logical components of improvement logic 254.

[0052] The preference setting logic 258 allows the user to select any preferences they may have for the operation of the mobile machine 100. For example, the user may prioritize grain quality over grain storage. The performance setting logic 258 may be operated with manual operator control, as indicated by block 259, or may automatically detect the operator's preferences, as indicated by block 261. Manual control of the performance setting logic 258 may include generating an interface (e.g., as described above) that allows the operator to select which settings they prioritize. In another example, manually controlling the performance setting logic 258 may include requiring a post-operation survey from the operator regarding the machine's performance during operation. Of course, preferences may also be set in other ways, as indicated by block 263.

[0053] Resource allocation logic 260 utilizes learning information storage 264 to allocate resources to meet user preferences (e.g., set by preference setting logic 258) and / or to improve overall system performance. For example, conflicts between subsystems competing for resources (e.g., engine power) may be identified and resources reallocated to the subsystems accordingly.

[0054] Algorithm selection logic 262 selects a machine learning algorithm for training a model that will later be used to generate optimized (or improved) machine controls. In some examples, algorithm selection logic 262 selects more than one algorithm or machine learning approach (e.g., different algorithms may be used to generate different parts of the overall machine control model). For example, the feed rate control subsystem may use a predictive control algorithm, while the crop harvesting subsystem may use a deep reinforcement learning algorithm.

[0055] Some examples of machine learning algorithms include (but are not limited to) attention mechanisms, memory networks, Bayesian systems, decision trees, eigenvectors, eigenvalues ​​and machine learning, evolutionary and genetic algorithms, expert systems / rule engines / symbolic reasoning, generative adversarial networks (GANs), graph analysis and ML, linear regression, logistic regression, LSTM and recurrent neural networks, Markov chain Monte Carlo methods (MCMC), neural networks, random forests, reinforcement learning, etc.

[0056] The learning information storage 264 stores the machine learning data and models generated by the machine learning process. As shown, the learning information storage 264 is on the mobile machine, but in other examples, the learning information storage 265 can be at a remote location.

[0057] The control system interface logic 272 interacts with the control system 186 to generate control signals according to the plan generated by the resource allocation logic 260 .

[0058] Initial training logic 274 receives historical contextual data and trains the machine learning system. For example, initial training logic 274 may generate / train a new neural network based on data available for runtime machine control (e.g., machine and operator sensor data) and results (e.g., performance metrics). In some examples, the neural network is further trained at runtime to update for changing conditions or reinforce consistent results.

[0059] Figure 4 is a flow chart illustrating example operations of mobile machine 100. Operation 300 begins at block 310, where the machine is initialized. Initializing the machine may include identifying the machine and its subsystems, as indicated at block 312. The work environment and operator are then identified, as indicated at block 314. Similar stored data is then retrieved and loaded into the machine or another system for storage. Similar historical data includes data collected during operation from the same or similar machines, the same or similar operators, the same or similar subsystems, and the like.

[0060] Operation 300 proceeds to block 320, where machine learning is applied to the retrieved similar historical data to generate a model that indicates which machine settings should be used under different scenarios (e.g., different operators, managers, machines, crops, weather, field conditions, etc.) to improve (e.g., optimize) the system. The machine learning may be processed locally (e.g., on the machine or a computer system near the machine), as indicated by block 322. The machine learning may be processed remotely (e.g., at a remote data processing center), as indicated by block 324. The machine learning may also be processed elsewhere, as indicated by block 326.

[0061] Operation 300 proceeds to block 330 where the machine learning data and / or model is stored. The machine learning data may be stored locally, as indicated by block 332, for example, on the mobile machine or a computer near the mobile machine. The machine learning data may be stored remotely, as indicated by block 334, for example, at a remote server. The machine data may also be stored elsewhere, as indicated by block 336, for example, partially on the machine and partially remotely.

[0062] Operation 300 proceeds to block 340 where the machine begins operating at a work site. For example, the machine begins harvesting crops in a field.

[0063] Operation 300 proceeds to block 350, where various characteristics are sensed or data is received during machine operation. As indicated by block 352, machine characteristics may be sensed. For example, engine RPM, engine temperature, engine power / torque consumption, travel speed, machine settings, etc. may be sensed. As indicated by block 354, operator characteristics may be sensed during machine operation. For example, operator heart rate, breathing, movement, gestures, eye movement, etc. may be sensed. As indicated by block 356, field characteristics may be sensed during machine operation. For example, field moisture, crop vegetation, standing water, etc. may be sensed. Environmental characteristics may be sensed during machine operation, as indicated by block 358. For example, ambient temperature, precipitation, barometric pressure, wind, etc. may be sensed. Other characteristics may also be sensed during machine operation, as indicated by block 360.

[0064] Operation 300 proceeds to block 370, where performance and consumption indicators are calculated based on the data sensed from block 350. Performance indicators indicate the performance of the machine, while consumption indicators indicate the resources being consumed by the unit. Indicators can be calculated by individual subsystems, as indicated in block 372. For example, separate indicators can be generated for the power / steering subsystem, the grain cleaning subsystem, the degranulation system, the friction subsystem, the spreading subsystem, and so on. Indicators can be calculated by modular subsystems or component collections. For example, the steering and propulsion subsystems can be grouped together, or the screen and chaff screen systems can be grouped together, or the header and feeder housing subsystems can be grouped together, and so on. Indicators can also be grouped together in other ways, as indicated in block 376.

[0065] Operation 300 proceeds to block 360, where the metrics are stored in conjunction with the data sensed from block 350. As indicated by block 362, the metrics and data may be stored locally, for example, on the machine. As indicated by block 364, the metrics and data may be stored remotely, for example, at a remote server. As indicated by block 366, the metrics are stored elsewhere in conjunction with the data.

[0066] At block 390, machine learning is applied to the sensed data and metrics to update the control model. For example, the sensed data is fed into a machine learning system as input, and the metrics it generates are fed into the machine learning system as results. In this case, the machine learning process discovers patterns or models that can be used to predict outcomes based on a given set of inputs. Once several patterns or models are found to represent correlations, machine learning occurs. These models are particularly useful for predicting outcomes (e.g., performance and consumption metrics) based on inputs (e.g., machine sensors).

[0067] In some examples, metrics are modified based on models or other types of machine learning data. For example, assuming all other factors and objective results are the same, a first operator may perceive the results as excellent, while a second operator may perceive the results as below their expectations. In this case, the performance metric generator can learn to tailor the performance metric to the operator, meaning that even if all other factors are the same, the performance metric generated by the second operator is lowered while the performance metric generated by the first operator remains the same. In other examples, if the user is dissatisfied with the results, the system will explore new and / or different machine settings to improve machine performance to satisfy the operator.

[0068] Operation 300 proceeds to block 400, where the runtime machine learning data is aggregated with historical and previously stored machine learning data. In one example, the aggregation includes reinforcement learning of the stored data with the runtime machine learning data. As indicated at block 402, during the aggregation, the runtime machine learning data that reinforces the historical / stored machine data may have its reinforcement weighted higher or lower. For example, if there is reason to believe that the current runtime data may be inaccurate or misleading (e.g., a possible sensor failure or field anomaly), this should be treated as an outlier and not overly impact the larger amount of past data.

[0069] Operation 300 proceeds to block 410, where a control signal is generated based on the machine learning data. As indicated by block 412, the control signal may be generated based on runtime machine learning data. As indicated by block 414, the control signal may be generated based on historical or stored machine learning data. As indicated by block 416, the control signal may be generated based on aggregated machine learning data. As indicated by block 418, the control signal may also be generated based on other data.

[0070] Operation 300 proceeds to block 420 where the machine is controlled based on the generated control signal.

[0071] Operation 300 proceeds to block 430 where it is determined whether more operations are to be performed. If no more operations are to be performed, then operation 300 ends. If additional operations are to be performed, then operation proceeds again to block 350.

[0072] Figure 5is a block diagram illustrating an example power allocation workflow. As shown, total engine power 450 is received by the supervisory and optimization system 202 and allocated to the various subsystems 208-218. The supervisory and optimization system 202 also receives power requests 454 from the various components 208-218. In one example, these requests are granted based on the priority of the subsystems requesting power. For example, the supervisory and optimization system 202 prioritizes subsystems based on which subsystems require power for the overall system to perform better. The supervisory and optimization system 202 can determine subsystem needs to improve overall machine performance based on, for example, one of the machine learning processes described above.

[0073] This discussion refers to processors and servers. In one embodiment, processors and servers include computer processors with associated memory and timing circuits (not shown separately). They are functional parts of the systems or devices to which they belong and are activated, and facilitate the functions of other components or items in those systems.

[0074] In addition, several user interface displays have been discussed. They can take a variety of different forms and can have a variety of different user-actuated input mechanisms arranged thereon. For example, the user-actuated input mechanism can be a text box, a check box, an icon, a link, a drop-down menu, a search box, etc. They can also be actuated in a variety of different ways. For example, they can be actuated using a pointing device (e.g., a trackball or a mouse). They can be actuated using hardware buttons, switches, joysticks, or keyboards, thumb switches, or thumb pads, etc. They can also be actuated using a virtual keyboard or other virtual actuators. In addition, when the screen displaying them is a touch-sensitive screen, they can be actuated using touch gestures. In addition, when the device displaying them has a speech recognition component, they can be actuated using speech instructions.

[0075] Several databases have also been discussed. It will be noted that each of these can be divided into multiple databases. All can be local to the system accessing them, all can be remote, or some can be local and others can be remote. All of these configurations are contemplated herein.

[0076] In addition, the accompanying drawings show several blocks, each of which has a function. It will be noted that fewer blocks can be used, and therefore fewer components can perform the function. In addition, more blocks can be used, with the function distributed between more components.

[0077] Figure 6 yes Figure 11. The block diagram of the machine 100 shown in FIG. 1 is similar to that of FIG. 1, except that it communicates with elements in a remote server architecture 500. In an example, the remote server architecture 500 can provide computing, software, data access, and storage services without requiring the end user to be aware of the physical location or configuration of the system delivering the services. In various examples, the remote server can deliver the services via a wide area network (e.g., the Internet) using an appropriate protocol. For example, the remote server can deliver applications via the wide area network, and they can be accessed through a web browser or any other computing component. Figure 2 and Figure 3 The software or components shown and corresponding data may be stored on a server at a remote location. The computing resources in a remote server environment may be consolidated at a remote data center location, or they may be dispersed. The remote server infrastructure may deliver services through a shared data center, even though they appear to be a single access point to the user. Thus, the components and functionality described herein may be provided from a remote server at a remote location using a remote server architecture. Alternatively, they may be provided from a conventional server, or they may be installed directly on the client device, or provided in other ways.

[0078] exist Figure 6 In the example shown, some items are similar to Figure 2 and Figure 3 those shown, and they are numbered similarly. Figure 6 Specifically shown is that the optimization and supervisory system 202 can be located at a remote server location 502. Thus, the harvester 100 accesses those systems through the remote server location 502.

[0079] Figure 6 Another example of a remote server architecture is also depicted. Figure 6 Show that you can also think of Figure 2 and Figure 3 Some elements of are located at the remote server location 502, while others are not. Figure 6 In this example, system 202 is located in cloud 502 and can be used by multiple machines (e.g., machine 100 and machine 501, which can be similar to machine 100 or different and have its own operator 503). Therefore, system 202 can generate an aggregate fleet value, which is similar to the value described above with respect to Figure 6 The aggregate in question changes value, but is aggregated across multiple different machines.

[0080] Similarly, Figure 6It is shown that some items can still be located in different locations. As an example, the database 241 or the reference data selection system 244 can be set at a location separated from the location 502 and accessed by a remote server at the location 502. Regardless of where they are located, they can be directly accessed by the harvester 100 and the machine 501 through a network (wide area network or local area network), they can be hosted at a remote site through a service, or they can be provided as a service, or accessed through a connection service residing in a remote location. In addition, data can be stored in essentially any location and intermittently accessed or forwarded to by interested parties. For example, instead of or in addition to electromagnetic wave carriers, physical carriers can be used. In an example where cell coverage is poor or non-existent, another mobile machine (e.g., a fuel truck) can have an automated information collection system. When a harvester approaches the fuel truck to refuel, the system automatically collects information from the harvester using any type of self-organizing wireless connection. Then, when the fuel truck arrives at a location with cellular coverage (or other wireless coverage), the collected information can be forwarded to the main network. For example, the fuel truck can enter a covered location when driving to refuel other machines or when it is in the main fuel storage location. All of these architectures are contemplated herein. Additionally, information can be stored on the harvester until the harvester enters a coverage location. The harvester itself can then send the information to the main network.

[0081] also, Figure 6 The illustrated architecture 500 may include a remote computing system 506 used by another user 508. Thus, the system 202 may send various values ​​and control outputs to the remote computing system 506, where they are exposed to the user 508. Thus, the system 506 may be a computing system remote from the machines 100, 501. It may be a farm or fleet manager's system, a vendor's system, a manufacturer's system, etc.

[0082] It will also be noted that Figure 2 The elements or parts thereof may be provided on a variety of different devices. Some of those devices include servers, desktop computers, laptop computers, tablet computers or other mobile devices (e.g., palmtop computers, cellular phones, smart phones, multimedia players, personal digital assistants, etc.).

[0083] Figure 7 1 is a simplified block diagram of one illustrative example of a handheld or mobile computing device that can be used as a user or customer's handheld device 16 in which the present system (or portions thereof) can be deployed. For example, the mobile device can be deployed in an operator's cabin of a harvester 100 for generating, processing, or displaying control operations. Figures 8 and 9 is an example of a handheld or mobile device.

[0084] Figure 7 Provide runnable Figure 2 and / or Figure 3 1 is a general block diagram of components of a client device 16 that interacts with some of the components shown, or both. In device 16, a communication link 13 is provided that allows the handheld device to communicate with other computing devices and, in some examples, provides a channel for automatically receiving information (e.g., by scanning). Examples of communication link 13 include allowing communication via one or more communication protocols (e.g., wireless services for providing cellular access to a network and protocols for providing local wireless connectivity to a network).

[0085] In other examples, the application may be received on a removable secure digital (SD) card connected to the interface 15. The interface 15 and the communication link 13 communicate with a processor 17 (which may also embody the processors or servers from the previous figures) along a bus 19, which is also connected to a memory 21 and input / output (I / O) components 23, as well as a clock 25 and a position system 27.

[0086] In one example, I / O components 23 are provided to facilitate input and output operations. Various examples of I / O components 23 for device 16 may include input components such as buttons, touch sensors, optical sensors, microphones, touch screens, proximity sensors, accelerometers, orientation sensors, and output components such as displays, speakers, and / or printer ports. Other I / O components 23 may also be used.

[0087] The clock 25 exemplarily includes a real-time clock component that outputs time and date. Exemplarily, it can also provide a timing function for the processor 17.

[0088] Position system 27 illustratively includes components for outputting the current geographic location of device 16. This may include, for example, a Global Positioning System (GPS) receiver, a LORAN system, a dead reckoning system, a cellular triangulation system, or other positioning systems. It may also include, for example, mapping software or navigation software that generates desired maps, navigation routes, and other geographic functions.

[0089] Memory 21 stores operating system 29, network settings 31, applications 33, application configuration settings 35, data repository 37, communication drivers 39, and communication configuration settings 41. Memory 21 may include all types of tangible, volatile, and non-volatile computer-readable memory devices. It may also include computer storage media (described below). Memory 21 stores computer-readable instructions that, when executed by processor 17, cause the processor to perform computer-implemented steps or functions in accordance with the instructions. Processor 17 may also be activated by other components to facilitate its functions.

[0090] Figure 8 The illustrated device 16 is an example of a tablet computer 600. Figure 8, computer 600 is shown with a user interface display screen 602. Screen 602 can be a touch screen or a pen-enabled interface that receives input from a pen or stylus. An on-screen virtual keyboard can also be used. Of course, it can also be attached to a keyboard or other user input device via a suitable attachment mechanism (e.g., a wireless link or a USB port), for example. Computer 600 can also illustratively receive voice input.

[0091] Figure 9 The device shown may be a smartphone 71. Smartphone 71 has a touch-sensitive display 73 that displays icons or tiles or other user input mechanisms 75. A user can use mechanism 75 to run applications, make calls, perform data transfer operations, etc. Typically, smartphone 71 is built on a mobile operating system and provides more advanced computing capabilities and connectivity than feature phones.

[0092] It is noted that other forms of the device 16 are possible.

[0093] Figure 10 Is deployable (for example) Figure 2 and / or Figure 3 An example of an element or portion of a computing environment. Figure 10 , an example system for implementing some embodiments includes a general purpose computing device in the form of a computer 810. Components of the computer 810 may include, but are not limited to, a processing unit 820 (including the processors or servers from the previous figures), a system memory 830, and a system bus 821 that couples various system components including the system memory to the processing unit 820. The system bus 821 may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. Figure 2 and / or Figure 3 The memory and program described can be deployed in Figure 10 in the corresponding part.

[0094] Computer 810 typically includes a variety of computer-readable media. Computer-readable media can be any available media accessible by computer 810 and includes both volatile and non-volatile media, removable and non-removable media. By way of example, and not limitation, computer-readable media can include computer storage media and communication media. Computer storage media is distinct from and does not include modulated data signals or carrier waves. It includes hardware storage media, including both volatile and non-volatile, removable and non-removable media implemented in any method or technology, for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVDs) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible by computer 810. Communication media can embody computer-readable instructions, data structures, program modules, or other data in a transmission mechanism and includes any information transmission media. The term "modulated data signal" means a signal whose one or more characteristics are set or changed in such a manner as to encode information in the signal.

[0095] The system memory 830 includes computer storage media in the form of volatile and / or nonvolatile memory, such as read-only memory (ROM) 831 and random-access memory (RAM) 832. A basic input / output system 833 (BIOS), containing the basic routines that help transfer information between elements within the computer 810 (e.g., during startup), is typically stored in ROM 831. RAM 832 typically contains data and / or program modules that are immediately accessible to and / or currently being operated on by the processing unit 820. By way of example, and not limitation, Figure 10 Operating system 834 , application programs 835 , other program modules 836 , and program data 837 are shown.

[0096] The computer 810 may also include other removable / non-removable volatile / non-volatile computer storage media. For example only, Figure 10 Shown are a hard disk drive 841 that reads from and writes to a non-removable non-volatile magnetic medium, an optical drive 855, and a non-volatile optical disk 856. The hard disk drive 841 is typically connected to the system bus 821 through a non-removable memory interface such as interface 840, and the optical drive 855 is typically connected to the system bus 821 through a removable memory interface such as interface 850.

[0097] Alternatively or additionally, the functions described herein may be at least partially performed by one or more hardware logic components. For example, and not limitation, exemplary types of hardware logic components that may be used include field programmable gate arrays (FPGAs), application specific integrated circuits (e.g., ASICs), application specific standard products (e.g., ASSPs), system-on-chip systems (SOCs), complex programmable logic devices (CPLDs), and the like.

[0098] discussed above and in Figure 10 The drives and their associated computer storage media shown in FIG. 8 provide storage of computer readable instructions, data structures, program modules and other data for the computer 810. Figure 10 844, application programs 845, other program modules 846, and program data 847. Note that these components may be the same as or different from operating system 834, application programs 835, other program modules 836, and program data 837.

[0099] A user can enter commands and information into the computer 810 through input devices such as a keyboard 862, a microphone 863, and a pointing device 861 (e.g., a mouse, trackball, or touch pad). Other input devices (not shown) may include a joystick, a game controller, a satellite dish, a scanner, or the like. These and other input devices are often connected to the processing unit 820 through a user input interface 860 that is coupled to the system bus, but may be connected through other interface and bus structures. A visual display 891 or other type of display device is also connected to the system bus 821 via an interface such as a video interface 890. In addition to the monitor, the computer may also include other peripheral output devices such as speakers 897 and a printer 896, which may be connected through an output peripheral interface 895.

[0100] The computer 810 operates in a networked environment using logical connections (eg, a local area network (LAN) or wide area network (WAN), a controller area network (CAN)) to one or more remote computers (eg, remote computer 880 ).

[0101] When used in a LAN networking environment, the computer 810 is connected to the LAN 871 through a network interface or adapter 870. When used in a WAN networking environment, the computer 810 typically includes a modem 872 or other means for establishing communications via the WAN 873 (e.g., the Internet). In a networking environment, program modules may be stored in the remote memory storage device. For example, Figure 10 Remote application programs 885 are shown as residing on remote computer 880 .

[0102] It should also be noted that the different examples described herein can be combined in different ways. That is, parts of one or more examples can be combined with parts of one or more other examples. All of these are contemplated herein.

[0103] Example 1 is a mobile machine comprising:

[0104] a propulsion subsystem, which propels the mobile machine across the operating environment;

[0105] machine monitoring logic that receives a sensor signal indicative of a value of a sensed machine variable;

[0106] operator monitoring logic that receives an operator sensor signal indicative of a value of a sensed operator variable;

[0107] performance indicator generator logic that receives the sensed machine variables and the sensed operator values ​​and generates a performance indicator based on the sensed machine variables and the sensed operator values;

[0108] an improvement system that accesses the historical contextual data and receives the performance indicator and generates an improvement signal based on the performance indicator and the historical contextual data; and

[0109] Control signal generator logic generates a control signal to perform a machine operation based on the improved signal.

[0110] Example 2 is a mobile machine according to any or all of the previous examples, and further comprising an operator sensor that generates an operator sensor signal.

[0111] Example 3 is the mobile machine of any or all of the previous examples, wherein the operator sensor comprises an eye tracking sensor, and wherein the performance indicator generator logic generates the performance indicator based at least in part on where the operator is looking.

[0112] Example 4 is the mobile machine of any or all of the previous examples, wherein the operator sensor comprises an operator movement sensor, and wherein the performance indicator generator logic generates the performance indicator based at least in part on how the operator moves.

[0113] Example 5 is the mobile machine of any or all of the previous examples, wherein the operator sensor includes an operator facial expression sensor, and wherein the performance indicator generator logic generates the performance indicator based at least in part on the operator's facial expression.

[0114] Example 6 is a mobile machine according to any or all of the previous examples, wherein the operator sensor includes an operator cardiac sensor or an operator respiratory sensor, and wherein the performance indicator generator logic generates the performance indicator based at least in part on operator cardiac characteristics or operator respiratory characteristics.

[0115] Example 7 is a mobile machine according to any or all of the previous examples, wherein the optimization system generates an optimization system to reallocate machine resources from overpowered machine subsystems to underpowered subsystems.

[0116] Example 8 is a mobile machine according to any or all of the previous examples, wherein the optimization system comprises:

[0117] A deep learning processor that performs machine learning on historical contextual data to generate optimization signals.

[0118] Example 9 is a mobile machine according to any or all of the previous examples, wherein the machine learning process includes generating a neural network.

[0119] Example 10 is a computer-implemented method comprising:

[0120] receiving a machine sensor signal indicative of a value of a sensed machine variable sensed on the mobile machine;

[0121] calculating a current performance indicator based on the sensed machine variables;

[0122] Retrieving historical context data and historical performance indicators corresponding to a subset of the historical context data;

[0123] identifying a machine performance gap between a current performance indicator and one of a plurality of historical performance indicators;

[0124] using a neural network or machine learning processor to identify machine change actions based on historical contextual data and sensed machine variables; and

[0125] Controls the machine to perform machine change operations.

[0126] Example 11 is a computer-implemented method according to any or all of the previous examples, and also includes receiving an environmental sensor signal indicating a value of a sensed environmental variable sensed proximate to the mobile machine, and wherein the current performance indicator is calculated at least in part based on the sensed environmental variable.

[0127] Example 12 is a computer-implemented method according to any or all of the previous examples, and further comprising:

[0128] receiving a second machine sensor signal indicative of machine subsystem resource consumption; and

[0129] A current consumption index is calculated based on the second machine sensor signal.

[0130] Example 13 is a computer-implemented method according to any or all of the previous examples, wherein identifying the machine change action using the neural network or machine learning processor is further based on the current consumption indicator.

[0131] Example 14 is a computer-implemented method according to any or all of the previous examples, and further comprising:

[0132] sensing an operator characteristic and generating an operator sensor signal indicative of a value of the operator characteristic sensed proximate to the operator;

[0133] An operator sensor signal is received, and wherein a current performance indicator is calculated based at least in part on the sensed operator variable.

[0134] Example 15 is a computer-implemented method according to any or all of the previous examples, wherein sensing an operator characteristic comprises:

[0135] Senses the operator's heart rate.

[0136] Example 16 is a computer-implemented method according to any or all of the previous examples, wherein sensing an operator characteristic comprises:

[0137] Sense the operator's eye position.

[0138] Example 17 is a computer-implemented method according to any or all of the previous examples, wherein sensing an operator characteristic comprises:

[0139] Sense the operator's body movement.

[0140] Example 18 is a computer-implemented method according to any or all of the previous examples, and further comprising:

[0141] An indication of a machine's changed operation is displayed on a display of the mobile machine.

[0142] Example 19 is a mobile machine comprising:

[0143] A collection of controllable subsystems;

[0144] a plurality of machine sensors that sense a plurality of machine variables and generate a plurality of machine sensor signals indicative of values ​​of the plurality of machine variables;

[0145] an operator sensor configured to sense an operator variable and generate an operator sensor signal indicative of a value of the operator variable;

[0146] optimization logic that receives the plurality of machine sensor signals and the operator sensor signals and generates a machine learning model based on the plurality of machine sensor signals and the operator sensor signals; and

[0147] Control signal generator logic identifies a machine control operation based on the machine learning model and generates a control signal to control at least one of the set of controllable subsystems to perform the machine control operation.

[0148] Example 20 is a mobile machine according to any or all of the previous examples, wherein the optimization logic comprises:

[0149] initial training logic that receives a plurality of historical machine sensor signals and a plurality of historical operator sensor signals and generates a first machine learning model using a machine learning method based on the plurality of historical machine sensor signals and the plurality of historical operator sensor signals; and

[0150] Wherein, the optimization logic generates a machine learning model based at least in part on the first machine learning model.

[0151] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. A mobile machine comprising: a propulsion subsystem that propels the mobile machine across the operating environment; machine monitoring logic that receives a sensor signal indicative of a value of a sensed machine variable; operator monitoring logic that receives an operator sensor signal indicative of a value of a sensed operator variable; performance indicator generator logic that receives the sensed machine variables and the sensed operator values ​​and generates a performance indicator indicative of how the mobile machine is performing based on the sensed machine variables and the sensed operator values; an improvement system that accesses the historical contextual data and receives the performance indicator and generates an improvement signal based on the performance indicator and the historical contextual data; as well as Control signal generator logic generates a control signal to perform a machine operation based on the improved signal. 2 . The mobile machine of claim 1 , further comprising an operator sensor that generates an operator sensor signal.

3. The mobile machine according to claim 2, wherein: The operator sensor comprises an eye tracking sensor, and wherein the performance indicator generator logic generates the performance indicator based at least in part on where the operator is looking.

4. The mobile machine according to claim 2, wherein: The operator sensor includes an operator movement sensor, and wherein the performance indicator generator logic generates a performance indicator based at least in part on how the operator moves.

5. The mobile machine according to claim 2, wherein: The operator sensor includes an operator facial expression sensor, and wherein the performance indicator generator logic generates the performance indicator based at least in part on the operator's facial expression.

6. The mobile machine according to claim 2, wherein: The operator sensor comprises an operator cardiac sensor or an operator respiratory sensor, and wherein the performance indicator generator logic generates the performance indicator based at least in part on an operator cardiac characteristic or an operator respiratory characteristic.

7. The mobile machine according to claim 1, wherein: Optimizing the System Generate an optimizing system to reallocate machine resources from overpowered machine subsystems to underpowered subsystems.

8. The mobile machine according to claim 1, wherein The optimization system includes: A deep learning processor that performs machine learning on historical contextual data to generate optimization signals.

9. The mobile machine according to claim 8, wherein: The machine learning process includes generating a neural network.

10. A computer-implemented method comprising: receiving a machine sensor signal indicative of a value of a sensed machine variable sensed on the mobile machine; calculating a current performance index based on the sensed machine variables, the current performance index indicating how the mobile machine is performing; Retrieving historical context data and historical performance indicators corresponding to a subset of the historical context data; identifying a machine performance gap between a current performance indicator and one of a plurality of historical performance indicators; using a neural network or machine learning processor to identify machine change actions based on historical contextual data and sensed machine variables; and Controls the machine to perform machine change operations.

11. The computer-implemented method of claim 10 , further comprising: An environmental sensor signal is received that indicates a value of a sensed environmental variable sensed proximate the mobile machine, and wherein a current performance indicator is calculated based at least in part on the sensed environmental variable.

12. The computer-implemented method of claim 10 , further comprising: receiving a second machine sensor signal indicative of machine subsystem resource consumption; as well as A current consumption index is calculated based on the second machine sensor signal.

13. The computer-implemented method of claim 12, wherein: Identifying machine change actions using a neural network or machine learning processor is also based on current consumption metrics.

14. The computer-implemented method of claim 10 , further comprising: sensing an operator characteristic and generating an operator sensor signal indicative of a value of the operator characteristic sensed proximate to the operator; An operator sensor signal is received, and wherein a current performance indicator is calculated based at least in part on the sensed operator variable.

15. The computer-implemented method of claim 14, wherein: Sensing operator features include: Senses the operator's heart rate.

16. The computer-implemented method of claim 14, wherein: Sensing operator features include: Sense the operator's eye position.

17. The computer-implemented method of claim 14, wherein: Sensing operator features include: Sense the operator's body movement.

18. The computer-implemented method of claim 10, further comprising: An indication of a machine's changed operation is displayed on a display of the mobile machine.

19. A mobile machine comprising: A collection of controllable subsystems; a plurality of machine sensors that sense a plurality of machine variables and generate a plurality of machine sensor signals indicative of values ​​of the plurality of machine variables; an operator sensor configured to sense an operator variable and generate an operator sensor signal indicative of a value of the operator variable; performance indicator generator logic that receives the sensed machine variables and the sensed operator values ​​and generates a performance indicator indicative of how the mobile machine is performing based on the sensed machine variables and the sensed operator values; optimization logic that receives the plurality of machine sensor signals and the operator sensor signals and generates a machine learning model based on the plurality of machine sensor signals and the operator sensor signals; as well as Control signal generator logic that identifies a machine control operation based on the machine learning model and generates a control signal to control at least one of the set of controllable subsystems to perform the machine control operation.

20. The mobile machine of claim 19, wherein: The optimization logic includes: initial training logic that receives a plurality of historical machine sensor signals and a plurality of historical operator sensor signals and uses a machine learning method to generate a first machine learning model based on the plurality of historical machine sensor signals and the plurality of historical operator sensor signals; and Wherein, the optimization logic generates a machine learning model based at least in part on the first machine learning model.

Images