Harvester header autonomous control method based on deep learning

Abstract

The present application relates to the technical field of automatic control of agricultural equipment, and discloses a self-control method for a harvesting machine header based on deep learning. The method acquires continuous operation images in front of the header, actual height of the header, driving speed, body posture and execution response data, inputs the time series of the multi-source data into a deep learning space-time recognition model after time series alignment, obtains crop canopy boundaries, ground boundaries, stubble trends and recognition confidence of a region to be harvested, maps the front region to a header arrival time sequence according to the driving speed, generates a target height sequence, and outputs a header height control instruction in combination with the confidence state by performing lag compensation according to rising response, falling response, instruction dead zone and lifting speed constraints. The method makes the front-view recognition, arrival time and execution response in the same control link, reduces height correction lag, missed cutting, low-position ground contact and repeated lifting and lowering.

Description

Technical Field

[0001] This invention relates to the field of automatic control technology for agricultural equipment, and discloses an autonomous control method for the header of a harvester based on deep learning. Background Technology

[0002] Autonomous control of the harvester header falls under the category of agricultural equipment operation process control technology. The controlled objects typically include the header lifting actuator, header height feedback signals, overall machine status, and operating environment information. Most existing harvesters employ a combination of manual setting of the stubble height and closed-loop height maintenance for header control. The operator pre-sets the target header height based on crop height, ground flatness, and operational experience. The controller then outputs lifting commands based on the deviation between the actual header height feedback and the target height. Some machines incorporate machine posture, travel speed, or header load signals to adjust control inputs, ensuring a relatively stable operating posture on slopes, uneven ground, or when speed changes. This type of control relies primarily on feedback data from the header's current position or its vicinity, representing a post-hoc deviation correction. When the ground is relatively undulating, crop height is relatively uniform, and travel speed is stable, this method can maintain a basic stubble height. However, in scenarios with abrupt terrain changes or continuous crop canopy shifts, the header often enters the changing area before generating significant feedback deviations, resulting in a time lag before the controller issues lifting commands.

[0003] To compensate for the shortcomings of simple height feedback control, existing technologies also include schemes that utilize image recognition or visual perception to assist header control. These schemes typically acquire images of the harvester in front of it, extracting crop canopy boundaries, ground areas, and crop distribution along the direction of travel through image segmentation, edge detection, or deep learning models. The target header height is then calculated based on the recognition results. Some schemes use continuous frame images to mitigate the effects of dust, occlusion, and lighting changes on single-frame images, and input the recognition results, along with the header's current height and travel speed, into the controller, allowing the controller to anticipate changes in the crop or ground in the area to be harvested. While these schemes add forward-looking information compared to traditional feedback control, they often directly convert visual recognition results into target height or simply add them to the existing height control values ​​using a fixed scaling factor. There is a conversion relationship between the forward area in the image coordinates and the actual time the header reaches that area. If this conversion relationship is not processed uniformly with travel speed, machine attitude, and header cutting position, the recognition results are difficult to reconcile with the execution time.

[0004] In the actual control of the header, the actuator itself also has constraints such as rising response, falling response, command dead zone, maximum lifting rate, and mechanical inertia. Existing controllers usually treat the actuator as an object that can respond immediately according to the target height, or only set fixed limits and smoothing at the control output, without modeling the execution response characteristics together with the forward recognition results. For the same section of undulating ground ahead, the harvester has a longer adjustment time when traveling at low speed, while the action time left for the header actuator is significantly shortened when traveling at high speed; for the same change in target height, the response time of the header rising and falling may also be different. If the target height is generated only based on the image recognition results, without advance compensation based on the header arrival time and execution lag, the control command will deviate from the actual needs on the time axis. If only smoothing at the actuator end is relied upon, necessary advance actions may be suppressed, causing the header to be in a continuous following state in the continuous undulating area, making it difficult to form a height control process that matches the changes in the ground ahead.

[0005] The lack of a unified closed-loop control link between the visual recognition results in front of the header, the time relationship between the harvester's arrival at the front area, and the lag in the response of the header's actuators results in difficulties in converting the crop canopy boundary, ground boundary, and stubble trend into height control commands corresponding to the actual execution time of the header, even if these are identified. The technical reason for this problem is that the visual recognition output usually remains at the spatial position or target height level. The controller lacks the processing to map the front area to an arrival time sequence based on the travel speed, and also lacks a mechanism to compensate for lead time and constrain actions based on the header's execution response data. When the harvester is operating under conditions of continuous ground undulations, changes in machine attitude, or speed fluctuations, the header control commands easily lag behind ground changes, causing misalignment between the header height correction and the actual cutting position. This leads to continuous occurrences of stubble height fluctuations, partial missed cuts, low-level ground contact, and repeated header rises and falls within the same work row. Summary of the Invention

[0006] The purpose of this invention is to provide an autonomous control method for the header of a harvester based on deep learning, which can effectively solve the problems mentioned in the background art.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows: The deep learning-based autonomous control method for harvester headers includes: acquiring continuous operation images in front of the header, actual header height, harvester travel speed, machine attitude, and header execution response data; After aligning the continuous operation images with the actual height, the driving speed, and the aircraft attitude in time, the images are input into a deep learning spatiotemporal recognition model to obtain the crop canopy boundary, ground boundary, stubble retention trend, and recognition credibility of the area to be harvested ahead. Based on the travel speed, the area to be harvested ahead is mapped to the time sequence of the header arrival, and a target height sequence is generated; Based on the execution response data of the cutting platform, hysteresis compensation and action constraints are applied to the target height sequence, and the cutting platform height control command is output in combination with the recognition confidence level.

[0008] Preferably, the time alignment includes: establishing a time index based on the acquisition time of the continuous operation image and according to the sampling time of the actual height of the cutting table, the travel speed and the body posture within the control cycle; Interpolate and resample the data within adjacent control cycles to form a state vector corresponding to each frame of the image; The spatiotemporal input window is constructed by combining multiple adjacent frames of images and their state vectors. The spatiotemporal input window is labeled with arrival time tags according to the longitudinal distance between the image acquisition position and the cutting position of the cutting table, and used as the input data of the deep learning spatiotemporal recognition model.

[0009] Preferably, generating the target height sequence includes: converting the crop canopy boundary, the ground boundary, and the stubble retention trend into segmented height curves under the working coordinates; The attitude correction is performed on the boundary displacement caused by changes in the image viewing angle based on the fuselage attitude. The segmented height curves are arranged into multiple prediction windows according to the arrival time labels; A height continuity constraint and a stubble variation constraint are introduced between adjacent prediction windows, and multiple candidate heights within the same control cycle are fused according to their recognition credibility to form a target height sequence corresponding to the header control cycle.

[0010] Preferably, the hysteresis compensation and action constraints include: establishing an execution constraint table for the rise response time, fall response time, command dead zone, and maximum rise and fall rate of the cutter head based on the cutter head execution response data; Each target height in the target height sequence is shifted forward according to the corresponding cutter arrival time to obtain the pre-execution height sequence; The pre-execution height sequence is subjected to dead zone removal, rate limiting, and adjacent instruction smoothing. Adjust the fusion ratio between the pre-executed height sequence and the actual height feedback of the cutter head according to the recognition confidence level, and generate the cutter head height control command.

[0011] Preferably, the construction of the spatiotemporal input window further includes: dividing adjacent multi-frame images into near-range windows, mid-range windows, and far-range windows based on the change in driving speed within a continuous control cycle; The near-range window retains data from each acquisition cycle, while the mid-range window and the far-range window retain the boundary features after resampling by the work coordinates. The historical commands, historical actual height, and response deviation from the cutter execution response data are written into the state vector of the corresponding window, and the forward features at different distances are written into the deep learning spatiotemporal recognition model according to a unified arrival time index.

[0012] Preferably, the fusion of the candidate heights includes: generating candidate heights based on ground boundaries, candidate heights based on stubble retention trends, and candidate heights derived from crop canopy boundaries within each prediction window; A minimum ground clearance constraint is set for the candidate heights based on the ground boundary, and a continuous adjacent window constraint is set for the candidate heights based on the stubble retention trend. When the three candidate heights deviate within the same control cycle, the fusion weight is determined according to the recognition confidence, attitude correction amount and historical response deviation, and a single target height is output.

[0013] Preferably, adjusting the fusion ratio according to the recognition confidence includes: forming a confidence state by combining the recognition confidence with image boundary continuity, pose correction stability and target height sequence change rate; When the confidence state meets the forward control conditions, the pre-executed height sequence is used as the main control input; When the confidence state meets the hybrid control conditions, the pre-execution height sequence and the actual height feedback of the cutter are synthesized in a weighted manner; When the confidence status meets the feedback hold condition, the restricted command of the previous control cycle is locked and the actual height of the cutting table is received for correction.

[0014] Preferably, the smoothing process of the pre-execution height sequence includes: calling the rise response time and fall response time respectively according to the increase or decrease direction of the adjacent target heights, and calculating the executable height change amount of the corresponding control cycle; Merge height changes smaller than the instruction dead zone with consecutive height changes in opposite directions; Height changes exceeding the maximum rate of ascent and descent are divided into multiple control cycles. Within the split control cycle, a highly progressive relationship consistent with the arrival time sequence of the cutter is maintained, forming continuous control instructions subject to execution constraints.

[0015] Preferably, the unified arrival time index is also updated according to the closed-loop verification result, which is generated by the pre-execution height corresponding to the prediction window, the actual height of the cutter, historical instructions, and the change in travel speed. When the actual height of the cutting table has a continuous time offset in the same direction relative to the pre-execution height, the longitudinal distance mapping value between the image acquisition position and the cutting position of the cutting table is adjusted. When the continuous same-direction time offset occurs along with changes in fuselage attitude, only the time label corresponding to the attitude correction amount is corrected, and the updated arrival time index is used for subsequent spatiotemporal input windows.

[0016] Preferably, the confidence status is also updated through a state holding sequence, which records the identification confidence, boundary displacement direction, pre-execution height change direction, and actual height change direction of the cutting table in multiple adjacent control cycles; When the direction of the pre-execution height change is continuously opposite to the direction of the actual height change of the cutting table, the forward control condition is converted into a mixed control condition. When the boundary displacement direction alternates within the state-maintaining sequence, the fusion weight of the newly added prediction window is frozen, and the current control cycle is output with continuous control instructions subject to execution constraints.

[0017] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. By acquiring continuous operation images in front of the header, the actual height of the header, travel speed, machine attitude, and header execution response data, and aligning the continuous operation images with the status data in time sequence, the deep learning spatiotemporal recognition model can output the crop canopy boundary, ground boundary, stubble trend, and recognition reliability of the area to be harvested ahead. After mapping the area to be harvested ahead to the header arrival time sequence based on the travel speed, the target height no longer corresponds only to the spatial position in the image, but also to the subsequent control cycle of the header. Furthermore, by combining the header's ascent response time, descent response time, command dead zone, and maximum lifting / lowering rate to perform lag compensation and motion constraints on the target height sequence, the control commands can be pre-executed before the header reaches the undulating ground area, ensuring that the forward perception, arrival time, and actuator actions form a closed-loop coordination on the same time axis, thereby reducing height correction misalignment caused by control lag.

[0018] 2. A spatiotemporal input window is constructed using near-range, mid-range, and far-range windows. Historical commands, historical actual heights, and response deviations are written into different distance windows, ensuring that the foreground features in continuous operation images correspond to the historical execution state of the header. By fusing candidate heights based on ground boundaries, stubble retention trends, and crop canopy boundaries, the target height sequence avoids control jumps caused by anomalies from a single identification source. A confidence level is established using identification confidence, image boundary continuity, attitude correction stability, and the target height sequence change rate, switching between forward-looking control, hybrid control, and feedback hold. When the foreground identification result is affected by occlusion, dust, or boundary fluctuations, the header control command can switch to a restricted output. By recording the pre-executed height change direction and the actual header height change direction using a state hold sequence, continuous opposite changes and alternating boundary displacements can be frozen or downweighted, maintaining the continuity of the header's ascent and descent process and reducing frequent reverse adjustments. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating the overall control process of the harvester header autonomous control method based on deep learning, as described in this invention. Figure 2 This is a flowchart illustrating the spatiotemporal input window construction and arrival time index generation process of the present invention. Figure 3 This is a flowchart of the target height sequence generation and candidate height fusion process of the present invention; Figure 4 This is a flowchart of the execution lag compensation, reliability classification control and closed-loop verification of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Please refer to Figure 1 This embodiment provides a deep learning-based autonomous control method for a harvester header, deployed within the harvester's operation control process. The controlled object is the header height control command. Input data includes continuous operation images in front of the header, the actual header height, the harvester's travel speed, the machine's attitude, and the header's execution response data. The continuous operation images describe the harvesting area not yet reached by the header; the actual header height describes the header's current execution position; the travel speed establishes a mapping relationship between the area in front and the header's arrival time; and the machine's attitude corrects for ground boundaries and crop canopy boundaries in the images. The relative displacement caused by pitch changes, and the header execution response data are used to describe the historical response process of the header actuator to the lifting command. The method collects the above data in each control cycle, performs time alignment, deep learning spatiotemporal recognition, arrival time mapping, target height sequence generation, execution lag compensation, and credibility constraint output. This makes the header height control command no longer determined solely by the current position feedback deviation, but jointly determined by the ground boundary of the area to be harvested ahead, the crop canopy boundary, the stubble retention trend, and the reachability of the header execution. This establishes a closed-loop control link from forward perception to header action.

[0022] In this embodiment, the continuous operation image is formed by image acquisition data in front of the harvester's direction of travel. The image acquisition range covers the uncut crop area in front of the header and the visible ground area. The actual height of the header is obtained from the header height feedback signal. The travel speed is obtained from the whole machine control bus or vehicle speed acquisition data. The machine attitude includes pitch and lateral attitude change information. The header execution response data includes historical height control commands, historical actual header height, response deviation within the same control cycle, and the time difference between the command issuance time and the height attainment time. After the data enters the controller, it does not directly generate control commands. Instead, a unified timeline is first established according to the acquisition time, so that the image frame, header height, travel speed, machine attitude, and execution response data are written into the same time sequence sample, avoiding mismatch between the forward visual results and the actual control cycle of the header.

[0023] Specifically, the deep learning spatiotemporal recognition model takes multiple consecutive frames of operational images and corresponding state vectors as input. The state vectors contain at least the actual height of the header, travel speed, machine attitude, and historical response deviation information. The model obtains crop canopy texture, ground boundary texture, and stubble area texture through an image feature extraction layer. Through a temporal feature aggregation layer, boundary changes in adjacent image frames are written into the same spatiotemporal feature. Through a multi-output decoding layer, the model outputs crop canopy boundary, ground boundary, stubble trend, and recognition confidence, respectively. The crop canopy boundary is used to characterize the spatial distribution of the upper or dense areas of the crop, the ground boundary is used to characterize the reference boundary for safe passage under the header, the stubble trend is used to characterize the change in the height of the target stubble in the cut or uncut area with the direction of travel, and the recognition confidence is used to describe the model's comprehensive judgment on boundary continuity, texture stability, and occlusion effects.

[0024] In this embodiment, the area to be harvested ahead is converted into a header arrival time sequence. The conversion process is based on the relationship between the travel speed and the distance between the area ahead and the header cutting position. Multiple longitudinal positions in the image are mapped to multiple future control cycles. The controller does not directly use the height result obtained from image recognition as the current control quantity. Instead, it arranges the recognition results of the area ahead according to the expected arrival time of the header into a target height sequence. Each target height in the target height sequence corresponds to a future control cycle or a group of control cycles. When the travel speed changes, the arrival time corresponding to the same image area changes accordingly. When the machine attitude changes, the boundary heights in the target height sequence are first corrected by attitude before participating in the sorting, ensuring that the spatial source of the target height is consistent with the header execution time.

[0025] In this embodiment, after the target height sequence enters the hysteresis compensation process, the controller establishes execution constraint information based on the cutter execution response data. The execution constraint information includes the cutter's rise response time, fall response time, command dead zone, and maximum rise / fall rate. The controller moves the future target height forward according to the cutter's arrival time to form a pre-execution height sequence, and performs dead zone elimination, rate limiting, and smoothing processing on the pre-execution height sequence. If the recognition confidence is stable, the pre-execution height sequence has a higher weight in the control output. If the recognition confidence decreases, the actual cutter height feedback and historical restricted commands have a higher weight in the control output. If the recognition confidence is continuously unstable, the controller restricts new forward-looking recognition results from entering the current control command, thereby enabling the cutter action to be adjusted in advance when the forward perception is reliable, and to maintain a restricted closed-loop output when the forward perception is disturbed.

[0026] In this embodiment, the arrival time series of the cutting platform can be calculated according to the following formula and used for sorting the target height sequence: ; in, Indicates the first in the image The estimated arrival time of the cutter platform corresponding to the area ahead. Indicates the current control cycle time. Indicates the first The working coordinate distance from the front area to the cutting position of the header. Indicates the current driving speed. This represents the speed correction amount used to avoid zero denominator when encountering low-speed pauses. , , , ,but This indicates that the foreground region should be positioned approximately 0.585 seconds after the current time, which physically means converting the foreground recognition result in the image space into a prediction node on the cutter control time axis.

[0027] In the above embodiment, the current control command is generated jointly by the target height ahead, the actual height of the header, and the execution constraints, rather than by a single visual height or a single feedback deviation. The crop canopy boundary, ground boundary, and stubble retention trend provide the source of changes ahead, the travel speed provides the basis for the conversion from space to time, the fuselage attitude provides the basis for boundary position correction, the execution response data provides the basis for pre-execution quantity constraints, and the recognition reliability provides the basis for switching forward control weights. The advantage of this embodiment is that it enables the header height command to correspond with the changes in the ground ahead on the time axis, reducing the lag control caused by correction after the header arrives.

[0028] In a preferred embodiment, reference Figure 2The temporal alignment process establishes a time index based on the acquisition time of continuous operation images. Since the actual height of the header, the travel speed, and the machine attitude may differ due to different sampling frequencies, the controller writes them into the buffer queue according to their original sampling time. When each frame of image enters the recognition process, it queries the adjacent sampling points from the buffer queue and uses linear interpolation or hold-at-the-moment interpolation to form a state vector corresponding to the acquisition time of the image. Linear interpolation is suitable for situations where the travel speed and attitude change slowly, while hold-at-the-moment interpolation is suitable for situations where the header height feedback remains unchanged within a short control cycle. After interpolation and resampling, adjacent frames of images and their state vectors are combined into a spatiotemporal input window, enabling the deep learning spatiotemporal recognition model to simultaneously read changes in image texture and changes in the harvester's motion state.

[0029] In this embodiment, the spatiotemporal input window is arranged according to the time sequence of the image frames, and an arrival time label is attached to each frame. The arrival time label is determined by the longitudinal distance between the image acquisition position and the cutting position of the header and the travel speed. If the same ground boundary in the continuous images moves between multiple frames, the model can use the spatiotemporal input window to determine whether the movement is caused by the forward movement of the harvester or by a change in the machine's attitude. If the actual height of the header changes in adjacent control cycles while the image boundary remains continuous, the model can use this change as execution response information and write it into the height estimation process. If the travel speed changes, the model can rearrange the image features to enter the position of the control sequence through the arrival time label.

[0030] Specifically, the state vector can be represented as a combination of the actual height of the header, the travel speed, the pitch of the fuselage, the lateral attitude of the fuselage, the historical command quantity, and the historical response deviation. In the model, the state vector and image features are not simply spliced ​​together to directly output the height. Instead, the position is first encoded in the time dimension so that the features of each frame of the image carry the acquisition time and the expected arrival time. Then, the features are fused with the state vector. The fused spatiotemporal features are then entered into boundary decoding and credibility decoding. Boundary decoding outputs the crop canopy boundary, ground boundary, and stubble retention trend. Credibility decoding outputs a comprehensive score for boundary continuity, image texture stability, occlusion degree, and attitude correction residual.

[0031] In a preferred embodiment, the temporal alignment process can obtain the state vector corresponding to the image frame according to the following formula:

[0032] ; in, Indicates the first The resampling state value corresponding to the frame image. Indicates the first The acquisition time of the frame image and These represent the state data at adjacent sampling times. and The value of , Indicates the time interpolation weights, if , , , , ,but , The physical meaning is to convert data from different sampling periods into control states corresponding to the same image frame.

[0033] Preferably, the spatiotemporal input window is further divided into a near-range window, a mid-range window, and a far-range window based on the change in driving speed within a continuous control cycle. The near-range window stores data for each acquisition cycle because the area in front of this window is about to reach the cutter, and data loss would directly affect the current control cycle. The mid-range window and the far-range window store the boundary features after resampling the work coordinates because this area is far from the cutter, and retaining all the original images would increase the computational load and contribute little to the current command. Historical commands, historical actual heights, and response deviations are written into the state vectors of each distance window, enabling the deep learning spatiotemporal recognition model to distinguish feature shifts caused by changes in the ground ahead, failure of the cutter to be executed, and changes in driving speed.

[0034] In this embodiment, the features of the near-range window, the medium-range window, and the far-range window are uniformly written into the arrival time index. The uniform arrival time index is not a fixed distance partition, but is dynamically updated according to the changes in driving speed and the closed-loop verification results. The closed-loop verification results are generated by the pre-execution height of the predicted window, the actual height of the cutter, historical instructions, and the change in driving speed. When there is a continuous same-direction time offset between the actual height of the cutter and the pre-execution height, the controller adjusts the longitudinal distance mapping value between the image acquisition position and the cutter cutting position. When the continuous same-direction time offset is accompanied by changes in the fuselage attitude, the controller only corrects the time label corresponding to the attitude correction amount to avoid misinterpreting the attitude error as the image distance mapping error.

[0035] To facilitate the organization of data in the spatiotemporal input window, this embodiment can adopt the data field definitions shown in Table 1 below. The field names in Table 1 are only used to describe the organization of data within the control period. Each field is associated with both the time index and the arrival time index during actual processing.

[0036] Table 1. Organization of Data Fields for Spatiotemporal Input Window ; Using the data organization method shown in Table 1 above, the spatiotemporal input window can place the foreground image features, the cutter state, and the execution response record into the same prediction link. The model can use image information when identifying boundaries, use state vectors when determining whether the boundary can enter the control command, and use arrival time index when arranging control. The advantage of this embodiment is that it enables a unified reference relationship between continuous images, state feedback, and execution response, reducing target height misalignment caused by inconsistent data sampling times.

[0037] In one embodiment, reference Figure 3 The target height sequence is generated by the crop canopy boundary, ground boundary, and stubble retention trend. The controller first converts the boundary results output by the deep learning spatiotemporal recognition model from image coordinates to working coordinates, and then corrects the boundary displacement caused by changes in image perspective based on the machine attitude. The longitudinal direction in the working coordinates corresponds to the harvester's travel direction, the lateral direction corresponds to the header width direction, and the height direction corresponds to the header lifting direction. After coordinate transformation and attitude correction, the crop canopy boundary, ground boundary, and stubble retention trend are written into the same segmented height curve. The segmented height curve corresponds to multiple prediction windows in the longitudinal direction and multiple fusionable areas within the header width in the lateral direction.

[0038] Specifically, the ground boundary is used to generate candidate heights based on safe ground passage, the stubble trend is used to generate candidate heights based on the continuity of the target stubble, and the crop canopy boundary is used to infer candidate heights under crop height changes. The three types of candidate heights are not directly averaged within the same prediction window, but are fused based on recognition confidence, attitude correction amount, and historical response deviation. If the ground boundary is continuous and the attitude correction amount is small, the candidate height based on the ground boundary has a higher weight. If the ground is occluded but the stubble trend is continuous, the candidate height based on the stubble trend has a higher weight. If the canopy boundary changes stably and the historical response deviation is low, the candidate height inferred from the crop canopy boundary participates in the supplement. After fusion, a single target height corresponding to the header control cycle is output.

[0039] In this embodiment, the target high-level fusion can be implemented according to the following formula:

[0040] ; in, Indicates the first The target height output by the prediction window This represents the candidate height obtained based on the ground boundary. This indicates the candidate height obtained based on the stubble retention trend. This represents the candidate height derived from the crop canopy boundary. , and These represent the fusion weights for the three candidate heights, respectively. , , , , , ,but The physical meaning is to transform the height of multiple source boundaries into a single control target height within the same prediction window.

[0041] Preferably, the three candidate heights are subject to different constraints before fusion. The candidate height based on the ground boundary is subject to a minimum ground clearance constraint to avoid misidentification of the boundary causing the target height to fall below the executable safe range. The candidate height based on the stubble retention trend is subject to an adjacent window continuity constraint to avoid short-term fluctuations in the stubble line causing frequent jumps in the target height. The candidate height based on the crop canopy boundary is subject to a canopy change consistency constraint to avoid local lodging or shading areas directly changing the overall target height. When the difference between the three candidate heights exceeds the fusion range allowed by the controller, the controller first reduces the weight of the candidate height with lower recognition confidence, and then outputs the remaining candidate heights as a single target height according to the constrained weights.

[0042] Furthermore, height continuity constraints and stubble variation constraints are introduced between adjacent prediction windows. Height continuity constraints are used to limit abrupt changes in target height before and after the same header control cycle. Stubble variation constraints are used to limit the deviation between the target height and the identified stubble variation trend. If a break occurs in the identification of the ground boundary in a prediction window but the stubble variation trend in adjacent windows remains continuous, the controller will move the target height of the window toward the continuous trend of the adjacent windows. If the crop canopy rises significantly in a prediction window but the ground boundary does not change, the controller will retain the candidate height of the ground boundary as the main reference and use the candidate height of the canopy as an auxiliary correction.

[0043] In this embodiment, the target height sequence can also be processed by partitioning the width direction of the cutter. The deep learning spatiotemporal recognition model outputs multiple boundary segments in the horizontal direction. The controller can obtain local candidate heights for each boundary segment and then synthesize them into the target height according to the overall height control method of the cutter. During synthesis, the horizontal local anomalies are not directly used as the basis for the overall lifting and lowering of the cutter. Instead, the controller detects whether the horizontal boundary segments maintain the same direction of change in consecutive image frames. If the local anomalies only appear in a few image frames or a few horizontal segments, the controller treats them as low-confidence inputs. If the local changes appear continuously in the near window and are consistent with the stubble retention trend, then the local changes participate in the target height fusion.

[0044] In the above embodiments, the target height sequence generation process converts the crop canopy boundary, ground boundary and stubble retention trend into control targets that are processed by attitude correction, candidate fusion and adjacent window continuous constraints, avoiding the direct mapping of a single image boundary to the header action. The advantage of this embodiment is that the source of the target height is traceable and has temporal continuity, enabling the header controller to generate a smooth target height sequence even when there is occlusion, boundary breakage or local changes in the canopy in the front area.

[0045] In one embodiment, reference Figure 4 The hysteresis compensation and motion constraints are established based on the cutter execution response data. The cutter execution response data comes from the height control command, actual cutter height feedback, command issuance time, actual height change time, target height arrival time, and response deviation in the historical control cycle. The controller distinguishes between the cutter's rising response and falling response based on these records, because the cutter's rising and falling may differ in terms of load, mechanical resistance, and control delay. If the target height sequence indicates that the cutter needs to rise, the rising response time and rising rate constraints are invoked. If the target height sequence indicates that the cutter needs to fall, the falling response time and falling rate constraints are invoked. If the target height change is less than the command dead zone, the change is merged into the subsequent control cycle.

[0046] In this embodiment, the target height sequence is compensated for hysteresis to form a pre-execution height sequence. The pre-execution height sequence is not simply issued at a fixed time in advance, but the pre-execution time is determined based on the arrival time of the cutter corresponding to each target height, the direction of the target height change, and the historical response deviation. If the cutter needs to reach the target height at a certain arrival time in the future, the controller moves the target height forward along the time axis to the time when the executable instruction is issued. If the change in target height exceeds the single-cycle change allowed by the maximum lifting rate, the controller splits the change into multiple consecutive control cycles. Each control cycle after splitting maintains a height progression relationship consistent with the cutter arrival time sequence.

[0047] In this embodiment, the smoothing process of the pre-execution height sequence includes merging height changes smaller than the instruction dead zone, suppressing continuous height changes in opposite directions, splitting height changes exceeding the maximum rise and fall rate across cycles, and maintaining a progressive relationship for height instructions changing in the same direction in adjacent control cycles. If adjacent target heights require rising, falling, and rising in sequence with small changes, the controller does not output instructions in the opposite direction in each control cycle. Instead, it determines whether to maintain the previous restricted instruction based on the recognition confidence and historical response deviation, thus preventing the cutter from being interrupted by a new reverse instruction before the actuator has completed its response.

[0048] Specifically, the pre-execution height subject to execution constraints can be calculated using the following formula:

[0049] ; in, Indicates the first The limited pre-execution height before the output of each control cycle Indicates the order after lag compensation. Reference height for each control cycle This indicates the actual height of the cutting table in the previous control cycle. This indicates the allowable height variation during the current control cycle. This indicates the permissible rate of ascent or descent selected based on the direction of ascent or descent. Indicates the duration of the control cycle. , , , ,but , The physical meaning is to limit the change in target height to the range that the cutting platform can execute in the current control cycle.

[0050] Preferably, the hysteresis compensation also incorporates historical response deviation correction. The controller compares the pre-execution height corresponding to the predicted window with the actual height of the cutter. If the actual height of the cutter lags behind the pre-execution height for a long period, it indicates that the current execution response is slower than the historical estimate, and the controller increases the subsequent pre-execution advance. If the actual height of the cutter reaches or exceeds the pre-execution height ahead of time, it indicates that the current pre-execution advance is too large, and the controller reduces the subsequent pre-execution advance. If the deviation is accompanied by changes in travel speed, the controller prioritizes correcting the arrival time mapping. If the deviation is accompanied by changes in fuselage attitude, the controller prioritizes correcting the time tag corresponding to attitude correction.

[0051] Furthermore, the fusion ratio between the pre-executed height sequence and the actual height feedback of the cutter is controlled by the recognition confidence level. The recognition confidence level is not only given by the deep learning spatiotemporal recognition model, but also forms a confidence level state together with the image boundary continuity, attitude correction stability and target height sequence change rate. When the confidence level state meets the forward control condition, the pre-executed height sequence is used as the main control input. When the confidence level state meets the mixed control condition, the pre-executed height sequence and the actual height feedback of the cutter are synthesized in a weighted manner. When the confidence level state meets the feedback holding condition, the controller locks the restricted instructions of the previous control cycle and receives the actual height feedback correction of the cutter.

[0052] In this embodiment, the controller also records the recognition confidence, boundary displacement direction, pre-execution height change direction, and actual cutter height change direction in adjacent control cycles through a state holding sequence. If the pre-execution height change direction and the actual cutter height change direction are continuously opposite, it indicates that there is a conflict between the actuator response and the forward prediction command. The controller converts the forward control condition into a hybrid control condition. If the boundary displacement direction changes alternately in the state holding sequence, it indicates that the image boundary is unstable. The controller freezes the fusion weight of the newly added prediction window and completes the current cycle output with the continuous control command that is already constrained by the execution constraints.

[0053] In the above embodiments, hysteresis compensation, rate limiting, dead zone elimination, reverse change merging, and reliability switching jointly limit the generation range of the cutter control command. The forward changes in the target height sequence are first converted into executable pre-execution heights, and then verified by the actual height feedback of the cutter and the state maintenance sequence. The advantage of this embodiment is that it can avoid the disconnect between the forward identification result and the response capability of the actuator, so that the cutter height command has reachability and continuity.

[0054] In a preferred embodiment, the recognition credibility state is composed of multiple pieces of information, including the recognition credibility output by the deep learning spatiotemporal recognition model, image boundary continuity, attitude correction stability, target height sequence change rate, historical response deviation, and boundary displacement direction stability. The image boundary continuity is calculated by the relative positional changes of the ground boundary, canopy boundary, and stubble trend in adjacent image frames. The attitude correction stability is calculated by the consistency of the boundary coordinate correction before and after the change in fuselage attitude. The target height sequence change rate is obtained by the difference in target height between adjacent prediction windows. The historical response deviation is obtained by the difference between the pre-executed height and the actual height of the cutter. The boundary displacement direction stability is determined by whether the boundary movement direction is continuous in the state holding sequence.

[0055] In this embodiment, the credibility status can be generated according to the following formula: ; in, Indicates the first The reliability status value for each control cycle. This indicates the recognition reliability of the output of the deep learning spatiotemporal recognition model. Indicates the continuity value of the image boundary. This represents the attitude correction stability value. Indicates the rate of change of the target height sequence. Indicates historical response bias. to This represents the weighting coefficients of each item, and their meaning remains unchanged within the same control task. , , , , , , , , , ,but The physical meaning is to unify visual recognition reliability, boundary continuity, posture stability and execution deviation as the basis for control weight switching.

[0056] Specifically, when the confidence state meets the forward control conditions, the controller generates control instructions based on the pre-executed height sequence and retains the actual height feedback of the cutter as boundary correction. When the confidence state meets the mixed control conditions, the controller allocates fusion weights between the pre-executed height sequence and the actual height feedback of the cutter according to the confidence state. When the confidence state meets the feedback retention conditions, the controller freezes the target height weight of the newly added prediction window and receives the actual height feedback correction of the cutter based on the restricted instructions of the previous control cycle. Logically, all three control conditions use the same confidence state and the same execution constraints, without introducing independent control routes.

[0057] In this embodiment, the control command under mixed control conditions can be calculated according to the following formula:

[0058] ; in, Indicates the first The target value corresponds to the header height control command output in each control cycle. Indicates the first The limited pre-execution height of each control cycle, Indicates the first Feedback value of the actual height of the cutting table for each control cycle This represents the look-ahead control fusion weights obtained by truncating the confidence state. , , ,but , The physical meaning is to generate a control target that takes into account both forward prediction and current position feedback when identifying a relatively stable target that still requires feedback correction.

[0059] Preferably, the state-maintaining sequence is updated using a first-in-first-out (FIFO) time-series record. In each control cycle, the identification confidence level, boundary displacement direction, pre-execution height change direction, and actual cutter height change direction are written to the end of the sequence, and old records exceeding the retention length are removed. The boundary displacement direction can be determined by the sign of the difference between the boundary positions of the current control cycle and the previous control cycle. The pre-execution height change direction can be determined by the sign of the difference between the current restricted pre-execution height and the previous restricted pre-execution height. The actual cutter height change direction can be determined by the sign of the difference between the current actual height and the previous actual height. If the direction signs in the sequence are continuously opposite or alternate, the controller performs weight freezing or condition downgrading processing on the newly added prediction window.

[0060] To facilitate the disclosure of the relationship between the credibility state and the control conditions, this embodiment can adopt the control condition organization method shown in Table 2 below. The condition names in Table 2 are used to describe the internal state transitions of the controller and do not represent independent control modules.

[0061] Table 2 Relationship between Reliability Status and Control Output Organization ; Using the control conditions shown in Table 2, the confidence status is directly used to control the output organization. The forward prediction result, feedback hold result, and weight freeze result all come from the same target height sequence, the same execution constraint, and the same state hold sequence. When the image is occluded, dusty, or affected by changes in lighting, the controller can reduce the control weight of the newly added forward recognition result and restore forward control after the boundary becomes continuous again. The advantage of this embodiment is that it forms a definite data path between the confidence judgment and the cutter output command, reducing the repeated rise and fall of the cutter caused by recognition fluctuations.

[0062] The primary weight under the forward-looking control conditions in the table refers to the proportion of the fusion weight corresponding to the pre-executed height sequence being higher than the proportion of the fusion weight corresponding to the actual height feedback of the cutter during the fusion calculation of the final cutter height control command. The pre-executed height sequence plays a dominant role in the generation of control commands. The specific value of the primary weight is determined based on the confidence state value: a confidence state threshold corresponding to the forward-looking control conditions is preset, and when the calculated... When the forward look control threshold is greater than or equal to the threshold, the forward look control condition is deemed met; at this point, the fusion weight is calculated using the same fusion weight calculation method as the hybrid control condition. Under forward-looking control conditions, the confidence level is high, therefore Take the higher value within the range of 0.7 to 1.0, and the weight corresponding to the actual height feedback of the cutter is... This ensures that the weights of the pre-executed height sequence are dominant, thus forming the sovereign weights.

[0063] In a preferred embodiment, the closed-loop verification result is used to update the unified arrival time index. The controller saves the original arrival time label, pre-execution height, actual height of the cutting platform, historical instructions, and changes in travel speed in each prediction window that has been passed. When the actual height of the cutting platform continuously deviates in the same direction from the pre-execution height, the controller determines that there is a deviation in the distance mapping between the image acquisition position and the cutting position of the cutting platform, and corrects the arrival time index of subsequent windows. When continuous time deviations in the same direction occur synchronously with changes in the aircraft attitude, the controller determines that the deviation mainly comes from the mismatch of the time labels of the attitude correction amount, and only corrects the time labels corresponding to the attitude correction amount without changing the image distance mapping value.

[0064] Specifically, closed-loop verification does not directly rewrite the output instructions within the current control cycle of the cutter. Instead, it updates the arrival time index when constructing the subsequent spatiotemporal input window to avoid secondary fluctuations caused by sudden changes in the verification results of the current cycle instructions. If the pre-execution height changes before the actual height of the cutter in multiple consecutive cycles, it indicates that the execution response time or arrival time mapping estimate is too early. The controller will appropriately advance the subsequent pre-execution time or adjust the distance mapping. If the change in pre-execution height is later than the change in the actual height of the cutter, it indicates that the arrival time mapping is too late or the feedback lag is small. The controller will reduce the subsequent pre-execution advance. If the deviation only occurs during the rapid change of fuselage attitude, the controller will write the attitude correction residual into the subsequent state vector and reduce the attitude correction stability value.

[0065] In this embodiment, closed-loop verification can also be used to update the input labels of the deep learning spatiotemporal recognition model. The image frames, recognition boundaries, target heights and actual heights in the prediction window have been associated with the task samples. However, these samples do not directly change the backbone parameters of the model. Instead, they are used in the current task to correct the arrival time labels, candidate height weights and state preservation sequences. If the same type of boundary is continuously judged as a low-confidence input in multiple windows, the controller reduces the weight of that type of boundary in the subsequent candidate height fusion. If a certain type of boundary remains consistent with the actual height feedback under low light or occlusion conditions, the controller retains its candidate weight.

[0066] In this embodiment, closed-loop verification, credibility status, and execution constraints together constitute a rolling update relationship. After the target height sequence is generated, it enters hysteresis compensation. After hysteresis compensation, a restricted pre-execution height is formed. The restricted pre-execution height and the actual height of the cutter are fed back to generate control commands. The actual height after the control commands are executed is then returned to the closed-loop verification. The closed-loop verification result is used to update the arrival time index and candidate height weights. The updated arrival time index enters the next round of spatiotemporal input window construction, thereby ensuring that the forward identification results, arrival time mapping, and execution response data are continuously aligned during continuous operation.

[0067] In the above embodiments, the closed-loop verification does not add a new control object, nor does it deviate from the cutter height control link. Instead, it establishes a time deviation correction mechanism between the target height sequence, the pre-execution height sequence, and the actual height feedback. The advantage of this embodiment is that it can maintain the correspondence between the front area and the cutter execution time when changes in driving speed, fuselage attitude, and execution response coexist, thereby reducing the control time offset caused by fixed mapping.

[0068] In a preferred embodiment, the training samples of the deep learning spatiotemporal recognition model can be constructed from historical harvesting operation data. The samples include continuous operation images, actual header height, driving speed, machine attitude, historical commands, execution response deviation, and manually or rule-generated crop canopy boundary labels, ground boundary labels, and stubble retention trend labels. During training, image frames are sorted according to acquisition time and appended with state vectors. The model simultaneously learns boundary recognition, stubble retention trend estimation, and recognition confidence estimation. The boundary recognition loss is used to constrain the crop canopy boundary and ground boundary, the stubble retention trend loss is used to constrain the height trend between adjacent prediction windows, and the confidence loss is used to constrain the model to give lower control weights in scenarios of occlusion, dust, boundary breakage, and attitude change.

[0069] In this embodiment, after the model training is completed, it enters the harvester control process. During online operation, the core parameters of the model can remain unchanged. The controller completes the adaptation through the spatiotemporal input window, arrival time index, candidate height weight, confidence status, and execution constraints. If it is necessary to adapt to different crops or different plot conditions, it can be done by updating the candidate height weight, attitude correction parameters, and state maintenance sequence judgment conditions. It is not necessary to change the header mechanical structure, nor is it necessary to separate the front image recognition results from the execution response data for separate control. The model output and the controller constraints maintain a definite data interface.

[0070] Specifically, the boundary recognition output can be represented by a pixel-level probability map or boundary curve parameters. If a pixel-level probability map is used, the controller obtains the ground boundary and canopy boundary under the working coordinates through connected component filtering and boundary fitting. If boundary curve parameters are used, the controller directly projects the curve parameters onto the working coordinates and then enters attitude correction. The stubble trend can be represented by the target height change curve of adjacent prediction windows. The recognition reliability can be obtained by the concentration of boundary probability distribution, the consistency of adjacent frame boundaries, and image texture occlusion features. All of the above outputs are written into the target height sequence generation process.

[0071] In this embodiment, the recognition confidence level output by the model is different from the confidence level state generated by the controller. The recognition confidence level represents the reliability of the deep learning spatiotemporal recognition model in recognizing image boundaries and stubble trends. The confidence level state also includes image boundary continuity, attitude correction stability, target height sequence change rate, and historical response deviation. The controller uses the recognition confidence level as an input to the confidence level state, rather than directly using the recognition confidence level to determine the control command. Therefore, when the image recognition result is stable but the actual height of the cutter table deviates from the pre-execution height for a long time, the controller can still reduce the forward control weight. When the image recognition result decreases temporarily but the boundary continuity and feedback consistency are maintained, the controller can maintain the hybrid control output.

[0072] In the above embodiment, the deep learning spatiotemporal recognition model and the controller are connected through boundary output, stubble retention trend, recognition confidence and state vector. The model is responsible for converting continuous operation images into controllable forward boundary information, and the controller is responsible for converting the boundary information into arrival time series and control commands subject to execution constraints. The advantage of this embodiment is that the boundary between model recognition and header control is clear, avoiding direct driving of header lifting and lowering with an uninterpretable single output.

[0073] In a preferred embodiment, the output of the header height control command can be further compatible with the harvester's original height closed-loop control interface. The controller outputs not the original electrical signal that directly drives the actuator, but the target height or constrained increment corresponding to the header height control interface. The original closed-loop control interface continues to complete the underlying execution based on the actual header height feedback. The method provides the original closed-loop control interface with constrained upper-level control targets through the target height sequence, the pre-execution height sequence, and the confidence status, thereby achieving forward predictive control without changing the header actuator structure.

[0074] In this embodiment, if the harvester's original control interface receives the target height, the controller will mix the... As the target height input, and with the existing control interface handling the ascent and descent. If the existing control interface receives a height increment, the controller will... The difference between the current actual height of the cutter and the current height of the cutter is converted into an incremental instruction, and dead zone elimination and rate limiting continue to be executed. If the original control interface receives discrete instructions such as hold, rise and fall, the controller generates discrete control requests based on the directional relationship between the limited pre-execution height and the actual height of the cutter, and limits continuous reverse requests through the state holding sequence.

[0075] Specifically, discrete control requests can be obtained from the sign of the difference between the restricted pre-execution height and the actual height of the cutter. If the difference is within the command dead zone, a hold request is output. If the difference is greater than the command dead zone and the confidence state allows look-ahead or hybrid control, an ascend request is output. If the difference is less than the negative command dead zone and the confidence state allows look-ahead or hybrid control, a descend request is output. If the state hold sequence shows that the execution direction of the previous cycle is continuously opposite to the current request direction, the controller will downgrade the current request to a hold request or a hybrid control request to prevent the actuator from being driven in reverse before completing the previous action.

[0076] In this embodiment, the interface data between the upper-level control target and the original closed-loop control interface is still written back to the execution response data. The actual height feedback, response time and response deviation of the cutter generated after the original closed-loop control interface completes the action are entered into the spatiotemporal input window of the next control cycle. The controller updates the execution constraint table and closed-loop verification results through these data, so that the upper-level forward prediction and the lower-level height closed loop maintain data interaction and avoid the separation of the upper-level target height from the actual execution process.

[0077] In the above embodiments, the cutter height control command can be output in the form of target height, limited increment or discrete control request, but its generation is based on continuous operation images, actual cutter height, travel speed, fuselage attitude, execution response data, target height sequence, hysteresis compensation and confidence status. The advantage of this embodiment is that it can adapt to different forms of cutter height control interface and maintain a unified closed loop between forward perception, arrival time mapping and execution feedback.

[0078] In a complete embodiment, after the harvester enters the work row, the controller continuously receives continuous working images and status data in front of the header. The time-series alignment process writes the image frames, actual header height, travel speed, machine attitude, and execution response data into the spatiotemporal input window. The deep learning spatiotemporal recognition model outputs the crop canopy boundary, ground boundary, stubble retention trend, and recognition confidence from the spatiotemporal input window. The controller generates the header arrival time sequence based on the travel speed and the work coordinates of the area in front, and converts the boundary output into a target height sequence. The target height sequence is constrained by the header execution response data to form a pre-execution height sequence. The confidence status determines the fusion ratio between the pre-execution height sequence and the actual header height feedback. The fused height control command enters the header height control interface. The executed actual header height, response deviation, and historical commands are returned to the spatiotemporal input window and the closed-loop verification process.

[0079] In this embodiment, when the harvester enters a region where the ground is slowly rising, the ground boundary in front gradually rises in the far-distance window and the medium-distance window. The target height sequence first shows an upward trend in the future arrival time. The lag compensation moves part of the target height forward according to the header's rise response time. The restricted pre-execution height rises continuously in multiple control cycles. The actual header height feedback returns with the execution process. If the direction of the actual height change is consistent with the direction of the pre-execution height, the state holding sequence maintains the forward-looking control conditions. If local dust in the operation image causes a short-term break in the boundary, the confidence state decreases, but the hybrid control output can still be maintained through the stubble retention trend and the actual header height feedback.

[0080] In this embodiment, when the harvester enters a rapidly concave area of ​​the ground, the ground boundary and stubble trend show a decrease in target height within the near-field window. However, the controller does not immediately output an unrestricted descent command. Instead, it breaks down the target height change based on the descent response time, command dead zone, and maximum lifting rate. If the target height change exceeds the executable range of the current control cycle, the limited pre-execution height only outputs the descent target within the reachable range. If the boundary displacement direction changes alternately within the state holding sequence, the controller freezes the weight of the newly added prediction window and maintains the already formed continuous control commands, so that the header does not exhibit continuous reverse actions due to boundary fluctuations when passing through the concave area.

[0081] In this embodiment, when the harvester's travel speed changes, the arrival time of the header corresponding to the same forward image area is recalculated. At low speeds, the prediction window of the target height sequence is spaced longer on the time axis, while at high speeds, the prediction window is spaced shorter on the time axis. The lag compensation rearranges the pre-executed height sequence according to the new arrival time. The closed-loop verification result also compares the pre-executed height of the window that has passed with the actual height of the header. If it is found that the actual height is continuously lagging behind the pre-executed height, the subsequent arrival time index or pre-execution advance is corrected, so that changes in travel speed will not fix the forward recognition result to the error control cycle.

[0082] In this embodiment, when the fuselage attitude changes pitch, the ground boundary and canopy boundary in the image will be relatively displaced in the field of view. The controller first uses the fuselage attitude to correct the boundary coordinates, and then enters the corrected boundary into the candidate height fusion. If the closed-loop verification result shows that the deviation occurs synchronously with the attitude change, the controller does not directly modify the distance mapping value from the image acquisition position to the cutting position of the cutting table, but corrects the time label corresponding to the attitude correction amount and reduces the attitude correction stability value to prevent the boundary displacement caused by the attitude from being misinterpreted as terrain undulation.

[0083] In this embodiment, if the working image is affected by occlusion or changes in lighting, the recognition credibility of the deep learning spatiotemporal recognition model output decreases, and the continuity of the image boundary also decreases. The controller switches the forward control condition to a hybrid control condition or a feedback hold condition based on the credibility state. Under the hybrid control condition, the pre-executed height sequence still participates in the output but its weight decreases. Under the feedback hold condition, the weight of the newly added prediction window is frozen. The previous restricted instruction and the actual height feedback of the cutter table together form the current output. After the continuity of the adjacent frame boundary is restored, the state hold sequence is gradually unfrozen, so that the controller can receive the newly added forward recognition result again.

[0084] In the complete embodiment described above, the harvester header autonomous control method forms a single closed loop around forward recognition, time mapping, execution compensation, and credibility constraints. The continuous operation image in front does not directly drive the header's lifting and lowering, the actual height feedback of the header does not independently determine the target height, the travel speed is not merely displayed or recorded data, the machine attitude is not merely status monitoring data, and the execution response data is not merely historical data. Instead, they are respectively incorporated into the target height sequence generation, hysteresis compensation, credibility status, and closed-loop verification process. The advantage of this embodiment is that it can convert spatial changes in the front into a temporally executable control process, enabling the header to maintain constrained height adjustments under continuous operation conditions.

Claims

1. A deep learning-based autonomous control method for the header of a harvester, characterized in that, include: Acquire continuous operation images in front of the header, actual header height, harvester travel speed, machine attitude, and header execution response data; The continuous operation images are time-aligned with the actual height, the driving speed and the body posture. Adjacent multiple frames of continuous operation images and their corresponding state vectors are used to form a spatiotemporal input window and labeled with the header arrival time label. The data are then input into a deep learning spatiotemporal recognition model to obtain the crop canopy boundary, ground boundary, stubble retention trend and recognition credibility of the area to be harvested ahead. Based on the driving speed, the area to be harvested ahead is mapped as the time series of the header arrival. Three types of candidate heights are generated based on ground boundary, stubble retention trend, and crop canopy boundary. Constraints are set for each type of candidate height and they are fused according to the recognition credibility to generate the target height sequence. Based on the rise response time, fall response time, command dead zone, and maximum rise / fall rate in the cutter execution response data, the target height sequence is shifted forward to obtain the pre-execution height sequence, and dead zone removal, rate limiting, and smoothing are performed. Combined with the recognition confidence level, a confidence state is formed and switched between forward control, hybrid control, and feedback hold mode to output the cutter height control command.

2. The deep learning-based autonomous control method for harvester headers according to claim 1, characterized in that, The timing alignment includes: establishing a time index based on the acquisition time of continuous operation images, according to the sampling time of the actual height of the cutting table, the travel speed and the body attitude within the control cycle; interpolating and resampling the data in adjacent control cycles to form a state vector corresponding to each frame image.

3. The deep learning-based autonomous control method for the header of a harvester according to claim 2, characterized in that, The generation of the target height sequence includes: converting the crop canopy boundary, the ground boundary, and the stubble retention trend into segmented height curves under the operational coordinates; The attitude correction is performed on the boundary displacement caused by changes in the image viewing angle based on the fuselage attitude. The segmented height curves are arranged into multiple prediction windows according to the arrival time labels; A height continuity constraint and a stubble variation constraint are introduced between adjacent prediction windows, and multiple candidate heights within the same control cycle are fused according to their recognition credibility to form a target height sequence corresponding to the header control cycle.

4. The deep learning-based autonomous control method for the header of a harvester according to claim 3, characterized in that, An execution constraint table is established based on the cutter execution response data, including cutter rise response time, fall response time, instruction dead zone, and maximum rise / fall rate. Each target height in the target height sequence is shifted forward according to the corresponding cutter arrival time to obtain a pre-execution height sequence. Dead zone removal, rate limiting, and adjacent instruction smoothing are performed on the pre-execution height sequence to generate cutter height control instructions.

5. The deep learning-based autonomous control method for the header of a harvester according to claim 2, characterized in that, The construction of the spatiotemporal input window also includes: dividing adjacent multi-frame images into near-range windows, mid-range windows, and far-range windows based on the change in driving speed within a continuous control cycle; The near-range window retains data from each acquisition cycle, while the mid-range window and the far-range window retain the boundary features after resampling by the work coordinates. The historical commands, historical actual height, and response deviation from the cutter execution response data are written into the state vector of the corresponding window, and the forward features at different distances are written into the deep learning spatiotemporal recognition model according to a unified arrival time index.

6. The deep learning-based autonomous control method for harvester headers according to claim 3, characterized in that, The fusion of candidate heights includes: generating candidate heights based on ground boundaries, candidate heights based on stubble retention trends, and candidate heights derived from crop canopy boundaries within each prediction window; setting minimum ground clearance constraints for candidate heights based on ground boundaries and setting continuous constraints for candidate heights based on stubble retention trends; when the three types of candidate heights deviate within the same control period, determining the fusion weights according to recognition confidence, attitude correction amount, and historical response deviation, and outputting a single target height.

7. The deep learning-based autonomous control method for the header of a harvester according to claim 4, characterized in that, The confidence level is jointly formed by recognition confidence, image boundary continuity, pose correction stability, and target height sequence change rate. When the confidence state meets the forward control conditions, the pre-executed height sequence is used as the main control input; When the confidence state meets the hybrid control conditions, the pre-execution height sequence and the actual height feedback of the cutter are synthesized in a weighted manner; When the confidence status meets the feedback hold condition, the restricted command of the previous control cycle is locked and the actual height of the cutting table is received for correction.

8. The deep learning-based autonomous control method for the header of a harvester according to claim 4, characterized in that, The smoothing process of the pre-execution height sequence includes: calling the rise response time and fall response time respectively according to the increase or decrease direction of the adjacent target heights, and calculating the executable height change amount of the corresponding control cycle; Merge height changes smaller than the instruction dead zone with consecutive height changes in opposite directions; Height changes exceeding the maximum rate of ascent and descent are divided into multiple control cycles. Within the split control cycle, a highly progressive relationship consistent with the arrival time sequence of the cutter is maintained, forming continuous control instructions subject to execution constraints.

9. The deep learning-based autonomous control method for the header of a harvester according to claim 5, characterized in that, The unified arrival time index is also updated according to the closed-loop verification result, which is generated by the pre-execution height corresponding to the prediction window, the actual height of the cutter, historical instructions, and the change in travel speed. When the actual height of the cutting table has a continuous time offset in the same direction relative to the pre-execution height, the longitudinal distance mapping value between the image acquisition position and the cutting position of the cutting table is adjusted. When the continuous same-direction time offset occurs along with changes in fuselage attitude, only the time label corresponding to the attitude correction amount is corrected, and the updated arrival time index is used for subsequent spatiotemporal input windows.

10. The deep learning-based autonomous control method for the header of a harvester according to claim 7, characterized in that, The confidence status is also updated through a state holding sequence, which records the recognition confidence, boundary displacement direction, pre-execution height change direction, and actual height change direction of the cutting table in multiple adjacent control cycles. When the direction of the pre-execution height change is continuously opposite to the direction of the actual height change of the cutting table, the forward control condition is converted into a mixed control condition. When the boundary displacement direction alternates within the state-maintaining sequence, the fusion weight of the newly added prediction window is frozen, and the current control cycle is output with continuous control instructions subject to execution constraints.

Images