Agricultural machinery operation self-adaptive control system and method based on vision intelligence
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- WEIFANG WANLONG ELECTRIC CO LTD
- Filing Date
- 2026-02-26
- Publication Date
- 2026-05-26
Smart Images
Figure CN122085683A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of adaptive control technology, and in particular to an adaptive control system and method for agricultural machinery operation based on visual intelligence. Background Technology
[0002] In recent years, intelligent agricultural machinery control technology has gradually evolved from mechanical sensing and rule-based control to an operation control paradigm centered on visual perception and data-driven decision-making. Common methods involve acquiring image information such as crop canopy, row structure, and operation boundaries through cameras at the front of the machine or near the working parts. This information is then combined with operating condition signals such as vehicle speed, working part rotation speed, and implement posture to complete operation status recognition, operation parameter setting, and actuator control, forming an automated operation process for multiple scenarios such as spraying and harvesting.
[0003] However, existing technologies mostly rely on direct regression of control quantities from single-frame or short-window visual features, or on mapping with fixed strategies and empirical rules, making it difficult to express the temporal coupling relationship of "operation state - working condition - action sequence". When farmland texture changes, inter-row structure is disturbed, or operation boundaries are blurred, control decisions are prone to problems of temporal discontinuity and incoordination of action segments. At the same time, there is a lack of interpretable pre-show mechanisms for future operation visual results, resulting in insufficient constraints on operation action selection on operation quality objectives such as coverage continuity, inter-row structure preservation, and boundary smoothness. Furthermore, it is difficult to form a traceable chain of operation evidence to support experience reuse and quality auditing. Summary of the Invention
[0004] In view of the aforementioned existing problems, the present invention is proposed.
[0005] Therefore, this invention provides an adaptive control method for agricultural machinery operations based on visual intelligence to solve the problem of difficulty in continuously generating and interpretably verifying the sequence of agricultural machinery operation actions under visual disturbances in complex farmland.
[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, the present invention provides an adaptive control method for agricultural machinery operations based on visual intelligence, comprising, The agricultural machinery operation process is continuously perceived through a multi-view visual acquisition unit, and the temporal consistency modeling and geometric correlation analysis of the operation image sequence are performed to generate an operation status token set. The set of operation status tokens is coupled with agricultural machinery operating condition signals in a time sequence, and then matched with the historical operation segment library to generate an action prior index corresponding to the current operation status. Based on the set of job status tokens and the prior index of actions, a sequence of job actions that can be directly issued and executed is generated through a conditional diffusion-type action generation model; Conditional visual evolution prediction is performed on the sequence of work actions to generate corresponding visual results of future work and determine the target execution sequence; The target execution sequence is segmented and distributed to agricultural machinery execution agencies, and a traceable chain of operational evidence is constructed simultaneously.
[0007] As a preferred embodiment of the visual intelligence-based adaptive control method for agricultural machinery operations described in this invention, the operation image sequence is synchronously acquired by a front-end camera, a near-end camera of the operation component, and a side camera within the same operation time period. The operational image sequence includes crop canopy images, interrow structure images, surface texture images, and images of the contact boundary between the operational component and the crop.
[0008] As a preferred embodiment of the visual intelligence-based adaptive control method for agricultural machinery operations described in this invention, the specific steps for generating the set of operation status tokens are as follows: Perform timestamp alignment and multi-view frame association on the job image sequence to form a multi-view time sequence frame group; Geometric consistency modeling is performed on multi-view temporal frame groups to generate a sequence of relative spatial relationships between agricultural machinery and crops; Temporal features reflecting inter-row offset, coverage continuity and job boundary changes are extracted from the relative spatial relationship sequence and discretized to obtain a set of job status tokens.
[0009] As a preferred embodiment of the visual intelligence-based adaptive control method for agricultural machinery operations described in this invention, the specific steps for temporally coupling the set of operation status tokens with the agricultural machinery operating condition signals are as follows: Acquire agricultural machinery operating condition signals synchronized with the operation image sequence and complete timestamp unification to obtain a set of operating condition time sequence signals; the agricultural machinery operating condition signals include vehicle speed, operating component rotation speed and implement posture; Align the set of work status tokens according to the token time slice with the set of work condition timing signals according to the sampling time slice, and establish a corresponding mapping relationship to obtain the token work condition correspondence table; Based on the token-operational-condition mapping table, the operation status tokens and operation condition signals of adjacent time slices are combined to obtain a joint segment of status and operation conditions used for matching and retrieval.
[0010] As a preferred embodiment of the visual intelligence-based adaptive control method for agricultural machinery operations described in this invention, the specific steps for generating the action prior index corresponding to the current operation state are as follows: Based on the state and operating conditions, a retrieval key and a fragment fingerprint are generated to obtain the current fragment fingerprint set; Retrieve historical fragment fingerprints that form the same structure as the current fragment fingerprint set from the historical task fragment library, and perform similarity matching to obtain a set of candidate historical task fragments; Extract historical action segments that are consistent with the operation control interface from the candidate historical operation segment set, and organize them in chronological order to obtain a historical action prototype set; The historical action prototype set is compressed and encoded into an action prior index, and the correspondence with the job status token set is retained to obtain the action prior index corresponding to the current job status.
[0011] As a preferred embodiment of the visual intelligence-based adaptive control method for agricultural machinery operations described in this invention, the historical operation segment library is obtained by archiving a set of operation evidence units formed during multiple operations according to operation type and working condition tags.
[0012] As a preferred embodiment of the visual intelligence-based adaptive control method for agricultural machinery operations described in this invention, the specific steps for generating a sequence of directly executable operational actions using a conditional diffusion-based action generation model are as follows: A motion generation domain is constructed within the motion representation space that is consistent with the agricultural machinery operation control interface, and multiple sets of initial motion trajectories consistent with the motion representation space are generated. The process involves progressively denoising multiple initial action trajectories, imposing job state constraints on the denoising process using the job state token set and action structure constraints on the denoising process using the action prior index, thus forming multiple candidate job action sequences constrained by both state and prior constraints. Perform control interface consistency verification and boundary pruning on the candidate job action sequences to obtain a set of candidate action sequences that can be executed by the interface. The timing coordination of the walking system actions and the actions of the working components is performed on the set of candidate action sequences that can be executed by the interface to obtain multiple sets of candidate operation action sequences that can be directly issued and executed.
[0013] As a preferred embodiment of the visual intelligence-based adaptive control method for agricultural machinery operations described in this invention, the specific steps for performing conditional visual evolution prediction on the sequence of operational actions are as follows: Call the task visual prediction unit and generate a short temporal visual prediction start frame group that is consistent with the current task scene to obtain the prediction start visual segment; The predicted initial visual segments are associated with multiple sets of candidate task action sequences to generate corresponding multiple sets of future task visual results. From the visual results of each group of future operations, we extract the coverage continuity feature, the inter-row structure preservation feature, and the cutting boundary smoothness feature to obtain multiple sets of visual discrimination features; Perform visual consistency adjudication on multiple sets of visual discriminative features to determine the target execution sequence that is consistent with the task objective.
[0014] As a preferred embodiment of the visual intelligence-based adaptive control method for agricultural machinery operations described in this invention, the specific steps for constructing a traceable chain of operational evidence are as follows: The target execution sequence is divided according to the vehicle control cycle to obtain a segmented execution table; Based on the segmented execution table, corresponding action segments are sequentially sent to the walking system controller and the working component controller, and the sending time is recorded to obtain an execution record set; The execution record set is bound and encapsulated with the job status token set, action fragments, and corresponding multi-view time frame groups of the job image sequence to generate a job evidence unit set; Through the Internet of Things (IoT) link, the collection of work evidence units is uploaded to the cloud data service and archived by work batch to form a traceable work evidence chain.
[0015] Secondly, the present invention provides an adaptive control system for agricultural machinery operation based on visual intelligence, including a visual modeling module, which continuously perceives the agricultural machinery operation process through a multi-view visual acquisition unit, performs temporal consistency modeling and geometric correlation analysis on the operation image sequence, and generates an operation status token set. The prior construction module couples the set of work status tokens with the agricultural machinery working condition signals in a time sequence and matches them in the historical work segment library to generate an action prior index corresponding to the current work status. The action generation module generates a sequence of job actions that can be directly issued and executed based on the job status token set and the action prior index through a conditional diffusion action generation model. The visual inference module performs conditional visual evolution prediction on the sequence of work actions, generates corresponding visual results of future work, and determines the target execution sequence. The operation execution module distributes the target execution sequence in segments to the agricultural machinery execution agency and simultaneously builds a traceable chain of operation evidence.
[0016] The beneficial effects of this invention are as follows: Multi-view operation image sequences are used to generate an operation status token set through temporal consistency modeling and geometric correlation analysis, enabling the operation status to be stably expressed using discrete temporal symbols, thus improving the distinguishability of states such as inter-row offset, coverage continuity, and operation boundary changes; the operation status token set is temporally coupled with agricultural machinery operating condition signals and matched with a historical operation segment library to obtain a priori action index, giving candidate actions empirical constraints and operating condition consistency; the conditional diffusion-type action generation model outputs directly executable operation action sequences under the combined effect of operation status constraints and action structure constraints, transforming control from single-point setting to executable action segments; conditional visual evolution prediction performs a visual preview of future operation results for candidate operation action sequences, and selects the target execution sequence through visual consistency adjudication, allowing action selection to directly constrain operation quality objectives such as coverage continuity, inter-row structure preservation, and cutting boundary smoothness. Attached Figure Description
[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0018] Figure 1 This is a flowchart of a vision-based adaptive control method for agricultural machinery operations.
[0019] Figure 2 A flowchart for generating action prior indexes.
[0020] Figure 3 A flowchart for generating a sequence of job actions.
[0021] Figure 4 A flowchart for determining the target execution sequence. Detailed Implementation
[0022] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.
[0023] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.
[0024] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.
[0025] Reference Figures 1-4 As one embodiment of the present invention, this embodiment provides an adaptive control method for agricultural machinery operations based on visual intelligence, comprising the following steps: S1. The agricultural machinery operation process is continuously perceived through a multi-view visual acquisition unit, and the temporal consistency modeling and geometric correlation analysis of the operation image sequence are performed to generate an operation status token set.
[0026] S1.1. The agricultural machinery operation process is continuously perceived through a multi-view visual acquisition unit to form an operation image sequence describing the operation status. The multi-view visual acquisition unit includes a front-end camera, a near-end camera of the operating component, and a side camera. The front-end camera is used to acquire overall scene images of the operation area in front of the agricultural machinery. The near-end camera of the operating component is used to acquire contact boundary images between the operating component and the crop. The side camera is used to acquire images of the inter-row structure and lateral spatial relationship. Each camera acquires operation images synchronously within the same operation time period, thereby forming an operation image sequence with temporal consistency.
[0027] To further explain, the image sequence includes crop canopy images, row structure images, surface texture images, and images of the contact boundary between the working component and the crop.
[0028] S1.2. Perform timestamp alignment processing on the job image sequence to rearrange job images from different perspectives according to a unified time base, thereby eliminating the time offset problem caused by sampling frequency differences and communication delays during multi-view acquisition. Through timestamp alignment processing, job images from different perspectives within the same time slice have consistent temporal semantics.
[0029] After completing timestamp alignment, multi-view frame association processing is performed on the operation image sequence. By matching feature points and mapping viewpoint correspondence, spatial correspondence between images from different viewpoints is established, so that multi-view operation images in the same time slice can form an alignable structure in space. Multi-view time sequence frame group is constructed, which is used to describe the spatial state of the agricultural machinery operation area in the same time slice under multiple viewpoints.
[0030] Geometric consistency modeling is performed on multi-view temporal frame groups. By analyzing the geometric correspondence between images from different perspectives, the relative spatial position, relative attitude, and spatial changes of the contact area between the agricultural machinery and the crop are calculated. A sequence of relative spatial relationships between the agricultural machinery and the crop is generated. This sequence describes the changes in the spatial relationship between the agricultural machinery and the crop over time during continuous operation, enabling the operation process to be expressed in the form of a continuous geometric structure.
[0031] S1.3. Extract time-series features reflecting row offset, coverage continuity and operation boundary changes from the relative spatial relationship sequence. The time-series features are obtained by calculating the changes in the relative spatial relationship between adjacent time slices and are used to describe the offset trend of crop row structure, the continuity of operation coverage area and the smooth changes of operation boundary during operation.
[0032] To further explain, the change in spatial relationship between adjacent time slices can be expressed as: ; in, Indicates time slice The relative spatial relationship between agricultural machinery and crops is described at any given time by geometric consistency modeling. Indicates time slice Description of the relative spatial relationships at any given time. It represents the change in spatial relationship between adjacent time slices, and is used to characterize the dynamic features of line offset, changes in coverage continuity, and changes in job boundaries.
[0033] Discretization encoding is performed on the temporal features, mapping the continuously changing spatial relationship changes to a finite number of discrete state intervals, so that each time slice corresponds to a discrete state representation, and they are arranged in chronological order, converting the continuous operation process into a temporal sequence composed of discrete symbols, and generating a set of operation state tokens.
[0034] It should be noted that, through discretization encoding, the temporal features are transformed into a set of operation status tokens, enabling the set of operation status tokens to fully describe the dynamic changes in the spatial structure during agricultural machinery operations.
[0035] S2. Couple the set of work status tokens with the agricultural machinery working condition signals in a time sequence, and match them in the historical work segment library to generate a priori index of actions corresponding to the current work status.
[0036] S2.1. After obtaining the set of operation status tokens, synchronously acquire the agricultural machinery operating condition signals corresponding to the operation image sequence, and perform unified timestamp processing on the agricultural machinery operating condition signals to form a set of operating condition time sequence signals.
[0037] Among them, the agricultural machinery operating condition signals include vehicle speed, working component rotation speed and implement posture, which are used to describe the changes in the operating status of agricultural machinery during operation.
[0038] It should be noted that the unified timestamp processing rearranges the timestamps of different sampling sources, so that the set of operation status tokens and the agricultural machinery operating condition signals have consistent time semantics under the same time base, thereby ensuring the accuracy of subsequent timing coupling operations.
[0039] Time slice alignment processing is performed on the set of operation status tokens and the set of operation condition timing signals. The set of operation status tokens is arranged according to the token time slice, and the set of operation condition timing signals is arranged according to the sampling time slice. A corresponding mapping relationship between operation status tokens and agricultural machinery operation condition signals is established to form a token operation condition correspondence table.
[0040] S2.2. Based on the token-operational-condition correspondence table, combine the operation status tokens and operation condition signals of adjacent time slices to form a joint segment of status and operation conditions for matching and retrieval.
[0041] It should be noted that the joint state-condition segment is used to describe the joint evolution process of changes in work state and operating conditions during operation, so that work state information and agricultural machinery operating state information can be continuously expressed in the time dimension.
[0042] By constructing joint state-condition segments from consecutive time slices, a sequence of joint state-condition segments is formed.
[0043] Based on the joint state and working condition fragments, a retrieval key and fragment fingerprint are generated to form the current fragment fingerprint set. The fragment fingerprint is used to encode the structural features of the joint state and working condition fragments, so that different work fragments can participate in subsequent similarity matching operations in a unified form.
[0044] To further explain, the retrieval key is composed of the order of the operation status tokens in the joint segment of state and condition and the time series arrangement of the agricultural machinery condition signal, which is used to represent the overall characteristics of the joint segment of state and condition in terms of time structure; the segment fingerprint is formed by combining the operation status token identifiers of each time slice in the joint segment of state and condition and the discrete interval representation of the agricultural machinery condition signal, which is used to express the structural characteristics of the joint segment of state and condition.
[0045] S2.3. Retrieve historical fragment fingerprints that form the same structure as the current fragment fingerprint set from the historical operation fragment library, and perform similarity matching between the historical fragment fingerprints and the current fragment fingerprint set to obtain a candidate historical operation fragment set. The similarity matching is to select historical operation fragments from the historical operation fragment library that are similar to the current operation process in terms of operation state evolution and working condition change characteristics.
[0046] To further explain, the historical operation fragment library is obtained by archiving the set of operation evidence units formed during multiple operations according to operation type and working condition tags.
[0047] Similarity matching compares the arrangement of the operation status tokens in each time slice of the segment fingerprint with the arrangement of the agricultural machinery operation signal item by item, calculates the degree of consistency between the current segment fingerprint and the historical segment fingerprint in terms of time structure, and selects historical operation segments with a consistency higher than a preset similarity threshold into the candidate historical operation segment set.
[0048] The similarity threshold is obtained by statistically analyzing the temporal structure consistency between historical operation segments of the same operation type and under the same working condition label in the historical operation segment library. By analyzing the concentrated intervals of the consistency distribution among historical operation segments, a lower bound value representing the structure of similar operation behaviors is determined, and this lower bound value is set as the similarity threshold. For example, in spraying operations and straight-line walking conditions, the statistical distribution of temporal structure consistency typically falls within the range of 0.70 to 1.00; therefore, 0.70 is used as the similarity threshold. In harvesting operations and frequent posture adjustment conditions, the statistical distribution of temporal structure consistency typically falls within the range of 0.60 to 1.00; therefore, 0.60 is used as the similarity threshold. Through this method, the similarity threshold is dynamically adjusted according to changes in operation type and working condition category, thereby ensuring that the historical operation segments selected for the candidate historical operation segment set have interpretable consistency with the current operation process in terms of the evolution structure of the operation state and the rhythm of working condition changes.
[0049] It should be noted that the consistency level is obtained by comprehensively comparing the matching relationship of the operation status tokens in the aligned time slices with the matching relationship of the agricultural machinery operating condition change trend, and the expression is: ; in, Indicates the fingerprint of the current segment fingerprints from historical fragments The degree of consistency in the time structure Indicates the fingerprint of the current segment; Fingerprints representing historical fragments; Indicates the number of time slices after alignment; Indicates the number of fingerprint segments in the current segment. A time-slice job status token; Indicating the first fingerprint fragment in historical data A time-slice job status token; Indicates the number of fingerprint segments in the current segment. Agricultural machinery operating condition signals in a time slice; Indicating the first fingerprint fragment in historical data Agricultural machinery operating condition signals in a time slice; This indicates the consistency judgment result of the job status tokens in discrete status categories. When both belong to the same discrete status interval, they are marked with the same flag; otherwise, they are marked with different flags. It indicates the consistency of agricultural machinery operating condition signals in terms of changing trends, and is used to reflect the degree of consistency between the direction and rhythm of operating condition changes within adjacent time slices.
[0050] To elaborate further, the consistency in trends of change includes consistency in the direction of change, consistency in the rhythm of change, and consistency in the structure of change.
[0051] Among them, the consistency of the direction of change is used to describe whether the direction of change of the working condition signal is the same in adjacent time slices. When the working condition signal of the current operation process and the historical operation process in the same time slice shows the same rising, falling or holding state, the direction of change is considered to be consistent.
[0052] The consistency of the change rhythm is used to describe the relative relationship of the rate of change of the working condition signal in adjacent time slices. When the current operation process and the historical operation process have the same rate of change in the time dimension, the change rhythm is considered to be consistent.
[0053] The consistency of the change structure is used to describe the overall evolution of the working condition signal over multiple consecutive time slices. When the current operation process and the historical operation process exhibit the same monotonic change, stage change, or periodic change structure in the time series, the change structure is considered to be consistent.
[0054] Extract historical action segments that are consistent with the operation control interface from the candidate historical operation segment set, and organize them in chronological order to form a historical action prototype set.
[0055] The historical action prototype set is compressed and encoded to map multiple historical action fragments into a unified expression form, forming an action prior index. The correspondence between the action prior index and the job status token set is maintained. Through compression and encoding, the action prior index can serve as a structured expression of historical job experience, describing the job action sequence pattern that may be adopted under specific job status conditions.
[0056] To further explain, compression encoding merges action patterns that recur in time structure from the historical action prototype set, categorizes action segments that are similar in action trends, and assigns a unified identifier to each type of action pattern, so that multiple historical action segments can be expressed in a compact form as a finite number of action structure patterns.
[0057] S3. Based on the set of job status tokens and the prior index of actions, generate a sequence of job actions that can be directly issued and executed through a conditional diffusion-type action generation model.
[0058] S3.1. After obtaining the set of operation status tokens and the action prior index, construct the action generation domain in the action representation space that is consistent with the agricultural machinery operation control interface.
[0059] The motion representation space is defined according to the control command format that can be issued in the agricultural machinery operation control interface. It is used to describe the continuous change of the motion of the walking system and the motion of the working parts in the time dimension. The motion of the walking system includes steering commands and drive commands, and the motion of the working parts includes control commands such as header height, boom posture, spray flow rate or valve opening.
[0060] The motion representation space is organized in the form of a time sequence, so that each time slice corresponds to a complete set of walking system motions and working component motions, ensuring that the motion sequence can be directly mapped to the agricultural machinery operation control interface in chronological order.
[0061] Based on the action representation space, according to the allowable value range and change rate limit of various control commands in the agricultural machinery operation control interface, the variable range of each action dimension in the time dimension is determined, and the action generation domain is constructed within the variable range.
[0062] To further explain, the action generation domain is used to limit the feasible area of the action sequence in terms of amplitude range, direction of change and rhythm of change, so that the subsequently generated action trajectory is physically executable.
[0063] S3.2. Generate multiple sets of initial motion trajectories consistent with the motion representation space within the motion generation domain. These initial motion trajectories are obtained through random sampling within a variable range defined by the motion generation domain, ensuring that each set of initial motion trajectories meets the format requirements of the motion representation space in terms of temporal structure. By generating multiple sets of initial motion trajectories, the subsequent progressive denoising generation process unfolds along different initial paths, providing diverse candidate evolutionary directions for motion generation under complex operational conditions.
[0064] S3.3. After obtaining multiple sets of initial motion trajectories, perform progressive denoising generation processing on the multiple sets of initial motion trajectories. The progressive denoising generation processing continuously corrects the initial motion trajectories in a way that evolves from a high uncertainty state to a low uncertainty state, so that the motion trajectories gradually converge into a structurally stable motion sequence in the time dimension.
[0065] During the stepwise denoising generation process, the job state token set applies job state constraints to the denoising process. The job state constraints are determined by the discrete state arrangement order of each time slice in the job state token set, which is used to limit the rhythm of the change of the action trajectory in the time structure, so that the change trend of the generated action between adjacent time slices is consistent with the job state change reflected by the job state token set.
[0066] At the same time, the action prior index applies action structure constraints to the denoising process; the action structure constraints are determined by the historical action prototype structure recorded in the action prior index, which is used to limit the evolution of the action trajectory in the overall trend, so that the generated action is consistent with the action structure pattern that has been verified in the historical operation process in terms of time series morphology.
[0067] By combining the constraints of job state and action structure, the stepwise denoising generation process is simultaneously restricted at both the temporal rhythm and structural form levels, thereby forming multiple sets of candidate job action sequences constrained by both the job state token set and the action prior index.
[0068] To further explain, the stepwise denoising generation process can be represented as: ; in, Indicates the first The action trajectory during the next iteration This represents the motion trajectory after noise reduction correction. Represents the set of job status tokens. Indicates the prior index of the action. This indicates a progressive denoising generation operation, used to correct motion trajectories under the combined effect of job state constraints and motion structure constraints.
[0069] S3.4. After obtaining multiple sets of candidate job action sequences, perform control interface consistency verification and boundary trimming processing on the candidate job action sequences.
[0070] The control interface consistency check examines each control instruction in the action sequence on a time-slice basis to see if it meets the requirements of the agricultural machinery operation control interface for instruction format, sampling period and time continuity, and eliminates action sequences that do not meet the interface specifications.
[0071] Boundary clipping processing corrects or truncates action values in candidate job action sequences that exceed the physical execution range based on the predefined action value range in the action generation domain, so that the action amplitude and rate of change both fall within the physical range that the agricultural machinery can execute, thus obtaining a set of candidate action sequences that can be executed by the interface.
[0072] The predefined range of action values is determined by the hardware safety thresholds of various control commands in the agricultural machinery operation control interface, the mechanical travel limits of the actuator, and the allowable range of the operation process specifications. It is obtained by statistically summarizing the effective action ranges in the agricultural machinery specification parameters, actuator calibration data, and historical operation evidence units, so that the generated action is always within the executable and safe physical range in terms of amplitude and rate of change.
[0073] The physical execution range is determined by the maximum and minimum allowable values of each control command in the agricultural machinery operation control interface, as well as the variation limits between adjacent time slices.
[0074] S3.5. Perform timing coordination processing on the set of executable candidate action sequences for the interface to coordinate the actions of the walking system and the actions of the working components. The timing coordination processing adjusts the timing and rhythm of the changes in the walking system actions and the changes in the working components within the same time slice, so that the changes in steering, drive and the changes in the header height, the spray boom attitude and the spray flow rate are coordinated in the time dimension.
[0075] Specifically, by limiting the overlap between the time slices of changes in the walking system's movements and the time slices of rapid changes in the movements of the working components, the posture change process of the walking system and the key movement change process of the working components are staggered or synchronized on the time axis, thereby avoiding movement conflicts and structural abrupt changes.
[0076] It should be noted that through time-series coordination processing, the candidate job action sequences form a smooth and continuous collaborative evolution process in the time dimension, ultimately forming multiple sets of candidate job action sequences that can be directly issued and executed. This transforms job control from single-point control value output to continuous action segment output, improving the continuity and stability of control results in the time dimension.
[0077] S4. Perform conditional visual evolution prediction on the sequence of work actions, generate corresponding visual results for future work, and determine the target execution sequence.
[0078] S4.1. After obtaining multiple sets of candidate job action sequences that can be directly issued and executed, the job visual prediction unit is called, and a short temporal visual prediction start frame group consistent with the current job scene is generated to form the prediction start visual segment.
[0079] To further explain, the predicted initial visual segment consists of a multi-view temporal frame group corresponding to the current moment, which is used to describe the real visual state of the agricultural machinery operation area at the start of the prediction, so that the subsequent prediction process has the same initial conditions as the real operation scenario.
[0080] The prediction of the initial visual segment and the multi-view temporal frame group are consistent in terms of image structure, spatial geometric relationship and temporal semantics, so that the prediction of the initial visual segment can directly inherit the visual representation results of the current working state, thereby ensuring the continuity of the visual evolution prediction process.
[0081] S4.2. After obtaining the predicted initial visual segment, the predicted initial visual segment is associated with multiple sets of candidate task action sequences, so that each set of candidate task action sequences has a one-to-one correspondence with the predicted initial visual segment in the time dimension, and each set of candidate task action sequences corresponds to an independent visual evolution path, forming multiple sets of future task visual results.
[0082] To further explain, conditional visual evolution prediction uses candidate task action sequences as time-driven conditions to cause spatial structure and texture changes in the prediction start visual segment over consecutive time slices, thereby obtaining future task visual results. Each set of future task visual results reflects the possible task visual state after executing the corresponding task action sequence.
[0083] After obtaining multiple sets of visual results for future tasks, the continuous coverage features, inter-row structure preservation features, and cutting boundary smoothness features are extracted from each set of visual results for future tasks to form multiple sets of visual discrimination features.
[0084] Among them, the coverage continuity feature is obtained by statistically analyzing the changes in the number of connected components, area changes, and spatial continuity of the operation coverage area in adjacent time slices, and is used to describe the integrity and stability of the operation coverage area in the time dimension.
[0085] The row structure retention characteristic is obtained by statistically analyzing the changes in direction, spacing, and arrangement of crop row centerlines or boundary lines in continuous time slices. It is used to describe the degree of geometric consistency of crop row structure during operation.
[0086] The smoothness feature of the cutting boundary is obtained by statistically analyzing the curvature distribution changes and abrupt changes of the operation boundary curve in continuous time slices, and is used to describe the smoothness of the operation boundary in the time series.
[0087] S4.3. Perform visual consistency adjudication on multiple sets of visual discriminant features. Visual consistency adjudication is used to determine which set of visual discriminant features is most consistent with the current operation objective in terms of structural stability, continuity and boundary smoothness. Through visual consistency adjudication, operation control is no longer based solely on numerical changes in actions, but rather on the interpretability of future visual results.
[0088] By using visual consistency adjudication, a target execution sequence that is consistent with the task objective is determined, so that the target execution sequence meets the requirements of task continuity, structure preservation and boundary smoothness in the visual performance of future tasks, thereby realizing the control logic of "first seeing the task result that will occur visually, and then deciding which task action sequence to execute".
[0089] S5. The target execution sequence is segmented and distributed to the agricultural machinery execution agency, and a traceable chain of operational evidence is constructed simultaneously.
[0090] S5.1. Divide the target execution sequence according to the vehicle control cycle to form a segmented execution table. The target execution sequence represents the relationship between the movement of the walking system and the movement of the working parts in a continuous time slice in the form of a time series. The vehicle control cycle is specified by the agricultural machinery operation control interface and is used to limit the minimum time interval for issuing control commands.
[0091] By dividing the target execution sequence into multiple consecutive action segments according to the vehicle control cycle, each action segment corresponds to a set of control instructions that can be issued within a control cycle, forming a segmented execution table.
[0092] To further explain, the segmented execution table is used to describe the segmented structural relationship of the target execution sequence in the time dimension, so that subsequent operations can be performed segment by segment in a predetermined time order.
[0093] S5.2. Based on the segmented execution table, the corresponding action segments are sequentially sent to the walking system controller and the working component controller, and the sending time is recorded to form an execution record set.
[0094] To further explain, according to the time sequence of the segmented execution table, each action segment is sequentially sent to the walking system controller and the working component controller, so that the walking system actions and the working component actions are executed synchronously within the same time slice.
[0095] When issuing each action segment, the corresponding issuance timestamp, action segment identifier, and control interface response status are recorded to form an execution record.
[0096] By recording the process of issuing all action segments, an execution record set is formed. The execution record set is used to describe the actual execution of the target execution sequence in the real operation process.
[0097] S5.3. Bind and encapsulate the execution record set with the job status token set, action fragments, and corresponding multi-view time frame groups of the job image sequence to form a job evidence unit set.
[0098] To further explain, within each control cycle, there is a one-to-one correspondence between the action segments recorded in the execution record set and the job status tokens and multi-view time sequence frame groups in the job image sequence within the corresponding time slice.
[0099] By aligning the timestamps, each record in the execution record set is jointly encapsulated with the job status token, action fragment, and multi-view time sequence frame group in the job image sequence within the same time slice to form a job evidence unit. Each job evidence unit describes a complete "visual state - action execution - visual result" relationship, making the job process reproducible in the time dimension.
[0100] S5.4. The set of work evidence units is remotely transmitted through the Internet of Things link, so that the set of work evidence units can be uniformly stored and managed in cloud data services.
[0101] In cloud data services, the set of work evidence units is archived according to work type, work time period and work condition label, so that the work evidence units in the same work process form a continuous structure in the time dimension, and the set of work evidence units forms a continuous chain of association on the time axis, thereby forming a traceable work evidence chain.
[0102] It should be noted that the traceable operational evidence chain is used to describe the complete correspondence between "visual state - operational action - operational result" in the process of agricultural machinery operation, so that the operational state, executed actions and operational visual results in any time slice can be traced back and reproduced.
[0103] This embodiment also provides a visual intelligence-based adaptive control system for agricultural machinery operations, comprising: a visual modeling module, which continuously perceives the agricultural machinery operation process through a multi-view visual acquisition unit, and performs temporal consistency modeling and geometric correlation analysis on the operation image sequence to generate an operation status token set; a priori construction module, which temporally couples the operation status token set with the agricultural machinery operating condition signal and matches it in a historical operation segment library to generate an action priori index corresponding to the current operation status; an action generation module, which generates an operation action sequence that can be directly issued and executed based on the operation status token set and the action priori index through a conditional diffusion action generation model; a visual inference module, which performs conditional visual evolution prediction on the operation action sequence, generates the corresponding future operation visual result, and determines the target execution sequence; and an operation execution module, which issues the target execution sequence in segments to the agricultural machinery execution mechanism and simultaneously constructs a traceable operation evidence chain.
[0104] This embodiment also provides a computer device applicable to the adaptive control method for agricultural machinery operations based on visual intelligence, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to realize the adaptive control method for agricultural machinery operations based on visual intelligence as proposed in the above embodiment.
[0105] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.
[0106] This embodiment also provides a storage medium storing a computer program. When executed by a processor, the program implements the adaptive control method for agricultural machinery operation based on visual intelligence as proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.
[0107] In summary, this invention generates a set of operation status tokens through temporal consistency modeling and geometric correlation analysis, enabling the operation status to be stably expressed using discrete temporal symbols, thus improving the distinguishability of states such as inter-row offset, coverage continuity, and operation boundary changes. The set of operation status tokens is temporally coupled with agricultural machinery operating condition signals and matched with a historical operation segment library to obtain a priori index of actions, giving candidate actions empirical constraints and operating condition consistency. The conditional diffusion-type action generation model outputs a sequence of operation actions that can be directly issued and executed under the combined effect of operation status constraints and action structure constraints, transforming control from single-point setting to executable action segments. Conditional visual evolution prediction performs a visual preview of future operation results for candidate operation action sequences and selects the target execution sequence through visual consistency adjudication, allowing action selection to directly constrain operation quality objectives such as coverage continuity, inter-row structure preservation, and cutting boundary smoothness.
[0108] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.
Claims
1. A visual intelligence-based adaptive control method for agricultural machinery operations, characterized by: include, The agricultural machinery operation process is continuously perceived through a multi-view visual acquisition unit, and the temporal consistency modeling and geometric correlation analysis of the operation image sequence are performed to generate an operation status token set. The set of operation status tokens is coupled with agricultural machinery operating condition signals in a time sequence, and then matched with the historical operation segment library to generate an action prior index corresponding to the current operation status. Based on the set of job status tokens and the prior index of actions, a sequence of job actions that can be directly issued and executed is generated through a conditional diffusion-type action generation model; Conditional visual evolution prediction is performed on the sequence of work actions to generate corresponding visual results of future work and determine the target execution sequence; The target execution sequence is segmented and distributed to agricultural machinery execution agencies, and a traceable chain of operational evidence is constructed simultaneously.
2. The adaptive control method for agricultural machinery operation based on visual intelligence as described in claim 1, characterized in that: The sequence of images of the operation is obtained synchronously by the front-end camera of the fuselage, the near-end camera of the operation component, and the side camera within the same operation time period; The operational image sequence includes crop canopy images, interrow structure images, surface texture images, and images of the contact boundary between the operational component and the crop.
3. The adaptive control method for agricultural machinery operation based on visual intelligence as described in claim 1, characterized in that: The specific steps for generating the job status token set are as follows: Perform timestamp alignment and multi-view frame association on the job image sequence to form a multi-view time sequence frame group; Geometric consistency modeling is performed on multi-view temporal frame groups to generate a sequence of relative spatial relationships between agricultural machinery and crops; Temporal features reflecting inter-row offset, coverage continuity and job boundary changes are extracted from the relative spatial relationship sequence and discretized to obtain a set of job status tokens.
4. The adaptive control method for agricultural machinery operation based on visual intelligence as described in claim 1, characterized in that: The specific steps for time-coupling the operation status token set with the agricultural machinery operating condition signal are as follows. Acquire agricultural machinery operating condition signals synchronized with the operation image sequence and complete timestamp unification to obtain a set of operating condition time sequence signals; the agricultural machinery operating condition signals include vehicle speed, operating component rotation speed and implement posture; Align the set of work status tokens according to the token time slice with the set of work condition timing signals according to the sampling time slice, and establish a corresponding mapping relationship to obtain the token work condition correspondence table; Based on the token-operational-condition mapping table, the operation status tokens and operation condition signals of adjacent time slices are combined to obtain a joint segment of status and operation conditions used for matching and retrieval.
5. The adaptive control method for agricultural machinery operation based on visual intelligence as described in claim 1, characterized in that: The specific steps for generating the action prior index corresponding to the current job state are as follows: Based on the state and operating conditions, a retrieval key and a fragment fingerprint are generated to obtain the current fragment fingerprint set; Retrieve historical fragment fingerprints that form the same structure as the current fragment fingerprint set from the historical task fragment library, and perform similarity matching to obtain a set of candidate historical task fragments; Extract historical action segments that are consistent with the operation control interface from the candidate historical operation segment set, and organize them in chronological order to obtain a historical action prototype set; The historical action prototype set is compressed and encoded into an action prior index, and the correspondence with the job status token set is retained to obtain the action prior index corresponding to the current job status.
6. The adaptive control method for agricultural machinery operation based on visual intelligence as described in claim 5, characterized in that: The historical operation fragment library is obtained by archiving a set of operation evidence units formed during multiple operations according to operation type and working condition tags.
7. The adaptive control method for agricultural machinery operation based on visual intelligence as described in claim 1, characterized in that: The specific steps for generating a sequence of job actions that can be directly issued and executed using a conditional diffusion-based action generation model are as follows. A motion generation domain is constructed within the motion representation space that is consistent with the agricultural machinery operation control interface, and multiple sets of initial motion trajectories consistent with the motion representation space are generated. The process involves progressively denoising multiple initial action trajectories, imposing job state constraints on the denoising process using the job state token set and action structure constraints on the denoising process using the action prior index, thus forming multiple candidate job action sequences constrained by both state and prior constraints. Perform control interface consistency verification and boundary pruning on the candidate job action sequences to obtain a set of candidate action sequences that can be executed by the interface. The timing coordination of the walking system actions and the actions of the working components is performed on the set of candidate action sequences that can be executed by the interface to obtain multiple sets of candidate operation action sequences that can be directly issued and executed.
8. The adaptive control method for agricultural machinery operation based on visual intelligence as described in claim 1, characterized in that: The specific steps for conditional visual evolution prediction of the task action sequence are as follows. Call the task visual prediction unit and generate a short temporal visual prediction start frame group that is consistent with the current task scene to obtain the prediction start visual segment; The predicted initial visual segments are associated with multiple sets of candidate task action sequences to generate corresponding multiple sets of future task visual results. From the visual results of each group of future operations, we extract the coverage continuity feature, the inter-row structure preservation feature, and the cutting boundary smoothness feature to obtain multiple sets of visual discrimination features; Perform visual consistency adjudication on multiple sets of visual discriminative features to determine the target execution sequence that is consistent with the task objective.
9. The adaptive control method for agricultural machinery operation based on visual intelligence as described in claim 1, characterized in that: The specific steps for constructing a traceable chain of evidence for the operation are as follows. The target execution sequence is divided according to the vehicle control cycle to obtain a segmented execution table; Based on the segmented execution table, corresponding action segments are sequentially sent to the walking system controller and the working component controller, and the sending time is recorded to obtain an execution record set; The execution record set is bound and encapsulated with the job status token set, action fragments, and corresponding multi-view time frame groups of the job image sequence to generate a job evidence unit set; Through the Internet of Things (IoT) link, the collection of work evidence units is uploaded to the cloud data service and archived by work batch to form a traceable work evidence chain.
10. A vision-based adaptive control system for agricultural machinery operations, based on the vision-based adaptive control method for agricultural machinery operations according to any one of claims 1 to 9, characterized in that: include, The visual modeling module continuously perceives the agricultural machinery operation process through a multi-view visual acquisition unit, performs temporal consistency modeling and geometric correlation analysis on the operation image sequence, and generates a set of operation status tokens. The prior construction module couples the set of work status tokens with the agricultural machinery working condition signals in a time sequence and matches them in the historical work segment library to generate an action prior index corresponding to the current work status. The action generation module generates a sequence of job actions that can be directly issued and executed based on the job status token set and the action prior index through a conditional diffusion action generation model. The visual inference module performs conditional visual evolution prediction on the sequence of work actions, generates corresponding visual results of future work, and determines the target execution sequence. The operation execution module distributes the target execution sequence in segments to the agricultural machinery execution agency and simultaneously builds a traceable chain of operation evidence.