Agricultural machinery operation track completion and path reconstruction method, system, equipment and medium

Abstract

The invention discloses an agricultural machine operation track completion and path reconstruction method, system and device and a medium, and belongs to the technical field of intelligent agriculture, the method comprises the steps that multi-source data is collected and preprocessed, and the multi-source data comprises GNSS positioning data, IMU data and agricultural implement image data of an agricultural machine; on the basis of multi-source data, through speed features, IMU signals and image features, agricultural machine operation state identification and confidence scoring are carried out on the original operation track; according to the operation state of the agricultural machine, an effective operation track in the original operation track is reserved, and missing section length identification is carried out on the effective operation track; and on the basis of the confidence of the missing segments and a length selection interpolation complementation mode or a path reconstruction mode, sequentially filling the missing segments of the effective operation track. According to the method, high completeness and high reliability of the agricultural machinery operation track are ensured by identifying the effective operation state and performing differential complementation on the track missing section.

Description

Technical Field

[0001] The present invention belongs to the technical field of smart agriculture, and particularly relates to a method, system, device and medium for completing agricultural machinery operation trajectories and reconstructing paths. Background Art

[0002] During the process of agricultural mechanization operations, accurately recording the operation trajectories of agricultural machinery is of great significance for farmland management, operation evaluation, and optimization of resource allocation. Currently, intelligent terminals are commonly used to collect GNSS data to obtain operation trajectories, but this method faces multiple challenges in the actual field environment, mainly including: (1) Risk of operation monitoring interruption: The monitoring application may be abnormally terminated due to system resource scheduling, misoperation, or background mechanisms, resulting in incomplete trajectory collection; (2) Unstable GNSS signals: In the farmland environment, there are often trees, terrain undulations, or building obstructions, which easily cause GNSS data loss or a decrease in positioning accuracy; (3) Discontinuous collected data: Limited by terminal storage, caching, and network, etc., the time interval of sampling points may be uneven, resulting in discontinuous trajectories; (4) Lack of the ability to distinguish operation states: Existing methods usually only complete trajectories based on position data, and it is difficult to effectively identify whether the agricultural machinery is in an effective operation state, resulting in problems such as miscompletion of non-operation trajectories and omission of operation segments.

[0003] Traditional trajectory completion methods usually ignore the differences in operation states and easily misjudge non-operation segments as areas to be completed, resulting in data pollution; In summary, there is an urgent need for a trajectory completion and path reconstruction method that can accurately identify effective operation states, have the ability to determine the types of trajectory missing, and a differential completion process to achieve high-integrity and high-reliability reconstruction and completion of agricultural machinery operation trajectories. Summary of the Invention

[0004] In view of the deficiencies in the prior art, the present invention provides a method, system, device and medium for completing agricultural machinery operation trajectories and reconstructing paths, which can integrate multi-source perception information such as GNSS and IMU, accurately identify effective operation states, have the ability to determine the types of trajectory missing, and a differential completion process to achieve high-integrity and high-reliability reconstruction and completion of agricultural machinery operation trajectories.

[0005] The present invention provides the following technical solutions: In the first aspect, a method for completing agricultural machinery operation trajectories and reconstructing paths is provided, including the following steps: S1: Collect multi-source data and preprocess it. The multi-source data includes GNSS positioning data, IMU data, and implement image data of the agricultural machinery; S2: Based on multi-source data, identify the agricultural machinery operation status and perform confidence scoring on the original operation trajectory formed by GNSS positioning data through speed characteristics, IMU signals, and image features; S3: According to the agricultural machinery operation status, retain the valid operation trajectory in the original operation trajectory, and identify the length of the missing segment of the valid operation trajectory; S4: Based on the confidence level and length of the missing segment, select the interpolation completion method or the path reconstruction method to fill the missing segments of the valid operation trajectory in sequence. Among them, when selecting path reconstruction to fill the missing segment m for filling, the missing segment m is determined as an unreasonable missing trajectory through double judgment of operation time deviation and IMU spectrum characteristics.

[0006] Optionally, the GNSS positioning data is spatial position coordinates including sampling timestamps; the IMU data includes: the three-axis acceleration of the agricultural machinery, the three-axis angular velocity of the agricultural machinery, and the heading angle, all including sampling timestamps; the preprocessing includes: timestamp alignment of multi-source data, filtering and denoising, and abnormal data elimination.

[0007] Optionally, step S2 specifically includes: S21: Based on the trajectory point sequence of the original operation trajectory , generate a speed sequence regarding the speed between adjacent trajectory points, and filter the speed sequence. If the speed within consecutive N set time windows after filtering is within the set reasonable speed range, the speed characteristic score of the current N set time window is 1, otherwise it is 0; S22: Based on the continuous IMU time series formed by IMU data, according to the set time window size, obtain the IMU signals within each time window, including the acceleration variance of the agricultural machinery along the driving direction, the acceleration peak , and the average angular velocity around the vertical axis. If and are satisfied, the acceleration vibration characteristic score of the current time window is 1, otherwise it is 0; if is satisfied, the IMU yaw angle characteristic score of the current time window is 1, otherwise it is 0, where , and are respectively , and corresponding set thresholds; S23: Fuse the speed feature score, acceleration vibration feature score, and IMU yaw angle feature score within the current time window into the confidence score of the current time window according to the set weights. α ; S24: If the confidence score of the current time window α is within the set high-confidence interval, the agricultural machinery operation state of the current time window is an effective working state with high confidence; If the confidence score of the current time window α is within the set low-confidence interval, further extract the image features of the implement image data and identify the implement opening state. If the output is that the implement is open, the agricultural machinery operation state of the current time window is an effective working state with low confidence. If the output is that the implement is closed, the agricultural machinery operation state of the current time window is a non-effective working state; If the confidence score of the current time window α is lower than the set low-confidence interval, the agricultural machinery operation state of the current time window is a non-effective working state.

[0008] Optionally, step S3 is specifically as follows: S31: Based on the trajectory point sequence of the original operation trajectory and the positioning data sampling timestamps, obtain the average sampling interval of the original operation trajectory , and set the first missing threshold and the second missing threshold according to the average sampling interval ;

[0009]

[0010] Among them, is the total number of trajectory points of the original operation trajectory, is the sampling interval between any two trajectory points, and are the sampling timestamps of the th and th trajectory points respectively; S32: Retain the operation trajectories in the effective working state as effective operation trajectories, and compare the sampling intervals between any two trajectory points in the effective operation trajectories. If , there is a short missing segment between the current two trajectory points. If , there is a long missing segment between the current two trajectory points.

[0011] Optionally, step S4 specifically includes: S41: If the current missing segment is of low confidence, no filling process is performed; S42: If the current missing segment is a high-confidence and large missing segment, then perform job time deviation judgment and IMU spectrum feature judgment on the current large missing segment in sequence. If both pass, it is a reasonable missing trajectory, and the interpolation completion method is selected to complete the current missing segment; otherwise, it is an unreasonable missing trajectory, and it is filled by path reconstruction; S43: If the current missing segment is a high-confidence and small missing segment, then select the interpolation completion method to complete the current missing segment; Among them, performing job time deviation judgment on the current large missing segment specifically includes: Obtain the average speed of multiple uniform straight-line segments in the effective job trajectory according to the GNSS positioning data corresponding to the effective trajectory curve and, according to the distance between the start point and the end point of the large missing segment obtain the expected job time of the large missing segment ; obtain the actual interval time between the start point and the end point of the large missing segment ; judge whether the difference between the expected job time and the actual interval time of the large missing segment meets the set threshold. If it meets, the job time deviation judgment passes; Performing IMU spectrum feature judgment on the current large missing segment specifically includes: Obtain the acceleration of the agricultural machinery along the driving direction of the large missing segment , perform FFT transformation on it to the frequency domain to obtain the frequency-amplitude distribution curve; find the main peak frequency and amplitude of the frequency-amplitude distribution curve. If both meet the set threshold interval, the IMU spectrum feature judgment passes.

[0012] Optionally, the interpolation completion method is used to complete the missing segment. Specifically: According to the position coordinates of the start point and the end point of the missing segment and , and the heading angles corresponding to the start point and the end point and , respectively obtain the positions of two control points and ;

[0013]

[0014] Among them, is the extended distance of the set control point; ​Use a cubic Bézier curve as the basic form of the interpolation trajectory. Based on the start point, end point, and the positions of two control points of the missing segment, generate a completed trajectory curve, and insert the completed trajectory curve into the valid operation trajectory in the form of trajectory points.

[0015] Optionally, complete the missing segment by means of path reconstruction, specifically: E1: Based on the start point position of the missing segment , the IMU data of the missing segment, and the sampling interval of the IMU data, obtain the positions of each sampling point of the missing segment, and generate a first path curve ;

[0016]

[0017]

[0018]

[0019] Among them, , , and are respectively the speed, position coordinates, and heading angle of the p rd sampling point, P is the total number of sampling points of the missing segment; is the sampling interval, the initial speed is the average speed within the set time before the missing segment, and the initial heading angle is the heading angle corresponding to the start time of the missing segment; is the sampling timestamp corresponding to the agricultural machinery combined acceleration; , and are respectively the speed and position coordinates of the rd sampling point; E2: Calculate the change in heading angle between the start point and the end point of the missing segment; if the change exceeds 150°, then use the set turning path template to generate several second candidate path curves , if the change is less than 150°, then use the set forward path template to generate several second candidate path curves ;

[0020]

[0021]

[0022]

[0023]

[0024]

[0025] Among them, is the length of the second candidate path curve, , is the average speed of multiple uniform linear segments in the effective operation trajectory, T is the missing time; is the course angle in the set forward path template, is the course angle relative to the initial course angle offset angle; D 1 and D 2 are respectively the forward path curve and the U-turn path curve of the second candidate path curve , and are respectively the set forward distance and the U-turn driving distance of the second candidate path curve ; and are respectively the set forward course angle and the course angle after U-turn of the second candidate path curve ; is the set of the set forward course angle and the course angle after U-turn of the second candidate path curve ; E3: Taking the minimum Euclidean distance integration error as the matching index, select the second candidate path curve with the smallest difference from the first path curve as the optimal second path curve ; E4: Fuse and reconstruct the first path curve and the second path curve to generate the reconstructed path of the missing segment; specifically: E41: Obtain the position coordinates of four control points , , and : and are the starting and ending positions of the missing segment; ; , among which, and are respectively the points at the 30% and 60% positions of the first path curve, and are respectively the points at the 30% and 60% positions of the second path curve; E42: Generate a reconstructed path using the Bezier curve formula and insert the reconstructed path into the effective operation trajectory in the form of trajectory points;

[0026] wherein, is a parameter of the Bezier curve, representing the position ratio of the curve between the control points.

[0027] In a second aspect, a farm machinery operation trajectory completion and path reconstruction system is provided, including: An acquisition module: acquire multi-source data and preprocess it. The multi-source data includes GNSS positioning data, IMU data, and farm tool image data of the farm machinery; An operation status recognition module: based on the multi-source data, identify the operation status of the farm machinery and perform a confidence score on the original operation trajectory formed based on the GNSS positioning data through speed characteristics, IMU signals, and image characteristics; A missing segment recognition module: according to the operation status of the farm machinery, retain the effective operation trajectory in the original operation trajectory and identify the length of the missing segment in the effective operation trajectory; A differential filling module: based on the confidence and length of the missing segment, select an interpolation completion method or a path reconstruction method to fill the missing segments of the effective operation trajectory in sequence. Among them, when selecting path reconstruction to fill the missing segment m the missing segment m is determined as an unreasonable missing trajectory through the dual determination of operation time deviation and IMU spectrum characteristics.

[0028] In a third aspect, a computer device is provided, including a processor and a memory; wherein, when the processor executes the computer program stored in the memory, the steps of the farm machinery operation trajectory completion and path reconstruction method described in any item of the first aspect are implemented.

[0029] In a fourth aspect, a computer-readable storage medium is provided for storing a computer program; when the computer program is executed by a processor, the steps of the farm machinery operation trajectory completion and path reconstruction method described in any item of the first aspect are implemented.

[0030] Compared with the prior art, the beneficial effects of the present invention are: (1) By introducing an agricultural machinery status recognition mechanism, a dual-criterion trajectory rationality analysis and an interpolation-reconstruction combination strategy, this application solves the problem of incomplete trajectories caused by discontinuous and severely missing trajectory data and unclear operation status in agricultural operations. By integrating GNSS speed characteristics, IMU dynamic characteristics, and image perception, this application accurately eliminates trajectory data in non-effective working states, improving the purity of the trajectory. In addition, for large segments of missing trajectories, this application proposes a "dual-criterion" judgment mechanism that combines operation time deviation and IMU spectrum characteristics, which can accurately judge whether the missing segment can be filled. For fillable segments, interpolation filling is used for filling, and for non-fillable segments, path reconstruction is used for path filling, enhancing the scientificity and robustness of path filling and completion.

[0031] (2) For small missing segments, this application uses a cubic Bezier curve interpolation method based on heading control points to achieve path geometric continuity and direction consistency. For non-fillable segments, a template path and IMU path fusion reconstruction strategy is introduced to support multi-path generation and evaluation, adapting to diverse missing situations. In the process of path reconstruction, this invention integrates the IMU heading estimation results, on the one hand, to improve the accuracy of path direction selection, and on the other hand, to achieve a high degree of matching between the path geometric structure and the actual movement behavior of agricultural machinery, enhancing the physical consistency of the reconstructed path. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 is a flowchart of the method for completing agricultural machinery operation trajectories and reconstructing paths of this invention; Figure 2 is a flowchart of the dual determination of missing segments by operation time deviation and IMU spectrum characteristics of this invention. DETAILED DESCRIPTION OF THE INVENTION

[0033] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and should not be used to limit the protection scope of the present invention.

[0034] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and do not necessarily need to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present invention described here can be implemented in an order other than those illustrated or described here. In addition, the term "including" and any of its variations are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or are inherent to these processes, methods, products, or devices.

[0035] Example 1 As shown Figure 1 in the figure, a method for completing the operation trajectory and reconstructing the path of agricultural machinery is provided, including the following steps: S1: Collect multi-source data and preprocess it. The multi-source data includes GNSS positioning data, IMU data, and agricultural implement image data of the agricultural machinery.

[0036] The GNSS positioning data is spatial position coordinates including sampling timestamps; the GNSS positioning data and IMU data can be collected in real time through multi-class sensors built-in or externally connected to an intelligent terminal (such as a smartphone) carried by the agricultural machinery; the GNSS positioning data optionally supports high-precision differential positioning or RTK solution; each piece of GNSS data synchronously records a timestamp.

[0037] The IMU data includes: the three-axis acceleration of the agricultural machinery, the three-axis angular velocity of the agricultural machinery, and the heading angle, all including sampling timestamps, where x the direction is the driving direction of the agricultural machinery, and the attitude angle can be estimated by fusing the data provided by the three-axis accelerometer and gyroscope built into the smartphone through the Attitude and Heading Reference System (AHRS) algorithm to obtain continuous three-axis attitude angle information, including the heading angle, pitch angle, and roll angle. Among them, the heading angle: represents the angle between the current direction and true north; the pitch angle: represents the front-back tilt angle of the agricultural machinery; the roll angle: represents the left-right tilt degree. The three-axis acceleration and three-axis angular velocity of the Inertial Measurement Unit (IMU) data are usually collected simultaneously. For specific reference to the prior art, the sampling frequency is usually set to 50 Hz - 100 Hz to ensure capturing the dynamic behavior changes of the agricultural machinery.

[0038] The agricultural implement image data is collected through the smartphone camera or an externally connected industrial camera to obtain the video or still images of the agricultural implement during the operation process.

[0039] To ensure the accuracy and temporal consistency of subsequent data analysis, the original multi-source data needs to go through the following preprocessing steps: S11: Timestamp alignment.

[0040] Based on the GNSS sampling frequency, unify the time axes of the IMU and image data to construct a unified temporal data sequence; if there are sampling rate differences among the various data, linear interpolation or spline interpolation is used for data alignment.

[0041] S12: Filtering and denoising processing.

[0042] Apply sliding mean filtering to the acceleration and angular velocity data to remove high-frequency jitter; remove noise points (such as sudden change values, obviously non-physical change points).

[0043] S13: Abnormal data elimination.

[0044] Delete the pseudo points where the GNSS speed is 0 but the position mutates; Eliminate the invalid data segments with abnormal timestamps and missing data.

[0045] S2: Based on multi-source data, identify the agricultural machinery operation status and confidence score for the original operation trajectory formed by GNSS positioning data through speed features, IMU signals, and image features.

[0046] The original operation trajectory is the GNSS sampling trajectory points (after preprocessing). Although the agricultural machinery has continuous GNSS trajectory records during operation, its behavior state may change frequently. For example: turning around between operation rows; the operation equipment is not turned on but still moving; waiting in a stationary state for refueling / feeding; the agricultural machinery is moving but not operating (empty vehicle transfer). The above states are manifested as continuous moving trajectories in the GNSS trajectory, but they do not represent actual operation behaviors. If not identified and filtered, it is easy to lead to misjudgments of key indicators such as operation area, coverage rate, and omission rate, thereby affecting the accuracy of trajectory completion and data analysis.

[0047] To improve the expression ability of trajectory data for real operation behaviors, identify the agricultural machinery operation status and confidence score for the original operation trajectory formed by GNSS positioning data through speed features, IMU signals, and image features, so as to use the trajectory of the effective working state of the agricultural machinery as the pre-judgment basis for trajectory missing identification and filling, significantly improving the effectiveness and filling quality of trajectory data; It should be noted that the operation status of the agricultural machinery in this application includes: effective working state and non-effective working state. In this application, the effective working state refers to the agricultural machinery being in effective operation processes such as sowing, spraying, fertilizing, etc., and the farming tools are in the working-on state; the non-effective working state includes non-operation state and stop state. The non-operation state means the agricultural machinery is in a moving but non-operating state, such as moving between rows, turning around or making a U-turn; the stop state indicates that the agricultural machinery is in a stationary state or the equipment stops the effective working state.

[0048] Step S2 specifically includes the following sub-steps: S21: Based on the trajectory point sequence of the original operation trajectory generate a speed sequence regarding the speed between adjacent trajectory points, and filter the speed sequence. If the speeds within consecutive N set time windows after filtering are all within the set reasonable speed range, the speed feature score of the current N set time window is 1, otherwise it is 0.

[0049] Specifically, in step S21, it is determined whether the agricultural machinery is in an effective working state by using speed characteristics. Through the analysis of the speed changes between GNSS trajectory points, the stationary stages and non-operation segments such as abnormal short-distance jitters can be effectively filtered out, providing an initial state framework for subsequent IMU dynamic analysis and image discrimination.

[0050] Based on the continuous sequence of trajectory points collected by the GNSS sensor , the speed between adjacent trajectory points is calculated using the following formula :

[0051] where is the spatial position coordinate of the th trajectory point; is the corresponding timestamp; the speed is the average moving speed of the nd segment, and the unit is usually meters per second (m / s) or kilometers per hour (km / h).

[0052] To improve the stability of speed estimation, the system uses a moving average filter to smooth the speed sequence, and the filter window size is set to 5 trajectory points. Optionally, the reasonable speed range set in this application is (0.2 m / s) and (3 m / s).

[0053] If the speed within consecutive time windows all satisfies: , then the current time period is determined as the "stationary state" or "non-operation state", that is, the non-effective working state, and the speed characteristic score is 0 at this time.

[0054] If the speed within consecutive time windows is stably maintained within the reasonable range (such as ), then the current time period is determined as a suspected effective working state, and the speed characteristic score is 1 at this time.

[0055] If the speed within consecutive time windows all satisfies: , then the current time period may be the "transition state" or "abnormally high moving segment", that is, the speed characteristic score is 0. Among them, both the "transition state" and the "abnormally high moving segment" are non-effective working states. In some other embodiments, according to actual needs, the "stationary state", "non-operation state", "transition state" or "abnormally high moving segment" can be further labeled.

[0056] S22: Based on the IMU continuous time series formed by the IMU data, according to the set time window size, obtain the IMU signal in each time window, including the acceleration variance of the agricultural machinery along the driving direction , peak acceleration , the mean angular velocity around the vertical axis , if satisfied and , then the acceleration vibration feature score of the current time window is 1, otherwise it is 0; if , then the IMU yaw angle feature score in the current time window is 1, otherwise it is 0, where , and They are , and The corresponding set threshold.

[0057] Specifically, step S22 utilizes the acceleration change and heading angle change characteristics of the agricultural machinery in the forward direction, combined with time domain statistics and frequency domain vibration characteristics for analysis, so as to identify whether the agricultural machinery is in an effective operating state.

[0058] Longitudinal acceleration : Indicates the linear acceleration of the agricultural machinery along the travel direction, reflecting the coupling characteristics of the forward rhythm and the ground.

[0059] Yaw angular velocity : Indicates the angular velocity around the vertical axis, reflecting whether the agricultural machinery has a change in direction (turn, reversal, turning).

[0060] Longitudinal acceleration and yaw rate The samples are sampled at a fixed frequency (such as 50~100Hz) and are processed by time alignment and low-pass filtering in the preprocessing stage to form a continuous time series.

[0061] 1. Acceleration variance of agricultural machinery along the direction of travel : It is used to measure the acceleration fluctuation degree of the agricultural machinery in the forward direction during this period, and is solved by the following formula:

[0062] Judgment logic: If < :If the vibration is stable, it may be an operating section. If the vibration is severe, it may be a U-turn or a non-operating section. Threshold Description: is the stability threshold, and the recommended initial value range is 0.02-0.1 (m / s 2 ) 2 , which can be adjusted according to the type of agricultural machinery or determined by fitting the historical trajectory; is the average acceleration in the driving direction of N sampling points.

[0063] (2) Peak acceleration : within the time window is the maximum absolute value, measuring the instantaneous intensity of vibration; the specific formula is:

[0064] Criterion logic: If , there is significant longitudinal acceleration fluctuation in the current time period, which may be caused by the operation of the working implement (such as the vibration of the seeding device, spraying pump); then it is determined as an effective operation vibration section, otherwise it is a weak vibration or static section.

[0065] is the acceleration intensity discrimination threshold, indicating the minimum instantaneous amplitude required to form an effective vibration behavior in the longitudinal acceleration signal. The initial value is 0.8 - 1.5 m / s 2 , and it should be specifically adjusted adaptively through historical sample fitting according to different agricultural machinery models and working tool characteristics.

[0066] (3) Average angular velocity : within the time window calculate the average value of the yaw angular velocity, reflecting the overall rotation trend. The specific formula is:

[0067] Discrimination logic: If , it indicates that there is a continuous turning action and may be in a U-turn / turning state; if , then it is considered that the current direction is stable and in a straight-line operation state. Optionally, the threshold: is , and it can be flexibly adjusted within the interval according to the agricultural machinery structure and turning radius to adapt to different equipment configurations and working scenarios.

[0068] At , the current time period is the "stable operation section", indicating that the agricultural machinery is performing an operation with a significant vibration amplitude and a stable direction; At and the change in the heading angular velocity is not significant ( ), then the current time period is the "non-operation abnormal vibration section", and such a state may be caused by equipment looseness, rough ground or the working implement not being closed; When Regardless of whether the acceleration characteristics change significantly, the system determines that the current section is the "U-turn / turning section", and this state often appears in the inter-row U-turn, turning around or turning operation stages (all are non-effective working states) and needs to be excluded to avoid double coverage.

[0069] S23: Fuse the speed feature score, acceleration vibration feature score, and IMU yaw angle feature score within the current time window into the confidence score of the current time window according to the set weights. α 。

[0070] Specifically, the formula is:

[0071] Where, is the speed feature score, is the acceleration vibration feature score, is the IMU yaw angle feature score, 、 and are respectively 、 and corresponding weights. As an option, is 0.4, and are both 0.3.

[0072] S24: If the confidence score α of the current time window is within the set high-confidence interval, the agricultural machinery operation state of the current time window is an effective working state and high confidence; if the confidence score α of the current time window is within the set low-confidence interval, further extract the image features of the implement image data and identify the implement opening state. If the output is that the implement is open, the agricultural machinery operation state of the current time window is an effective working state and low confidence; if the output is that the implement is closed, the agricultural machinery operation state of the current time window is a non-effective working state; if the confidence score α of the current time window is lower than the set low-confidence interval, the agricultural machinery operation state of the current time window is a non-effective working state.

[0073] As an option, the high-confidence interval is: α > 0.8; the low-confidence interval is 0.5 - 0.8; lower than the set low-confidence interval is lower than 0.5.

[0074] When the agricultural machinery is at a stable speed, with a stable acceleration and no direction deflection, but the operating implement is not turned on (e.g., the sprinkler is not spraying, the seeder is not rotating), it is difficult to accurately identify whether it is an effective operation section by relying solely on GNSS and IMU signals. To improve the integrity and accuracy of state recognition, the present invention introduces image-assisted recognition as an optional enhanced channel for the state recognition system, and determines whether the agricultural implement is in the "on / off" state based on visual perception to assist in confirming "whether it is operating". The extraction of image features from the agricultural implement image data and the recognition of the implement's on state can be carried out after training through existing neural network algorithms. The feature extraction method and the on state recognition method can refer to the prior art.

[0075] S3: According to the operating state of the agricultural machinery, retain the effective operating trajectories in the original operating trajectory and identify the length of the missing segments in the effective operating trajectories.

[0076] Step S3 is specifically as follows: S31: Based on the trajectory point sequence of the original operating trajectory and the positioning data sampling timestamps, obtain the average sampling interval of the original operating trajectory , and set the first missing threshold and the second missing threshold according to the average sampling interval ;

[0077]

[0078] Among them, is the total number of trajectory points of the original operating trajectory, is the sampling interval between any two trajectory points, and are the sampling timestamps of the th and th trajectory points respectively.

[0079] As an option, set the first missing threshold and the second missing threshold to be and respectively.

[0080] S32: Retain the operating trajectories in the effective working state as the effective operating trajectories, and compare the sampling interval between any two trajectory points in the effective operating trajectories. If , then the segment between the current two trajectory points is a short missing segment. If , then the segment between the current two trajectory points is a long missing segment.

[0081] S4: Based on the confidence level and length of the missing segment, select the interpolation completion method or the path reconstruction method to fill the missing segments of the effective operation trajectory in sequence. When selecting path reconstruction to fill the missing segment m for filling, the missing segment m is determined as an unreasonable missing trajectory through double judgment of operation time deviation and IMU spectrum characteristics.

[0082] Step S4 is specifically as follows: S41: If the large missing segment of the current missing segment has a low confidence level, mark the large missing segment as an invalid trajectory and do not perform filling processing; if the current missing segment is a small missing segment and has a low confidence level, do not perform filling processing.

[0083] S42: If the current missing segment has a high confidence level and is a large missing segment, perform judgment on operation time deviation and IMU spectrum characteristics on the current large missing segment in sequence. If both pass, it is a reasonable missing trajectory, and select the interpolation completion method to complete the current missing segment; otherwise, it is an unreasonable missing trajectory, and then fill it through the path reconstruction method.

[0084] Among them, as Figure 2 shown, a: Perform judgment on operation time deviation on the current large missing segment, specifically: a1: According to the GNSS positioning data corresponding to the effective trajectory curve, obtain the average speed of multiple uniform linear segments in the effective operation trajectory , and according to the distance between the start point and the end point of the large missing segment , obtain the expected operation time of the large missing segment .

[0085]

[0086]

[0087] Among them, is the total distance of multiple uniform linear segments, is the total duration of multiple uniform linear segments.

[0088] a2: Obtain the actual interval time between the start point and the end point of the large missing segment .

[0089]

[0090] The actual interval time is equivalent to the actual missing time of the missing segment. The distance between the two trajectory points at both ends of the missing segment is determined by the position coordinates of the start point and the end point of the missing segment, and the actual missing time is determined according to the sampling timestamps of the start point and the end point.

[0091] a3: Judge the expected operation time of the large missing segment and the actual interval time whether the difference meets the set threshold. If it meets, it is judged by the operation time deviation; otherwise, it is an unreasonable missing trajectory.

[0092]

[0093] As an option, set the error tolerance threshold , which is 10% - 15% of the estimated time, or fixed ±5 seconds.

[0094] b: Judge the IMU spectrum characteristics of the current large missing segment, specifically: b1: Obtain the acceleration of the agricultural machinery along the driving direction in the large missing segment , perform FFT transformation on it to the frequency domain, and obtain the frequency - amplitude distribution curve.

[0095] Specifically, the fast Fourier transform (FFT) refers to the prior art.

[0096] Acceleration The frequency - amplitude distribution curve of is:

[0097] The spectrum range is generally set to 0 - 2 Hz, covering most of the conventional action frequencies of agricultural machinery.

[0098] b2: Find the main peak frequency and amplitude of the frequency - amplitude distribution curve. If both meet the set threshold interval, it is judged by the IMU spectrum characteristics; otherwise, it is an unreasonable missing trajectory.

[0099] For typical behaviors of agricultural machinery such as turning around, spraying conversion, and left - right micro - swing, their main frequencies are concentrated in 0.15 - 0.35 Hz; set the frequency matching interval as Hz; find the main peak frequency in the spectrum and its amplitude ; if it meets:

[0100] That is, it is considered that this segment has the characteristics of operation behavior and is judged as "reasonable trajectory missing". Among them is the dynamically set energy threshold (such as more than 20% of the total energy of this segment, or 3 times the average value).

[0101] The IMU spectrum feature judgment can be used to identify whether the missing segment of the trajectory conforms to the characteristics of typical agricultural machinery operation actions (such as turning around, uniform seeding, and micro-amplitude steering). During the actual operation of agricultural machinery, different actions will exhibit different inertial motion patterns. Specifically: When turning around: there will be periodic direction changes and acceleration oscillations; when driving in a straight line at a constant speed: the acceleration and angular velocity are relatively stable; when in a stationary or waiting state: the inertial signal fluctuations are weak; these motion characteristics can be manifested as different frequency energy distributions in the spectrum. In some scenarios, the duration of the missing segment of the trajectory is consistent with the historical operation rhythm, and the time criterion may pass, but the agricultural machinery is actually in a non-operating state (such as waiting, parking); or, the IMU signal spectrum of the missing segment conforms to the characteristics of the operation behavior, but the actual duration of this segment far exceeds the reasonable range, and there may be situations such as turning around or path deviation; if only relying on a single-dimensional criterion for judgment, it is easy to cause incorrect filling, overfilling, or wrong filling, affecting the overall accuracy of trajectory reconstruction. Therefore, to improve the accuracy and robustness of trajectory missing determination, this application incorporates a dual-criterion joint judgment mechanism that fuses the judgment of operation time deviation and IMU spectrum features, so as to accurately understand the missing situation of the current missing segment and select a more accurate trajectory filling method.

[0102] S43: If the current missing segment is a high-confidence and small-segment missing segment, select the interpolation completion method to complete the current missing segment.

[0103] S431: Complete the missing segment by interpolation completion, specifically: According to the starting and ending position coordinates of the missing segment and , and the heading angles corresponding to the starting and ending points and , respectively obtain the positions of two control points and ;

[0104]

[0105] Among them, is the extended distance of the set control point; usually set to 20% - 30% of the Euclidean distance between the start and end points. This proportional range can effectively control the natural bending degree of the path, so that the completed trajectory can avoid a rigid broken line for the path and will not have a situation of a drawn-out or fluctuating and deviated path.

[0106] Use the cubic Bezier curve as the basic form of the interpolation trajectory, and use the positions of the starting point, ending point, and two control points of the missing segment to generate the completed trajectory curve, and insert the completed trajectory curve into the effective operation trajectory.

[0107] The present invention uses the cubic Bezier curve as the basic form of the interpolation trajectory. The shape of the Bezier curve is determined by the starting point, the end point and two control points, and the position of the control point directly affects the bending direction and degree of the trajectory. In order to make the path transition in the trajectory completion process more natural and in line with the actual operation mode of agricultural machinery, the present invention uses the heading angle information to derive the positions of the two control points, so as to control the "starting direction" and "ending convergence direction" of the Bezier curve. Specifically, the control points of the trajectory are no longer automatically generated by the traditional average position or curve fitting, but are calculated through the direction vectors of the starting and ending points. Taking the starting point as an example, the control point starts from the starting point and follows its heading angle. The first section of the direction guidance is formed by extending a certain distance d in the direction of The control points are set symmetrically in the opposite direction of the curve. This method forms a smooth curve with a minimum curvature and a starting and ending point connection in space.

[0108] The Bezier path constructed by this application can not only strictly meet the position boundary conditions, but also achieve a natural transition of the direction boundary, forming a trajectory path with continuous curvature, no mutation, no return, and no unreasonable fluctuations. Subsequently, the number of sampling points is evenly divided according to time on the generated curve (based on the missing time of the trajectory and the sampling frequency), and the completion point sequence is inserted into the original trajectory to fill the data gap.

[0109] S432: Completing the missing segment by path reconstruction, specifically: E1: Based on the starting position of the missing segment , the IMU data of the missing segment and the sampling interval of the IMU data, obtain the position of each sampling point of the missing segment, and generate the first path curve ;

[0110]

[0111]

[0112]

[0113] in, , , and Respectively p The speed, position coordinates and heading angle of each sampling point, P is the total number of sampling points in the missing segment; is the sampling interval, the initial speed is the average speed within the set time before the missing segment, the initial heading angle is the heading angle corresponding to the starting time of the missing segment; is the sampling timestamp corresponding to the combined acceleration of the agricultural machinery, which is obtained by synthesizing the three-axis acceleration vectors of the agricultural machinery; 、 and are respectively the speed and position coordinates of the th sampling point.

[0114] E2: Calculate the change in the heading angle between the starting point and the ending point of the missing segment ; If the change exceeds 150°, then use the set U-turn path template to generate several second candidate path curves , if the change is less than 150°, then use the set forward path template to generate several second candidate path curves .

[0115] E2-1: Use the set forward path template to generate several second candidate path curves .

[0116] Starting from the starting point of the missing segment, emit direction lines along the direction, and each direction line is a second candidate path curve .

[0117]

[0118]

[0119]

[0120] Among them, is the length of the second candidate path curve, , is the average speed of multiple uniform straight-line segments in the effective operation trajectory, T is the missing time; is the heading angle in the set forward path template, is the heading angle relative to the initial heading angle offset angle.

[0121] E2-2: Use the set U-turn path template to generate several second candidate path curves .

[0122] The change exceeds 150°, then the agricultural machinery has completed a U-turn operation in the missing segment. At this time, in order to simulate the real turning behavior of the agricultural machinery moving forward first and then making a U-turn, the second candidate path curve It is divided into a forward section and a turning section .

[0123]

[0124]

[0125]

[0126]

[0127] Among them, D 1 and D 2 are respectively the forward path curve and the U-turn path curve of the second candidate path curve , and are respectively the set forward distance and the U-turn driving distance of the second candidate path curve ; As an option ; and are respectively the set forward heading angle and the heading angle after U-turn of the second candidate path curve ; is the set of the set forward heading angle and the heading angle after U-turn of the second candidate path curve E3: Using the minimum Euclidean distance integral error as the matching index, select the second candidate path curve with the smallest difference from the first path curve as the optimal second path curve .

[0128] Specifically, linearly resample a number of points in both the first path curve and the second candidate path curve to obtain two corresponding sets of points, and then calculate the matching degree between the first path curve and the second candidate path through the sum of squared Euclidean distances. The second candidate path curve with the smallest sum of squared Euclidean distances is taken as the second path curve

[0129] E4: Fuse and reconstruct the first path curve and the second path curve to generate a reconstructed path for the missing segment; Specifically: E41: Obtain the position coordinates of four control points , , and : and are the starting and ending positions of the missing segment; ; ​, where and are the points at the 30% and 60% positions of the first path curve respectively, and are the points at the 30% and 60% positions of the second path curve respectively; E42: Generate a reconstructed path using the Bezier curve formula , and insert the reconstructed path into the valid operation trajectory in the form of trajectory points;

[0130] where is a parameter of the Bezier curve, representing the position ratio of the curve between the control points.

[0131] In this embodiment, the interpolated and completed path or the reconstructed path is evenly sampled into trajectory points according to the missing time period, and then these trajectory points are inserted into the valid operation trajectory to complete the valid operation trajectory.

[0132] Embodiment 2 An agricultural machinery operation trajectory completion and path reconstruction system, comprising: A collection module: Collect multi-source data and preprocess it. The multi-source data includes GNSS positioning data, IMU data, and agricultural implement image data of the agricultural machinery; An operation status recognition module: Based on the multi-source data, identify the operation status of the agricultural machinery and perform a confidence score on the original operation trajectory formed based on the GNSS positioning data through speed characteristics, IMU signals, and image characteristics; A missing recognition module: According to the operation status of the agricultural machinery, retain the valid operation trajectory in the original operation trajectory, and identify the length of the missing segment of the valid operation trajectory; A differential filling module: Based on the confidence and length of the missing segment, select an interpolation completion method or a path reconstruction method to fill the missing segments of the valid operation trajectory in sequence. Among them, when selecting path reconstruction to fill the missing segment m , the missing segment m is determined as an unreasonable missing trajectory through double determination of operation time deviation and IMU spectrum characteristics.

[0133] For a more specific process of the above method, reference can be made to the corresponding content disclosed in the foregoing embodiments, and details will not be elaborated here.

[0134] Embodiment 3 The present invention provides a computer device, comprising a processor and a memory; wherein, when the processor executes the computer program stored in the memory, the steps of the above agricultural machinery operation trajectory completion and path reconstruction method are implemented.

[0135] For a more specific process of the above method, reference may be made to the corresponding content disclosed in the foregoing embodiments, and details will not be repeated herein.

[0136] Embodiment 4 The present invention provides a computer-readable storage medium for storing a computer program; when the computer program is executed by a processor, the steps of the above-mentioned method for completing the agricultural machinery operation trajectory and reconstructing the path are implemented.

[0137] For a more specific process of the above method, reference may be made to the corresponding content disclosed in the foregoing embodiments, and details will not be repeated herein.

[0138] In this specification, the various embodiments are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the various embodiments, reference can be made to each other. For the systems, devices, and storage media disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple. For the relevant parts, reference can be made to the description in the method section.

[0139] Those skilled in the art can clearly understand that the technologies in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solutions in the embodiments of the present invention, in essence, or the parts that contribute to the prior art can be embodied in the form of a software product. The computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of the present invention.

[0140] The above is only the preferred embodiment of the present invention. The protection scope of the present invention is not limited to the above embodiments. All technical solutions falling within the idea of the present invention belong to the protection scope of the present invention. It should be noted that for those of ordinary skill in the art in this technical field, several improvements and refinements made without departing from the principle of the present invention should be regarded as within the protection scope of the present invention.

Claims

1. A method for completing the operation trajectory and reconstructing the path of agricultural machinery, characterized in that, It includes the following steps: S1: Collect multi-source data and preprocess it. The multi-source data includes GNSS positioning data, IMU data, and implement image data of agricultural machinery; S2: Based on the multi-source data, identify the operation state of agricultural machinery and perform confidence scoring on the original operation trajectory formed based on GNSS positioning data through speed characteristics, IMU signals, and image characteristics; S3: According to the operation state of agricultural machinery, retain the valid operation trajectories in the original operation trajectory and identify the length of the missing segments in the valid operation trajectories; S4: Based on the confidence level and length of the missing segment, select the interpolation completion method or the path reconstruction method, and fill in the missing segments of the valid operation trajectory in sequence. When selecting path reconstruction to fill in the missing segment m for filling, the missing segment m is determined to be an unreasonable missing trajectory through the dual judgment of operation time deviation and IMU spectrum characteristics.

2. The method for completing the agricultural machinery operation trajectory and reconstructing the path according to claim 1, wherein The GNSS positioning data is spatial position coordinates including sampling timestamps; the IMU data includes: the three-axis acceleration of agricultural machinery, the three-axis angular velocity of agricultural machinery, and the heading angle, all including sampling timestamps; the preprocessing includes: timestamp alignment of multi-source data, filtering and denoising, and abnormal data elimination.

3. The method for completing agricultural machinery operation trajectories and reconstructing paths according to claim 1, wherein Step S2 specifically includes: S21: Trajectory point sequence based on the original operation trajectory , generate a speed sequence regarding the speed between adjacent trajectory points , and filter the speed sequence. If the speeds within consecutive N set time windows after filtering are all within the set reasonable speed range, the speed feature score for the current N set time window is 1, otherwise it is 0; S22: Based on the continuous time series of IMU data formed, according to the set time window size, obtain the IMU signals within each time window, including the acceleration variance of the agricultural machinery along the driving direction , the acceleration peak , the average angular velocity around the vertical axis . If it satisfies and , then the acceleration vibration feature score of the current time window is 1, otherwise it is 0; if it satisfies , then the IMU yaw angle feature score of the current time window is 1, otherwise it is 0, where , and are respectively , and corresponding set thresholds; S23: Fuse the speed feature score, acceleration vibration feature score, and IMU yaw angle feature score within the current time window into the confidence score of the current time window according to the set weights α ; S24: If the confidence score of the current time window α is within the set high-confidence interval, the agricultural machinery operation status of the current time window is in an effective working state and with high confidence; If the confidence score of the current time window α is within the set low-confidence interval, then further extract the image features of the farm implement image data and identify the opening state of the farm implement. If the output is that the farm implement is open, the agricultural machinery operation state of the current time window is an effective working state with low confidence. If the output is that the farm implement is closed, the agricultural machinery operation state of the current time window is a non-effective working state; If the confidence score of the current time window α is lower than the set low confidence interval, the agricultural machinery operation status of the current time window is in a non-effective working state.

4. The method for supplementing agricultural machinery operation trajectories and reconstructing paths according to claim 1, wherein Step S3 specifically is: S31: Obtain the average sampling interval of the original operation trajectory based on the trajectory point sequence of the original operation trajectory and the positioning data sampling timestamp , and set a first missing threshold and a second missing threshold according to the average sampling interval ; Among them, is the total number of trajectory points of the original operation trajectory, is the sampling interval between any two trajectory points, and are respectively the sampling timestamps of the -th and the -th trajectory points; S32: Retain the job trajectory in the effective working state as the effective job trajectory, and compare the sampling intervals of any two trajectory points in the effective job trajectory , if , then there is a short missing segment between the current two trajectory points. If , then there is a long missing segment between the current two trajectory points.

5. The method for supplementing agricultural machinery operation trajectories and reconstructing paths according to claim 1, wherein Step S4 specifically includes: S41: If the current missing segment has low confidence, no filling process is performed; S42: If the current missing segment has high confidence and is a large missing segment, successively perform operation time deviation judgment and IMU spectrum feature judgment on the current large missing segment. If both pass, it is a reasonable missing trajectory, and the interpolation filling method is selected to fill the current missing segment. Otherwise, it is an unreasonable missing trajectory, and it is filled through path reconstruction; S43: If the current missing segment has high confidence and is a small missing segment, the interpolation filling method is selected to fill the current missing segment; Among them, performing operation time deviation judgment on the current large missing segment specifically is: Obtain the average speed of multiple uniform straight-line segments in the effective operation trajectory according to the GNSS positioning data corresponding to the effective trajectory curve , and according to the distance between the starting point and the ending point of the large missing segment , obtain the expected operation time of the large missing segment ; obtain the actual interval time between the starting point and the ending point of the large missing segment ; judge whether the difference between the expected operation time and the actual interval time of the large missing segment meets the set threshold. If it meets, judge through the operation time deviation; Performing IMU spectrum feature judgment on the current large missing segment specifically is: Obtain the acceleration of the agricultural machinery in the driving direction for large missing segments , perform FFT transformation on it to the frequency domain, and obtain the frequency-amplitude distribution curve; find the main peak frequency and amplitude of the frequency-amplitude distribution curve. If both of them meet the set threshold interval, then judge through the IMU spectrum characteristics.

6. The method for completing agricultural machinery operation trajectories and reconstructing paths according to claim 5, wherein Filling the missing segment through the interpolation filling method specifically is: According to the starting and ending position coordinates of the missing segment and , as well as the course angles corresponding to the starting and ending points and , respectively obtain the positions of two control points and ; Among them, is the extension distance of the set control point; Using a cubic Bézier curve as the basic form of the interpolation trajectory, and using the positions of the starting point, ending point, and two control points of the missing segment to generate a filled trajectory curve, and inserting the filled trajectory curve into the valid operation trajectory in the form of trajectory points.

7. The method for supplementing agricultural machinery operation trajectories and reconstructing paths according to claim 5, wherein Filling the missing segment through path reconstruction specifically is: E1: Based on the starting position of the missing segment , the IMU data of the missing segment, and the sampling interval of the IMU data, obtain the position of each sampling point of the missing segment and generate a first path curve ; Among them, , , and are the velocity, position coordinates, and heading angle of the p -th sampling point, respectively; P is the total number of sampling points in the missing segment; is the sampling interval, and the initial velocity is the average velocity within the set time before the missing segment, and the initial heading angle is the heading angle corresponding to the starting time of the missing segment; is the sampling timestamp corresponding to the combined acceleration of the agricultural machinery; , and are the velocity and position coordinates of the -th sampling point, respectively; E2: Calculate the change in the heading angle between the starting point and the ending point of the missing segment ; If the change exceeds 150°, then use the set U-turn path template to generate several second candidate path curves ; If the change is less than 150°, then use the set forward path template to generate several second candidate path curves ; Among them, is the length of the second candidate path curve, , is the average speed of multiple uniform linear segments in the effective operation trajectory, T is the missing time; is the course angle in the set forward path template, is the course angle relative to the initial course angle offset angle; D 1 and D 2 are respectively the forward path curve and the U-turn path curve of the second candidate path curve , and are respectively the set forward distance and the U-turn driving distance of the second candidate path curve ; and are respectively the set forward course angle and the course angle after U-turn of the second candidate path curve ; is the set of the set forward course angle and the course angle after U-turn of the second candidate path curve ; E3: Using the minimum Euclidean distance integral error as the matching criterion, select the second candidate path curve with the smallest difference from the first path curve as the optimal second path curve ; E4: Fuse and reconstruct the first path curve and the second path curve to generate a reconstructed path for the missing segment; specifically: E41: Obtain the position coordinates of four control points , , and : and are the starting and ending positions of the missing segment; ; , where and are the points at the 30% and 60% positions of the first path curve respectively, and are the points at the 30% and 60% positions of the second path curve respectively; E42: Generate a reconstruction path using the Bezier curve formula and insert the reconstruction path into the valid operation trajectory in the form of trajectory points; Among them, is the parameter of the Bessel curve, representing the position ratio of the curve between the control points.

8. An agricultural machinery operation trajectory completion and path reconstruction system, characterized in that, It includes: Collection module: Collect multi-source data and preprocess it. The multi-source data includes GNSS positioning data, IMU data, and implement image data of agricultural machinery; Operation state identification module: Based on the multi-source data, identify the operation state of agricultural machinery and perform confidence scoring on the original operation trajectory formed based on GNSS positioning data through speed characteristics, IMU signals, and image characteristics; Missing identification module: According to the operation state of agricultural machinery, retain the valid operation trajectories in the original operation trajectory and identify the length of the missing segments in the valid operation trajectories; Differential filling module: Based on the confidence level and length of the missing segment, select the interpolation completion method or the path reconstruction method to fill the missing segments of the valid operation trajectory in sequence. Among them, when selecting path reconstruction to fill the missing segment m for filling, the missing segment m is determined as an unreasonable missing trajectory through the dual judgment of operation time deviation and IMU spectrum characteristics.

9. A computer device, characterized in that, It includes a processor and a memory; among them, when the processor executes the computer program saved in the memory, it implements the steps of the agricultural machinery operation trajectory filling and path reconstruction method described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, It is used to store a computer program; when the computer program is executed by the processor, it implements the steps of the agricultural machinery operation trajectory filling and path reconstruction method described in any one of claims 1-7.

Images