Agricultural machinery field road trajectory classification method fusing space-time clustering and semantic segmentation

Abstract

The invention discloses an agricultural machinery field and road track classification method fusing spatial-temporal clustering and semantic segmentation. The requirements for farmland track and road track classification precision and operation efficiency in an actual agricultural machinery operation scene are met at the same time. The method mainly comprises three stages: firstly, track preprocessing; and removing abnormal track points by using four preprocessing operations of null value filling, attribute filtering, speed cleaning and linear interpolation, and complementing missing tracks. And 2, track clustering. A space-time neighbor track segment clustering method based on track position, time sequence and direction information is adopted to form a track group on the basis of removing a long-distance road driving track. And 3, track segmentation. And combining a channel and space mixed attention mechanism and a focus loss function to construct a trajectory diagram segmentation model based on U-Net so as to realize segmentation of a road driving trajectory, a drifting trajectory and a farmland trajectory close to a farmland plot in each trajectory group. Experiments show that the method provided by the invention has the highest classification precision in a real agricultural machine operation scene data set, and the average reasoning time is superior to that of a method with the highest precision in latest classification methods.

Description

Technical Field

[0001] The present invention belongs to the field of computer technology, and in particular relates to a method for classifying agricultural machinery field roads trajectories by integrating spatiotemporal clustering and semantic segmentation. Background Art

[0002] With the integration and application of agricultural machinery information equipment and Internet of Things technology, especially the development of positioning technologies such as GPS and Beidou, agricultural machinery has gradually accumulated a large amount of trajectory data during driving. Agricultural machinery trajectory data records the location information, time series information and machine status of agricultural machinery when it operates in the field, drives on the road and parks. Agricultural machinery field and road trajectory classification is the process of using these trajectory data to judge the behavior of agricultural machinery. The classification process converts trajectory points into field trajectory points and road trajectory points. Among them, field points refer to the trajectory points generated by agricultural machinery operating in the field, and road points include trajectory points generated by other behaviors such as driving and parking on the road. The set of road trajectory points is divided into three types: long-distance road driving trajectories (generally generated by agricultural machinery driving on the road), drifting trajectories (generally generated by agricultural machinery parking) and road driving trajectories close to farmland plots (generally generated by agricultural machinery driving on farm roads). The classification results and processing time are of great practical significance for applications such as operation efficiency evaluation and agricultural machinery operation area calculation.

[0003] At present, the traditional field road trajectory classification method mainly relies on manual recording of farmland and road boundary information when agricultural machinery enters the farmland. However, this method not only consumes manpower and material resources in real agricultural machinery operation scenarios, but is also prone to false alarms and missed reports, which is not conducive to the statistical supervision of effective agricultural machinery operation information.

[0004] In recent years, scholars have successively tried the field track classification method based on clustering, the field track classification method based on fully supervised machine learning, and the field track classification method based on deep learning:

[0005] 1. Clustering-based field and road trajectory classification method: mainly uses point density information to achieve trajectory clustering. Based on the characteristics of low farmland trajectory speed and close distance between continuous trajectory points, the point set with dense spatial distribution is classified as farmland trajectory segment, and the remaining points are classified as road trajectory segment. Although this method has high computational efficiency, due to the single point density feature and strong parameter dependence, it is difficult to correctly detect road driving trajectories and drift trajectories close to farmland plots based solely on point density features.

[0006] 2. Field track classification method based on fully supervised machine learning: It mainly relies on manually designed feature extraction, including motion features such as speed, acceleration and direction angle, and uses machine learning algorithms to identify the feature similarity between adjacent points to achieve agricultural machinery field track classification. This method also has faster computing efficiency, but the manually extracted features are difficult to effectively capture the spatiotemporal correlation information between track points, resulting in low classification accuracy.

[0007] 3. Field and road trajectory classification method based on deep learning: On the basis of the input features of fully supervised machine learning, automatically extracted advanced features are added, such as extracting the local spatiotemporal relationship between trajectory points through graph convolutional networks, extracting the global spatial features of the entire trajectory using convolutional neural networks, or using recurrent neural networks to capture the sequence information of the trajectory, and using deep neural networks as classifiers to label each trajectory point as "farmland" and "road". This method has a wider range of application features and higher recognition accuracy. However, the method based on graph convolutional networks will generate large time calculation costs and space utilization requirements in the stage of constructing spatiotemporal relationship graphs, resulting in slower calculation speeds, and when the total number of trajectory points in the data set is too large, training work is difficult to carry out. The method based on recurrent neural networks needs to construct feature vectors for each trajectory point during training, and there is also a problem of high resource consumption, which makes it difficult to train in a data set with a large number of trajectory points, and the ability to extract local spatial features is weak, affecting the calculation accuracy. In addition, it is a feasible idea to convert trajectory data into trajectory graphs and use convolutional neural networks with low computational costs for trajectory classification. Convolutional neural networks have strong spatial feature extraction capabilities and parallel processing capabilities, and are suitable for batch training of multiple data with different numbers of trajectory points. However, directly mapping the complete trajectory data with a large spatial span will easily cause a large number of road points and farmland points to overlap at the same position, which cannot achieve the expected purpose of classification based on the trajectory pixel-level visual features, resulting in reduced classification accuracy.

[0008] In summary, studying a high-precision and high-computational efficiency agricultural machinery field road trajectory classification method is of great significance for actual agricultural machinery operation scenarios. Summary of the invention

[0009] The present invention provides an efficient agricultural machinery field road trajectory classification method, which integrates the advantages of a clustering method with a simple input structure and high computational efficiency and the advantage of a deep learning semantic segmentation method that can automatically extract effective features from trajectory images, thereby meeting the requirements for method accuracy and computational efficiency in actual agricultural machinery operation scenarios.

[0010] The technical solution adopted by the present invention is as follows:

[0011] A method for classifying agricultural machinery field road trajectories by integrating spatiotemporal clustering and semantic segmentation. The process is as follows Figure 1 The processing results are shown as Figure 2 As shown, the operation process is as follows Figure 3 Specifically, it is divided into three stages: trajectory preprocessing, trajectory clustering and trajectory segmentation. Step 1: Trajectory preprocessing. Four trajectory preprocessing methods are used, namely null value filling, attribute filtering, velocity cleaning, and linear interpolation, to eliminate abnormal trajectory points and maintain a balanced sampling frequency; Step 2: Trajectory clustering. Design a spatiotemporal neighbor trajectory segment clustering method to fully utilize the spatiotemporal density information of the trajectory to quickly and effectively identify and remove long-distance road driving trajectories. Then, group the trajectories for processing, which will help to accurately capture the spatial characteristics of farmland trajectories in the subsequent semantic segmentation stage. Step 3: Trajectory segmentation. By improving the traditional U-Net trajectory segmentation model, combining the channel and spatial hybrid attention mechanism and the focal loss function, the model is adapted to the field and road trajectory classification task, and the farmland trajectory, the road driving trajectory close to the farmland plot, and the drift trajectory can be segmented end-to-end based on clustering and grouping.

[0012] Furthermore, in step 1, the trajectory preprocessing process is as follows: Step 1.1: Fill the corresponding attributes of the trajectory points with empty values ​​for timestamp, longitude, latitude and speed to 0; Step 1.2: According to the timestamp, longitude and latitude of the trajectory points, clean the resampled trajectories with the same time in consecutive moments, repeated trajectories with the same longitude and latitude coordinates in consecutive moments, static drift trajectories with different longitude and latitude coordinates and 0 speed in consecutive moments, and abnormal longitude and latitude trajectories whose longitude and latitude range exceeds the territory of China; Step 1.3: Each type of agricultural machinery has a fixed maximum operating speed speed_max, which is obtained by analyzing the labeled data of different types of agricultural machinery. When the agricultural machinery is operating normally in the field, it will not exceed this rated maximum speed. Therefore, the agricultural machinery trajectory points with speed values ​​greater than the maximum operating speed speed_max are eliminated one by one; Step 1.4: If the current environmental signal is insufficient or the positioning system fails, the sampling trajectory is easily lost, that is, the time interval between two adjacent points in the trajectory is greater than the sampling frequency, resulting in an increase in the spatial distance between the two points. At the same time, in order to avoid invalid interpolation for situations such as parking of agricultural machinery or shutting down while driving on the road, linear interpolation is performed only between two trajectory points whose time interval is greater than the sampling frequency f and less than the average waiting time for red lights t (preferably t is 60) with the sampling frequency as the interval.

[0013] Furthermore, in step 2, the trajectory clustering process is as follows: Step 2.1: Recalculate the directional angle feature and the directional angle difference feature between each trajectory point after trajectory preprocessing and the subsequent trajectory point in the sequence. Decompose the agricultural machinery trajectories with a directional angle difference less than or equal to dir (preferably dir is 30) into line segments of r (preferably r is 54) points, and label the serial number no for each trajectory segment according to the original trajectory time series; Step 2.2: At the starting point, 1 / 4 position point, midpoint, 3 / 4 position point, and end point of each trajectory segment, draw 5 fixed-length perpendicular line segments according to the directional angle of the corresponding trajectory point. Drawing five perpendicular line segments is to avoid the decomposed trajectory segment being a trajectory segment at the junction of the field road or a turning trajectory segment at the edge of the field. In this case, if only one perpendicular line segment is drawn, this perpendicular line segment may cause detection errors because it does not intersect with other line segments. The length of the perpendicular line segment is set to extend l (preferably l is 8) times the width of the agricultural machinery on both sides of the trajectory point to reduce the judgment error. The width value of the agricultural machinery has been manually sampled and recorded in the trajectory attributes in advance; Step 2.3: Detect the spatio-temporal neighbor relationship of the trajectory segments. Let the number of trajectory segments be n (n>1). For a trajectory segment with the serial number no being x (0≤x<n), if one of the five perpendicular line segments of this trajectory segment intersects with the trajectory segment with the serial number y (max(0, x - m)≤y≤min(x + m, n - 1) and y≠x, preferably m is 3), then retain this trajectory segment, as shown in Figure 4 Figure a; Step 2.4: Detect the spatial neighbor relationship of the trajectory segments. Let the number of trajectory segments be n (n>1). For a trajectory segment with the serial number no being x (0≤x<n), if none of the five perpendicular line segments of this trajectory segment intersects with the trajectory segment with the serial number y (max(0, x - m)≤y≤min(x + m, n - 1) and y≠x, preferably m is 3), then determine whether the perpendicular line segment intersects with other trajectory segments. If the five perpendicular line segments intersect with i trajectory segments in total (1≤i<n - m), and the serial number of each trajectory segment is x j (1≤j≤i), then take the corresponding trajectory segment whose serial number conforms to the formula min|x j -x| (assuming x k meets the conditions, 1≤k≤i), and form a new set X with all the trajectory points within the range of the trajectory segment serial numbers [min(x, x k ), max(x, x k )], and calculate the maximum longitude difference lon_diff max , maximum latitude difference lat_diff max and maximum time difference ts_diff max among the trajectory points in set X. By analyzing the labeled data, take the maximum longitude and latitude difference spatial max and the maximum time difference temporal maxAs a threshold, if set X simultaneously satisfies max(lon_diff max , lat_diff max ) ≤ spatial max and ts_diff max ≤ temporal max , then retain the trajectory segment with the serial number x, as shown in Figure 4 b. Otherwise, it is determined that this trajectory may be a long-distance road reciprocating driving trajectory, and this trajectory segment needs to be removed, as shown in Figure 4 c; Step 2.5: Detect the spatio-temporal non-neighbor relationship of the trajectory segments. Let the number of trajectory segments be n (n≥1). For a trajectory segment with the serial number no being x (0≤x<n), if there is no perpendicular segment intersecting any other trajectory segment, then it is determined that this trajectory may be a long-distance road one-way driving trajectory, and this trajectory segment needs to be removed, as shown in Figure 4 d; Step 2.6: Group the retained trajectory segments, recalculate the distance between each trajectory point and the next trajectory point in the sequence, and calculate the distance threshold d according to the following formula. Use the trajectory points with a distance greater than d as split points to achieve trajectory clustering. d = speed_max × f × 2 where speed_max is the maximum operating speed of the agricultural machinery obtained by analyzing the labeled data of different types of agricultural machinery, and f is the sampling frequency.

[0014] Furthermore, in step 3, the trajectory segmentation process is as follows: Step 3.1: Track image generation. The Pillow library is used to convert the track data into track images, and the final image is presented in the form of a scatter plot. First, the image size is fixed to 256×256. Referring to the rule of WGS84 system projection tiles, the minimum longitude and latitude difference lon_lat_diff of the track points in the image coverage area is calculated according to the following formula, and the tracks are processed in two cases: (a) If the longitude and latitude span of each group of tracks is less than the minimum longitude and latitude difference, two boundary track points are added to make the longitude and latitude range of the filled track reach the minimum longitude and latitude difference. (b) If at least one of the longitude and latitude spans of each group of tracks is greater than the minimum longitude and latitude difference, two boundary track points are added to make the longitude and latitude differences of the filled track consistent. It is worth noting that the track images generated by this method do not have the same resolution. For case (b), the track is essentially compressed, resulting in the loss of some detail information. However, the clustering stage has achieved the removal of long-distance road driving tracks and the grouping of the entire track, which greatly reduces the overlap of farmland points and road points. At the same time, farmland trajectories have the characteristics of dense distribution and clear boundaries, so the lost detail information in the image does not affect the extraction of global spatial features. When the semantic segmentation method determines the boundary of the farmland trajectory, all points within the boundary can be classified as farmland trajectories. Where 2×π×6378137 is the circumference of the equatorial plane on the ellipsoid, zoom_level is the zoom level of the tile, preferably 16, 111195.07973436874 is the distance of 1 degree of longitude and latitude on the earth's surface, and image_size is the image size, preferably 256. After filling the trajectory, the pixel coordinates of each trajectory point mapped to the image are calculated according to the following formula. Where lon(i) and lat(i) are the longitude and latitude coordinates of the trajectory point p(i), lon_mmin, lon_max, lat_min, and lat_max are the minimum longitude, maximum longitude, minimum latitude, and maximum latitude of the entire trajectory, and long and wide are the length and width of the image, which are the same as the image size. The color channel of traditional images is composed of three colors: red, green, and blue. In order to fully integrate the speed, acceleration, and angular difference motion characteristics of the trajectory data, a channel expression method suitable for distinguishing between farmland trajectories and road trajectories is defined. Since the trajectory of agricultural machinery in the field is usually slower than the speed and acceleration of the road driving trajectory, and is not prone to large changes in direction, the speed, acceleration characteristics of the trajectory points and the angular difference between adjacent trajectory points are normalized according to the following formula to replace the pixels in the traditional red, green, and blue channels, and are input as parameters into the drawing function. In order to distinguish the farmland trajectory from the background color, the three normalized eigenvalues ​​are complemented, so the lower speed, acceleration, and angular difference between adjacent trajectory points appear as brighter colors in the image, such as Figure 5 As shown in a. Where speed(i) is the speed of trajectory point p(i), speed max is the maximum velocity of the entire trajectory, acceleration(i) is the acceleration of the trajectory point p(i), acceleration max is the maximum acceleration of the entire trajectory, direction_diff(i) is the difference in direction between the trajectory point p(i) and the previous trajectory point p(i-1), direction_diff max is the maximum angular difference of the overall trajectory. In summary, the generation of trajectory images can not only reflect the spatial features through the pixel positions, but also encode the motion features of the trajectory points through the pixel values. Step 3.2: Segmentation image generation. The segmentation image contains the semantic annotation information of the trajectory image, that is, it provides the ground-truth category of each pixel. Its image size is similar to the trajectory. Figure 1 The encoding category of each pixel only needs one color channel. When training the image model, the two images need to be matched one by one. The pixel coordinates and pixel values ​​of the segmentation map will be determined by the latitude and longitude and label of each track point. In order to simplify the difficulty of classifying field and road tracks, two colors are selected to represent the two categories, that is, the pixel value corresponding to the farmland track point is "1" (black), the pixel value corresponding to the road track point and the pixel value of other background are "0" (white), such as Figure 5 As shown in b. Step 3.3: Track image segmentation. Based on the original U-Net framework, an attention mechanism is added to make it more accurate in the task of agricultural machinery field road track classification. The improved semantic segmentation model is named CBGAM U-Net (Convolutional Block and Gated Attention Module U-Net). Its architecture is as follows: Figure 6As shown in the figure. The Convolutional Block Attention Module (CBAM) is a general and lightweight module that can extract the channel and spatial dimension information of the input data to set the weight matrix, allowing the network to focus more attention on the target area to achieve the simultaneous extraction of global features and detail features of the trajectory points. In addition, integrating the attention mechanism in the jump connection stage not only helps to obtain the original image information, but also the weight can be learned by the network. Therefore, CBAM is used in the jump connection stage. Specifically, the channel attention module uses global maximum pooling and global average pooling operations on the input feature map, calculates the maximum eigenvalue and average eigenvalue on each channel, and generates two 1×1×C feature maps. Then the two feature maps are used as the input of the shared network, and the important channels are adaptively screened through convolution and ReLU activation functions, and the sum operation is performed. Finally, the channel attention weight vector is generated using the Sigmoid function. The spatial attention module takes the output feature map of the channel attention module as input, performs maximum pooling and average pooling operations along the channel dimension and splices them, and outputs a feature map of size H×W×2C. Finally, the spatial attention weight vector is generated through the convolution layer and Sigmoid. In CBGAM, the channel and spatial attention weight calculations are performed on the downsampled and upsampled feature maps corresponding to each layer of skip connection at the same time. The two vectors are added and activated by ReLU. Finally, the final overall weight matrix is ​​calculated through the convolution layer, normalization layer and Sigmoid function, and the final attention enhancement feature map is obtained after multiplication with the downsampled feature map. The principle diagram of CBGAM is shown in the figure. Figure 7 shown. The single-channel segmentation map obtained by the CBGAM U-Net semantic segmentation network after downsampling and upsampling is consistent with the size of the original input trajectory map. Next, Sigmoid is selected as the activation function to control the pixel value of the segmentation map within the range of [0, 1], and the pixel value corresponding to each trajectory point is located using the coordinate mapping relationship. Pixel values ​​greater than 0.5 are mapped to the label "1", indicating that the trajectory point belongs to the farmland trajectory, and pixel values ​​less than or equal to 0.5 are mapped to the label "0", indicating that the trajectory point belongs to the road trajectory. Finally, the labeled agricultural machinery operation trajectory is output. Generally, the number of points in a daily operation trajectory does not exceed 43,200, and farmland trajectories often account for a small part. Therefore, for a trajectory map of size H×W (256×256 in this article), there is often an imbalance between the foreground color (white) and the background color (black). In order to solve the imbalance problem, a weighted focal loss function is used as the training set loss function. As shown in the following formula, the weighted focal loss function introduces a dynamic scaling factor into the standard cross entropy loss to increase the contribution of difficult-to-distinguish pixels to the loss. FL=-α(1-p t ) γ log(p t ) Wherein α is a predefined parameter for balancing positive and negative samples, γ is a hyperparameter for adjusting the weight factor, and pt represents the probability value of the t-th pixel being predicted as a farmland trajectory. In the present invention, α is assigned a value of 0.75 and γ is assigned a value of 2.

[0015] Compared with the existing technical solutions, the beneficial effects of the present invention are:

[0016] 1. The present invention eliminates abnormal trajectory points and maintains a balanced sampling frequency through four trajectory preprocessing methods, namely null value filling, attribute filtering, speed cleaning and linear interpolation.

[0017] 2. The present invention makes full use of the spatiotemporal density information of the trajectory, adopts the spatiotemporal neighbor trajectory segment clustering method to quickly and effectively identify and eliminate long-distance road driving trajectories, and then groups the trajectories for processing, which helps to accurately capture the spatial characteristics of the farmland trajectory in the subsequent semantic segmentation stage.

[0018] 3. This paper combines the channel and space mixed attention mechanism and the focal loss function to improve the traditional U-Net trajectory graph segmentation model, making the model suitable for field road trajectory classification tasks and effectively improving the classification accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a schematic diagram of the process of the present invention.

[0020] Figure 2 It is a schematic diagram of the processing results of the present invention.

[0021] Figure 3 It is a schematic diagram of the operation process of the present invention.

[0022] Figure 4 Schematic diagram of four possible situations using the spatiotemporal neighbor trajectory segment clustering method.

[0023] Figure 5 It is the structure diagram of trajectory image segmentation model.

[0024] Figure 6 This is the CBGAM U-Net semantic segmentation network structure diagram.

[0025] Figure 7 This is the principle diagram of the attention mechanism. DETAILED DESCRIPTION

[0026] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. The technical solutions of the present invention are further illustrated in conjunction with the accompanying drawings and embodiments.

[0027] For a set of n-point daily operation trajectory data of agricultural machinery, each trajectory point includes six attribute information, namely, timestamp (ms), coordinates (latitude and longitude, expressed in WGS84 coordinate system), speed (m / s), direction (°) and width (m). After the proposed field and road trajectory classification method, the "farmland" and "road" classification labels corresponding to each trajectory point are finally output, where label "1" represents farmland trajectory and label "0" represents road trajectory. The operation process is as follows Figure 3 As shown. It includes three stages:

[0028] Step 1: Trajectory preprocessing. Four different preprocessing operations are performed on the original agricultural machinery trajectory data in turn, including null value filling, attribute filtering, speed cleaning, and linear interpolation, to avoid abnormal points and missing points from affecting the classification effect. The trajectory data after preprocessing is named cleaning point data.

[0029] Step 2: Trajectory clustering. The trajectories are clustered using the direction and distance features of temporally adjacent trajectory segments to eliminate long-distance road driving trajectories and form trajectory groups.

[0030] Step 3: Trajectory segmentation. Convert the clustered trajectory group into a trajectory map and establish the mapping relationship between position coordinates and pixel coordinates. Build a semantic segmentation model, extract the global spatial features of the trajectory image and perform pixel-level classification to achieve the segmentation of farmland trajectories, road driving trajectories close to farmland plots, and drift trajectories.

[0031] Example

[0032] The experimental data used in this embodiment is named the Precision Agriculture Application Project Data Service Platform Dataset. This dataset comes from the Precision Agriculture Application Project Data Service Platform and consists of 150 trajectory data. The sampling time is from June 4, 2021 to June 23, 2021. It covers 8 provinces including Jiangsu Province, Shandong Province and Hebei Province in China, and the agricultural machinery type is only wheat harvester. The sampling interval of this dataset is 5s, and the number of trajectory points for each trajectory ranges from 539 to 17223. The total number of trajectory points for 150 trajectories is 885279. Each trajectory point has seven parameters: time (YYYY / MM / DD hh:MM:ss), coordinates (latitude and longitude, expressed in WGS84 coordinate system), speed (m / s), direction (°), height (m) and label. The width (m) parameter is taken as the standard value 2.75 by analyzing the wheat harvester data. This embodiment includes the following steps:

[0033] Step 1: Data division. The 150 daily running trajectory data samples are randomly divided into 10 parts, and training, validation and testing are performed in a ratio of 8:1:1. Note that since each trajectory data sample will be divided into different groups after the trajectory clustering stage, one trajectory dataset will correspond to multiple 256×256 image blocks, which results in the training set, validation set and test set images of the semantic segmentation model not actually being divided in a ratio of 8:1:1. Table 1 provides the number of 10 datasets divided by the precision agriculture application project data service platform dataset and the number of trajectory images generated. Table 1 Division of the data set of the precision agriculture application project data service platform Dataset number Number of tracks Number of image blocks 1 15 56 2 15 63 3 15 56 4 15 80 5 15 66 6 15 74 7 15 66 8 15 76 9 15 82 10 15 73

[0034] Step 2: Trajectory preprocessing. Affected by factors such as weather and geographical environment signal interference, positioning equipment failure, and frequent stationary and turning of agricultural machinery, the positioning equipment may collect abnormal data, including trajectory with missing attribute values, resampled trajectory, repeated trajectory, static drift trajectory, abnormal latitude and longitude trajectory, and lost sampling trajectory. At the same time, to ensure effective operation, different types of agricultural machinery should not exceed the specified maximum speed when operating in the field. Therefore, four methods for preprocessing the original agricultural machinery trajectory data are used, including null value filling, attribute filtering, speed cleaning, and linear interpolation.

[0035] Step 3: Trajectory clustering. When driving agricultural machinery to farmlands in different locations, road driving trajectories with a large span of longitude and latitude are often generated. Compared with farmland trajectories, road driving trajectories close to farmland plots, and drifting trajectories, these long-distance road driving trajectories often have the characteristics of large spatial distances between temporal neighbor trajectory points and large temporal ranges between spatial neighbor trajectory points. By clustering trajectories, dense trajectories can be quickly aggregated into a group based on attributes such as the timing and position coordinates of the trajectories, and long-distance road driving trajectories can be classified as noise points to reduce the consumption of computing resources and storage resources. To this end, the spatiotemporal neighbor trajectory segment clustering method is adopted to make full use of the directional distribution characteristics of the trajectory segments and the spatiotemporal characteristics between the trajectory segments, eliminate long-distance road driving trajectories, and group trajectories according to the distance between the first and last points of the trajectory segments. The four possible situations of using the spatiotemporal neighbor trajectory segment clustering method are as follows: Figure 4 shown.

[0036] Step 4: Trajectory segmentation. In order to further classify the road driving trajectories close to farmland plots and drift trajectories based on the clustering results, a trajectory image segmentation model based on the improved U-Net is constructed. Farmland trajectories, road driving trajectories close to farmland plots, and drift trajectories can show obvious boundary differences in the image, which is difficult to reflect using the original trajectory sequence. Using images to represent trajectories can not only use adjacent pixels to reflect the spatial relationship of trajectories, but also help to extract high-dimensional features using computer vision technology. By representing the trajectory data as a two-dimensional image data structure for processing, and using a semantic segmentation model to directly classify each pixel in the trajectory map, the accuracy of trajectory classification can be effectively improved.

[0037] This method first converts the trajectory data into a two-dimensional trajectory map, thereby transforming the farm machinery field road trajectory classification problem into a pixel classification problem. The pixel part corresponding to the farmland trajectory is biased towards the foreground color (white) for classification, and the remaining road trajectory and the pixel part corresponding to the background fill are biased towards the background color (black) for classification. This task mainly includes three stages: trajectory image generation, segmentation image generation and trajectory image segmentation. The trajectory image segmentation model structure is as follows: Figure 5 As shown in Figure 2, the trajectory image and the segmentation image together constitute the input image data for semantic segmentation model training, verification, and testing.

[0038] This experiment was completed in an NVIDIA GeForce RTX 3060Laptop GPU, where the initial learning rate of the image segmentation model was defined as 0.001, the parameters of the model were updated using the Adam optimizer, the weight decay was 0, and the batch size and number of iterations were defined as 4 and 20. Training was stopped when the validation set loss value no longer increased for five consecutive epochs.

[0039] The present invention uses dice loss as the validation set loss function. Dice loss measures the similarity between pixels by calculating the overlap between the predicted result and the true result. The formula is as follows: Where A is the pixel value matrix of the predicted segmentation image, and B is the pixel value matrix of the actual segmentation image. This paper uses four evaluation indicators to evaluate the performance of the agricultural machinery field road trajectory classification method that integrates spatiotemporal clustering and semantic segmentation on the test set, namely precision, recall, F1 score and accuracy. Precision mainly focuses on the accuracy of the model's prediction of positive examples, recall mainly focuses on the model's coverage of positive examples, and F1 score is the harmonic mean of precision and recall. Accuracy indicates the proportion of samples correctly predicted by the model to the total number of samples. The formula is as follows: In the present invention, the result in which both the predicted category and the true category are farmland trajectories is called TP, the result in which the predicted category is a road trajectory and the true category is a farmland trajectory is called FP, the result in which the predicted category is a farmland trajectory and the true category is a road trajectory is called FN, and the result in which both the predicted category and the true category are road trajectories is called TN.

[0040] Step 5: Comparative experiment. In order to further evaluate the overall performance of the agricultural machinery field road trajectory classification method integrating spatiotemporal clustering and semantic segmentation, two traditional machine learning methods and three advanced deep learning methods were selected as baselines, namely, density-based spatial clustering of applications with noise (DBSCAN) and direction distribution-based reasoning method (DBSCAN+Rules), decision tree (DT), graph convolutional network (GCN), statistical features (STF) and visual features extracted with attention U-Net (VFAU) and bidirectional long short-term memory network (BiLSTM) fusion method (STF+VFAU+BiLSTM) and generative adversarial network-bidirectional long short-term memory network (GAN-BiLSTM). The parameter settings of various methods are consistent with the original paper.

[0041] The precision agriculture application project data service platform data set is used to compare the agricultural machinery field road trajectory classification method (STNNTC+CBGAM U-Net), DBSCAN+Rules, DT, GCN, STF+VFAU+BiLSTM and GAN-BiLSTM that integrate spatio-temporal clustering and semantic segmentation. Among them, STNNTC (Spatio-Temporal Nearest Neighbor Trajectory Clustering) is a spatio-temporal neighbor trajectory segment clustering method, and CBGAM U-Net is the semantic segmentation model used. Using the same training set, validation set and test set division and trajectory preprocessing operations, it is verified that the proposed method can achieve a balance between accuracy and computational efficiency compared with advanced models, as shown in Table 2. Table 2 Overall performance of different methods on the precision agriculture application project data service platform dataset.

[0042] The results show that STNNTC+CBGAM U-Net achieves the highest F1 score of 90.41% and the highest accuracy of 92.03% in the precision agriculture application project data service platform dataset, where the F1 score is 3.04% (DBSCAN+Rules), 5.89% (DT), 2.07% (GCN), 0.68% (STF+VFAU+BiLSTM) and 6.63% (GAN-BiLSTM) higher than the other five methods. The accuracy is 2.26% (DBSCAN+Rules), 4.45% (DT), 1.70% (GCN), 0.65% (STF+VFAU+BiLSTM) and 5.53% (GAN-BiLSTM) higher than the other five methods.

[0043] By comprehensively analyzing the accuracy and average inference time, DBSCAN+Rules has the fastest calculation speed, but is limited by the incomplete feature extraction and poor parameter versatility, resulting in low classification accuracy. DT also has the problem of insufficient feature extraction, especially the difficulty in obtaining spatial information between trajectory points, and each trajectory point needs to be binary searched during inference, resulting in low accuracy and computational efficiency. Although GCN effectively improves the classification accuracy, it takes a lot of computing time to build a spatiotemporal relationship graph for input. STF+VFAU+BiLSTM extracts rich features by integrating multiple models, and its accuracy ranks first among all the comparison methods, but it also leads to an increase in inference time. GAN-BiLSTM focuses on processing time series information, insufficient extraction of spatial features between trajectory points, and a complex structure, which affects classification accuracy and computational efficiency. STNNTC+CBGAM U-Net not only captures the key spatial local features and global features of trajectory points, but also combines the fast-computing clustering method with the semantic segmentation model, and finally achieves a balance between classification accuracy and computational efficiency.

[0044] The above-described embodiments are merely examples for clearly illustrating the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for classifying agricultural machinery field and road trajectories by integrating spatiotemporal clustering and semantic segmentation, characterized by: The following steps are included: Step 1: trajectory preprocessing: four trajectory preprocessing methods are used, namely null value filling, attribute filtering, velocity cleaning and linear interpolation, to eliminate abnormal trajectory points and maintain a balanced sampling frequency; Step 2: Trajectory clustering: Design a spatiotemporal neighbor trajectory segment clustering method to fully utilize the spatiotemporal density information of the trajectory to quickly and effectively identify and remove long-distance road driving trajectories. Then, the trajectories are grouped to help accurately capture the spatial characteristics of farmland trajectories in the subsequent semantic segmentation stage. Step 3: trajectory segmentation; By improving the traditional U-Net trajectory graph segmentation model and combining the channel and spatial mixed attention mechanism and focal loss function, the model is made suitable for the field road trajectory classification task, and the farmland trajectory and the road driving trajectory and drift trajectory close to the farmland plots can be segmented end-to-end based on clustering grouping.

2. The agricultural machinery field road trajectory classification method integrating spatiotemporal clustering and semantic segmentation as claimed in claim 1, characterized in that: In step 1, the trajectory preprocessing process is as follows: The first step is to fill the corresponding attributes of the trajectory points with empty values ​​for timestamp, longitude, latitude and speed with 0; The second step is to clean the resampled trajectories with the same time in consecutive moments, repeated trajectories with the same longitude and latitude coordinates in consecutive moments, static drift trajectories with different longitude and latitude coordinates and 0 speed in consecutive moments, and abnormal longitude and latitude trajectories beyond the territory of China according to the timestamp, longitude and latitude and speed of the trajectory points; The third step is to remove the agricultural machinery trajectory points whose speed values ​​are greater than the maximum operating speed speed_max one by one; The fourth step is to perform linear interpolation between two trajectory points whose time interval is greater than the sampling frequency f and less than the average waiting time for the red light t, with the sampling frequency as the interval.

3. The agricultural machinery field road trajectory classification method integrating spatiotemporal clustering and semantic segmentation as claimed in claim 1, characterized in that: In step 2, the trajectory clustering process is as follows: The first step is to recalculate the angular features and angular difference features of each trajectory point after trajectory preprocessing and the next trajectory point in the sequence, decompose the agricultural machinery trajectory with angular difference less than or equal to dir into r point segments, and mark each trajectory segment with serial number no according to the original trajectory time series; The second step is to make five fixed-length vertical segments at the starting point, 1 / 4 point, midpoint, 3 / 4 point and end point of each trajectory segment according to the direction angle of the corresponding trajectory point; the length of the vertical segment is set to extend along both sides of the trajectory point by 1 times the width of the agricultural machinery to reduce the judgment error; the width value of the agricultural machinery has been recorded in the trajectory attributes in advance by manual sampling; The third step is to detect the temporal and spatial neighbor relationship of the trajectory segments; determine whether at least one of the five vertical segments of each trajectory segment intersects with other trajectory segments. If there is an intersecting trajectory segment, then the maximum longitude difference lon_diff of all trajectory points within the time range of the two trajectory segments is calculated based on the sequence numbers of the two trajectory segments and the maximum longitude difference lon_diff of all trajectory points within the time range of the two trajectory segments. max , maximum latitude difference lat_diff max and the maximum time difference ts_diff max The attribute determines whether to keep the trajectory segment; if there is no intersecting trajectory segment, the trajectory segment needs to be eliminated; The fourth step is to group the retained trajectory segments, recalculate the distance between each trajectory point and the next trajectory point in the sequence, and calculate the distance threshold d according to the maximum operating speed speed_max of the agricultural machinery and the sampling frequency f. The trajectory points with a distance greater than d are used as segmentation points to achieve trajectory clustering.

4. The agricultural machinery field road trajectory classification method integrating spatiotemporal clustering and semantic segmentation as claimed in claim 1, characterized in that: In step 3, the trajectory segmentation process is as follows: The first step is to generate a trajectory image. The trajectory data is converted into a trajectory map whose pixel values ​​are determined by speed, acceleration and angular difference, and a mapping relationship between position coordinates and pixel coordinates is established. The second step is to generate segmented images. The segmented images are generated using the ground-truth categories of each labeled trajectory point. The third step is trajectory image segmentation. The segmentation model selects the U-Net framework as the backbone network, and adds a channel and spatial mixed attention mechanism on this basis. The improved semantic segmentation model is named CBGAM U-Net. The semantic segmentation model is trained using trajectory images and segmented images, and the training set loss function selects the weighted focal loss function. Finally, the network with the optimal parameters for classifying pixels into "farmland" and "road" is output as the image segmentation model.

Images