Agricultural machine track analysis method based on remote sensing image enhancement

Abstract

The invention provides an agricultural machine track analysis method based on remote sensing image enhancement, and relates to the technical field of artificial intelligence, and the method comprises the steps: obtaining track point data of an agricultural machine, a corresponding remote sensing image and a corresponding image description text, respectively extracting the features of three modes of a track, an image and a text, and carrying out the multi-mode fusion, and finally, classifying the fusion features, and respectively obtaining corresponding analysis results under three different trajectory analysis tasks of trajectory quality classification, field road trajectory point classification and turning-around classification. Through the method, the defects of an agricultural machinery track analysis method in the prior art are overcome, and the precision of agricultural machinery track analysis is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and particularly to a method for analyzing agricultural machinery trajectories based on remote sensing image enhancement. Background Art

[0002] By analyzing the operation process of agricultural machinery, the production cost of agriculture can be effectively reduced and its working efficiency can be improved. By installing a global navigation satellite system (GNSS) on agricultural machinery, its position at each moment can be determined. Therefore, a GNSS trajectory with continuous time can be generated during the driving of the agricultural machinery. Analyzing the driving trajectory of the agricultural machinery can identify its state at each moment (such as driving on the road, operating in the field or turning around). Further classifying the trajectory generated during the driving of the agricultural machinery into driving on the road, operating in the field and turning around in the field can provide a data basis for deeper analysis (such as operation area calculation, agricultural machinery transfer efficiency, etc.).

[0003] Currently, the existing methods for analyzing agricultural machinery trajectories have the following defects: First, the agricultural machinery trajectories lack consideration of background remote sensing images. Second, they do not use text to more deeply obtain the information contained in the remote sensing images and trajectory data. Third, when processing the characteristics of various types of trajectory data, only simple multi-modal feature fusion is performed for trajectory analysis and prediction. The above defects lead to low accuracy in agricultural machinery trajectory analysis. Summary of the Invention

[0004] The present invention provides a method for analyzing agricultural machinery trajectories based on remote sensing image enhancement to overcome the defects of the existing methods for analyzing agricultural machinery trajectories and improve the accuracy of agricultural machinery trajectory analysis.

[0005] The present invention provides a method for analyzing agricultural machinery trajectories based on remote sensing image enhancement, including: Obtaining trajectory point data of the driving of agricultural machinery, collecting corresponding remote sensing images for the trajectory point data according to the trajectory analysis task, and generating corresponding image description texts according to the remote sensing images; Respectively extracting the trajectory characteristics of the trajectory point data, the image characteristics of the remote sensing image, and the text characteristics of the image description text; Performing multi-modal feature fusion on the trajectory characteristics, the image characteristics, and the text characteristics to obtain fusion characteristics; When the trajectory analysis task is a trajectory-level trajectory quality classification task, performing classification processing on the fusion characteristics to obtain the quality category of the agricultural machinery trajectory; When the trajectory analysis task is a road - field trajectory point classification task and a U - turn classification task at the trajectory point level, classify the fused features to obtain corresponding road - field classification results and U - turn classification results.

[0006] In some embodiments, after obtaining the trajectory point data of the agricultural machine running, the method further includes: Pre - process the trajectory point data; The pre - processing process of the trajectory point data includes at least one of the following: Clear the duplicate trajectory points in the trajectory point data; Remove consecutive multiple trajectory points in the trajectory point data that represent the agricultural machine being stationary and having the same longitude and latitude; Clear consecutive multiple trajectory points in the trajectory point data that represent the agricultural machine being stationary and having different longitudes and latitudes; Delete the trajectory points in the trajectory point data whose longitude and latitude values exceed the preset value range.

[0007] In some embodiments, the collecting of the corresponding remote sensing image according to the trajectory analysis task for the trajectory point data includes: When the trajectory analysis task includes a road - field trajectory point classification task and a U - turn classification task at the trajectory point level, for each trajectory point in the trajectory point data, collect a corresponding remote sensing image, and the trajectory point is located at the center of the remote sensing image; When the trajectory analysis task includes a trajectory quality classification task at the trajectory level, determine the number of trajectory points of each trajectory in the trajectory point data; For each trajectory, determine the sampling positions of the trajectory point data according to the number of trajectory points and the sampling interval, and at each sampling position, collect the corresponding remote sensing image.

[0008] In some embodiments, the generating of the corresponding image description text according to the remote sensing image includes: For each remote sensing image, call a pre - trained large - scale vision - language model to generate text for the remote sensing image to obtain the corresponding image description text.

[0009] In some embodiments, performing multi - modal feature fusion on the trajectory feature, the image feature, and the text feature to obtain a fused feature includes: Input the trajectory feature, the image feature, and the text feature respectively integrated with position encoding into a Transformer model; In the Transformer model, construct a query vector according to the trajectory feature, and construct a key vector and a value vector according to the image feature and the text feature; Perform cross-modal attention mechanism calculation based on the query vector, the key vector, and the value vector to obtain a fused feature.

[0010] In some embodiments, the classifying the fused feature to obtain corresponding field-road classification results and U-turn classification results includes: Invoke a bidirectional long short-term memory network to perform serialization processing on the fused feature to obtain bidirectional sequence features; Perform mapping processing on the bidirectional sequence features through the activation function of a fully connected layer neural network to obtain field-road classification results corresponding to the field-road trajectory point classification task and U-turn classification results corresponding to the U-turn classification task.

[0011] The present invention also provides an agricultural machinery trajectory analysis device based on remote sensing image enhancement, including: An acquisition module, configured to acquire trajectory point data of agricultural machinery traveling, collect corresponding remote sensing images for the trajectory point data according to a trajectory analysis task, and generate corresponding image description texts according to the remote sensing images; An extraction module, configured to extract trajectory features of the trajectory point data, image features of the remote sensing image, and text features of the image description text respectively; A fusion module, configured to perform multi-modal feature fusion on the trajectory features, the image features, and the text features to obtain a fused feature; An analysis module, configured to, when the trajectory analysis task is a trajectory-level trajectory quality classification task, classify the fused feature to obtain the quality category of the agricultural machinery trajectory; When the trajectory analysis task is a trajectory point-level field-road trajectory point classification task and a U-turn classification task, classify the fused feature to obtain corresponding field-road classification results and U-turn classification results.

[0012] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where when the processor executes the computer program, it implements the agricultural machinery trajectory analysis method based on remote sensing image enhancement as described in any one of the above.

[0013] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the agricultural machinery trajectory analysis method based on remote sensing image enhancement as described in any one of the above.

[0014] The present invention also provides a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the agricultural machinery trajectory analysis method based on remote sensing image enhancement as described in any one of the above.

[0015] The agricultural machinery trajectory analysis method based on remote sensing image enhancement provided by the present invention uses the corresponding remote sensing images on the basis of trajectory prediction, and further supplements with image description texts to extract multi-modal features for agricultural machinery trajectory analysis, realizing a general agricultural machinery trajectory analysis framework based on remote sensing image enhancement, which can adapt to three different trajectory analysis tasks: trajectory quality classification, field-road trajectory point classification, and U-turn classification, and effectively improves the accuracy of agricultural machinery trajectory analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the accompanying drawings required for use in the description of the embodiments or the prior art one by one. Obviously, the accompanying drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other accompanying drawings can be obtained based on these drawings without creative efforts.

[0017] Figure 1 It is a flowchart of the agricultural machinery trajectory analysis method based on remote sensing image enhancement provided by the present invention.

[0018] Figure 2 It is a principle framework diagram of the agricultural machinery trajectory analysis method based on remote sensing image enhancement provided by the present invention.

[0019] Figure 3 It is a framework schematic diagram of multi-modal feature fusion provided by the present invention.

[0020] Figure 4 It is a framework diagram for realizing the classification task at the trajectory level provided by the present invention.

[0021] Figure 5 It is a framework diagram for realizing the classification task at the trajectory point level provided by the present invention.

[0022] Figure 6 It is a structural schematic diagram of the agricultural machinery trajectory analysis device based on remote sensing image enhancement provided by the present invention.

[0023] Figure 7 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0024] To make the objectives, technical solutions, and advantages of the present invention clearer, the following will clearly and completely describe the technical solutions in the present invention with reference to the accompanying drawings in the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art without creative efforts based on the embodiments in the present invention belong to the scope of protection of the present invention.

[0025] The method for analyzing agricultural machinery trajectory based on remote sensing image enhancement of the present invention will be described below with reference to the accompanying drawings. Figure 1 is a schematic flowchart of the method for analyzing agricultural machinery trajectory based on remote sensing image enhancement provided by the present invention. As Figure 1 shown, the method includes the following steps 101 to 105.

[0026] Step 101: Obtain the trajectory point data of the agricultural machinery traveling, collect the corresponding remote sensing images from the trajectory point data according to the trajectory analysis task, and generate the corresponding image description text based on the remote sensing images.

[0027] During the process of agricultural machinery operating and traveling in the farmland, the trajectory point data of the agricultural machinery traveling can be collected through sensors, GNSS and other positioning terminals, so as to construct a trajectory data set.

[0028] Since the GNSS signal is vulnerable to the influence of the farmland environment, such as vegetation, crop height and density, which may cause signal reflection or absorption, thus affecting the collection of trajectory data. Secondly, the irregularity of the farmland terrain and the changes on the ground during agricultural machinery operation will also cause signal multipath effects, resulting in measurement errors. In addition, there may be signal occlusion during the agricultural machinery operation process, such as due to the presence of large agricultural machinery or dense vegetation, which will affect the quality of the satellite signal received by the agricultural machinery positioning terminal. And due to agricultural machinery failures, insufficient or inconsistent sampling rates, the accuracy of the trajectory data may also be relatively low.

[0029] Based on the above scenarios, as Figure 2 shown, after the trajectory point data of the agricultural machinery traveling is obtained in the embodiment of the present invention, preprocessing is performed on the trajectory point data. The preprocessing process of the trajectory point data includes at least one of the following: clearing duplicate trajectory points, removing stationary trajectory points, processing static drift trajectory points, and deleting abnormal longitude and latitude trajectory points, which will be described one by one below.

[0030] Specifically, clearing duplicate trajectory points is to clear the duplicate trajectory points existing in the trajectory point data. Duplicate trajectory points refer to the trajectory points that continuously appear in the trajectory data and have the same longitude and latitude and a non-zero speed. These trajectory points may be data redundancies caused by data acquisition equipment errors or other factors. To ensure the accuracy and reliability of the data, these duplicate trajectory points should be cleared during the data preprocessing stage, and only the first trajectory point is retained. This helps to avoid repeated calculations of the data at the same position in subsequent data analysis, improving the efficiency and credibility of data processing.

[0031] Removing stationary trajectory points specifically refers to removing multiple consecutive trajectory points in the trajectory point data that represent the agricultural machinery being stationary and having the same longitude and latitude. A stationary trajectory point refers to a trajectory point that continuously appears in the trajectory data, has the same longitude and latitude, and a speed of zero. A speed of zero indicates that the agricultural machinery is stationary. These trajectory points are usually generated due to the long-term stay of the agricultural machinery at a certain position and have no practical significance for the analysis of the agricultural machinery operation mode. Therefore, during the data preprocessing process, these stationary trajectory points should be removed, and only the first trajectory point within consecutive time moments should be retained. This can avoid introducing invalid or misleading information in subsequent model training and ensure the accuracy and reliability of the experimental results.

[0032] Processing static drift trajectory points specifically refers to clearing multiple consecutive trajectory points in the trajectory point data that represent the agricultural machinery being stationary and having different longitudes and latitudes. A static drift trajectory point refers to a trajectory point at consecutive time moments with a speed of zero and different longitudes and latitudes. This may be due to signal interference causing errors in the data acquisition device. Such an abnormal situation may lead to inaccurate position information and mislead the analysis of the agricultural machinery driving trajectory. Processing static drift trajectory points helps to ensure the spatial continuity and stability of the data.

[0033] Deleting trajectory points with abnormal longitude and latitude specifically refers to deleting trajectory points in the trajectory point data whose longitude and latitude values exceed the preset value range. A trajectory point with abnormal longitude and latitude refers to a trajectory point in the trajectory data whose collected longitude and latitude values exceed the reasonable range, that is, the longitude and latitude values exceed the preset value range. This may be caused by GNSS signal drift, equipment failure, or other problems during the data acquisition process. During the data preprocessing stage, these trajectory points with abnormal longitude and latitude should be deleted to ensure the accuracy of the data. Specifically, if the longitude and latitude values of a trajectory point exceed the geographical range of the region or differ significantly from the longitude and latitude ranges of most trajectory points in the trajectory sequence, then these abnormal trajectory points should be deleted. This can effectively eliminate abnormal data and improve the accuracy of the experimental results.

[0034] Thus, through the process of trajectory data preprocessing, the finally obtained cleaned trajectory dataset can be used as trajectory point data for subsequent trajectory analysis. When conducting trajectory analysis in the embodiments of the present invention, the background remote sensing image corresponding to the trajectory point is used. However, due to different trajectory analysis tasks, there are also some differences in the background remote sensing images used.

[0035] Such as Figure 2As shown in the figure, in data engineering, remote sensing images corresponding to trajectory point data are collected according to trajectory analysis tasks. First, trajectory analysis tasks are divided into trajectory-level tasks and trajectory-point-level tasks. Trajectory-level tasks are specifically trajectory quality classification tasks, whose purpose is to classify the quality of the trajectories of agricultural machinery operations, specifically into three categories: high quality, medium quality, and low quality. Trajectory-point-level tasks include field-road trajectory point classification tasks and turning classification tasks, which belong to sequence characterization tasks, and their purpose is to classify trajectory points and predict the categories of the driving trajectories in the field road (such as straight trajectories, curved trajectories) and the turning (i.e., changing the operation direction to the opposite) situation during agricultural machinery operations.

[0036] Before obtaining remote sensing images, the corresponding trajectory analysis tasks need to be determined. When the trajectory analysis tasks include field-road trajectory point classification tasks and turning classification tasks at the trajectory-point level, for each trajectory point in the trajectory point data, a corresponding remote sensing image is collected, and the trajectory point is located at the center of the remote sensing image.

[0037] Here, a corresponding remote sensing image needs to be collected for each trajectory point. The collection method is to call the remote sensing image acquisition API according to the longitude and latitude position information of the trajectory data, and obtain the corresponding remote sensing image from the public service platform for geographic information (Tianditu). And the trajectory point is located at the center of the remote sensing image, so as to ensure that the remote sensing image can express the complete trajectory point features. Since the public service platform for geographic information provides relevant API services, when collecting remote sensing images, the background remote sensing image data is obtained according to the longitude and latitude corresponding to each trajectory point. On this basis, due to the huge number of trajectory points, in order to ensure the operation efficiency of the subsequent trajectory analysis process, a 128 128 small image is obtained as the remote sensing image for each trajectory point.

[0038] When the trajectory analysis task includes the trajectory quality classification task at the trajectory level, determine the number of trajectory points of each trajectory in the trajectory point data. The trajectory point data includes multiple trajectories, and only the number of corresponding trajectory points can be counted for each trajectory. Because the number of trajectory points collected in the trajectory-level classification task needs to be sufficient to represent the characteristics of the entire trajectory, the collected remote sensing images need to include multiple trajectory points. For this reason, in the embodiment of the present invention, for each trajectory, the sampling positions of the trajectory point data are determined according to the number of trajectory points and the sampling interval. This sampling position is essentially also the position of the trajectory point. Since the number of sampling positions of each trajectory is known, the ratio of the number of trajectory points to the number of sampling positions is the sampling interval, and each sampling position can be determined in turn according to the sampling interval. Finally, at each sampling position, the corresponding remote sensing image is collected, that is, the corresponding remote sensing image is collected at the trajectory point position corresponding to the sampling position.

[0039] For example, for a certain trajectory in the trajectory point data, the corresponding number of trajectory points is counted as 100, and the sampling number N of this trajectory is generally taken as 10. Then the sampling interval of this trajectory is 100 / 10 = 10. Then, based on the number of trajectory points 100 and the sampling interval 10, 11 sampling positions can be determined in sequence, which are 1, 10, 20, 30, ..., 100. Finally, according to the determined 11 sampling positions, the remote sensing images corresponding to each sampling position are obtained respectively. The acquisition method of the remote sensing image is the same as that of the image acquisition at the trajectory point level, and it is still obtained by calling the API service provided by the geographic information public service platform.

[0040] After collecting the corresponding remote sensing images through the above process, the corresponding image description text is generated based on the remote sensing images, and the generation method can borrow a pre-trained large vision-language model. Specifically, for each remote sensing image, the pre-trained large vision-language model is called to generate text for the remote sensing image, and the corresponding image description text is obtained. The image description text is used to describe the image content of the remote sensing image, which can help the remote sensing image present richer semantic information and summarize the image features at the semantic level. The large vision-language model is specifically a large language model based on Transformer, for example, it can be BLIP2.

[0041] The embodiment of the present invention uses a large vision-language model to generate text descriptions about remote sensing images, further expanding the modal types of agricultural machinery trajectory analysis data and enriching the feature types contained in agricultural machinery trajectory data.

[0042] Step 102: Extract the trajectory features of the trajectory point data, the image features of the remote sensing image, and the text features of the image description text respectively.

[0043] As Figure 2 shown, after obtaining the remote sensing image and generating the corresponding image description text through data engineering, the next step is the process of trajectory analysis. The first step of trajectory analysis is feature extraction, that is, to extract the trajectory features of the trajectory point data, the image features of the remote sensing image, and the text features of the image description text respectively. The cleaned trajectory data set is encoded through a trajectory encoder to obtain the corresponding trajectory features. The trajectory encoder can be a time series encoder or a hybrid Fourier encoder. The trajectory features contain important information of the trajectory data, such as the direction, speed, acceleration, etc. of the trajectory. The remote sensing image can be used to extract image features through a vision encoder, and the vision encoder can be a Transformer model or a convolutional neural network. The image description text extracts text features through a text encoder, and the text encoder can also be a Transformer model.

[0044] Step 103: Perform multi-modal feature fusion on the trajectory features, image features, and text features to obtain fusion features.

[0045] As Figure 2 shown, the second step of trajectory analysis is to perform multi-modal feature fusion. The process of multi-modal fusion is to fuse the trajectory features, text features, and image features obtained from feature extraction to enhance the representation ability of the data and more efficiently utilize multi-modal data to complete the agricultural machinery trajectory analysis task. The following specifically describes the process of multi-modal feature fusion.

[0046] First, the trajectory features, image features, and text features respectively incorporated with positional encoding are input into the Transformer model.

[0047] Here, referring to Figure 3 , first, the trajectory features, image features, and text features to be fused are respectively input through a linear layer, and then the corresponding positional encoding is incorporated into the trajectory features, image features, and text features respectively, so as to ensure that the features to be fused carry time information. When performing feature fusion, the trajectory features, image features, and text features respectively incorporated with positional encoding are processed into the same dimension and then input into the Transformer model to utilize the cross-modal attention mechanism to achieve multi-modal fusion.

[0048] The embodiment of the present invention uses a Transformer model based on the cross-modal attention mechanism to achieve multi-modal feature fusion. The Transformer model specifically has N layers of networks. In each layer, a normalization layer is used to normalize each feature respectively. Next, a query vector is constructed based on the trajectory features, and key vectors and value vectors are constructed based on the image features and text features. In the cross-modal attention mechanism, the normalized image features and text features are both used to construct the corresponding key vectors and value vectors, while the normalized trajectory features are used to construct the query vector.

[0049] Finally, cross-modal attention mechanism calculation is performed based on the query vector, key vector, and value vector to obtain the fused features. Specifically, in the cross-modal attention mechanism of the Transformer model, a cross-attention form is adopted. On the one hand, cross-modal attention mechanism calculation is performed based on the key vector and value vector constructed from the image features and the query vector constructed from the trajectory features to focus on the key feature information of the remote sensing image. On the other hand, cross-modal attention mechanism calculation is performed based on the key vector and value vector constructed from the text features and the query vector constructed from the trajectory features to generate a more accurate trajectory representation in combination with the text features. Subsequently, the feature information generated in the two parts is respectively input into a multi-layer perceptron through a normalization layer for feed-forward calculation, and finally, mapping processing is performed through the multi-layer perceptron to output the final fused features.

[0050] In the embodiments of the present invention, by performing multi-modal feature fusion on trajectory features, image features, and text features, and enhancing the representation ability of trajectory features with the aid of image-text features, multi-modal data is utilized more efficiently to complete the agricultural machinery trajectory analysis task.

[0051] Step 104: When the trajectory analysis task is a trajectory-level trajectory quality classification task, classify the fused features to obtain the quality category of the agricultural machinery trajectory.

[0052] As Figure 2 shown, after achieving multi-modal feature fusion, finally use the obtained fused features to perform subsequent trajectory analysis tasks. According to the above description, the trajectory analysis tasks are divided into trajectory level and trajectory point level, corresponding to trajectory classification and trajectory point classification.

[0053] When the trajectory analysis task is a trajectory-level trajectory quality classification task, at this time, trajectory classification needs to be performed. Classify the fused features to obtain the quality category of the agricultural machinery trajectory, which is divided into three categories: high quality, medium quality, and low quality.

[0054] Specifically, referring to Figure 4 shown, Figure 4 shows the implementation framework of the trajectory quality classification task. For trajectory data, remote sensing images, and image description texts, use a trajectory encoder, a visual encoder, and a text encoder to perform trajectory point encoding, image encoding, and text encoding respectively, and then perform multi-modal feature fusion on the obtained trajectory features, image features, and text features to output fused features. In the trajectory quality classification task, first perform mapping processing on the fused features through a multi-layer perceptron, then input the mapping processing result into a fully connected layer neural network for processing, and finally perform final mapping processing on the output result of the fully connected layer through the activation function softmax to obtain the quality category of the agricultural machinery trajectory, thus realizing the quality classification of trajectory point data, including three categories: high quality, medium quality, and low quality.

[0055] Step 105: When the trajectory analysis task is a trajectory point-level field-road trajectory point classification task and a U-turn classification task, classify the fused features to obtain the field-road classification result and the U-turn classification result.

[0056] When the trajectory analysis task is a trajectory point-level field-road trajectory point classification task and a U-turn classification task, at this time, trajectory point classification needs to be performed. Classify the fused features to obtain the corresponding field-road classification result and the U-turn classification result.

[0057] Specifically, referring to Figure 5 shown, Figure 5It shows the implementation framework of the trajectory quality classification task. For trajectory data, remote sensing images, and image description texts, a trajectory encoder, a visual encoder, and a text encoder are used to perform trajectory point encoding, image encoding, and text encoding respectively. Then, the obtained trajectory features, image features, and text features are fused for multi-modal feature fusion, and the fused features are output. Further, classification processing is performed on the fused features, and the obtained classification results are the corresponding field-road classification results and U-turn classification results.

[0058] Specifically, before classification processing, a bidirectional long short-term memory network is first called to serialize the fused features to obtain bidirectional sequence features. Trajectory point classification needs to consider the feature information existing between trajectory points. Since each trajectory point and its adjacent trajectory points in the trajectory data are collected sequentially, there is a temporal relationship between each trajectory point. That is to say, the category of a certain trajectory point can affect the category prediction of its surrounding trajectory points. Therefore, a bidirectional long short-term memory network is used here to extract bidirectional sequence features containing the temporal relationship between trajectory points for field-road trajectory segmentation.

[0059] As Figure 5 shown, during classification processing, the bidirectional sequence features output by the bidirectional long short-term memory network are input into a fully connected layer neural network for processing, and then the output result of the fully connected layer is mapped through the activation function softmax to obtain the final classification results, including the field-road classification results corresponding to the field-road trajectory point classification task and the U-turn classification results corresponding to the U-turn classification task.

[0060] In the embodiments of the present invention, for different trajectory analysis tasks, based on multi-modal feature fusion, different classification methods are adopted to obtain the corresponding analysis results, so as to improve the performance of trajectory quality classification, field-road trajectory segmentation, and in-field U-turn area trajectory recognition tasks.

[0061] In the embodiments of the present invention, corresponding remote sensing images are used based on trajectory and trajectory point classification, and further supplemented with image description texts to extract multi-modal features for agricultural machinery trajectory analysis, realizing a general agricultural machinery trajectory analysis framework based on remote sensing image enhancement, which can adapt to three different trajectory analysis tasks: trajectory quality classification, field-road trajectory point classification, and U-turn classification, and effectively improves the accuracy of agricultural machinery trajectory analysis.

[0062] Next, the agricultural machinery trajectory analysis device based on remote sensing image enhancement provided by the present invention will be described. The agricultural machinery trajectory analysis device based on remote sensing image enhancement described below can be mutually referenced with the agricultural machinery trajectory analysis method based on remote sensing image enhancement described above.

[0063] As Figure 6As shown in the figure, the agricultural machinery trajectory analysis device based on remote sensing image enhancement includes an acquisition module 601, an extraction module 602, a fusion module 603, and an analysis module 604. Specifically, the acquisition module 601 is used to obtain the trajectory point data of the agricultural machinery traveling, collect the corresponding remote sensing images for the trajectory point data according to the trajectory analysis task, and generate the corresponding image description text according to the remote sensing images; the extraction module 602 is used to extract the trajectory features of the trajectory point data, the image features of the remote sensing images, and the text features of the image description text respectively; the fusion module 603 is used to perform multi-modal feature fusion on the trajectory features, the image features, and the text features to obtain fusion features; the analysis module 604 is used to perform classification processing on the fusion features to obtain the quality category of the agricultural machinery trajectory when the trajectory analysis task is a trajectory-level trajectory quality classification task; when the trajectory analysis task is a trajectory point-level field-road trajectory point classification task and a U-turn classification task, perform classification processing on the fusion features to obtain the corresponding field-road classification result and U-turn classification result.

[0064] It should be noted that the beneficial effects of the agricultural machinery trajectory analysis device based on remote sensing image enhancement here correspond to those of the agricultural machinery trajectory analysis method based on remote sensing image enhancement in the above text. Therefore, the beneficial effects of the agricultural machinery trajectory analysis device based on remote sensing image enhancement will not be elaborated here.

[0065] Figure 7 An entity structure diagram of an electronic device is exemplified, as Figure 7 shown, the electronic device may include: a processor 710, a communication interface 720, a memory 730, and a communication bus 740. Among them, the processor 710, the communication interface 720, and the memory 730 complete communication with each other through the communication bus 740. The processor 710 can call the logical instructions in the memory 730 to execute the agricultural machinery trajectory analysis method based on remote sensing image enhancement. The method includes: obtaining the trajectory point data of the agricultural machinery traveling, collecting the corresponding remote sensing images for the trajectory point data according to the trajectory analysis task, and generating the corresponding image description text according to the remote sensing images; extracting the trajectory features of the trajectory point data, the image features of the remote sensing images, and the text features of the image description text respectively; performing multi-modal feature fusion on the trajectory features, the image features, and the text features to obtain fusion features; when the trajectory analysis task is a trajectory-level trajectory quality classification task, perform classification processing on the fusion features to obtain the quality category of the agricultural machinery trajectory; when the trajectory analysis task is a trajectory point-level field-road trajectory point classification task and a U-turn classification task, perform classification processing on the fusion features to obtain the corresponding field-road classification result and U-turn classification result.

[0066] In addition, when the logical instructions in the above-mentioned memory 730 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0067] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the agricultural machinery trajectory analysis method based on remote sensing image enhancement provided by the above-mentioned various methods. The method includes: acquiring trajectory point data of the agricultural machinery traveling, collecting corresponding remote sensing images for the trajectory point data according to the trajectory analysis task, and generating corresponding image description texts according to the remote sensing images; respectively extracting the trajectory features of the trajectory point data, the image features of the remote sensing images, and the text features of the image description texts; performing multi-modal feature fusion on the trajectory features, the image features, and the text features to obtain fusion features; when the trajectory analysis task is a trajectory-level trajectory quality classification task, performing classification processing on the fusion features to obtain the quality category of the agricultural machinery trajectory; when the trajectory analysis task is a trajectory point-level field-road trajectory point classification task and a U-turn classification task, performing classification processing on the fusion features to obtain corresponding field-road classification results and U-turn classification results.

[0068] In another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements a method for analyzing agricultural machinery trajectories based on remote sensing image enhancement provided by the above-mentioned various methods. The method includes: obtaining trajectory point data of the agricultural machinery traveling, collecting corresponding remote sensing images for the trajectory point data according to the trajectory analysis task, and generating corresponding image description texts according to the remote sensing images; respectively extracting the trajectory features of the trajectory point data, the image features of the remote sensing images, and the text features of the image description texts; performing multi-modal feature fusion on the trajectory features, the image features, and the text features to obtain fusion features; when the trajectory analysis task is a trajectory-level trajectory quality classification task, performing classification processing on the fusion features to obtain the quality category of the agricultural machinery trajectory; when the trajectory analysis task is a trajectory point-level field-road trajectory point classification task and a U-turn classification task, performing classification processing on the fusion features to obtain corresponding field-road classification results and U-turn classification results.

[0069] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0070] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disc, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0071] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of each embodiment of the present invention.

Claims

1. A method for analyzing agricultural machinery trajectories based on remote sensing image enhancement, characterized in that, Including: Obtain the trajectory point data of the agricultural machinery, collect the corresponding remote sensing images for the trajectory point data according to the trajectory analysis task, and generate the corresponding image description text according to the remote sensing images; Extract the trajectory features of the trajectory point data, the image features of the remote sensing images, and the text features of the image description text respectively; Perform multi-modal feature fusion on the trajectory features, the image features, and the text features to obtain fusion features; When the trajectory analysis task is a trajectory-level trajectory quality classification task, perform classification processing on the fusion features to obtain the quality category of the agricultural machinery trajectory; When the trajectory analysis task is a trajectory point-level field-road trajectory point classification task and a U-turn classification task, perform classification processing on the fusion features to obtain the corresponding field-road classification result and U-turn classification result.

2. The method for analyzing agricultural machinery trajectories based on remote sensing image enhancement according to claim 1, wherein After obtaining the trajectory point data of the agricultural machinery traveling, the method further includes: Perform preprocessing on the trajectory point data; The preprocessing process of the trajectory point data includes at least one of the following: Clear the trajectory points with duplicates in the trajectory point data; Remove consecutive multiple trajectory points in the trajectory point data that represent the agricultural machinery being stationary and having the same longitude and latitude; Clear consecutive multiple trajectory points in the trajectory point data that represent the agricultural machinery being stationary and having different longitudes and latitudes; Delete the trajectory points in the trajectory point data whose longitude and latitude values exceed the preset value range.

3. The method for analyzing agricultural machinery trajectories based on remote sensing image enhancement according to claim 1, wherein, The collecting the corresponding remote sensing images for the trajectory point data according to the trajectory analysis task includes: When the trajectory analysis task includes a trajectory point-level field-road trajectory point classification task and a U-turn classification task, for each trajectory point in the trajectory point data, collect a corresponding remote sensing image, and the trajectory point is located at the center position of the remote sensing image; When the trajectory analysis task includes a trajectory-level trajectory quality classification task, determine the number of trajectory points of each trajectory in the trajectory point data; For each trajectory, determine the sampling positions of the trajectory point data according to the number of trajectory points and the sampling interval, and at each sampling position, collect the corresponding remote sensing image.

4. The agricultural machinery trajectory analysis method based on remote sensing image enhancement according to claim 1, wherein The generating the corresponding image description text according to the remote sensing images includes: For each remote sensing image, call a pre-trained large-scale vision-language model to generate text for the remote sensing image to obtain the corresponding image description text.

5. The method for analyzing agricultural machinery trajectories based on remote sensing image enhancement according to claim 1, wherein Performing multi-modal feature fusion on the trajectory features, the image features, and the text features to obtain fusion features includes: Input the trajectory features, image features, and text features respectively incorporated with position encoding into a Transformer model; In the Transformer model, construct a query vector according to the trajectory features, and construct key vectors and value vectors according to the image features and the text features; Perform cross-modal attention mechanism calculation according to the query vector, the key vectors, and the value vectors to obtain fusion features.

6. The method for analyzing agricultural machinery trajectories based on remote sensing image enhancement according to claim 1, characterized in that The performing classification processing on the fusion features to obtain the corresponding field-road classification result and U-turn classification result includes: Call a bidirectional long short-term memory network to perform serialization processing on the fusion features to obtain bidirectional sequence features; The bidirectional sequence features are mapped through the activation function of the fully connected layer neural network to obtain the field path classification result corresponding to the field path point classification task and the U-turn classification result corresponding to the U-turn classification task.

7. An agricultural machinery trajectory analysis device based on remote sensing image enhancement, characterized in that, It includes: An acquisition module, configured to acquire the trajectory point data of the agricultural machinery driving, collect the corresponding remote sensing image for the trajectory point data according to the trajectory analysis task, and generate the corresponding image description text according to the remote sensing image; An extraction module, configured to extract the trajectory features of the trajectory point data, the image features of the remote sensing image, and the text features of the image description text respectively; A fusion module, configured to perform multi-modal feature fusion on the trajectory features, the image features, and the text features to obtain fusion features; An analysis module, configured to perform classification processing on the fusion features to obtain the quality category of the agricultural machinery trajectory when the trajectory analysis task is a trajectory-level trajectory quality classification task; When the trajectory analysis task is a trajectory point-level field path point classification task and a U-turn classification task, perform classification processing on the fusion features to obtain the corresponding field path classification result and U-turn classification result.

8. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the agricultural machinery trajectory analysis method based on remote sensing image enhancement according to any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is executed by the processor, it implements the agricultural machinery trajectory analysis method based on remote sensing image enhancement according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the agricultural machinery trajectory analysis method based on remote sensing image enhancement according to any one of claims 1 to 6.

Images