Agricultural machinery intelligent driving data automatic labeling method based on visual large model and related equipment

Abstract

The invention discloses an agricultural machinery intelligent driving data automatic labeling method based on a visual large model and related equipment, and relates to the technical field of agricultural machinery intelligent driving data labeling. The method comprises the following steps: constructing a labeling model based on the visual large model; the method comprises the steps of obtaining a to-be-labeled data set in an agricultural machine intelligent driving scene, selecting a part of samples from the to-be-labeled data set for manual labeling to form a labeled data set, and forming an unlabeled data set by the remaining samples; performing fine tuning training on the annotation model by using the annotated data set, and in the fine tuning training, freezing parameters of a front part network layer of a backbone network of the annotation model, and only updating parameters of a rear part network layer and a task output module; and automatically labeling the unlabeled data set by using the fine-tuned labeling model. According to the method, on the premise of ensuring the labeling quality, the dependence on manual labeling can be greatly reduced, and the labeling cost and period can be reduced.

Description

Technical Field

[0001] This invention relates to the field of agricultural machinery intelligent driving data annotation technology, and in particular to an automatic annotation method and related equipment for agricultural machinery intelligent driving data based on a large visual model. Background Technology

[0002] In recent years, with the rapid development of artificial intelligence technology, intelligent and automated agricultural machinery and equipment have emerged and are gradually being put into practical application in agricultural production. As a key technological component of intelligent agricultural machinery, the training of intelligent driving algorithms has long relied on manual data annotation. This reliance results in a very long algorithm development cycle and high human resource costs, severely limiting the practical implementation and widespread application of related technology projects.

[0003] To address the aforementioned issues, automatic labeling methods based on dedicated detection models have gradually matured and begun to be applied in the processing of agricultural machinery intelligent driving data. The typical workflow of this existing method is as follows: First, a small portion of the data to be labeled, approximately 10% to 20% of the total, is manually labeled; then, a dedicated target detection model is trained using this labeled data; finally, this trained dedicated detection model is used to predict the labeling results of the remaining large amount of data, and the prediction results are corrected by manual review. The parts with better prediction results are directly adopted, while the parts with poorer prediction results require manual intervention for modification.

[0004] However, while this automatic annotation method based on a dedicated detection model can effectively shorten the overall annotation time, it still has two prominent drawbacks. The first is the generally low automatic annotation rate, and the second is the poor robustness of the model. Specifically, when the environmental conditions of the data to be annotated, such as light intensity, weather conditions, and farmland background, differ from the data used to train the dedicated detection model, the model's annotation performance drops significantly. This results in a low usability of the automatic annotation results, ultimately requiring significant manpower for subsequent checking and correction, failing to truly achieve the goal of high-efficiency, low-cost annotation.

[0005] Therefore, the field of intelligent agricultural machinery driving data annotation urgently needs a new method that can maintain high annotation rates and strong robustness in changing environments. The development of large visual model technology provides a new solution to this need. Large visual models, through self-supervised learning on massive unlabeled real-world image datasets, have mastered extremely rich general visual features and possess powerful feature extraction and generalization capabilities. Building automatic annotation tools based on large visual models holds the promise of fundamentally improving the automation level and adaptability of intelligent agricultural machinery driving data annotation. Summary of the Invention

[0006] The technical problem to be solved by this invention is to address the shortcomings of existing technologies. Specifically, it provides an automatic annotation method and related equipment for intelligent agricultural machinery driving data based on a large visual model, as detailed below: 1) In a first aspect, the present invention provides an automatic annotation method for agricultural machinery intelligent driving data based on a large visual model, the specific technical solution of which is as follows: A labeled model based on a large visual model is constructed. The large visual model is pre-trained on a real-world image dataset through self-supervised learning and has the ability to extract visual features. Obtain the unlabeled dataset in the intelligent driving scenario of agricultural machinery, select a portion of the samples from the unlabeled dataset for manual labeling to form a labeled dataset, and combine the remaining samples into an unlabeled dataset; The labeled model is fine-tuned using the labeled dataset. During fine-tuning, the parameters of the first part of the backbone network of the labeled model are frozen, and only the parameters of the last part of the network layer and the task output module are updated. We use a finely tuned annotation model to automatically annotate unlabeled datasets.

[0007] The beneficial effects of the automatic annotation method for intelligent agricultural machinery driving data based on a large visual model provided by this invention are as follows: By constructing a labeling model based on a large visual model and leveraging its powerful visual feature extraction capabilities obtained through pre-training on massive real-world image datasets, this method can identify more general and stable features from the unlabeled data. This enables the automatic labeling process to maintain high accuracy and consistency even in the context of agricultural machinery intelligent driving, despite changes in lighting, weather, and farmland background, thereby improving the overall automatic labeling rate. Furthermore, by using a small amount of labeled dataset to perform targeted fine-tuning training on the labeling model, and employing a strategy of freezing the first part of the backbone network layers and only updating the parameters of the last part of the network layers and the task output module, this method can retain the original generalization ability of the large visual model while introducing specific knowledge of agricultural machinery scenarios. Finally, using the fine-tuned labeling model to automatically label unlabeled datasets can significantly reduce the reliance on manual labeling while ensuring labeling quality, reducing labeling costs and time, and providing efficient and reliable data support for the large-scale training of agricultural machinery intelligent driving algorithms.

[0008] Based on the above scheme, the automatic annotation method for intelligent agricultural machinery driving data based on a large visual model of the present invention can be further improved as follows.

[0009] Furthermore, it also includes: filtering the automatic annotation results based on a preset confidence threshold, saving the annotation results with a confidence level higher than the preset confidence threshold, and manually checking and correcting the annotation results with a confidence level lower than the preset confidence threshold.

[0010] The beneficial effects of adopting the above-mentioned further approach are: by saving annotation results with confidence levels higher than the preset confidence threshold, a large number of high-quality annotations can be automatically adopted, directly reducing the workload of manual review. Simultaneously, annotation results with confidence levels below the preset confidence threshold are manually checked and corrected, ensuring the overall reliability of the final annotation dataset. This method optimizes the allocation of human resources while ensuring annotation quality, avoiding manual inspection of all results one by one, thereby reducing the time cost of the quality inspection stage and improving the efficiency of the entire automatic annotation process.

[0011] Furthermore, the large visual model is the DINOv2 model. During fine-tuning training, the parameters of the first 20 layers of the backbone network of the DINOv2 model are frozen, and the parameters of layers 21 to 24 and the task output module are updated.

[0012] The beneficial effects of adopting the above-mentioned further approach are as follows: By freezing the parameters of the first 20 layers of the DINOv2 model backbone network, the general visual feature extraction capabilities learned by the model during the pre-training phase are fully preserved. Simultaneously, by updating the parameters of layers 21 to 24 of the DINOv2 model backbone network and the task output module, the model can adaptively adjust to the specific needs of agricultural machinery intelligent driving scenarios. This strategy allows the model to utilize its existing strong generalization foundation while efficiently learning specific patterns in agricultural machinery data during fine-tuning training. This improves annotation accuracy while effectively avoiding overfitting caused by over-adjustment, ensuring the stability and reliability of the labeled model in real-world, variable environments.

[0013] Furthermore, before fine-tuning the labeled model using the labeled dataset, the process includes: preprocessing the labeled dataset and performing adaptive resolution adjustment on the unlabeled dataset.

[0014] The beneficial effects of adopting the above-mentioned further solutions are as follows: By preprocessing the labeled dataset, the quality and diversity of the labeled data can be effectively improved, thereby enhancing the learning effect and generalization ability of the labeled model in fine-tuning training. Simultaneously, performing adaptive resolution adjustment on the unlabeled dataset ensures that all images in the unlabeled dataset have a uniform input size, meeting the input format requirements of the fine-tuned labeled model. These two operations together provide a standardized and high-quality data foundation for subsequent fine-tuning training and automatic annotation processes, ensuring the stability of model training and the smooth progress of the automatic annotation process, thereby improving the reliability and consistency of the overall annotation results.

[0015] 2) Secondly, the present invention also provides an automatic labeling system for intelligent agricultural machinery driving data based on a large visual model, the specific technical solution of which is as follows: It includes a model building module, a dataset processing module, a fine-tuning training module, and an automatic annotation module; The model building module is used to build labeled models based on large visual models. These large visual models are pre-trained on real-world image datasets through self-supervised learning and have the ability to extract visual features. The dataset processing module is used to: acquire the dataset to be labeled in the intelligent driving scenario of agricultural machinery, select a portion of the samples from the dataset to be labeled for manual labeling to form a labeled dataset, and combine the remaining samples into an unlabeled dataset; The fine-tuning training module is used to fine-tune the labeled model using the labeled dataset. During fine-tuning training, the parameters of the first part of the backbone network of the labeled model are frozen, and only the parameters of the last part of the network layer and the task output module are updated. The automatic annotation module is used to automatically annotate unlabeled datasets using a fine-tuned annotation model.

[0016] Based on the above scheme, the automatic labeling system for intelligent agricultural machinery driving data based on a large visual model of the present invention can be further improved as follows.

[0017] Furthermore, it also includes an inspection and correction module, which is used to: filter the automatic annotation results based on a preset confidence threshold, save the annotation results with a confidence level higher than the preset confidence threshold, and manually inspect and correct the annotation results with a confidence level lower than the preset confidence threshold.

[0018] Furthermore, the large visual model is the DINOv2 model. During fine-tuning training, the parameters of the first 20 layers of the backbone network of the DINOv2 model are frozen, and the parameters of layers 21 to 24 and the task output module are updated.

[0019] Furthermore, it also includes a preprocessing module, which is used to preprocess the labeled dataset before fine-tuning the labeled model using the labeled dataset, and to perform adaptive resolution adjustment on the unlabeled dataset.

[0020] 3) In a third aspect, the present invention also provides an electronic device, the electronic device including a processor coupled to a memory, the memory storing at least one computer program, the at least one computer program being loaded and executed by the processor, so that the electronic device implements any of the above-mentioned methods for automatic labeling of agricultural machinery intelligent driving data based on a large visual model.

[0021] 4) In a fourth aspect, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the above-mentioned methods for automatic annotation of agricultural machinery intelligent driving data based on a large visual model.

[0022] It should be noted that the beneficial effects of the technical solutions of the second to fourth aspects of the present invention and their corresponding possible implementations can be found in the above description of the technical effects of the first aspect and its corresponding possible implementations, and will not be repeated here. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments of the present invention will be briefly introduced below: Figure 1 This is one of the flowcharts illustrating an automatic data annotation method for intelligent agricultural machinery driving based on a large visual model, according to an embodiment of the present invention. Figure 2 This is a second flowchart illustrating an automatic data annotation method for intelligent agricultural machinery driving based on a large visual model, according to an embodiment of the present invention. Figure 3 Example image of the annotation results; Figure 4 This is a schematic diagram of the structure of an automatic labeling system for intelligent agricultural machinery driving data based on a large visual model, according to an embodiment of the present invention. Detailed Implementation

[0024] The principles and features of the present invention are described below. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0025] The technical solution of the present invention and how the technical solution of the present invention solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of the present invention will now be described with reference to the accompanying drawings.

[0026] like Figure 1 As shown in the figure, an automatic annotation method for intelligent agricultural machinery driving data based on a large visual model according to an embodiment of the present invention includes the following steps: S1. Construct a labeled model based on a large visual model. The large visual model is pre-trained on a real-world image dataset through self-supervised learning and has the ability to extract visual features. The specific implementation process is as follows: S10. Select a suitable large-scale visual model as the foundational architecture for the entire annotation model. A large-scale visual model is a deep learning model with a massive parameter scale that gains powerful visual understanding capabilities through training on massive amounts of data. Based on the technical solution, the specific model chosen here is the DINOv2 model. The DINOv2 model has been pre-trained on an extremely wide range of image data through self-supervised learning. The DINOv2 model was chosen as the foundation because it performs excellently in learning general visual features, providing a good starting point for subsequent adaptation to agricultural machinery scenarios. The output of this step is a basic model framework with general visual knowledge, without being tailored to a specific task.

[0027] S11. The core of the large-scale visual model is a deep convolutional neural network, often referred to as the backbone network. It is responsible for extracting visual features from the input image layer by layer, from low to high levels. Self-supervised learning is a training method that does not require manual labeling. It allows the model to learn the inherent patterns and feature representations from the data itself through cleverly designed prediction tasks. For the DINOv2 model, its backbone network has already undergone this self-supervised learning pre-training on a real-world image dataset containing hundreds of millions of images, learning rich and transferable visual features. In practical implementations, the backbone network parameters, pre-trained on massive amounts of data provided by the model publisher, are usually directly loaded. If conditions permit and sufficient unlabeled data from agricultural machinery scenarios is available, this data can also be used to perform additional self-supervised pre-training on the backbone network to make its visual feature distribution more closely resemble the agricultural working environment; however, this is not a necessary condition. After completing this step, the labeled model possesses a pre-trained backbone network with powerful feature extraction capabilities.

[0028] S12. After building a backbone network with powerful feature extraction capabilities in the first two steps, this general capability needs to be applied to specific agricultural machinery intelligent driving annotation tasks. Agricultural machinery intelligent driving annotation tasks typically include object detection and semantic segmentation. Therefore, it is necessary to design and assemble corresponding task output modules based on the selected task. Taking semantic segmentation as an example, the purpose of semantic segmentation is to predict the category of each pixel in an image. The general visual large model backbone network outputs high-level feature maps and does not directly produce pixel-level classification results. Therefore, a semantic segmentation output head needs to be constructed. This output head typically consists of a series of convolutional layers and upsampling layers, responsible for converting the feature maps extracted by the backbone network into prediction maps corresponding to the input image size, where each pixel contains the probability of each category. In implementation, the pre-trained backbone network is first loaded and its parameters are frozen to prevent modification during the initial construction phase. Then, the newly designed task output modules are connected to the end of the backbone network, thus forming a complete visual large model annotation model oriented towards a specific annotation task. At this point, the model has been built, integrating the generalization ability of a general vision model with a structural design tailored to a specific task, thus preparing it for fine-tuning using a small amount of agricultural machinery labeled data in the next step.

[0029] Visual feature extraction capability refers to the ability of a deep learning model to automatically identify and extract meaningful information patterns or features from raw image data. These features range from simple edges, corners, and textures to complex object parts and structured representations of complete objects. Models with strong visual feature extraction capabilities can understand the content of images, distinguish different objects from the background, and exhibit a certain degree of stability against disturbances such as changes in lighting and perspective shifts. This is a fundamental prerequisite for completing advanced computer vision tasks such as image recognition and segmentation.

[0030] Real-world image datasets refer to large-scale collections of images gathered and organized from various real-world scenes and application environments. These datasets typically contain millions or even hundreds of millions of images, covering common objects, scenes, and human activities in daily life, as well as diverse shooting conditions such as different times, weather, lighting, and geographical locations. Training with real-world image datasets, especially through self-supervised learning, enables models to learn very broad and general visual patterns and knowledge, rather than being limited to a narrow domain. This is the key foundation for giving large-scale visual models powerful generalization capabilities.

[0031] S2. Obtain the unlabeled dataset for the intelligent driving scenario of agricultural machinery. Select a portion of the samples from the unlabeled dataset for manual labeling to form a labeled dataset. Combine the remaining samples to form an unlabeled dataset. The specific implementation process is as follows: S20. Collect raw data for intelligent driving scenarios of agricultural machinery. This step is accomplished through visual acquisition devices deployed at key locations on the agricultural machinery, such as waterproof and dustproof high-resolution cameras, which continuously capture images during the autonomous driving operation of rice harvesters. The acquisition process needs to cover various typical scenarios that intelligent driving may encounter, including different light intensities at dawn, noon, and dusk; different weather conditions such as sunny, cloudy, rainy, and foggy days; and different background environments such as rice paddies, field ridges, and roads. The collected raw data is stored in the vehicle's storage device in the form of timestamped sequence image files or video streams, forming the initial data raw material library.

[0032] S21. All collected raw data are inspected and processed one by one to remove invalid image samples caused by camera damage, strong vibration blur, transmission interruption, or extreme overexposure or underexposure. Simultaneously, data format standardization is performed, such as extracting image frames from the video stream at a fixed frame rate, standardizing image resolution and file format, and initially classifying them according to the collection time and agricultural machinery operation stage. After cleaning and processing, all qualified image samples are systematically organized into a structured dataset, which is the unlabeled dataset for the agricultural machinery intelligent driving scenario, where each image is an unlabeled sample.

[0033] S22. Using computer-generated random numbers, a certain proportion of image samples are unbiasedly selected from the dataset to be labeled in the entire agricultural machinery intelligent driving scenario. The sampling proportion is determined based on specific project needs and labeling resources, and is usually controlled within the range of 5% to 10%. The random sampling strategy ensures that the selected sample set can approximately represent the statistical characteristics of the dataset to be labeled in the entire agricultural machinery intelligent driving scenario in terms of illumination distribution, weather type, and scene complexity, thus establishing a representative foundation for subsequent manual labeling.

[0034] S23. Organize a professional annotation team to manually annotate each extracted sample image using image annotation software with functions such as polygon delineation and pixel-level painting. For semantic segmentation tasks common in agricultural machinery intelligent driving, annotators need to carefully identify each visual element in the image, accurately depict the outlines of different categories of objects such as rice paddies, road areas, agricultural machinery bodies, and background obstacles, and assign a corresponding preset category label to each pixel within the outline. After completing the annotation work for all extracted samples, these images and their corresponding pixel-level label files together constitute the labeled dataset.

[0035] S24. All unselected image samples in the unlabeled dataset under the agricultural machinery intelligent driving scenario retain their original unlabeled state. These remaining samples are separated from the unlabeled dataset under the agricultural machinery intelligent driving scenario, summarized and stored separately, thus directly forming the unlabeled dataset. The unlabeled dataset will be used as the object for automated prediction processing of the fine-tuned labeling model in subsequent steps. Its samples no longer require manual labeling, thereby supporting a highly efficient automatic labeling process.

[0036] The unlabeled dataset for intelligent agricultural machinery driving scenarios refers to the collection of raw images or video frames continuously acquired by onboard visual sensors, such as cameras, during the intelligent driving operation of agricultural machinery. This data comprehensively records various visual information in the farmland operating environment, including crop growth status, field road outlines, the structure of the agricultural machinery itself, surrounding dynamic obstacles, and scene changes under different lighting and weather conditions. Each sample in the dataset is a raw image without any manual annotation or labeling, and together they constitute the raw input that subsequent automatic annotation methods need to process. The size, diversity, and quality of this dataset directly determine the learning effect of the fine-tuning stage of the annotation model and the reliability of the final automatic annotation results.

[0037] S3. Fine-tune the labeled model using the labeled dataset. During fine-tuning, freeze the parameters of the first part of the backbone network of the labeled model, and only update the parameters of the later network layers and the task output module. The specific implementation process is as follows: S30. Before starting fine-tuning training, the previously constructed labeled model based on the large visual model needs to be loaded into the computer's memory. Then, a series of key hyperparameters controlling the training process are configured. According to the technical solution, the batch size is set to 12, meaning that 12 samples are randomly selected from the labeled dataset each time to form a batch input to the model. The number of training epochs is set to 160, representing that the entire labeled dataset will be reused 160 times to train the model. Simultaneously, the initial learning rate is set to 0.005; the learning rate is an important parameter in the optimization algorithm that determines the step size of parameter updates. Furthermore, a detailed learning rate decay strategy needs to be set: in the first 60 training epochs, the current learning rate is multiplied by 0.8 after every 4 training epochs; in the subsequent 100 training epochs, the current learning rate is multiplied by 0.9 after every 5 training epochs. These parameters together constitute the operational framework of the fine-tuning training.

[0038] S31. To achieve the goal of preserving general knowledge and adapting to specific tasks, it is necessary to differentiate the parameters of the labeled model. Specifically, this involves traversing all layers of the backbone network of the labeled model. Based on the technical solution, the first 20 layers of the backbone network are identified. For these first 20 layers, their parameters are explicitly set to be unupdable by the optimization algorithm during training, i.e., their state is "frozen." In deep learning frameworks, this is typically done by setting the trainable properties of the corresponding network layer to false. These frozen network layers mainly contain convolutional layers used to extract low- and mid-level general visual features (such as edges, textures, and basic shapes). By freezing them, it is ensured that the valuable and generalized feature representations learned by the large visual model from massive amounts of real-world image data during pre-training are not destroyed or forgotten during fine-tuning for agricultural machinery scenarios.

[0039] S32. Set the parameters of the later layers of the backbone network and the task output module of the labeled model to a trainable state. After completing the freeze operation, it is necessary to explicitly specify the range of parameters that need to be updated and optimized during fine-tuning training. This includes the later layers of the backbone network of the labeled model, from layer 21 to layer 24. These deep network layers are usually responsible for combining basic features to form more complex and abstract high-level semantic features. At the same time, all parameters of the task output module of the labeled model also need to be set to trainable. In the program, this is achieved by setting the trainable attribute of the corresponding network layer to true. In this way, the optimization algorithm will only calculate the gradients of the parameters of these network layers that are specified as trainable during training and update the parameters accordingly. This strategy allows the model to flexibly adjust the deep feature combination method and task output logic while maintaining strong general feature extraction capabilities, so as to better fit the labeled dataset in the agricultural machinery intelligent driving scenario.

[0040] S33. The training process takes place within a loop structure, with the total number of loops equal to the preset number of training rounds. At the beginning of each training round, the labeled dataset is randomly shuffled to ensure that the order in which the model sees samples is different in different rounds. Then, the labeled dataset is divided into multiple batches. For each batch, the following sub-process is executed: The image data of the batch is input into the labeling model, which performs forward propagation calculation based on the current parameters, passing through the backbone network and the task output module, and finally outputs the category prediction corresponding to each pixel. Next, the model's prediction output is compared with the real human-annotated labels corresponding to the images in that batch, and a preset loss function is used to calculate the difference between the predicted value and the true value. This difference value is called the loss value. For semantic segmentation tasks, the commonly used loss function is the cross-entropy loss function. After calculating the loss value, the backpropagation algorithm is started to automatically calculate the gradient of the loss value with respect to all parameters set to trainable states. Finally, an optimization algorithm, such as the Adam optimizer, a variant of stochastic gradient descent, is called to update the parameters of the later layers of the backbone network and the task output module of the labeling model based on the calculated gradient and the current learning rate. After each training round, determine whether the learning rate needs to be reduced based on the current round and the preset strategy, and perform the decay operation as planned.

[0041] S34. After all 160 training iterations are completed, the fine-tuning training process ends. At this point, the parameters of the first 20 layers of the backbone network of the labeled model remain consistent with the initially loaded pre-trained parameters, while the parameters of the last 4 layers of the backbone network and the parameters of the task output module have been optimized and adjusted multiple times based on the labeled dataset in the agricultural machinery intelligent driving scenario. This optimized model has the ability to combine general visual features with the specificity of the agricultural machinery scenario. Finally, the complete labeled model with updated parameters, including all frozen and unfrozen network layer structures and their corresponding parameters, is serialized and saved to a file on the computer's hard drive. This saved model file is the fine-tuned labeled model, which will be used in the next stage for the automatic labeling task of the unlabeled dataset.

[0042] The task output module of the annotation model is a dedicated network structure component responsible for mapping the general visual features extracted by the backbone network to the format required by the specific task. In the context of agricultural machinery intelligent driving data annotation, the task typically involves pixel-level prediction, such as semantic segmentation. For semantic segmentation tasks, the typical structure of the task output module is a decoder, which consists of a series of convolutional layers and upsampling layers connected sequentially. This module receives a deeply abstracted feature map from the backbone network as input, further fuses feature context information through convolutional operations, and gradually restores the spatial size of the feature map to the size of the original input image through upsampling operations. Finally, the output of the task output module is a feature map of the same size as the input image, where each spatial location (corresponding to a pixel in the original image) contains a vector, and each value in the vector represents the predicted probability or score of the pixel belonging to various preset categories (such as rice, road, and background). Therefore, the task output module is a key conversion component for realizing the transformation from general feature recognition to specific task execution.

[0043] The large visual model is the DINOv2 model. During fine-tuning training, the parameters of the first 20 layers of the DINOv2 model's backbone network are frozen, and the parameters of layers 21 to 24 and the task output module are updated. The fine-tuning training process is as follows: 1) Load the pre-trained DINOv2 model file. This DINOv2 model contains a backbone network composed of a Vision Transformer architecture. Next, based on the specific requirements of the agricultural machinery intelligent driving annotation task, instantiate a corresponding task output module, such as a decoder network designed for semantic segmentation, and connect this task output module to the output of the DINOv2 model backbone network to form a complete annotation model to be fine-tuned. Subsequently, configure the hyperparameters required for fine-tuning training: set the training batch size to 12, determine the number of training epochs to 160, select an initial learning rate of 0.005, and specify the learning rate decay plan as follows: in the first 60 training epochs, the learning rate is multiplied by a factor of 0.8 every 4 epochs, and in the subsequent 100 training epochs, the learning rate is multiplied by a factor of 0.9 every 5 epochs.

[0044] 2) Traverse all layers of the DINOv2 model backbone. The DINOv2 model backbone typically consists of multiple consecutive Transformer blocks. The program identifies and locates the first 20 layers of the backbone, which contain the initial attention mechanism and feature transformation modules. All learnable parameters in these first 20 layers, including the query, key-value mapping weight matrix in the self-attention module, and the weights and biases in the feedforward neural network, are explicitly set to a frozen state. In the deep learning framework, this is achieved by setting the "requires_grad" attribute of the corresponding parameters to "false". This ensures that during subsequent backpropagation calculations, these frozen parameters will not have their gradients calculated, and their values ​​will remain unchanged from the initial pre-trained values ​​throughout training, thus preserving the general visual representation learned by the DINOv2 model from massive amounts of data.

[0045] 3) After completing the freezing operation, the range of parameters to be optimized needs to be explicitly specified. Locate the deep network structure in the DINOv2 model backbone from layer 21 to layer 24. Simultaneously, locate all network layers in the task output module of the entire DINOv2 model. Set the "requires_grad" attribute of all learnable parameters for these specified parts to "true". This means that during training, the optimization algorithm will update parameters such as self-attention weights and feedforward network weights in layers 21 to 24 of the DINOv2 model backbone, and simultaneously update the weights and bias parameters of each convolutional or upsampling layer in the task output module of the DINOv2 model. This setting allows the model to fine-tune deeper feature interaction methods and specialize task decoding logic for agricultural machinery scenarios.

[0046] 4) The configured DINOv2 model is trained iteratively through multiple rounds using the labeled dataset. Specifically, the preprocessed labeled dataset is loaded into the data loader. The training loop executes for 160 rounds. In each round, the data loader divides the labeled dataset into multiple batches, each containing 12 images and their corresponding ground truth labels. For each batch, forward propagation is performed: the batch images are input into the DINOv2 model, and the data sequentially passes through the frozen first 20 layers, the trainable layers 21 to 24, and the task output module of the trainable DINOv2 model, ultimately outputting a pixel-level prediction image. The cross-entropy loss between the prediction image and the ground truth label image is calculated. Backpropagation is performed: the gradient of the loss value is calculated for all parameters in the network whose "requires_grad" attribute is "true", i.e., gradients are only generated for the parameters of layers 21 to 24 of the DINOv2 model backbone network and the parameters of the DINOv2 model's task output module. The optimizer, such as the Adam optimizer, is invoked to update these trainable parameters based on the calculated gradients and the current learning rate. After every 4 or 5 rounds of training (depending on the current stage), the learning rate is lowered according to the predetermined strategy.

[0047] 5) After all 160 training cycles have been completed, the training process terminates. At this point, the parameters of the first 20 layers of the DINOv2 model backbone network are completely consistent with the initial pre-training state, while the parameters of layers 21 to 24 of the DINOv2 model backbone network and the parameters of the entire DINOv2 model's task output module have been fully adjusted based on the labeled dataset for intelligent agricultural machinery driving. The complete model state, including all frozen and updated parameters, is exported and saved as a separate model file. This file is the dedicated labeled model obtained after fine-tuning the training, suitable for the target agricultural machinery scenario, which can be loaded and executed subsequently for automatic labeling tasks on unlabeled datasets.

[0048] The task output module of the DINOv2 model is a dedicated network structure designed for data annotation tasks in agricultural machinery intelligent driving and connected to the end of the DINOv2 model backbone. In semantic segmentation tasks, the DINOv2 model's task output module is typically a decoder network, consisting of multiple transposed convolutional layers or upsampling layers stacked alternately with regular convolutional layers. This module receives a feature map output from the last layer of the DINOv2 model backbone, which contains rich semantic information but has a small spatial size. The DINOv2 model's task output module progressively expands the spatial size of the feature map through a series of upsampling operations, while simultaneously fusing features from different levels through convolutional operations to recover detailed information. Finally, the DINOv2 model's task output module outputs a feature map with the same spatial resolution as the input image, where each pixel location corresponds to a vector containing the prediction scores for each category, thus completing the transformation from the input image to pixel-level category prediction.

[0049] Before fine-tuning the labeled model using the labeled dataset, the process includes: preprocessing the labeled dataset and performing adaptive resolution adjustment on the unlabeled dataset. The specific implementation process is as follows: 1) Preprocessing the labeled dataset aims to enhance its diversity and robustness, thereby improving the learning performance of the labeled model during fine-tuning training. Preprocessing begins with adaptive resolution adjustment. Since the original sizes of images acquired in agricultural machinery intelligent driving scenarios may vary, each image and its corresponding pixel-level label image need to be uniformly scaled to a fixed base resolution, for example, adjusting both the image width and height to 512 pixels. The adjustment process uses bilinear interpolation to process the image data and nearest-neighbor interpolation to process the label image data, ensuring the integrity of the category labels is not compromised. Next, image enhancement operations are performed, applying random transformations to simulate visual changes in the real-world environment. Common transformations include random horizontal flipping to simulate different driving directions; random brightness and contrast adjustments to simulate changes in light intensity; and random addition of Gaussian noise to simulate sensor noise. These transformations are applied only to the image data, while the corresponding label images undergo synchronized spatial transformations without pixel value modification. Finally, a multi-size random cropping operation is performed, randomly selecting multiple sub-regions of different sizes from each adjusted and enhanced image, for example, generating crop blocks of 256×256 pixels and 384×384 pixels simultaneously. During cropping, it is ensured that the sub-regions contain meaningful labeled objects, such as rice paddy areas or road areas, and the images are cropped to correspond with the label images. All generated cropped samples are collected to form a preprocessed labeled dataset for subsequent fine-tuning training.

[0050] 2) Perform adaptive resolution adjustment on the unlabeled dataset. The goal of this process is to standardize the image sizes in the unlabeled dataset to meet the input requirements of the annotation model during the automatic annotation stage, while avoiding unnecessary enhancement interference to ensure prediction consistency. The process is relatively simplified, involving only adaptive resolution adjustment. Each original image in the unlabeled dataset is examined for its width and height. The image scaling ratio is calculated based on a preset model input size, such as 512×512 pixels. Each image is uniformly scaled to the target size using bilinear interpolation, ensuring that the aspect ratio of the image content is not distorted. If necessary, gray pixels are padded at the image edges to complete the size. Unlike the preprocessing of the labeled dataset, no image enhancement or multi-size random cropping is performed on the unlabeled dataset because it is only used for model inference and prediction and does not require enhancement to increase training diversity. After adjusting all images, each image in the unlabeled dataset has a uniform size format, facilitating batch input into the fine-tuned annotation model for efficient automatic annotation and prediction.

[0051] S4. Automatically label the unlabeled dataset using the fine-tuned annotation model. The specific implementation process is as follows: S40. Read the model file saved after the previous fine-tuning training from the storage system, and load the model structure and parameters into the computer memory or graphics card memory. Simultaneously, read the unlabeled dataset after adaptive resolution adjustment. Each image in the unlabeled dataset now has a uniform size that meets the model input requirements. Check and ensure that the paths of all image files in the unlabeled dataset are correctly indexed so that the program can sequentially access each image to be labeled. This step prepares the necessary software model and data raw materials for automatic annotation.

[0052] S41. Set the fine-tuned labeled model to evaluation mode. In evaluation mode, the model disables randomization operations specific to the training phase, such as dropping layers, ensuring that the prediction result for the same image is deterministic each time. Next, build a data loader to efficiently read the unlabeled dataset. The data loader reads image data from the unlabeled dataset in batches according to the file index order and the set batch size. For example, the batch size can be set to 8 or 16, and the specific value is adjusted according to the graphics card memory capacity. The data loader is also responsible for converting the read image pixel values ​​from integer format to floating-point format and normalizing them so that their numerical range is consistent with the data distribution used during model training.

[0053] S42. Iterate through each batch of images generated by the data loader. For each image in the current batch, the data loader converts it to tensor format and feeds it into the fine-tuned annotation model. The image data flows through the backbone network and task output module of the fine-tuned annotation model, performing forward propagation computation. For semantic segmentation tasks, the final output of the model is a three-dimensional tensor with the same height and width as the input image, and the depth dimension equal to the number of classes. The value at each position represents the predicted probability or score of the corresponding pixel belonging to each class. After the model completes the computation for all images in the entire batch, it outputs a tensor containing the prediction results for all images in that batch. This step is the core computational process of automatic annotation, which is completely automated by the fine-tuned annotation model.

[0054] S43. Obtain the model's raw prediction output tensor for each image. For semantic segmentation tasks, find the category index with the highest prediction score for each pixel along the category dimension. This index value represents the category label predicted by the model for that pixel. For example, a value of zero might represent the background category, a value of one might represent the rice category, and a value of two might represent the road category. Through this operation, the three-dimensional prediction tensor containing the probability distribution is converted into a two-dimensional integer label map, where each pixel value is a specific category number. This two-dimensional label map is the model's automatic annotation result for the input image. Then, the two-dimensional label map is converted from tensor format to an array format that can be processed by the image processing library, ready for saving.

[0055] S44. For each original image in the unlabeled dataset, generate a corresponding automatically labeled result file. Typically, the automatically labeled results are saved as a single-channel grayscale image file format, such as PNG, where different grayscale values ​​represent different predicted categories. Alternatively, they can be saved in other formats that are easier for subsequent algorithms to read, such as JSON files, recording the pixel coordinates of each category. When saving, the naming of the automatically labeled result files should strictly correspond to the original image filenames, or the relationship should be explicitly stated through a metadata table. All generated automatically labeled result files are systematically organized in a separate directory, forming the final automatically labeled result set, which can be used for subsequent quality detection or directly as training data.

[0056] Optionally, the above technical solution also includes: S5. Based on a preset confidence threshold, the automatic annotation results are filtered, and annotation results with a confidence level higher than the preset confidence threshold are saved. Annotation results with a confidence level lower than the preset confidence threshold are manually checked and corrected. The specific implementation process is as follows: S50. For each image in the unlabeled dataset, while automatically labeling and generating pixel-level category prediction maps using the fine-tuned labeling model, the model also outputs the predicted probability distribution of each pixel for each category. Taking the rice harvester semantic segmentation task as an example, confidence calculation requires traversing every pixel position in the prediction result map. At each pixel position, the probability value with the largest value is selected from all category probability values ​​output by the model, and this maximum value is taken as the confidence of that pixel. After extracting the confidence of all pixels in the entire image, the confidence of all pixels is added together and then divided by the total number of pixels to calculate an arithmetic mean. This mean is defined as the overall confidence of the automatic labeling result of the image. This confidence value ranges from 0 to 1, reflecting the model's degree of confidence in the overall labeling quality of the current image.

[0057] S51. The preset confidence threshold ranges from 0 to 1, and can specifically be 0.7, 0.8, or 0.9, etc. The value can be chosen based on practical experience. Taking a preset confidence threshold of 0.9 as an example, the program compares the calculated overall confidence of each image with this preset confidence threshold of 0.9. The comparison operation produces a Boolean logic judgment result. Based on this judgment result, the program needs to distribute all automatically labeled images and their corresponding confidence values ​​into two different subsequent processing queues. The first queue stores automatically labeled results with an overall confidence value greater than the preset confidence threshold of 0.9; the second queue stores automatically labeled results with an overall confidence value less than or equal to the preset confidence threshold of 0.9. This sorting operation achieves preliminary automated classification of massive amounts of labeled results.

[0058] S52. For automatically labeled results assigned to the first queue—images with an overall confidence level higher than 0.9—the program determines their labeling quality is acceptable, requiring no manual review. The program moves or copies the corresponding automatically labeled result files, such as segmentation label maps, directly from the temporary cache directory or working directory to the final determined "high-quality automatically labeled results" storage directory. Simultaneously, it records the metadata information of these files, including the original image name, the corresponding automatically labeled result file name, and the calculated confidence level value, forming a quality control pass list. This step enables rapid acceptance and archiving of high-quality outputs, significantly saving manpower.

[0059] S53. For automatically labeled images assigned to the second queue—those with an overall confidence level less than or equal to 0.9—the program determines that they contain uncertainties and require manual quality checks. The program prepares the original image files and automatically labeled result files corresponding to these images and imports them into a dedicated manual quality inspection platform or software interface. On this platform, quality inspectors can simultaneously view the original images and the automatically labeled result overlay maps generated by the model for visual comparison. Based on predefined labeling specifications, quality inspectors determine whether the automatically labeled results accurately identify and segment the target objects, check for large-area mislabeling, omissions, or boundary errors, and make a "qualified" or "unqualified" judgment.

[0060] S54. Based on the manual judgment results of quality inspectors, different operations are performed. For automatically labeled results deemed acceptable by manual inspection, even if their confidence level is not higher than the threshold, their quality is considered acceptable. The program saves such result files to the "Manual Review Passed" storage directory. For automatically labeled results deemed unacceptable by manual inspection, a correction operation needs to be initiated. Quality inspectors use the labeling tools on the manual quality inspection platform to manually modify the original automatically labeled results, such as erasing erroneous areas, filling in missing areas, and correcting boundaries. After correction, a final version of the manually corrected labeled result file is generated. The program saves this corrected file to the "Manual Correction Completed" storage directory and marks its version status. Thus, all automatically labeled results have undergone a complete quality control process based on confidence thresholds and manual inspection, forming the final usable labeled dataset.

[0061] This invention addresses a long-standing challenge in the field of intelligent agricultural machinery driving: model training requires a massive amount of data, while traditional manual annotation methods are inefficient and time-consuming, resulting in high overall annotation costs and severely hindering the practical application and promotion of related technologies. To solve this problem, this invention proposes an automatic annotation method for intelligent agricultural machinery driving data based on a large visual model. This method aims to leverage the powerful feature learning and generalization capabilities of large visual models to achieve high-precision, automated processing of large amounts of data to be labeled, thereby reducing reliance on manual annotation, improving annotation efficiency, and ensuring that the intelligent driving algorithm maintains good robustness and generalization in complex and changing farmland operating environments. The following section will use autonomous driving data from rice harvesters as an example to elaborate on the complete implementation process and technical details of this automatic annotation method.

[0062] The tool used for developing the automatic labeling software for intelligent driving data of agricultural machinery was an HP Omen laptop, model 17-ck0055TX. This computer is equipped with an Intel i9-13900H processor and an NVIDIA RTX 4090 graphics card, with 31GB of RAM and 16GB of dedicated video memory. Figure 2 As shown, automatic annotation is achieved through the following steps: 1) The construction of the annotation model involves three consecutive steps.

[0063] ① Visual Large-Scale Model Construction. This patent selects the DINOv2 model as the basic visual large-scale model. Developed by Meta, the DINOv2 model is trained using self-supervised learning based on 142 million images covering various life scenes. Considering the limitations of actual hardware computing power, this solution uses the DINOv2-L version of the DINOv2 model, which has 300 million parameters, as the visual foundation for the agricultural machinery automatic driving annotation model.

[0064] ② Self-supervised pre-training of the backbone network. Meta provides pre-trained backbone network parameter files based on a wide range of real-life scenarios. Since real-life scenarios differ from agricultural machinery operation scenarios, ideally, if a sufficient amount of unlabeled agricultural machinery data is available, the backbone network can be further pre-trained using this data to improve the model's automatic labeling performance in the target agricultural machinery scenario. If this is not possible, the officially released pre-trained parameters can be directly loaded as the model's foundation.

[0065] ③ Visual large-scale model annotation model construction. By loading pre-trained backbone network parameters and combining them with the specific requirements of agricultural machinery autonomous driving tasks, corresponding task output modules are constructed. For example, if the annotation task is semantic segmentation, a semantic segmentation output head needs to be designed and assembled to replace the output part of the original model, thereby constructing a dedicated annotation model suitable for agricultural machinery autonomous driving tasks.

[0066] 2) Fine-tuning and automatic annotation of the labeled model, comprising five consecutive steps: ① Prepare the data for the automated driving of agricultural machinery to be labeled. Clean the raw image data collected by cameras during the operation of agricultural machinery, remove blurry and invalid samples, and construct a high-quality dataset to be labeled.

[0067] ② Manually label a portion of the data. From the prepared dataset to be labeled, randomly select approximately 5% to 10% of the image samples, and have them meticulously labeled manually by annotators to form the labeled dataset. The remaining image samples in the dataset to be labeled, which were not selected, constitute the unlabeled dataset.

[0068] ③ Perform a series of preprocessing operations on the labeled dataset, including adaptive resolution adjustment, image enhancement, and multi-size random cropping, to improve the effectiveness of subsequent model fine-tuning training. For the unlabeled dataset, only adaptive resolution adjustment is performed to ensure that all images meet the size requirements of the labeled model input.

[0069] ④ Fine-tune the already constructed labeled model using the labeled dataset. The specific training strategy is as follows: freeze the parameters of the first 20 layers of the backbone network of the labeled model, and train only the parameters of the last 4 layers of the backbone network and the task output module; set the batch size to 12 and the total number of training rounds to 160; set the initial learning rate to 0.005, and in the first 60 training rounds, multiply the current learning rate by 0.8 after every 4 training rounds, and in the subsequent 100 training rounds, multiply the current learning rate by 0.9 after every 5 training rounds.

[0070] ⑤ Using the fine-tuned training model, automatically annotate all remaining images in the unannotated dataset, generating corresponding annotation result files. Some annotation results are shown below. Figure 3 As shown.

[0071] 3) Annotation quality inspection combines confidence assessment with manual review to determine the quality of the labeled data. Different confidence calculation methods need to be designed for different autonomous driving tasks. Taking the semantic segmentation task of a rice harvester as an example, firstly, the maximum probability value of each pixel in the image's classification prediction result is obtained from the automatic annotation model, and this value is used as the confidence of that pixel. Then, the arithmetic mean of the confidence values ​​of all pixels in the entire image is calculated as the overall confidence of the automatic annotation result for that image. A fixed confidence threshold of 0.9 is set. When the overall confidence of an image is higher than 0.9, its annotation quality is considered acceptable, and the automatic annotation result is directly saved. When the overall confidence of an image is lower than or equal to 0.9, the result is submitted for manual review. Labeling results deemed acceptable by manual review are saved, while those deemed unacceptable are manually corrected based on the model's pre-annotation.

[0072] This invention presents a complete solution for automatic data annotation in agricultural machinery autopilot systems. The main innovations of this method are reflected in three aspects, specifically: Firstly, this invention innovatively introduces large-scale visual models into the data annotation task for intelligent driving of agricultural machinery, constructing a complete automated annotation process that includes model building, data preparation, model fine-tuning, automatic prediction, and quality inspection. This process can improve annotation efficiency, reduce manpower consumption and time costs, and enhance the feasibility of agricultural machinery autonomous driving projects in practice.

[0073] Secondly, a training strategy was designed to adapt the annotation model for agricultural machinery autonomous driving. This strategy proposes freezing the parameters of the first 20 layers of the backbone network and training only the last 4 layers of the backbone network and the differentiated fine-tuning of the task output module parameters. This strategy achieves high-precision annotation for specific agricultural machinery scenarios while retaining the powerful generalization ability of the large visual model, thus balancing training efficiency and model performance.

[0074] Thirdly, it integrates a confidence-based automatic screening mechanism with a manual review mechanism to build an efficient and reliable quality control process. This process reduces the time cost consumed in the quality inspection stage while ensuring the quality of the final labeled data.

[0075] This invention innovatively applies a large-scale visual model to the automatic annotation task of agricultural machinery intelligent driving data, addressing the problems of high annotation costs, long cycles, low automatic annotation rates, and poor model robustness in existing technologies. Specifically, the key points include the following: First, a large-scale visual model-based annotation model is constructed. Taking the DINOv2 model as an example, this model is pre-trained on a massive real-world image dataset through self-supervised learning, learning a large number of general visual features. This enables the model to have a high automatic annotation rate and strong generalization ability, adapting to changes in different lighting conditions, weather conditions, and surrounding environments in agricultural machinery operation scenarios. Second, a fine-tuning training strategy for the annotation model of agricultural machinery data is designed. This strategy freezes the parameters of the first 20 layers of the backbone network, training only the parameters of the last 4 layers of the backbone network and the task output module. While retaining the powerful feature extraction capabilities of the large-scale visual model, it improves the annotation accuracy for specific agricultural machinery data and helps avoid model overfitting. Third, a novel annotation quality detection method is proposed. This method combines confidence threshold screening with manual inspection. For example, it automatically saves the results when the overall confidence level of the image is greater than 0.9, efficiently ensuring annotation quality and reducing the time required for manual correction. This ensures the uniqueness and practicality of this invention in the field of automatic annotation, providing technical support for the implementation of intelligent driving projects for agricultural machinery. At the same time, the methodological innovation improves annotation efficiency and model robustness.

[0076] The advantages derived from the reasoning in this invention are mainly reflected in the following aspects. Addressing the problems of low automatic annotation rates and poor robustness in existing automatic annotation methods, this invention uses a large visual model as the basis for reasoning. Large visual models, such as the DINOv2 model, have completed pre-training on a large number of real-world images through self-supervised learning, and have learned rich general visual features. Therefore, when applied to agricultural machinery scenarios, they can more accurately identify objects in different environments. Even if the lighting, weather, or surrounding environment changes, the model can maintain relatively stable performance, thereby improving the automatic annotation rate and enhancing the model's generalization ability. Through a designed fine-tuning training strategy—freezing the first 20 layers of the backbone network and training only the last 4 layers and the output module—it can be inferred that this strategy utilizes the pre-trained knowledge of the large visual model, avoiding the high cost of training a model from scratch, and also performs targeted optimization for agricultural machinery data. This strategy ensures that the model retains strong generalization ability while improving the annotation accuracy of agricultural machinery data and reducing reliance on a large amount of manually labeled data. By employing a labeling quality detection method that combines confidence thresholding and manual inspection, it can be deduced that this method can quickly filter out high-quality labeling results. For example, it automatically saves labeling results with a confidence level greater than 0.9, and only manually corrects those with low confidence levels. This reduces the time cost of comprehensive manual inspection, thereby lowering the overall cycle and cost of the labeling task. Therefore, starting from the technical solution itself, this invention demonstrates its advantages in efficiency, cost, and environmental adaptability through logical deduction, helping to solve the bottleneck problem of data labeling for intelligent agricultural machinery and promoting the practical application of intelligent agricultural machinery.

[0077] In the above embodiments, although the steps are numbered S1, S2, etc., they are only specific embodiments given by the present invention. Those skilled in the art can adjust the execution order of S1, S2, etc. according to the actual situation. The scheme after adjusting the order is also within the protection scope of the present invention. It can be understood that in some embodiments, some or all of the above embodiments may be included.

[0078] like Figure 4 As shown, an embodiment of the present invention provides an automatic labeling system 200 for intelligent agricultural machinery driving data based on a large visual model, which includes a model building module 201, a dataset processing module 202, a fine-tuning training module 203, and an automatic labeling module 204. The model building module 201 is used to: build a labeled model based on a large visual model. The large visual model is pre-trained on a real-world image dataset through self-supervised learning and has the ability to extract visual features. The dataset processing module 202 is used to: acquire the dataset to be labeled in the intelligent driving scenario of agricultural machinery, select some samples from the dataset to be labeled for manual labeling to form a labeled dataset, and combine the remaining samples into an unlabeled dataset; The fine-tuning training module 203 is used to: fine-tune the labeled model using the labeled dataset. During fine-tuning training, the parameters of the first part of the backbone network of the labeled model are frozen, and only the parameters of the last part of the network layer and the task output module are updated. The automatic annotation module 204 is used to automatically annotate unlabeled datasets using a fine-tuned annotation model.

[0079] Optionally, the above technical solution also includes an inspection and correction module, which is used to: filter the automatic annotation results based on a preset confidence threshold, save the annotation results with a confidence level higher than the preset confidence threshold, and manually inspect and correct the annotation results with a confidence level not higher than the preset confidence threshold.

[0080] Optionally, in the above technical solution, the large visual model is the DINOv2 model. During fine-tuning training, the parameters of the first 20 layers of the backbone network of the DINOv2 model are frozen, and the parameters of layers 21 to 24 and the task output module are updated.

[0081] Optionally, the above technical solution also includes a preprocessing module, which is used to: preprocess the labeled dataset before fine-tuning the labeled model using the labeled dataset, and perform adaptive resolution adjustment on the unlabeled dataset.

[0082] It should be noted that the beneficial effects of the automatic labeling system 200 for intelligent agricultural machinery driving based on a large visual model provided in the above embodiments are the same as those of the automatic labeling method for intelligent agricultural machinery driving based on a large visual model, and will not be repeated here. Furthermore, the system provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the system can be divided into different functional modules according to the actual situation to complete all or part of the functions described above. In addition, the system and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process is detailed in the method embodiments, and will not be repeated here.

[0083] An electronic device according to an embodiment of the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements any of the above-mentioned methods for automatic labeling of agricultural machinery intelligent driving data based on a large visual model.

[0084] An embodiment of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the above-mentioned methods for automatic annotation of agricultural machinery intelligent driving data based on a large visual model.

[0085] The above description is merely a preferred embodiment of the present invention and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this invention is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-disclosed concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this invention.

[0086] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. An automatic annotation method for agricultural machinery intelligent driving data based on a large visual model, characterized in that, include: A labeled model based on a large visual model is constructed. The large visual model is pre-trained on a real-world image dataset through self-supervised learning and has the ability to extract visual features. Obtain the unlabeled dataset under the intelligent driving scenario of agricultural machinery, select a portion of the samples from the unlabeled dataset for manual labeling to form a labeled dataset, and combine the remaining samples into an unlabeled dataset; The labeled model is fine-tuned using the labeled dataset. During fine-tuning, the parameters of the first part of the backbone network of the labeled model are frozen, and only the parameters of the last part of the network layer and the task output module are updated. The unlabeled dataset is automatically labeled using a finely tuned labeling model.

2. The automatic annotation method for agricultural machinery intelligent driving data based on a large visual model according to claim 1, characterized in that, Also includes: The automatic annotation results are filtered based on a preset confidence threshold. Annotations with a confidence level higher than the preset threshold are saved, while annotations with a confidence level lower than the preset threshold are manually checked and corrected.

3. The automatic annotation method for agricultural machinery intelligent driving data based on a large visual model according to claim 1 or 2, characterized in that, The large visual model is the DINOv2 model. During fine-tuning training, the parameters of the first 20 layers of the backbone network of the DINOv2 model are frozen, and the parameters of layers 21 to 24 and the task output module are updated.

4. The automatic annotation method for agricultural machinery intelligent driving data based on a large visual model according to claim 1 or 2, characterized in that, Before fine-tuning the labeled model using the labeled dataset, the method further includes: preprocessing the labeled dataset and performing an adaptive resolution adjustment operation on the unlabeled dataset.

5. An automatic data annotation system for intelligent agricultural machinery driving based on a large visual model, characterized in that, It includes a model building module, a dataset processing module, a fine-tuning training module, and an automatic annotation module; The model building module is used to: build a labeled model based on a large visual model, which is obtained by pre-training on a real-world image dataset through self-supervised learning and has the ability to extract visual features; The dataset processing module is used to: acquire the dataset to be labeled in the intelligent driving scenario of agricultural machinery, select a portion of the samples from the dataset to be labeled for manual labeling to form a labeled dataset, and combine the remaining samples into an unlabeled dataset; The fine-tuning training module is used to: fine-tune the labeled model using the labeled dataset. During fine-tuning training, the parameters of the first part of the backbone network of the labeled model are frozen, and only the parameters of the last part of the network layer and the task output module are updated. The automatic annotation module is used to automatically annotate the unannotated dataset using a fine-tuned annotation model.

6. The automatic labeling system for intelligent agricultural machinery driving data based on a large visual model according to claim 5, characterized in that, It also includes an inspection and correction module, which is used to: filter the automatic annotation results based on a preset confidence threshold, save the annotation results with a confidence level higher than the preset confidence threshold, and manually inspect and correct the annotation results with a confidence level lower than the preset confidence threshold.

7. The automatic labeling system for intelligent agricultural machinery driving data based on a large visual model according to claim 5 or 6, characterized in that, The large visual model is the DINOv2 model. During fine-tuning training, the parameters of the first 20 layers of the backbone network of the DINOv2 model are frozen, and the parameters of layers 21 to 24 and the task output module are updated.

8. The automatic labeling system for intelligent agricultural machinery driving data based on a large visual model according to claim 5 or 6, characterized in that, It also includes a preprocessing module, which is used to: preprocess the labeled dataset before using the labeled dataset to fine-tune the labeled model, and perform an adaptive resolution adjustment operation on the unlabeled dataset.

9. An electronic device, characterized in that, The system includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the automatic labeling method for intelligent agricultural machinery driving data based on a visual large model as described in any one of claims 1 to 4.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the automatic annotation method for intelligent agricultural machinery driving data based on a visual large model as described in any one of claims 1 to 4.

Images