Real-time detection method for lawn objects, electronic device and storage medium

Abstract

This invention provides a real-time lawn object detection method, electronic device, and storage medium. By adopting different screening criteria for data of varying acquisition difficulty, a classification model is introduced to expand samples in the tail category, simultaneously improving the overall dataset and annotation accuracy. This reduces the cost of manual annotation while minimizing the loss of annotation accuracy. Training a lightweight object detector (FCOS) on the expanded dataset effectively achieves real-time lawn object detection.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence and image processing technology, and in particular to a real-time detection method for lawn objects, an electronic device, and a storage medium. Background Art

[0002] In computer vision tasks, the detection and recognition of specific targets is one of the most common and important tasks.

[0003] Object detection and recognition tasks require large datasets with high annotated accuracy. While massive amounts of data are available online, methods are needed to effectively screen image quality for specific tasks. Furthermore, since many directly available images lack labels, extensive manual annotation is required, which is time-consuming and labor-intensive.

[0004] In the process of target positioning and recognition, there are problems of uneven category distribution and long-tail distribution of data. That is, the most common objects occupy the majority of the data set, while the number of objects in many rare or difficult-to-obtain categories is seriously insufficient, resulting in serious category imbalance problems, which seriously affects the performance of the target detection model.

[0005] In addition, in reality, datasets obtained through the Internet and field photography often have a natural long-tail distribution characteristic. There are often more common categories, but objects of specific categories required for some specific tasks are more difficult to obtain and are less in number. At the same time, a high-precision detection model cannot do without a large dataset, but for specific tasks, the manual labels that can be directly obtained are not sufficient, and a large amount of additional manual labeling costs are required. Summary of the Invention

[0006] In light of this, the present invention aims to address the imbalanced class distribution in the original dataset by introducing a classification model to augment the samples in the tail categories. This improves both the overall dataset and the annotation accuracy, reducing the cost of manual annotation while minimizing the loss of annotation accuracy. Finally, a lightweight object detector (FCOS) is trained on the augmented dataset to effectively perform real-time lawn object detection.

[0007] A first aspect of the present invention provides a method for real-time detection of lawn objects, the method comprising:

[0008] Step S1, obtaining a category list of objects to be detected, and dividing the category list into a first category set and a second category set;

[0009] Step S2, collecting pictures of different categories to form a data set, wherein the data set includes pictures belonging to the category list;

[0010] Step S3, integrating the images in the data set according to whether they have labels or not to obtain a labeled data set and an unlabeled data set, and filtering the labeled data set and the unlabeled data set according to predetermined filtering rules respectively;

[0011] Step S4: performing category statistics on the images in the labeled dataset to obtain a subcategory set whose number of labels in the category list is lower than a predetermined threshold; performing manual supplementary annotation on only the images belonging to the subcategory set in the unlabeled dataset to obtain an updated unlabeled dataset; and merging the updated labeled dataset and the unlabeled dataset into a first updated dataset;

[0012] Step S5, cross-validating the first object detection model based on the updated data set to obtain an optimal object detection model, and updating the images belonging to the first category set and the second category set in the updated data set according to the optimal object detection model to obtain a second updated data set;

[0013] Step S6: Perform training on the second updated data set according to the second target detection model to obtain a final target detection model, and perform lawn object detection.

[0014] Preferably, in step S1, the first category set and the second category set are obtained and divided according to whether the categories are common;

[0015] Common or easily accessible object categories are determined as a first category set; uncommon or difficult to obtain object categories are determined as a second category set.

[0016] Preferably, in step S2, collecting pictures of different categories to form a data set includes:

[0017] Different data acquisition paths are divided into obtaining from public datasets, obtaining from web crawlers, and obtaining corresponding datasets through field photography;

[0018] For data sets obtained from different acquisition paths, preprocessing is performed on the corresponding data sets, and the preprocessed data sets are merged to form a data set.

[0019] Preferably, the step S4 further includes:

[0020] Perform category statistics on the images in the above labeled dataset, and count the subcategory sets whose number of labels in the category list is 0 or lower than the median number of all category labels;

[0021] Only the images corresponding to the subcategory set are supplementally annotated on the unlabeled dataset to obtain an updated unlabeled dataset;

[0022] Save the category labels Ia corresponding to all images in the first updated dataset.

[0023] Preferably, the first target detection model includes a Faster-RCNNr50+fpn model;

[0024] The step S5 further includes: performing cross-validation on the first updated data set according to the Faster-RCNN r50+fpn model; the cross-validation uses a 5-fold cross-validation method to evaluate the performance of the model parameters, thereby obtaining the best model.

[0025] Preferably, the cross-validation is performed on the first updated dataset according to the Faster-RCNN r50+fpn model, comprising:

[0026] The first updated dataset is divided into 5 equal parts. Four sub-datasets are selected for training each time, and the remaining sub-dataset is used as the test set evaluation indicator. This is repeated 5 times. The mAP indicator is used, and the model with the highest average indicator is selected as the best object detection model.

[0027] Preferably, in step 5, updating the images belonging to the first category set and the second category set in the updated dataset according to the optimal object detection model to obtain a second updated dataset includes:

[0028] Step 5.1: Filter the target object frames of the pictures in the first category set and the second category set according to the confidence threshold to obtain a set of candidate frames Ic for all pictures;

[0029] Step 5.2: For each image, compare the candidate boxes in the candidate box set Ic with the category labels in the category labels Ia. If, for the same image, the IoU (intersection over union) ratio of the box in Ic to the box in Ia is greater than 0.8 and the categories are the same, the box is considered a duplicate and removed from the candidate box set Ic. Merge the box information of Ic and Ia to obtain the candidate box information It for all images.

[0030] Step 5.3: Resize the image area enclosed by the candidate box corresponding to the category label Ia to a size of 224x224 and feed it into the ViT-L model pre-trained on ImageNet for fine-tuning;

[0031] In step 5.4, all the candidate boxes selected from the candidate box information It of all the pictures are fed into the fine-tuned ViT-L model. Based on the characteristics of the long-tail distribution, the candidate boxes whose final category belongs to the first category set and whose confidence is greater than 0.8, and the candidate boxes whose final category belongs to the second category set and whose confidence is greater than 0.6 are removed and retained;

[0032] In step 5.5, all the candidate box information retained above is used as the final category label to obtain the second updated dataset.

[0033] Preferably, the step 6 further includes: the second target detection model includes an FCOS model;

[0034] Training the FCOS model according to the second updated data set to obtain a trained FCOS model;

[0035] The trained FCOS model is used to perform real-time detection on the input lawn scene image to be detected, and a detection result is generated.

[0036] In addition, in a second aspect, the present invention also proposes an electronic device, which includes: one or more processors, and a memory, wherein the memory is used to store one or more computer programs; characterized in that the computer program is configured to be executed by the one or more processors, and the program includes steps for executing the real-time lawn object detection method as described in the first aspect.

[0037] In addition, in a third aspect, the present invention further proposes a storage medium storing a computer program; the program is loaded and executed by a processor to implement the steps of the real-time lawn object detection method described in the first aspect.

[0038] In the solution of the present invention, a category list of objects to be detected is obtained, and the category list is divided into a first category set and a second category set; pictures of different categories are collected to form a data set, and the data set includes pictures belonging to the category list; the pictures in the data set are integrated according to whether they have labels to obtain a labeled data set and an unlabeled data set, and the labeled data set and the unlabeled data set are respectively filtered according to predetermined filtering rules; category statistics are performed on the pictures in the labeled data set to count the subcategory sets whose number of labels in the category list is lower than a predetermined threshold; and manual supplementary annotation is performed on only the pictures belonging to the subcategory sets on the unlabeled data set to obtain an updated unlabeled data set; the updated labeled data set and the unlabeled data set are merged into a first updated data set; a first target detection model is cross-validated based on the updated data set to obtain an optimal target detection model, and the pictures belonging to the first category set and the second category set in the updated data set are updated according to the optimal target detection model to obtain a second updated data set; training is performed on the second updated data set according to the second target detection model to obtain a final target detection model, and detection of lawn objects is performed. Compared with existing technologies, on the one hand, by adopting different screening criteria for data with different acquisition difficulties, and by introducing a classification model ViT-L combined with a relatively excellent two-stage object detector Faster-RCNN r50+fpn to eliminate candidate boxes, it is possible to construct a dataset with relatively high annotation quality and good category balance while requiring very little manual correction work. By combining network data with field-captured data, the cost of capturing data can be reduced. On the other hand, using FCOS for final object detection has high application value and can be applied to lightweight computing platforms with high real-time requirements. It can detect objects relatively quickly and is better suited for real-time tasks such as robot obstacle avoidance and automatic lawn mower operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0040] Figure 1 is a flow chart of a method for real-time detection of lawn objects disclosed in an embodiment of the present invention;

[0041] Figure 2 is a schematic diagram of a process for obtaining a second updated data set disclosed in an embodiment of the present invention;

[0042] Figure 3 This is a flow chart of a processing method based on frame screening rules disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0043] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be thorough and complete and will fully convey the concepts of the example embodiments to those skilled in the art.

[0044] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner.In the following description, many specific details are provided so as to provide a full understanding of the embodiments of the present application. However, it will be appreciated by those skilled in the art that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps etc. can be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0045] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically separate entities. That is, these functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0046] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, while others may be combined or partially combined. Therefore, the actual execution order may vary depending on the actual situation.

[0047] It should be noted that the “plurality” mentioned in this article refers to two or more.

[0048] The following is a detailed description of the implementation details of the technical solution of the embodiment of the present application:

[0049] In reality, datasets obtained online or through field photography often have a naturally long-tail distribution. Common categories are often abundant, while the specific categories required for specific tasks are more difficult to obtain and are therefore rare. Furthermore, a high-precision detection model requires a large dataset. However, for specific tasks, readily available manual labels are insufficient, necessitating significant additional manual labeling costs.

[0050] This example addresses the imbalanced nature of the original dataset by introducing a classification model to expand the samples in the tail categories. This improves both the overall dataset and the annotation accuracy, reducing the cost of manual annotation while minimizing the loss of annotation accuracy. Finally, a lightweight object detector (FCOS) is trained on the expanded dataset to effectively perform real-time lawn object detection.

[0051] like Figure 1 As shown, a first aspect of this embodiment provides a real-time detection method for lawn objects, the method comprising:

[0052] Step S1: Obtain a category list of objects to be detected, and divide the category list into a first category set and a second category set.

[0053] Preferably, in step S1, the first category set and the second category set are divided according to whether the categories are common; common or easily accessible object categories are determined as the first category set; uncommon or difficult to obtain object categories are determined as the second category set.

[0054] Specifically, in this embodiment, the category list O to be detected in this embodiment is determined, and the category list is divided into two categories. One category is a first category set containing common and easy-to-obtain categories - category set A (such as cats, dogs, people, etc. categories); the other category is a second category set containing uncommon and difficult-to-obtain categories - category set B (such as hedgehog sprinklers, etc. categories).

[0055] Step S2: Collect pictures of different categories to form a data set, where the data set includes pictures belonging to a category list.

[0056] Preferably, in step S2, different categories of pictures are collected to form a data set, including: for different data acquisition paths, the corresponding data sets are obtained from public data sets, web crawlers, and field photography; for data sets from different acquisition paths, the corresponding data sets are preprocessed, and the preprocessed data sets are merged to form a data set.

[0057] Specifically, in this embodiment, according to different data acquisition paths, data sets are obtained from public data sets, web crawlers, and field shooting. Field recorded videos or photographs are used to collect pictures containing the above categories. A large number of pictures containing the above category set A and category B are collected through web crawlers or public data sets containing the above categories, and the pictures are screened based on rules.

[0058] The required images are obtained through public datasets, and the public datasets are divided into three categories: classification datasets, detection datasets, and segmentation datasets. Here, the classification dataset uses ImageNet1K, the detection dataset uses VOC2012 and COCO2017 datasets, and the segmentation dataset uses Cityscapes and ADE20K datasets.

[0059] First, the acquired detection and segmentation datasets, as well as the classification datasets, need to be filtered for the required images according to category O. Further processing is then performed based on the dataset category. The acquired detection dataset images only need to be unified into the VOC format, meaning that each image corresponds to an XML file with the same name, which records the object category and its location in the image.

[0060] Secondly, for the acquired segmentation dataset, first unify the mask format of the segmentation annotation, convert the image into a binary image, use the connected domain for analysis, extract the contour of the required category in each connected domain, and then use the convex hull method to fit the minimum circumscribed rectangular box of the target object, obtain the position information of the specified category object in each connected domain, repeat the above operation for each required category in the set O, and you can get the information of all required category objects, and convert this information into the above VOC format for export.

[0061] Classification datasets are not processed for the time being.

[0062] Step S3: Integrate the images in the data set according to whether they have labels or not to obtain a labeled data set and an unlabeled data set, and perform filtering on the labeled data set and the unlabeled data set according to predetermined filtering rules.

[0063] Specifically, in this embodiment, after the process of the above-mentioned step S2, this embodiment can obtain a large number of pictures of the A set category, but there are fewer pictures of the B category. Here, it is necessary to supplement the pictures of the B category through a web crawler. Since there are also fewer pictures of uncommon and difficult-to-obtain categories, 150 pictures are crawled for each category in the B category. The 150 crawled pictures of each category are directly checked for duplication through file size and picture size, and duplicate pictures are deleted.

[0064] To address the problem that the datasets collected from the Internet are not widely applicable in various scenarios, we use pictures or videos taken on site to supplement the dataset. We extract frames from the video data according to the video bit rate, adjust the resolution of the pictures obtained after extraction, and filter out only the pictures in set O.

[0065] The above datasets are integrated according to whether they have labels or not, and divided into labeled dataset L1 and unlabeled dataset U1.

[0066] Specifically, in this embodiment, the above datasets L1 and U1 are both screened according to rule 3.1 to obtain the filtered labeled dataset L2 and unlabeled dataset U2, respectively.

[0067] The main screening rules include: 1) eliminating images with a resolution smaller than 300x300; 2) removing images with solid backgrounds or large logos; 3) removing images with extremely small objects or blurred objects; and 4) removing images with drastically different aspect ratios. The overall principle is to apply stricter rules to images belonging to Set A and more relaxed rules to images belonging to Set B, thereby achieving a better balance between categories.

[0068] Step S4: perform category statistics on the images in the labeled dataset, and count the subcategory sets whose number of labels in the category list is lower than a predetermined threshold; and perform manual supplementary annotation on only the images belonging to the subcategory sets in the unlabeled dataset to obtain an updated unlabeled dataset; merge the updated labeled dataset and the unlabeled dataset into a first updated dataset.

[0069] Preferably, the step S4 further includes: counting the labels of the above-mentioned labeled data set according to categories, and counting the subcategory sets whose number of labels in the category list is 0 or lower than the median number of all category labels; supplementing the annotation of only the pictures corresponding to the subcategory sets in the unlabeled data set to obtain an updated unlabeled data set; and saving the category labels Ia corresponding to all pictures in the first updated data set.

[0070] Specifically, in this embodiment, the labels of the above-mentioned labeled data set L2 are counted according to categories, and a category set Y in which the number of labels in category O is 0 or lower than the median number of labels of all categories is counted. Only the Y category set is supplemented and labeled on the unlabeled data set U2 to obtain an updated data set U2' of U2.

[0071] At this point, the U2' dataset and the L2 dataset are merged into the dataset D1, and the existing category labels Ia corresponding to all images are retained.

[0072] Step S5: cross-validate the first target detection model based on the updated data set to obtain the best target detection model, and update the pictures belonging to the first category set and the second category set in the updated data set according to the best target detection model to obtain a second updated data set.

[0073] Preferably, the first target detection model includes a Faster-RCNN r50+fpn model; the step S5 further includes: performing cross-validation on the first updated data set according to the Faster-RCNN r50+fpn model; the cross-validation uses a 5-fold cross-validation method to evaluate the performance of the model parameters, thereby obtaining the optimal model.

[0074] Preferably, the cross-validation based on the Faster-RCNN r50+fpn model on the first updated dataset includes: dividing the first updated dataset into 5 equal parts, selecting 4 sub-datasets for training each time, and using the remaining sub-dataset as the test set evaluation index, repeating 5 times in total, using the mAP index, and taking the model with the highest average index as the best target detection model.

[0075] Preferably, if Figure 2 As shown, in step 5, updating the pictures belonging to the first category set and the second category set in the update data set according to the optimal target detection model to obtain a second updated data set includes:

[0076] Step 5.1: Filter the target object frames of the pictures in the first category set and the second category set according to the confidence threshold to obtain a set of candidate frames Ic for all pictures;

[0077] Step 5.2: For each image, compare the candidate boxes in the candidate box set Ic with the category labels in the category labels Ia. If, for the same image, the IoU (intersection over union) ratio of the box in Ic to the box in Ia is greater than 0.8 and the categories are the same, the box is considered a duplicate and removed from the candidate box set Ic. Merge the box information of Ic and Ia to obtain the candidate box information It for all images.

[0078] Step 5.3: Resize the image area enclosed by the candidate box corresponding to the category label Ia to a size of 224x224 and feed it into the ViT-L model pre-trained on ImageNet for fine-tuning;

[0079] In step 5.4, all the candidate boxes selected from the candidate box information It of all the pictures are fed into the fine-tuned ViT-L model. Based on the characteristics of the long-tail distribution, the candidate boxes whose final category belongs to the first category set and whose confidence is greater than 0.8, and the candidate boxes whose final category belongs to the second category set and whose confidence is greater than 0.6 are removed and retained;

[0080] In step 5.5, all the candidate box information retained above is used as the final category label to obtain the second updated dataset.

[0081] Specifically, this example uses Faster-RCNN r50+fpn for cross-validation on dataset D1. A 5-fold cross-validation method is used to evaluate the performance of model parameters and obtain the optimal model. The dataset is divided into five equal parts. Training is performed on four of these sub-datasets each time, with the remaining sub-dataset used as the test set for evaluation. This is repeated five times, using the mAP metric. The model with the highest average metric is selected as the final model.

[0082] like Figure 3 The following figure shows the processing method based on the box-based screening rule of this implementation. The best model is tested and inferred on the D1 dataset, and the following rules are used for screening: different screening criteria are adopted for objects in sets A and B. Due to the characteristics of the long-tail distribution, the recognition accuracy of rare classes is high, while the recall rate of common classes is low. Therefore, for the boxes belonging to the categories of set A, candidate boxes with a category confidence of confidence>0.8 are retained, and for the categories in set B, candidate boxes with a confidence>0.6 are retained. The candidate box set Ic for all images is obtained. The candidate boxes in Ic are compared with the annotation information in Ia for each individual image. If, for the same image, the IoU intersection over union ratio of the box in Ic and the box in Ia is greater than 0.8 and the category is the same, it is considered a duplicate box and the box is removed from the candidate box set Ic. Finally, the box information of Ic and Ia is merged to obtain the candidate box information It for all images.

[0083] The image area framed by the Ia candidate box is resized to 224x224 and fed into the ViT-L model pre-trained on ImageNet for fine-tuning. The specific operation is to freeze the weight part of the backbone network to retain a strong feature extraction capability, and only adjust the parameters of the last fully connected layer. The fully connected layer originally used to classify 1000 categories is changed to classify objects in the set O.

[0084] The images selected from all candidate boxes in It are fed into the fine-tuned ViT-L model. Based on the long-tail distribution, candidate boxes whose final categories belong to set A and whose confidence level is greater than 0.8 are retained, as are those whose final categories belong to set B, but whose confidence level is greater than 0.6. The information on all retained candidate boxes is used as the final annotation information (labels), resulting in the second updated dataset, the new dataset D2.

[0085] Step S6: Perform training on the second updated data set according to the second target detection model to obtain a final target detection model, and perform lawn object detection.

[0086] Preferably, step 6 further includes: the second target detection model includes an FCOS model; training the FCOS model according to the second updated data set to obtain a trained FCOS model; performing real-time detection on the input image to be detected of the lawn scene according to the trained FCOS model, and generating a detection result.

[0087] Specifically, in this embodiment, FCOS is used to train on the D2 dataset to obtain the final lightweight lawn object model recognition model; the FCOS model obtained above is converted into the onnx intermediate format, and the accuracy is retained as 16-bit floating point numbers, which speeds up the actual inference speed compared to the original 32 bits.

[0088] It's important to note that ONNX is an open file format designed for machine learning, used to store trained models. It enables different AI frameworks (such as Pytorch and MXNet) to store and interact with model data in the same format. The ONNX specification and code were developed jointly by companies like Microsoft, Amazon, Facebook, and IBM, and are hosted on Github as open source.

[0089] This embodiment eliminates the need for multiple iterations; only a single annotation of the original image is required to obtain a large amount of tail category data, significantly reducing manual annotation costs. By incorporating the characteristics of long-tail distributions and adopting different screening criteria for data of varying difficulty, a classification model, ViT-L, combined with a relatively advanced two-stage object detector, Faster-RCNN r50+fpn, is introduced to eliminate candidate boxes. This allows for the construction of a dataset with relatively high annotation quality and good category balance, while minimizing manual correction work.

[0090] Furthermore, by combining network data with field data, the cost of capturing data can be reduced. Using FCOS for final object detection has high application value and can be applied to lightweight computing platforms with high real-time requirements. It can detect objects relatively quickly and is better suited for real-time tasks such as robot obstacle avoidance and automatic lawn mower operations.

[0091] In addition, the second aspect of this embodiment also proposes an electronic device, which includes: one or more processors, and a memory, wherein the memory is used to store one or more computer programs; it is characterized in that the computer program is configured to be executed by the one or more processors, and the program includes steps for executing the real-time detection method of lawn objects as described in the first aspect.

[0092] In addition, the third aspect of this embodiment further proposes a storage medium, which stores a computer program; the program is loaded and executed by a processor to implement the steps of the real-time lawn object detection method described in the first aspect.

[0093] The processor and memory are used together to execute the program stored in the memory. When the program is executed by the computer, the methods, steps or functions described in the above embodiments can be implemented.

[0094] Although not shown, in some embodiments, a storage medium is further provided, storing a computer program configured to execute any method of the embodiments of the present invention when executed.

[0095] Storage media in embodiments of the present invention include permanent and non-permanent, removable and non-removable items that can be used to store information using any method or technology. Examples of storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0096] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, those skilled in the art will appreciate that the functional modules / units or controllers and related method steps described in the above embodiments may be implemented using software, hardware, or a combination of software / hardware.

[0097] Unless explicitly stated, the actions or steps of the methods, procedures, and methods described in accordance with the embodiments of the present invention do not have to be performed in a specific order and can still achieve the desired results. In some implementations, multitasking and parallel processing are also possible or may be advantageous.

[0098] In this document, multiple embodiments of the present invention are described, but for the sake of brevity, the description of each embodiment is not exhaustive, and the same or similar features or parts between the embodiments may be omitted. In this document, "one embodiment", "some embodiments", "example", "specific example", or "some examples" are intended to apply to at least one embodiment or example according to the present invention, but not all embodiments. The above terms do not necessarily mean to refer to the same embodiment or example. Those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are mutually contradictory.

[0099] While the exemplary systems and methods of the present invention have been specifically shown and described with reference to the foregoing embodiments, these are merely examples of the best modes for implementing the present systems and methods. Those skilled in the art will appreciate that various changes may be made to the embodiments of the systems and methods described herein when implementing the present systems and / or methods without departing from the spirit and scope of the present invention as defined in the appended claims.

Claims

1. A real-time detection method for lawn objects, characterized in that: The method comprises: Step S1, obtaining a category list of objects to be detected, and dividing the category list into a first category set and a second category set; Step S2, collecting pictures of different categories to form a data set, wherein the data set includes pictures belonging to the category list; Step S3, integrating the images in the data set according to whether they have labels or not to obtain a labeled data set and an unlabeled data set, and filtering the labeled data set and the unlabeled data set according to predetermined filtering rules respectively; Step S4: performing category statistics on the images in the labeled dataset to obtain a subcategory set whose number of labels in the category list is lower than a predetermined threshold; performing manual supplementary annotation on only the images belonging to the subcategory set in the unlabeled dataset to obtain an updated unlabeled dataset; and merging the updated labeled dataset and the unlabeled dataset into a first updated dataset; Step S5, cross-validating the first object detection model based on the updated data set to obtain an optimal object detection model, and updating the images belonging to the first category set and the second category set in the updated data set according to the optimal object detection model to obtain a second updated data set; Step S6, performing training on the second updated data set according to the second object detection model to obtain a final object detection model, and performing lawn object detection; In step S5, updating the images belonging to the first category set and the second category set in the updated dataset according to the optimal object detection model to obtain a second updated dataset includes: Step 5.1: Filter the target object frames of the pictures in the first category set and the second category set according to the confidence threshold to obtain a set of candidate frames Ic for all pictures; Step 5.2: For each image, compare the candidate boxes in the candidate box set Ic with the category labels in the category labels Ia. If, for the same image, the IoU (intersection over union) ratio of the box in Ic to the box in Ia is greater than 0.8 and the categories are the same, the box is considered a duplicate and removed from the candidate box set Ic. Merge the box information of Ic and Ia to obtain the candidate box information It for all images. Step 5.3: Resize the image area enclosed by the candidate box corresponding to the category label Ia to a size of 224x224 and feed it into the ViT-L model pre-trained on ImageNet for fine-tuning; In step 5.4, all the candidate boxes selected from the candidate box information It of all the pictures are fed into the fine-tuned ViT-L model. Based on the characteristics of the long-tail distribution, the candidate boxes whose final category belongs to the first category set and whose confidence is greater than 0.8, and the candidate boxes whose final category belongs to the second category set and whose confidence is greater than 0.6 are removed and retained; In step 5.5, all the candidate box information retained above is used as the final category label to obtain the second updated dataset.

2. The real-time lawn object detection method according to claim 1, characterized in that: In step S1, the first category set and the second category set are obtained and divided according to whether the categories are common; Common or easily accessible object categories are determined as a first category set; uncommon or difficult to obtain object categories are determined as a second category set.

3. The real-time lawn object detection method according to claim 2, characterized in that: In step S2, collecting pictures of different categories to form a data set includes: Different data acquisition paths are divided into obtaining from public datasets, obtaining from web crawlers, and obtaining corresponding datasets through field photography; For data sets obtained from different acquisition paths, preprocessing is performed on the corresponding data sets, and the preprocessed data sets are merged to form a data set.

4. The real-time lawn object detection method according to claim 3, characterized in that: The step S4 further includes: The labels of the above labeled dataset are counted by category, and the sub-category sets whose number of labels in the category list is 0 or lower than the median number of labels of all categories are counted; Only the images corresponding to the subcategory set are supplementally annotated on the unlabeled dataset to obtain an updated unlabeled dataset; Save the category labels Ia corresponding to all images in the first updated dataset.

5. The real-time lawn object detection method according to claim 4, characterized in that: The first target detection model includes a Faster-RCNNr50+fpn model; The step S5 further includes: performing cross-validation on the first updated data set according to the Faster-RCNN r50+fpn model; the cross-validation uses a 5-fold cross-validation method to evaluate the performance of the model parameters, thereby obtaining the best model.

6. The real-time lawn object detection method according to claim 5, characterized in that: The cross-validation is performed on the first updated dataset according to the Faster-RCNNr50+fpn model, comprising: The first updated dataset is divided into five equal parts. Four of the sub-datasets are selected for training each time, and the remaining sub-dataset is used as the test set evaluation indicator. This is repeated five times. The mAP indicator is used, and the model with the highest average indicator is selected as the best object detection model.

7. The real-time lawn object detection method according to claim 1, characterized in that: The step S6 further includes: the second target detection model includes an FCOS model; Training the FCOS model according to the second updated data set to obtain a trained FCOS model; The trained FCOS model is used to perform real-time detection on the input lawn scene image to be detected, and a detection result is generated.

8. An electronic device, comprising: One or more processors, a memory, the memory being used to store one or more computer programs; characterized in that the computer program is configured to be executed by the one or more processors, and the program includes steps for executing the real-time lawn object detection method according to any one of claims 1 to 7.

9. A storage medium storing a computer program; the program is loaded and executed by a processor to implement the steps of the method for real-time detection of lawn objects according to any one of claims 1 to 7.

Images