Paddy field seedling stage weed small sample target detection method based on multi-mode large model
By constructing an image-text pair dataset of weeds during the rice seedling stage and using the G-DINO multimodal large model for self-supervised learning, the problem of insufficient feature learning in the rice seedling weed detection model under small sample conditions was solved. This enabled high-precision weed detection and optimization of weeding schemes, and improved the intelligence level of rice weed detection and weeding operations.
Patent Information
- Application Number
- CN202511519485.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-23
- Publication Date
- 2026-01-02
AI Technical Summary
Existing deep learning-based weed detection models for rice seedlings have poor generalization performance under small sample conditions, making it difficult to effectively identify weeds with low contrast, partial occlusion, or complex growth morphology. Furthermore, existing methods lack a multimodal feature space mapping mechanism, resulting in insufficient robustness of the detection models.
We constructed an image-text dataset of weeds during the rice seedling stage, used the G-DINO multimodal large model for self-supervised learning to generate a shared feature space for images and text, and combined it with a few-shot learning algorithm to train the model. Through self-supervised training and few-shot learning, we improved the model's feature representation ability and recognition generalization.
It achieves high-precision real-time detection of weeds in paddy fields, generates and optimizes weeding plans, and improves the intelligence level and overall efficiency of weed detection and weeding operations in paddy fields.
Smart Images
Figure CN121259488A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the field of agricultural intelligent sensing technology, and in particular to a rice field seedling stage weed small sample target detection method based on a multi-modal large model. BACKGROUND
[0002] With the continuous development of intelligent agriculture and computer vision technology, the deep learning-based farmland weed identification and detection method has gradually become an important research direction of agricultural automation. In recent years, target detection algorithms have been introduced into the rice field scene for seedling stage weed detection and crop growth monitoring. However, the rice field environment has strong background interference, a variety of weed species and similar morphology, etc., which makes the traditional supervised learning-based detection model limited in small sample conditions. The existing technology relies on a large number of manually labeled sample data for model training, but in the agricultural scene, due to the influence of season, region and growth stage on sample collection conditions, it is difficult to construct a high-quality and balanced training dataset. The traditional detection model usually only uses the visual features of the image, and cannot fully combine the semantic, morphological or growth description information of the weed, resulting in poor generalization performance and insufficient robustness of the detection model, which is difficult to meet the application requirements of real-time and accurate weeding.
[0003] The existing deep learning-based weed detection method still has significant deficiencies in small sample learning and cross-modal feature fusion. On the one hand, although small sample learning research has been applied in medical imaging, industrial detection and other fields, in the agricultural scene, especially for rice field seedling stage weed detection, the characteristics of the model transfer learning effect are limited due to the difference in characteristics and the complexity of the scene. On the other hand, although some research has tried to jointly train images and text descriptions, most methods use simple semantic embedding or label enhancement, lack a unified multi-modal feature space mapping mechanism, making it difficult for the model to effectively align from text semantics to visual features. The existing G-DINO detection framework is prone to overfitting or detection omission under small sample conditions, and cannot stably identify low-contrast, partially occluded or complex growth form weed targets in the rice field, limiting the execution accuracy and feedback optimization efficiency of subsequent automatic weeding equipment. SUMMARY
[0004] In view of the above existing problems, the present application is proposed.
[0005] Therefore, the present application provides a rice field seedling stage weed small sample target detection method based on a multi-modal large model, which solves the problem of insufficient feature learning caused by small sample data in rice field seedling stage weed detection, which is prone to errors.
[0006] To solve the above technical problems, the present application provides the following technical solutions:
[0007] In a first aspect, the present invention provides a method for detecting small-sample targets of weeds in paddy field seedlings based on a multimodal large model, comprising,
[0008] Image data of weeds during the seedling stage of rice paddies were collected, and text annotations were added to the images to generate an image-text pair dataset. The G-DINO multimodal large model was used for self-supervised learning to generate feature representations of images and text, and mapped to a shared feature space.
[0009] By combining the shared feature space, a few-shot learning algorithm is used to train the G-DINO multimodal large model, and the trained G-DINO multimodal large model is used to detect weeds in real-time images.
[0010] Weed control plans are generated and implemented based on weed detection results. The weed control plans are then adjusted based on the weed control results, and the experience data generated during the weed control process is uploaded to the database for storage.
[0011] As a preferred embodiment of the small sample target detection method for paddy field seedling weeds based on a multimodal large model described in this invention, the step of collecting image data of paddy field seedling weeds and annotating the images with text to generate an image-text pair dataset refers to collecting image data of paddy field seedling weeds, using the CLIP image recognition tool to recognize the collected images, generating text data containing weed species, status, and location, and uploading the images and corresponding text data as an image-text pair dataset to the database.
[0012] As a preferred embodiment of the method for detecting small-sample targets of weeds in paddy fields during the seedling stage based on a multimodal large model as described in this invention, the method of using a G-DINO multimodal large model for self-supervised learning to generate feature representations of images and text and mapping them to a shared feature space refers to removing noise and enhancing contrast in the acquired images, using a Tokenizer to segment the text data into tokens, inputting the tokens into a word embedding layer to convert them into word vector sequences, using a Vit visual encoder to extract the image feature matrix, and using a BERT text encoder to extract the text feature matrix.
[0013] Normalize the image feature matrix and the text feature matrix, and calculate the cosine similarity between the normalized image feature matrix and the text feature matrix.
[0014] The G-DINO multimodal large model is self-supervised by using a contrastive loss function. The gradient descent method is repeatedly used to minimize the loss function until the set upper limit is reached. The trained G-DINO multimodal large model is then used to extract image and text features, and the extracted features are mapped to a shared feature space.
[0015] As a preferred embodiment of the small sample target detection method for weeds in paddy fields during the seedling stage based on a multimodal large model described in this invention, wherein: the step of combining the shared feature space and using a small sample learning algorithm to train the G-DINO multimodal large model refers to using the K-means clustering algorithm to divide the text feature matrix in the shared feature space into K categories, and averaging all text feature matrices and image feature matrices corresponding to each category;
[0016] The text center features and image center features of each category are weighted and fused to obtain the prototype of each category;
[0017] The classification head of the G-DINO multimodal large model is initialized using the category prototype, local features of the image are extracted, and the similarity between the local features of the image and the category prototype is calculated. The local region with the highest similarity is used as the pseudo-target box.
[0018] The G-DINO multimodal large model is trained using pseudo-boundaries, and the object detection loss is minimized using gradient descent. When the loss converges, the trained G-DINO multimodal large model is obtained.
[0019] As a preferred embodiment of the small-sample target detection method for weeds in paddy fields during the seedling stage based on a multimodal large model as described in this invention, the step of using the trained G-DINO multimodal large model to detect weeds in real-time images refers to inputting the real-time acquired image data into the G-DINO multimodal large model, and the G-DINO multimodal large model outputting a set of predicted candidate boxes. The candidate boxes include the position of the candidate box in the image, the probability of weeds existing within the candidate box, and the probability distribution of the category prediction results. The confidence score of each candidate box is then calculated. m ;
[0020] Set the confidence score threshold. thr If Score m ≥Score thr If the Score is high, then the candidate bounding box is retained as the object detection result. m <Score thr If it is, then it will not be retained.
[0021] As a preferred embodiment of the multimodal large model-based small sample target detection method for paddy field seedling weeds described in this invention, the step of generating and executing weed control plans based on weed detection results involves pre-establishing a mapping relationship between each type of weed and its corresponding weed control plan as a weed control plan library, defining recommended weed control methods for each type of weed, and matching weed control plans from the predefined weed control plan library based on the identified weed types and their characteristics.
[0022] As a preferred embodiment of the multimodal large model-based small sample target detection method for paddy field seedlings described in this invention, the step of adjusting the weeding plan according to the weeding results refers to collecting regional image data after weeding, analyzing and calculating the weeding rate of the weeding plan, setting a weeding rate threshold, and if the weeding rate of the plan is greater than or equal to the weeding rate threshold, it indicates that the weeding plan is qualified; if the weeding rate of the plan is less than the weeding rate threshold, the weeding plan is executed again and the intensity of the weeding plan is increased.
[0023] As a preferred embodiment of the multimodal large model-based small sample target detection method for weeds in paddy fields during the seedling stage described in this invention, the step of uploading the empirical data generated during the weeding process to the database for storage refers to saving the data of the collected images, weed detection, cleaning execution, and cleaning results to the database.
[0024] In a second aspect, the present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, it implements any step of the method for detecting small samples of weeds in paddy field seedlings based on a multimodal large model as described in the first aspect of the present invention.
[0025] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the method for detecting small sample targets of weeds in paddy field seedlings based on a multimodal large model as described in the first aspect of the present invention.
[0026] The beneficial effects of this invention are as follows: By constructing an image-text pair dataset of weeds during the seedling stage in paddy fields and using the G-DINO multimodal large model for self-supervised learning, deep fusion of image and text features and the establishment of a shared feature space are achieved, improving the feature expression ability and recognition generalization of the G-DINO multimodal large model under small sample conditions; by combining the shared feature space and using a small sample learning algorithm to train the G-DINO multimodal large model, high-precision real-time detection of weeds in paddy fields is achieved, and weeding schemes are automatically generated and dynamically optimized based on the detection results, thereby improving the intelligence level and overall operational efficiency of weed detection and weeding operations in paddy fields. Attached Figure Description
[0027] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0028] Figure 1 This is a flowchart of a method for detecting small samples of weeds in paddy field seedlings based on a multimodal large model.
[0029] Figure 2 This diagram illustrates the self-supervised training of the G-DINO multimodal large model and the training of the G-DINO multimodal large model. Detailed Implementation
[0030] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.
[0031] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.
[0032] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.
[0033] Reference Figure 1 and Figure 2 This is one embodiment of the present invention, which provides a method for detecting small samples of weeds in paddy field seedlings based on a multimodal large model, including the following steps:
[0034] Image data of weeds during the seedling stage of rice paddies were collected, and text annotations were added to the images to generate an image-text pair dataset. The G-DINO multimodal large model was used for self-supervised learning to generate feature representations of images and text, which were then mapped to a shared feature space.
[0035] Specifically, image data of weeds during the rice seedling stage is collected, and the CLIP image recognition tool is used to identify the collected images to generate text data containing weed species, status, and location. The images and corresponding text data are then uploaded to the database as an image-text pair dataset.
[0036] By using the CLIP image recognition tool to identify image content, the automatic extraction and semantic description of weed species, status, and spatial location information were achieved, thus forming a highly correlated image-text pair dataset. This process effectively combines visual and semantic information, reduces the workload of manual annotation, and improves the scalability and expressive richness of the dataset.
[0037] Furthermore, noise removal and contrast enhancement are performed on the acquired images. A Tokenizer is used to segment the text data into tokens, which are then input into a word embedding layer to convert them into word vector sequences. Finally, a Vit visual encoder is used to extract the image feature matrix, using the following formula:
[0038] E image =Vit(X) image );
[0039] Among them, E image The extracted image feature matrix is represented by Vit, where Vit is the visual encoder and X is the image feature matrix. image Image data;
[0040] The text feature matrix is extracted using the BERT text encoder, with the following formula:
[0041] E text =BERT(F text );
[0042] Among them, E text The extracted text feature matrix is denoted by F. BERT is the text encoder. text It is a sequence of word vectors.
[0043] Normalize the image feature matrix and the text feature matrix, and calculate the cosine similarity between the normalized image feature matrix and the text feature matrix using the following formula:
[0044]
[0045] Where i is the i-th image, j is the j-th group of text, and S ij Let be the similarity between the i-th image and the j-th text. z is the transpose of the i-th image feature matrix after normalization. text,j Let be the j-th text feature matrix after normalization.
[0046] The G-DINO multimodal large model is trained using a contrastive loss function, as shown in the following formula:
[0047]
[0048] in, For the calculated loss, N is the total number of images acquired, and S... ii For a correctly matched image-text pair, S ij The total number of image-text matches is given, and τ is a parameter that adjusts the learning difficulty.
[0049] Gradient descent is repeatedly used to minimize the loss function until the set upper limit for the number of iterations is reached. The trained G-DINO multimodal large model is then used to extract image and text features, and the extracted features are mapped to a shared feature space.
[0050] By removing noise and enhancing contrast in the acquired images, image quality was optimized and feature information was strengthened. Then, by tokenizing text data using a tokenizer and inputting it into a word embedding layer, and simultaneously extracting image and text features using the ViT visual encoder and BERT text encoder respectively, deep expression and structured representation of multimodal information were achieved. Subsequently, the extracted image and text feature matrices were normalized and cosine similarity was calculated. The G-DINO multimodal large model was then trained under self-supervised conditions using a contrastive loss function and gradient descent, achieving unified alignment and shared space mapping of cross-modal features. This enhanced the model's ability to fuse semantic and visual information about paddy field weeds, improving feature generalization and recognition accuracy in small sample scenarios.
[0051] By combining a shared feature space, a few-shot learning algorithm is used to train the G-DINO multimodal large model, and the trained G-DINO multimodal large model is used to detect weeds in real-time images.
[0052] Specifically, the K-means clustering algorithm is used to divide the text feature matrices in the shared feature space into K categories, and the average of all text feature matrices corresponding to each category is calculated using the following formula:
[0053]
[0054] in, C represents the text center feature of the k-th category cluster. k denoted as the number of text feature matrix and image feature matrix pairs in the k-th category.
[0055] The average of all image feature matrices corresponding to each category is calculated using the following formula:
[0056]
[0057] in, The image center feature is the k-th category cluster.
[0058] The text center features and image center features of each category are weighted and fused to obtain the prototype P of each category. k .
[0059] Using category prototype P kInitialize the classification head of the G-DINO multimodal large model, extract local features of the image, calculate the similarity between the local features of the image and the class prototype, and use the local region with the highest similarity as the pseudo-target box.
[0060] The G-DINO multimodal large model is trained using pseudo-boundaries, and the object detection loss is minimized using gradient descent. When the loss converges, the trained G-DINO multimodal large model is obtained.
[0061] K-means clustering is used to divide text features in the shared feature space into K classes, and the average of text and image features in each class is calculated to achieve semantically consistent cross-modal feature alignment, effectively suppressing noise and building stable class priors. Weighted fusion of text and image center features in each class generates class prototypes, achieving compact representation of multimodal semantics and providing transferable semantic anchors for small sample weed categories. The G-DINO classification head is initialized using class prototypes, and pseudo-target boxes are generated by calculating the similarity between local image features and prototypes, achieving weakly supervised localization guidance without manual box annotation. Finally, a large G-DINO multimodal model is trained based on pseudo-boxes, and the detection loss is optimized through gradient descent until convergence, ensuring that the model fully learns weed features under small sample conditions.
[0062] Furthermore, the real-time acquired image data is input into the G-DINO multimodal large model. The G-DINO multimodal large model outputs a set of predicted candidate boxes, where each candidate box includes its location in the image, the probability of weeds within the candidate box, and the probability distribution of the category prediction results. The confidence score of each candidate box is calculated using the following formula:
[0063] Score m =Pro m ×max a U m,a ;
[0064] Among them, Score m For the confidence score of the m-th candidate box, Pro m Let max be the probability that the m-th candidate box contains weeds. a U m,a Let be the probability that the target in the m-th candidate box belongs to category a.
[0065] Set the confidence score threshold. thr If Score m ≥Score thr If the Score is high, then the candidate bounding box is retained as the object detection result. m <Score thr If it is, then it will not be retained.
[0066] By inputting real-time images into the G-DINO multimodal large model and calculating the confidence scores of each candidate box, a refined scoring of the detection results is achieved, providing a quantifiable and reliable basis for subsequent decision-making. Furthermore, by setting a confidence score threshold for dynamic screening, high-confidence detection results are retained while low-confidence interference boxes are filtered out.
[0067] Weed control plans are generated and implemented based on weed detection results. The weed control plans are then adjusted based on the weed control results, and the experience data generated during the weed control process is uploaded to the database for storage.
[0068] Specifically, a mapping relationship between each type of weed and its corresponding weeding scheme is pre-established as a weeding scheme library, and a recommended weeding method is defined for each type of weed. Based on the identified weed types and their characteristics, weeding schemes are matched from the predefined weeding scheme library.
[0069] By pre-establishing a mapping relationship between weed types and weeding solutions and constructing a weeding solution library, a structured association between weed types and optimal management strategies is realized, providing an executable decision support path for the detection results. Then, by dynamically matching the optimal weeding solution from the solution library based on the identified weed types and their characteristics, personalized and differentiated management based on accurate identification is achieved.
[0070] Furthermore, image data of the area after weeding is collected, the weeding rate of the weeding plan is analyzed and calculated, and a weeding rate threshold is set. If the weeding rate of the plan is greater than or equal to the weeding rate threshold, the weeding plan is considered qualified. If the weeding rate of the plan is less than the weeding rate threshold, the weeding plan is executed again and the intensity of the weeding plan is increased.
[0071] By collecting regional image data after weeding and calculating the actual weeding rate, a quantitative closed-loop evaluation of the weeding plan's effectiveness was achieved. By setting a weeding rate threshold and making a qualification judgment, the effectiveness of the plan was dynamically verified. For plans that did not meet the standards, a second execution was automatically triggered and the treatment intensity was increased, effectively addressing the problem of insufficient treatment caused by weed resistance, environmental interference, or initial identification bias.
[0072] Furthermore, the data from the collected images, weed detection, cleaning operations, and cleaning results are saved to a database.
[0073] By uniformly saving the collected image data, weed detection results, weeding process and cleaning effect data into the database, a complete closed-loop feedback chain from perception to decision-making to verification is constructed, providing a reliable data foundation for continuous learning and optimization.
[0074] This embodiment also provides a computer device applicable to the small sample target detection method for weeds in paddy fields during the seedling stage based on a multimodal large model, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to realize the small sample target detection method for weeds in paddy fields during the seedling stage based on a multimodal large model as proposed in the above embodiment.
[0075] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.
[0076] This embodiment also provides a storage medium storing a computer program. When executed by a processor, the program implements the method for detecting small samples of weeds in paddy field seedlings based on a multimodal large model, as proposed in the above embodiment. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.
[0077] In summary, this invention achieves deep fusion of image and text features and the establishment of a shared feature space by constructing an image-text pair dataset of weeds during the rice paddy seedling stage and using the G-DINO multimodal large model for self-supervised learning. This enhances the feature representation ability and recognition generalization of the G-DINO multimodal large model under small sample conditions. By combining the shared feature space and using a small sample learning algorithm to train the G-DINO multimodal large model, high-precision real-time detection of weeds in rice paddies is achieved. Based on the detection results, weeding schemes are automatically generated and dynamically optimized, improving the intelligence level and overall efficiency of weed detection and weeding operations in rice paddies.
[0078] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.
Claims
1. A method for detecting small-sample targets of weeds in paddy field seedlings based on a multimodal large model, characterized in that: include, Image data of weeds during the seedling stage of rice paddies were collected, and text annotations were added to the images to generate an image-text pair dataset. The G-DINO multimodal large model was used for self-supervised learning to generate feature representations of images and text, and mapped to a shared feature space. By combining the shared feature space, a few-shot learning algorithm is used to train the G-DINO multimodal large model, and the trained G-DINO multimodal large model is used to detect weeds in real-time images. Weed control plans are generated and implemented based on weed detection results. The weed control plans are then adjusted based on the weed control results, and the experience data generated during the weed control process is uploaded to the database for storage.
2. The method for detecting small-sample targets of weeds in paddy field seedlings based on a multimodal large model as described in claim 1, characterized in that: The process of collecting image data of weeds during the seedling stage in paddy fields and annotating the images with text to generate an image-text pair dataset refers to collecting image data of weeds during the seedling stage in paddy fields, using the CLIP image recognition tool to recognize the collected images, generating text data containing weed species, status, and location, and uploading the images and corresponding text data as an image-text pair dataset to the database.
3. The method for detecting small-sample targets of weeds in paddy field seedlings based on a multimodal large model as described in claim 2, characterized in that: The process of using the G-DINO multimodal large model for self-supervised learning to generate feature representations of images and text and mapping them to a shared feature space refers to removing noise and enhancing contrast in the acquired images, using a Tokenizer to segment the text data into tokens, inputting the tokens into a word embedding layer to convert them into word vector sequences, using a Vit visual encoder to extract the image feature matrix, and using a BERT text encoder to extract the text feature matrix. Normalize the image feature matrix and the text feature matrix, and calculate the cosine similarity between the normalized image feature matrix and the text feature matrix. The G-DINO multimodal large model is self-supervised by using a contrastive loss function. The gradient descent method is repeatedly used to minimize the loss function until the set upper limit is reached. The trained G-DINO multimodal large model is then used to extract image and text features, and the extracted features are mapped to a shared feature space.
4. The method for detecting small-sample targets of weeds in paddy field seedlings based on a multimodal large model as described in claim 3, characterized in that: The method of combining the shared feature space and using the few-shot learning algorithm to train the G-DINO multimodal large model refers to using the K-means clustering algorithm to divide the text feature matrix in the shared feature space into K categories, and averaging all the text feature matrices and image feature matrices corresponding to each category. The text center features and image center features of each category are weighted and fused to obtain the prototype of each category; The classification head of the G-DINO multimodal large model is initialized using the category prototype, local features of the image are extracted, and the similarity between the local features of the image and the category prototype is calculated. The local region with the highest similarity is used as the pseudo-target box. The G-DINO multimodal large model is trained using pseudo-boundaries, and the object detection loss is minimized using gradient descent. When the loss converges, the trained G-DINO multimodal large model is obtained.
5. The method for detecting small-sample targets of weeds in paddy fields during the seedling stage based on a multimodal large model as described in claim 4, characterized in that: The process of using the trained G-DINO multimodal large model to detect weeds in real-time images involves inputting the acquired image data into the G-DINO multimodal large model. The G-DINO multimodal large model outputs a set of predicted candidate boxes, which includes the position of the candidate box in the image, the probability of weeds being present in the candidate box, and the probability distribution of the category prediction result. A confidence score is then calculated for each candidate box. m ; Set the confidence score threshold. thr If Score m ≥Score thr If the Score is not high enough, the candidate bounding box is retained as the object detection result. m <Score thr If it is, then it will not be retained.
6. The method for detecting small-sample targets of weeds in paddy field seedlings based on a multimodal large model as described in claim 5, characterized in that: The process of generating and executing a weeding plan based on weed detection results involves pre-establishing a mapping relationship between each type of weed and its corresponding weeding plan as a weeding plan library, defining recommended weeding methods for each weed category, and matching weeding plans from the predefined weeding plan library based on the identified weed categories and their characteristics.
7. The method for detecting small-sample targets of weeds in paddy fields during the seedling stage based on a multimodal large model as described in claim 6, characterized in that: The process of adjusting the weeding plan based on the weeding results involves collecting image data of the area after weeding, analyzing and calculating the weeding rate of the weeding plan, setting a weeding rate threshold, and determining that the weeding plan is qualified if the weeding rate of the plan is greater than or equal to the weeding rate threshold. If the weeding rate of the plan is less than the weeding rate threshold, the weeding plan is executed again and the intensity of the weeding plan is increased.
8. The method for detecting small-sample targets of weeds in paddy fields during the seedling stage based on a multimodal large model as described in claim 7, characterized in that: Uploading the experience data generated during the weeding process to the database for storage refers to saving the data of collected images, weed detection, cleaning execution, and cleaning results to the database.
9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the method for detecting small samples of weeds in paddy field seedlings based on a multimodal large model as described in any one of claims 1 to 8.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the method for detecting small samples of weeds in paddy fields based on a multimodal large model as described in any one of claims 1 to 8.