Navigation line detection system based on agricultural scene
Through a high-resolution RGB color camera and a semantic segmentation model based on Transformer, combined with an anchor point clustering algorithm with triangle fitting and dynamic search direction, and the least squares method fitting navigation routes, the accuracy and efficiency problems of rice seedling row detection in rice fields are solved, and the precise navigation of agricultural machinery is realized, which helps agricultural automation production.
Patent Information
- Application Number
- CN202510461517.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-14
- Publication Date
- 2025-08-08
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
When detecting rice seedlings in rice fields, there are problems such as noise interference from herbs and cyanobacteria, influence of water surface reflection, and uneven distribution of rice leaves, resulting in low detection accuracy and poor generalization ability of traditional image processing technology. The method based on convolutional neural network failed to effectively overcome leaf morphological interference.
The high-resolution RGB color camera is used to collect images, combine image denoising, grayscale and normalization processing, and use a semantic segmentation model based on Transformer to segment rice stems, extract anchor points through triangle fitting, combine anchor point clustering algorithm with dynamic search direction, and least squares method to fit navigation lines to achieve accurate positioning of rice rows.
It improves the accuracy and efficiency of rice row detection, overcomes complex environmental noise interference, reduces costs, provides accurate navigation support for agricultural machinery in rice fields, and improves the level of agricultural production automation.
Smart Images
Figure CN120451775A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of agricultural automation technology, and in particular to a navigation line detection system based on an agricultural scenario. Background Art
[0002] As the global population grows, the demand for food continues to rise. According to statistics, global rice consumption is projected to increase from 480 million tons in 2014 to nearly 550 million tons in 2030. However, in some major rice-producing regions, the growth rate of rice production has gradually slowed, even reaching a plateau. At the same time, the decline in the agricultural labor force has become a major constraint on agricultural development. Against this backdrop, increasing rice yields through automated production has become an effective way to address the challenges of growing global rice demand and a shrinking labor force.
[0003] Automating routine rice paddy management tasks, such as spraying pesticides, fertilizing, and weeding, would not only improve production efficiency but also reduce labor costs. The key to achieving these automated operations lies in precise, automated machine guidance. Visual navigation technology, with its low cost, high performance, flexibility, and ease of use, is an ideal choice for rice field row tracking. However, before the deployment of visual navigation systems, accurately detecting rows of young rice seedlings remained a pressing challenge.
[0004] Detecting rice seedling rows from early-stage rice images presents numerous challenges. For one thing, weed noise from herbaceous plants (such as sheath grass) and algae from cyanobacteria can interfere with rice seedling recognition. Furthermore, reflections from water can affect image acquisition. Furthermore, the growth characteristics of rice plants themselves complicate detection. Rice leaves expand in different directions during growth, resulting in an uneven distribution of plant pixels in the image. Relying solely on leaf recognition makes it difficult to accurately fit rice rows. Traditional image processing techniques, such as extracting feature points from rice plants, separating plants using blob extraction, and combining color indices with algorithms for plant segmentation, can identify rice seedlings. However, due to their reliance on limited domain knowledge, they suffer from low detection accuracy and poor generalization in complex field environments.
[0005] Although the convolutional neural network (CNN)-based method has improved the row detection performance to a certain extent, it still has the problem of using the pixels of the entire rice plant image to extract crop rows, and fails to fully consider the interference of rice leaf morphological characteristics on row detection. Therefore, it is necessary to design a navigation line detection system based on agricultural scenarios. Summary of the Invention
[0006] The present invention provides a navigation line detection system based on agricultural scenarios to solve the problems raised in the background technology, thereby improving the accuracy and efficiency of rice row detection, providing a reliable basis for the automated navigation of agricultural machinery in complex farmland environments, and thus promoting the automation and intelligent development of agricultural production.
[0007] The present invention provides a navigation line detection system based on agricultural scenarios, comprising an image acquisition module, a data preprocessing module, a semantic segmentation model module, a triangle fitting module, an anchor point aggregation module and a least squares fitting module;
[0008] The image acquisition module is connected to the data preprocessing module, the data preprocessing module is connected to the semantic segmentation model module, and the semantic segmentation model module is sequentially connected to the triangle fitting module, the anchor point aggregation module and the least squares fitting module;
[0009] An image acquisition module is used to acquire rice seedling images using a high-resolution RGB color camera and transmit the acquired image data to a data preprocessing module;
[0010] The data preprocessing module preprocesses the received image data, including image denoising, grayscale conversion, and normalization operations. The preprocessed image data is output to the semantic segmentation model module;
[0011] The semantic segmentation model module uses a Transformer-based semantic segmentation model to segment rice seedling stems. The semantic segmentation model includes three architectures: segmentation Transformer, shifted window Transformer, and SegFormer. The model outputs the segmentation mask of the rice seedling stem to the triangle fitting module;
[0012] The triangle fitting module uses the limited triangle fitting technology to extract anchor points from the segmented stem mask, uses the bottom endpoint of the triangle as the cropping reference, and outputs the extracted anchor points to the anchor point aggregation module;
[0013] The anchor point clustering module divides the predicted rice stalk mask binary image into several blocks, determines the basic search direction for clustering the initial row of rice seedlings through vertical projection, and searches both up and down from the initial point. The anchor point closest to the fitting line is used as the new basic search point, and the anchor points of adjacent rice rows are clustered. The clustered anchor points are output to the least squares fitting module.
[0014] The least squares fitting module performs least squares fitting on the clustered anchor point rows to obtain the navigation lines of the rice rows.
[0015] Preferably, a medium to high resolution RGB color camera is provided in the image acquisition module, and the high resolution RGB color camera is installed on the agricultural machinery to collect rice seedling images in real time as the agricultural machinery moves in the rice field.
[0016] Preferably, in the data preprocessing module, Gaussian filtering algorithm is used for image denoising, weighted averaging method is used for grayscale conversion, and normalization is performed to map image pixel values into the range of [0, 1].
[0017] Preferably, during training, the semantic segmentation model module uses a morphology-based superpixel masking method to annotate rice seedling images, generates an annotated data set, and uses the annotated data set to train three models: segmentation Transformer, shifted window Transformer, and SegFormer.
[0018] Preferably, when extracting anchor points, the triangle fitting module first calculates the convex hull of the stem mask, then fits a triangle on the convex hull, uses the bottom endpoint of the triangle as the anchor point, performs quality assessment on the extracted anchor points, and removes erroneous or noisy anchor points.
[0019] Preferably, when determining the initial search direction, the anchor point aggregation module performs vertical projection on each image block, calculates the cumulative number of pixels, determines the basic search direction based on the vertical projection result, and continuously updates the search direction and search range during the search process.
[0020] Preferably, when fitting the anchor row, the least squares fitting module selects a suitable fitting model such as a linear fitting or a quadratic curve fitting according to the actual shape of the rice row.
[0021] Preferably, a detection method based on a navigation line detection system in an agricultural scenario includes the following steps:
[0022] A. Using a high-resolution RGB color camera installed on an agricultural machine, as the machine moves in the rice field, real-time images of rice seedlings are collected and the collected image data is transmitted to the data preprocessing module;
[0023] B. After receiving the image data, the data preprocessing module performs image denoising, grayscale conversion, and normalization operations in sequence. It uses the Gaussian filter algorithm to remove noise from the image; it converts the color image into a grayscale image using the weighted average method; and through normalization, it maps the image pixel values to the range of [0, 1] to enhance image contrast and stability, providing high-quality input images for the semantic segmentation model module.
[0024] C. The preprocessed image is input into the Transformer-based semantic segmentation model module. The model performs deep feature extraction and analysis on the rice seedling stems. After training, it learns the characteristic patterns of the stems and outputs an accurate rice seedling stem segmentation mask, providing key data for subsequent anchor point extraction.
[0025] D. The triangle fitting module receives the segmentation mask, calculates the convex hull of the stem mask and fits the triangle, using the triangle bottom endpoints as the cropping reference to extract anchor points. At the same time, the quality of the extracted anchor points is evaluated to remove erroneous anchor points or those that are severely affected by noise.
[0026] E. The anchor point clustering module divides the predicted rice stalk mask binary image into blocks, performs vertical projection on each block, and determines the basic search direction for clustering the initial row of rice seedlings based on the cumulative number of pixels. Starting from the initial point, a synchronous search is performed up and down, using the anchor point closest to the fitting line as the new basic search point. This process is iterated continuously until all anchor points are clustered. The scattered anchor points are then clustered by row to provide ordered anchor point data for navigation line fitting.
[0027] F. The least squares fitting module receives the clustered anchor point data and, based on the least squares principle, selects an appropriate fitting model to accurately fit the anchor point rows. This generates navigation lines representing the location and direction of the rice rows, guiding agricultural machinery to accurately navigate the rice fields and complete automated operations such as spraying, fertilizing, and weeding.
[0028] Beneficial effects:
[0029] (1) The present invention uses rice stalks as the identification object, effectively overcoming the interference of leaves and the influence of complex environmental noise. At the same time, it adopts an innovative labeling method to reduce costs, improve detection accuracy and system robustness, provide strong support for the precise navigation of agricultural machinery in rice fields, and promote efficient agricultural production.
[0030] (2) This invention addresses the problem of a small number of rice stalk anchor points and their sparse distribution. The proposed clustering algorithm based on dynamic search direction can effectively cluster the anchor points. This algorithm combines vertical projection and iterative search methods, does not require prior knowledge, such as the number of crop rows and the spacing between rows, and improves the robustness of the system in actual field scenarios. This algorithm can accurately group anchor points into rows, providing a reliable data foundation for subsequent navigation line fitting.
[0031] The above description is only an overview of the technical solutions of the embodiments of the present invention. In order to more clearly understand the technical means of the embodiments of the present invention, they can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the embodiments of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are specifically listed below. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.
[0033] Figure 1 This is a block diagram of the system principle of the present invention;
[0034] Figure 2 It is a flowchart of the present invention;
[0035] Explanation of the accompanying symbols: image acquisition module 1, data preprocessing module 2, semantic segmentation model module 3, triangle fitting module 4, anchor point aggregation module 5, least squares fitting module 6. DETAILED DESCRIPTION
[0036] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.
[0037] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.
[0038] See also Figure 1-Figure 2 , the present invention discloses a navigation line detection system based on an agricultural scene, comprising an image acquisition module 1, a data preprocessing module 2, a semantic segmentation model module 3, a triangle fitting module 4, an anchor point aggregation module 5 and a least squares fitting module 6;
[0039] The image acquisition module 1 is connected to the data preprocessing module 2, the data preprocessing module 2 is connected to the semantic segmentation model module 3, and the semantic segmentation model module 3 is sequentially connected to the triangle fitting module 4, the anchor point aggregation module 5 and the least squares fitting module 6;
[0040] Image acquisition module 1, used to capture rice seedling images using a high-resolution RGB color camera and transmit the captured image data to the data preprocessing module;
[0041] Data preprocessing module 2 preprocesses the received image data, including image denoising, grayscale conversion and normalization operations, and outputs the preprocessed image data to the semantic segmentation model module;
[0042] Semantic segmentation model module 3, which uses a Transformer-based semantic segmentation model to segment the rice seedling stems. The semantic segmentation model includes three architectures: segmentation Transformer, shifted window Transformer, and SegFormer. The model outputs the segmentation mask of the rice seedling stems to the triangle fitting module;
[0043] Triangle fitting module 4, which uses limited triangle fitting technology to extract anchor points from the segmented stem mask, uses the bottom endpoints of the triangle as the cropping reference, and outputs the extracted anchor points to the anchor point aggregation module;
[0044] Anchor point clustering module 5 divides the predicted rice stalk mask binary image into several blocks, determines the basic search direction for clustering the initial row of rice seedlings through vertical projection, searches both up and down from the initial point, uses the anchor point closest to the fitting line as the new basic search point, clusters the anchor points of adjacent rice rows, and outputs the clustered anchor points to the least squares fitting module;
[0045] The least squares fitting module 6 performs least squares fitting on the clustered anchor point rows to obtain the navigation lines of the rice rows.
[0046] The image acquisition module 1 includes a medium- to high-resolution RGB color camera, which is mounted on an agricultural machine and collects images of rice seedlings in real time as the machine moves through the paddy field. These images capture features such as the color and shape of the rice seedlings, providing basic data for subsequent processing and analysis. In practical applications, the camera is installed in a suitable position to ensure that clear and complete images of the rice seedlings can be captured. For example, the camera can be mounted on the front end of the agricultural machine, collecting image data of the rice seedlings in real time as the machine moves through the paddy field. The collected image data is transmitted to the image acquisition module via a data transmission interface for preprocessing.
[0047] In the data preprocessing module of this invention, image denoising uses a Gaussian filter algorithm, grayscale conversion uses a weighted average method, and normalization maps image pixel values to the range of [0, 1] to improve image contrast and stability. These preprocessing operations can improve image quality and provide a better data foundation for subsequent model training and analysis.
[0048] In the present invention, during training, the semantic segmentation model module uses a morphology-based superpixel-level masking method to annotate rice seedling images, generating an annotated dataset. This annotated dataset is then used to train three models: the segmentation Transformer, the shifted window Transformer, and the SegFormer. The annotated data is annotated using a morphology-based superpixel-level masking method. This annotation method simplifies the semantic annotation process in complex natural scenes, reduces the workload and cost of manual annotation, and maintains an appropriate description of the stem posture. Through training, the model is able to learn the characteristics of the rice seedling stems, thereby achieving accurate segmentation of the stems.
[0049] In this invention, the triangle fitting module first calculates the convex hull of the stalk mask to extract anchor points. It then fits triangles on the convex hull, using the bottom endpoints of the triangles as anchor points. The extracted anchor points are then quality-assessed and any erroneous or noisy anchor points are removed. This method ensures that the bottom endpoints of the fitted triangles can be used as cropping anchor points, reducing noise interference and improving the accuracy and convenience of anchor point detection. The anchor points represent the location where rice seedlings emerge from the soil, providing critical information for subsequent row detection and navigation.
[0050] In this invention, the anchor point clustering module determines the initial search direction by vertically projecting each image block, calculating the cumulative number of pixels, and then determining the basic search direction based on the vertical projection results. The search direction and search range are continuously updated during the search process. This clustering algorithm based on dynamic search directions effectively addresses the problem of a small number of sparsely distributed anchor points, enabling efficient clustering of sparse anchor points.
[0051] Furthermore, in the present invention, the least squares fitting module selects an appropriate fitting model, either a linear linear fit or a quadratic curve fit, based on the actual rice row shape when fitting the anchor point rows. In this invention, the least squares method is used to fit the anchor point rows to obtain a rice row navigation line. This navigation line accurately reflects the position and direction of the rice rows, providing a reliable basis for agricultural machinery navigation in rice fields. Based on this navigation line, agricultural machinery can perform automated driving, pesticide spraying, and fertilization operations, thereby improving the automation and efficiency of agricultural production.
[0052] Working principle: A detection method based on a navigation line detection system in an agricultural scenario includes the following steps:
[0053] A. Using a high-resolution RGB color camera installed on an agricultural machine, as the machine moves in the rice field, real-time images of rice seedlings are collected and the collected image data is transmitted to the data preprocessing module;
[0054] B. After receiving the image data, the data preprocessing module performs image denoising, grayscale conversion, and normalization operations in sequence. It uses the Gaussian filter algorithm to remove noise from the image; it converts the color image into a grayscale image using the weighted average method; and through normalization, it maps the image pixel values to the range of [0, 1] to enhance image contrast and stability, providing high-quality input images for the semantic segmentation model module.
[0055] C. The preprocessed image is input into the Transformer-based semantic segmentation model module. The model performs deep feature extraction and analysis on the rice seedling stems. After training, it learns the characteristic patterns of the stems and outputs an accurate rice seedling stem segmentation mask, providing key data for subsequent anchor point extraction.
[0056] D. The triangle fitting module receives the segmentation mask, calculates the convex hull of the stem mask and fits the triangle, using the triangle bottom endpoints as the cropping reference to extract anchor points. At the same time, the quality of the extracted anchor points is evaluated to remove erroneous anchor points or those that are severely affected by noise.
[0057] E. The anchor point clustering module divides the predicted rice stalk mask binary image into blocks, performs vertical projection on each block, and determines the basic search direction for clustering the initial row of rice seedlings based on the cumulative number of pixels. Starting from the initial point, a synchronous search is performed up and down, using the anchor point closest to the fitting line as the new basic search point. This process is iterated continuously until all anchor points are clustered. The scattered anchor points are then clustered by row to provide ordered anchor point data for navigation line fitting.
[0058] F. The least squares fitting module receives the clustered anchor point data and, based on the least squares principle, selects an appropriate fitting model to accurately fit the anchor point rows. This generates navigation lines representing the location and direction of the rice rows, guiding agricultural machinery to accurately navigate the rice fields and complete automated operations such as spraying, fertilizing, and weeding.
[0059] In summary, the present invention uses rice stalks as the identification object, effectively overcoming the interference of leaves and the influence of complex environmental noise. At the same time, it adopts innovative labeling methods to reduce costs, improve detection accuracy and system robustness, provide strong support for the precise navigation of agricultural machinery in rice fields, and promote efficient agricultural production.
[0060] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions described in the above embodiments can still be modified, or some of the technical features thereof can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A navigation line detection system based on agricultural scenarios, characterized in that: It includes an image acquisition module (1), a data preprocessing module (2), a semantic segmentation model module (3), a triangle fitting module (4), an anchor point aggregation module (5) and a least squares fitting module (6); The image acquisition module (1) is connected to the data preprocessing module (2), the data preprocessing module (2) is connected to the semantic segmentation model module (3), and the semantic segmentation model module (3) is sequentially connected to the triangle fitting module (4), the anchor point aggregation module (5) and the least squares fitting module (6); An image acquisition module (1) is used to acquire rice seedling images using a high-resolution RGB color camera and transmit the acquired image data to a data preprocessing module; The data preprocessing module (2) preprocesses the received image data, including image denoising, grayscale conversion and normalization operations, and outputs the preprocessed image data to the semantic segmentation model module; Semantic segmentation model module (3), which uses a Transformer-based semantic segmentation model to segment rice seedling stems. The semantic segmentation model includes three architectures: segmentation Transformer, shift window Transformer, and SegFormer. The model outputs the segmentation mask of the rice seedling stems to the triangle fitting module; The triangle fitting module (4) extracts anchor points from the segmented stem mask using a limited triangle fitting technique, uses the bottom endpoints of the triangles as the cropping reference, and outputs the extracted anchor points to the anchor point aggregation module; Anchor point clustering module (5) divides the predicted rice stalk mask binary image into several blocks, determines the basic search direction of the initial row rice seedling cluster by vertical projection, searches up and down from the initial point simultaneously, takes the anchor point closest to the fitting line as the new basic search point, clusters the anchor points of adjacent rice rows, and outputs the clustered anchor points to the least squares fitting module; The least square fitting module (6) performs least square fitting on the clustered anchor point rows to obtain the navigation line of the rice row.
2. The navigation line detection system based on agricultural scenarios according to claim 1 is characterized in that: The image acquisition module (1) is provided with a medium- and high-resolution RGB color camera, which is mounted on the agricultural machine and collects images of rice seedlings in real time as the agricultural machine moves in the rice field.
3. The navigation line detection system based on agricultural scenarios according to claim 1 is characterized in that: In the data preprocessing module, Gaussian filtering algorithm is used for image denoising, weighted averaging method is used for grayscale conversion, and normalization is used to map image pixel values to the range of [0, 1].
4. The navigation line detection system based on agricultural scenarios according to claim 1 is characterized in that: During training, the semantic segmentation model module uses a morphology-based superpixel masking method to annotate rice seedling images, generates annotated datasets, and uses the annotated datasets to train three models: the segmentation Transformer, the shifted window Transformer, and the SegFormer.
5. The navigation line detection system based on agricultural scenarios according to claim 1 is characterized in that: When extracting anchor points, the triangle fitting module first calculates the convex hull of the stem mask, then fits a triangle on the convex hull, uses the bottom endpoint of the triangle as the anchor point, performs quality assessment on the extracted anchor points, and removes erroneous or noisy anchor points.
6. The navigation line detection system based on agricultural scenarios according to claim 1 is characterized in that: When determining the initial search direction, the anchor point aggregation module performs vertical projection on each image block, calculates the cumulative number of pixels, determines the basic search direction based on the vertical projection result, and continuously updates the search direction and search range during the search process.
7. The navigation line detection system based on agricultural scenarios according to claim 1 is characterized in that: When fitting the anchor row, the least squares fitting module selects a suitable fitting model of linear fitting or quadratic curve fitting according to the actual shape of the rice row.
8. Implementing the detection method of a navigation line detection system based on an agricultural scenario as described in claim 1, characterized in that: The following steps are involved: A. Using a high-resolution RGB color camera installed on an agricultural machine, as the machine moves in the rice field, real-time images of rice seedlings are collected and the collected image data is transmitted to the data preprocessing module; B. After receiving the image data, the data preprocessing module performs image denoising, grayscale conversion and normalization operations in sequence, and uses the Gaussian filtering algorithm to remove noise in the image; The weighted averaging method is used to convert color images into grayscale images. Through normalization, the image pixel values are mapped to the range of [0, 1] to enhance the image contrast and stability, providing high-quality input images for the semantic segmentation model module. C. The preprocessed image is input into the Transformer-based semantic segmentation model module. The model performs deep feature extraction and analysis on the rice seedling stems. After training, it learns the characteristic patterns of the stems and outputs an accurate rice seedling stem segmentation mask, providing key data for subsequent anchor point extraction. D. The triangle fitting module receives the segmentation mask, calculates the convex hull of the stem mask and fits the triangle, using the triangle bottom endpoints as the cropping reference to extract anchor points. At the same time, the quality of the extracted anchor points is evaluated to remove erroneous anchor points or those that are severely affected by noise. E. The anchor point clustering module divides the predicted rice stalk mask binary image into blocks, performs vertical projection on each block, and determines the basic search direction for clustering the initial row of rice seedlings based on the cumulative number of pixels. Starting from the initial point, a synchronous search is performed up and down, using the anchor point closest to the fitting line as the new basic search point. This process is iterated continuously until all anchor points are clustered. The scattered anchor points are then clustered by row to provide ordered anchor point data for navigation line fitting. F. The least squares fitting module receives the clustered anchor point data and, based on the least squares principle, selects an appropriate fitting model to accurately fit the anchor point rows. This generates navigation lines representing the location and direction of the rice rows, guiding agricultural machinery to accurately navigate the rice fields and complete automated operations such as spraying, fertilizing, and weeding.