Open field vegetable row identification method, electronic equipment, device and storage medium
By utilizing crop segmentation and recognition models during the seedling and plant adhesion stages respectively, combined with the YOLOv8s model and feature pyramid network, accurate crop row recognition throughout the entire growth cycle of open-field vegetables was achieved, solving the recognition problem during the plant adhesion stage and improving recognition accuracy and robustness.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- INTELLIGENT EQUIPMENT RESEARCH CENTER BEIJING ACADEMY OF AGRICULTURE AND FORESTRY SCIENCES
- Filing Date
- 2025-12-25
- Publication Date
- 2026-05-15
AI Technical Summary
Existing technologies struggle to achieve accurate crop row identification throughout the entire growth cycle of open-field vegetables, especially during the period when plants are sticking together, resulting in low accuracy or no identification at all.
A crop segmentation model is used to obtain individual mask contours and calculate center points during the seedling stage, and parallel scan lines are used to determine crop row boundaries. During the plant adhesion stage, a crop row recognition model is used to obtain the midpoints of crop row mask contours and fit center lines using vertical scan lines. Image processing is performed by combining the YOLOv8s model and feature pyramid network.
It has achieved accurate crop row identification throughout the entire growth cycle of open-field vegetables, solved the identification problem during the period of plant adhesion, and improved identification accuracy and robustness.
Smart Images
Figure CN122049889A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of crop cultivation technology, and in particular to a method, electronic device, apparatus, and storage medium for identifying rows of open-field vegetables. Background Technology
[0002] With the rapid development of agricultural automation and intelligence, precision agriculture has become a key direction for improving agricultural production efficiency and reducing resource waste. Among them, accurate crop row identification of field crops is the foundation for realizing automated operations such as sowing, fertilization, spraying, weeding, and yield estimation.
[0003] Current methods for identifying crop rows in open-field vegetables primarily rely on convolutional operations to extract features such as color, texture, and shape from images. However, the morphological characteristics of open-field vegetables change significantly throughout their growth cycle, especially in the later stages when plants often stick together and occlude, resulting in blurred individual boundaries. This makes existing crop row identification methods based on individual identification or simple geometric analysis inapplicable, leading to low accuracy or even ineffective identification during periods of crop sticking together.
[0004] Therefore, how to achieve accurate identification of crop rows throughout the entire growth cycle of open-field vegetables, especially solving the problem of identifying crop rows during the sticky period, is a key technical challenge that urgently needs to be addressed. Summary of the Invention
[0005] This invention provides a method, electronic device, apparatus, and storage medium for identifying rows of open-field vegetables, which solves the problem that existing technologies cannot accurately identify crop rows throughout the entire growth cycle of open-field vegetables, and achieves accurate identification of crop rows throughout the entire growth cycle of open-field vegetables.
[0006] This invention provides a method for identifying rows of open-field vegetables, comprising: Acquire images of open-field vegetables at different growth stages; When the open-field vegetables are in the seedling stage and the individual plants are not stuck together, a crop segmentation model is used to obtain the individual mask contour of each open-field vegetable in the image, and the center point of each individual mask contour is calculated; the open-field vegetable image is traversed by a first scan line parallel to the main direction of the crop row to determine the two side boundaries of all crop rows; all the center points contained in the two side boundaries of each crop row are fitted to obtain the crop row center line corresponding to the two side boundaries of each crop row; When open-field vegetables are in the stage of stalk-adhesive growth, a crop row recognition model is used to obtain multiple crop row mask contours in the open-field vegetable image; the open-field vegetable image is traversed by a second scan line perpendicular to the main direction of the crop row to obtain the midpoint of each crop row mask contour in each scan step; all the midpoints contained in each crop row mask contour are fitted to obtain the crop row centerline corresponding to each crop row mask contour.
[0007] In some embodiments, the step of traversing the open-field vegetable image using a first scan line parallel to the main direction of the crop rows to determine the lateral boundaries of all crop rows includes: Starting from one side of the open-field vegetable image, when the first scan line first intersects with two or more individual mask contours, the current position of the first scan line is marked as one side boundary of the first crop row; Continue scanning to the other side of the open-field vegetable image. When the number of intersections between the first scan line and the individual mask contour decreases to one, mark the current position of the first scan line as the other side boundary of the first crop row. Repeat the above steps until you have traversed to the other side of the open-field vegetable image and obtained the two side boundaries of all crop rows.
[0008] In some embodiments, the step of traversing the open-field vegetable image using a second scan line perpendicular to the main direction of the crop rows to obtain the midpoint of the mask contour of each crop row in each scan step includes: In each scan step, the leftmost and rightmost intersection points of the second scan line with the mask contour of each crop row are obtained; Calculate the center coordinates of the leftmost and rightmost intersection points to obtain the midpoint of each crop row mask contour in each scan step.
[0009] In some embodiments, the method further includes: A virtual horizontal line is preset in the camera's field of view of the open-field vegetable image; The coordinates of the intersection of the crop row centerline and the virtual horizontal line are used as the location information; The location information is sent to the nozzle of the agricultural machinery through a fixed distance compensation mechanism.
[0010] In some embodiments, the crop segmentation model is constructed based on the YOLOv8s model by introducing an efficient multi-scale attention module in the C2f module of the 3rd and 5th layers of the backbone network, a convolutional block attention module in the 10th layer of the backbone network, and a weighted bidirectional feature pyramid network in the neck network.
[0011] In some embodiments, the CBS module in the YOLOv8s model uses the SiLU function.
[0012] The present invention also provides a field vegetable row identification device, comprising: The acquisition module is used to acquire images of open-field vegetables at different growth stages; The first recognition module is used to obtain the individual mask contour of each open-field vegetable in the open-field vegetable image using a crop segmentation model when the open-field vegetables are in the seedling stage and the individual plants are not stuck together, and to calculate the center point of each individual mask contour; to perform a traversal scan of the open-field vegetable image using a first scan line parallel to the main direction of the crop row, and to determine the two side boundaries of all crop rows; and to fit all the center points contained in the two side boundaries of each crop row to obtain the crop row center line corresponding to the two side boundaries of each crop row. The second recognition module is used to obtain multiple crop row mask contours in the open-field vegetable image when the open-field vegetables are in the period of stalk sticking together; to perform a traversal scan of the open-field vegetable image using a second scan line perpendicular to the main direction of the crop row, and to obtain the midpoint of each crop row mask contour in each scan step; and to fit all the midpoints contained in each crop row mask contour to obtain the crop row centerline corresponding to each crop row mask contour.
[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the open-field vegetable row identification method as described above.
[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the open-field vegetable row identification method as described above.
[0015] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the open-field vegetable row identification method as described above.
[0016] The present invention provides a method, electronic device, apparatus, and storage medium for identifying rows of open-field vegetables. When open-field vegetables are in the seedling stage and the individual plants are not adhered together, the method obtains the individual mask contour of each plant through a crop segmentation model and calculates the center point of the individual mask contour. A first scan line parallel to the main direction of the crop row is used to traverse the open-field vegetable image, determining the two side boundaries of all crop rows. The center points of all individuals within the two side boundaries of each crop row are fitted to generate the crop row center line corresponding to the two side boundaries of each crop row. This method fully utilizes the advantage of clear individual plants in the seedling stage, ensuring the accuracy of crop row identification during the individual plant dispersion stage through precise positioning of the center of each plant and subsequent statistical fitting. When open-field vegetables are in the adhering growth stage, the crop row identification model is used to identify and segment the entire adhering crop row as a whole, obtaining multiple crop row mask contours. A second scan line perpendicular to the main direction of the crop row is used to traverse the open-field vegetable image, obtaining the midpoint of each crop row mask contour in each scan step. By fitting the midpoints of the mask contours of each crop row acquired in all scanning steps, a centerline corresponding to each crop row mask contour is generated. This method no longer insists on identifying individual plants, but treats adhered crop rows as a continuous region. By stably acquiring and fitting the midpoints on multiple cross-sections, it effectively solves the problem of blurred individual boundaries during the adhesion period, and can accurately delineate the center trajectory of crop rows even in complex adhesion environments. Ultimately, it achieves accurate crop row identification throughout the entire growth cycle of open-field vegetables. Attached Figure Description
[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0018] Figure 1 This is a flowchart illustrating the open-field vegetable row identification method provided by the present invention.
[0019] Figure 2 This is a schematic diagram of the crop segmentation model provided by the present invention.
[0020] Figure 3 This is a schematic diagram of the structure of the open-field vegetable row identification device provided by the present invention.
[0021] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation
[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.
[0023] Figure 1 This is a flowchart illustrating the open-field vegetable row identification method provided by the present invention, as shown below. Figure 1 As shown, this embodiment of the invention provides a method for identifying rows of open-field vegetables. The executing entity can be an edge computing device, and includes the following steps: Step 110: Obtain images of open-field vegetables at different growth stages.
[0024] Specifically, images of open-field vegetables at different growth stages are acquired. These images can be digital images captured above or to the side of the open-field vegetable field using optical sensors (such as cameras), intended to serve as input data for subsequent crop row identification processing.
[0025] Step 120: When the open-field vegetables are in the seedling stage and the individual plants are not stuck together, use the crop segmentation model to obtain the individual mask contour of each open-field vegetable in the image and calculate the center point of each individual mask contour; use the first scan line parallel to the main direction of the crop row to perform a traversal scan of the open-field vegetable image to determine the two side boundaries of all crop rows; fit all the center points contained in the two side boundaries of each crop row to obtain the crop row center line corresponding to the two side boundaries of each crop row.
[0026] Specifically, when open-field vegetables are in the seedling stage and individual plants are not stuck together, a crop row identification strategy based on individual segmentation is adopted, which includes: A pre-trained crop segmentation model is used to process images of open-field vegetables to obtain the individual mask contour of each vegetable plant. Subsequently, the center point of each individual mask contour is calculated and recorded as the representative position of each vegetable plant in the image. Based on the distribution of all identified individual mask contours, it can be determined whether the main direction of the crop rows in the open-field vegetable image is vertical or horizontal.
[0027] Set a first scan line parallel to the main direction of the crop rows. Then, scan the open-field vegetable image pixel by pixel or gradually increasing in length along the first scan line in a direction perpendicular to the main direction of the crop rows. During the scan, determine the two side boundaries of all crop rows by statistically analyzing the changes in the number of intersections between the first scan line and individual mask contours.
[0028] If the main direction of the crop row is vertical, then the first scan line is a vertical scan line with a length equal to the height of the open-field vegetable image. The first scan line traverses the open-field vegetable image pixel by pixel or gradually increasing in length from left to right or from right to left. During the traversal, the left and right boundaries of each crop row are determined by statistically analyzing the changes in the number of intersections between the scan line and the individual mask contours.
[0029] If the main direction of the crop row is horizontal, then the first scan line is a horizontal scan line with a length equal to the width of the open-field vegetable image. The first scan line traverses the open-field vegetable image horizontally from top to bottom or from bottom to top, pixel by pixel or gradually increasing in length. During the traversal, the upper and lower boundaries of each crop row are determined by statistically analyzing the changes in the number of intersections between the scan line and the individual mask contours.
[0030] Determine the two sides of the crop row where the center point of each individual mask contour falls. Using the least squares method, fit all the center points contained in the two sides of each crop row to obtain the crop row center line corresponding to the two sides of each crop row, which is the crop row center line of seedling open field vegetables.
[0031] Step 130: When the open-field vegetables are in the period of stalk-adhesive growth, use the crop row recognition model to obtain multiple crop row mask contours in the open-field vegetable image; use the second scan line perpendicular to the main direction of the crop row to perform a traversal scan of the open-field vegetable image to obtain the midpoint of each crop row mask contour in each scan step; fit all the midpoints contained in each crop row mask contour to obtain the crop row centerline corresponding to each crop row mask contour.
[0032] Specifically, when open-field vegetables are in a period of intertwined growth, an identification strategy based on the overall crop row is adopted, which includes: A pre-trained crop row recognition model is used to process open-field vegetable images to obtain the overall mask contours of multiple interconnected crop rows. These crop row mask contours represent continuous, lush crop strips in the open-field vegetable image. Based on the distribution of all identified crop row mask contours, it can be determined whether the main direction of the crop rows in the open-field vegetable image is vertical or horizontal. Set a second scan line perpendicular to the main direction of the crop rows. The second scan line is then used to traverse the open-field vegetable image pixel by pixel or step by step in a direction parallel to the main direction of the crop rows. During the traversal, the midpoint of each crop row mask contour in each scan step is determined by statistically analyzing the intersection points of the second scan line with the mask contour of each crop row.
[0033] If the main direction of the crop rows is vertical, the second scan line is a horizontal scan line with a length equal to the width of the open-field vegetable image, allowing the second scan line to traverse the open-field vegetable image vertically from top to bottom or from bottom to top. If the main direction of the crop rows is horizontal, the second scan line is a vertical scan line with a length equal to the height of the open-field vegetable image, allowing the second scan line to traverse the open-field vegetable image horizontally from left to right or from right to left.
[0034] Identify all midpoints included in the mask outline of each crop row. Using the least squares method, obtain the crop row centerline corresponding to each crop row mask outline for all midpoints included in each crop row mask outline, thus obtaining the crop row centerline of open-field vegetables during the adhesion period.
[0035] The open-field vegetable row recognition method provided by this invention, when the open-field vegetables are in the seedling stage and the individual plants are not sticking together, obtains the individual mask contour of each plant through a crop segmentation model and calculates the center point of the individual mask contour. A first scan line parallel to the main direction of the crop row is used to traverse the open-field vegetable image, determining the two side boundaries of all crop rows. All individual center points within the two side boundaries of each crop row are fitted to generate the crop row center line corresponding to the two side boundaries of each crop row. This method fully utilizes the advantage of clear individual plants in the seedling stage, ensuring the accuracy of crop row recognition during the individual plant dispersion stage through precise positioning of the center of each plant and subsequent statistical fitting. When the open-field vegetables are in the sticking growth stage, the crop row recognition model is used to identify and segment the sticking crop rows as a whole, obtaining multiple crop row mask contours. A scan line perpendicular to the main direction of the crop row is used to traverse the open-field vegetable image, obtaining the midpoint of each crop row mask contour in each scan step. By fitting the midpoints of the mask contours of each crop row acquired in all scanning steps, a centerline corresponding to each crop row mask contour is generated. This method no longer insists on identifying individual plants, but treats adhered crop rows as a continuous region. By stably acquiring and fitting the midpoints on multiple cross-sections, it effectively solves the problem of blurred individual boundaries during the adhesion period, and can accurately delineate the center trajectory of crop rows even in complex adhesion environments. Ultimately, it achieves accurate crop row identification throughout the entire growth cycle of open-field vegetables.
[0036] In some embodiments, a first scan line parallel to the main direction of the crop rows is used to perform a traversal scan of the open-field vegetable image to determine the lateral boundaries of all crop rows, including: Starting from one side of the open-field vegetable image, when the first scan line first intersects with the mask contour of two or more individuals, the current position of the first scan line is marked as one side boundary of the first crop row; Continue scanning to the other side of the open-field vegetable image. When the number of intersections between the first scan line and the individual mask contour decreases to one, mark the current position of the first scan line as the other side boundary of the first crop row. Repeat the above steps until you have traversed to the other side of the open-field vegetable image and obtained the two side boundaries of all crop rows.
[0037] Specifically, taking a vertical scan line as an example, when the first scan line intersects with two or more individual mask contours for the first time, the current position of the first scan line is marked as the "left boundary" of the first crop row. As the first scan line continues to move to the other side, when the number of intersections between the first scan line and the individual mask contours decreases to one, the current position of the first scan line is marked as the "right boundary" of the first crop row. This process is repeated until the entire width of the open-field vegetable image has been traversed, thereby determining the left and right boundaries of all crop rows and achieving image region division.
[0038] The open-field vegetable row identification method provided by this invention determines the two side boundaries of the crop row by measuring the change in the number of intersections between the first scan line and the individual mask outline during the seedling stage when individual crops are clear but may not be completely regular. This enables efficient and robust automatic determination of the spatial separation between crop rows, thus providing an accurate zoning basis for subsequent precise crop row centerline fitting.
[0039] In some embodiments, a second scan line perpendicular to the main direction of the crop rows is used to perform a traversal scan of the open-field vegetable image to obtain the midpoint of the mask contour of each crop row in each scan step, including: In each scan step, the leftmost and rightmost intersection points of the second scan line with the mask contour of each crop row are obtained; Calculate the center coordinates of the leftmost and rightmost intersection points to obtain the midpoint of the mask contour for each crop row in each scan step.
[0040] Specifically, in each scan step, the intersection points of the second scan line and each crop row mask contour are recorded. For each crop row mask contour, the center coordinates of the leftmost and rightmost intersection points on the current second scan line are recorded, and the center coordinates of the leftmost and rightmost intersection points are calculated. These center coordinates are recorded as the midpoint of the crop row mask contour at the current scan height.
[0041] The open-field vegetable row identification method provided by this invention, when individual crops have stuck together and blurred boundaries, obtains the leftmost and rightmost intersection points of the second scan line with the mask contour of each crop row, and performs "section cutting" on the crop row in a direction perpendicular to the main direction of the crop row. This achieves efficient and stable extraction of the center axis points of each crop row at different vertical positions, providing reliable data support for finally fitting a continuous crop row centerline.
[0042] In some embodiments, the open-field vegetable row recognition method provided by the present invention further includes: presetting a virtual horizontal line in the camera field of view of the open-field vegetable image; The coordinates of the intersection of the crop row centerline and the virtual horizontal line are used as the location information; The location information is sent to the nozzles of agricultural machinery through a fixed distance compensation mechanism.
[0043] Specifically, a fixed virtual horizontal line is pre-set within the camera's field of view for images of open-field vegetables. This virtual horizontal line has a constant Y-axis coordinate in the image coordinate system. Its function is to provide a standardized and fixed vertical reference for subsequent crop row positioning. Regardless of how the crop row moves up and down within the camera's field of view, the final positional information used for control will be anchored to this virtual horizontal line.
[0044] Based on the identified and fitted crop row centerline, the coordinates of the intersection point between the crop row centerline and the virtual horizontal line are calculated. Since the Y-axis coordinate of the virtual horizontal line is fixed, the Y-axis coordinate of the intersection point is the Y-value of the virtual horizontal line. The X-axis coordinate of the intersection point is calculated based on the equation of the crop row centerline and the Y-value of the virtual horizontal line. This intersection point coordinate represents the precise intersection point of the crop row centerline and the preset reference plane (represented by the virtual horizontal line) from the camera's perspective. This intersection point coordinate is defined as the position information used for agricultural machinery control.
[0045] The acquired location information is processed through a fixed-distance compensation mechanism and then sent to the execution unit of agricultural machinery, such as the nozzle of a spraying device. Specifically: Because there is a fixed spatial offset between the camera and the nozzle on the agricultural machinery in terms of their physical installation positions (i.e., the position where the camera captures the image is different from the position where the nozzle performs the operation), this offset is fixed during equipment installation and can be obtained in advance through calibration, represented as a fixed two-dimensional or three-dimensional compensation vector. When transmitting position information, this compensation vector is superimposed on the intersection coordinates to correct the physical deviation between the camera and the nozzle.
[0046] The open-field vegetable row identification method provided by this invention anchors location information onto virtual horizontal lines, so that the location information sent to the sprinkler head only needs to focus on the changes in the crop row in the horizontal direction, while the reference point in the vertical direction remains constant. This greatly simplifies the sprinkler head control logic, reduces the complexity of real-time calculations, and allows for the compensation of physical installation errors between the sensor and the actuator through simple fixed-distance compensation, thereby ensuring that the sprinkler head can accurately target the crop row for operation.
[0047] In some embodiments, the crop segmentation model is constructed based on the YOLOv8s model by introducing an efficient multi-scale attention module in the C2f module of the 3rd and 5th layers of the backbone network, a convolutional block attention module in the 10th layer of the backbone network, and a weighted bidirectional feature pyramid network in the neck network.
[0048] Specifically, Figure 2 This is a schematic diagram of the structure of the crop segmentation model provided by the present invention, as shown below. Figure 2 As shown, the crop segmentation model is based on the YOLOv8s framework and consists of five parts: Input, Backbone, Neck, Head, and Output.
[0049] The input is an image of open-field vegetables. The backbone network, consisting of 11 layers, is responsible for extracting various useful visual features from the original image. The neck network's task is to cleverly fuse the features extracted from different layers by the backbone network, thereby improving detection accuracy. The head receives the fused feature maps from the neck network and makes a final prediction based on these feature maps. The output filters out the messy predictions generated by the head, obtaining a clean and accurate final result.
[0050] Efficient Multi-Scale Attention (EMA) is an efficient multi-scale attention mechanism that uses parallel substructures to reduce network depth and preserves information in each channel through cross-dimensional interactions. Introducing EMA modules into the C2f modules of layers 3 and 5 of the model allows the model to focus more on the location information of the target region, thereby improving the detection accuracy of the target region.
[0051] A Convolutional Block Attention Module (CBAM) is introduced at layer 10 of the model to perform self-attention in the channel and spatial dimensions, enhancing the model's ability to extract features of open-field vegetables, reducing model computational parameters, and effectively improving the model's recognition speed. The neck network introduces a Weighted Bi-directional Feature Pyramid Network (BiFPN) structure. The BiFPN structure has strong robustness and maintains good prediction accuracy for open-field vegetables of different stages and sizes. Table 1 shows the parameters of the crop segmentation model; the crop segmentation model should be set according to Table 1.
[0052] Table 1. Parameters of the Crop Segmentation Model
[0053] Layer 17 outputs the first fused feature map, with a size of 80×80×256. Layer 20 outputs the second fused feature map, with a size of 40×40×512. Layer 23 outputs the third fused feature map, with a size of 20×20×1024. The model head makes predictions on these three feature maps simultaneously to ensure that broccoli of all sizes can be identified. However, this generates a large number of overlapping candidate boxes for the same target. Non-Maximum Suppression (NMS) in the output part filters these overlapping candidate boxes, retaining the "best" box with the highest score for each target and removing all other redundant boxes. Finally, the model outputs clean and accurate individual mask contours of open-field vegetables.
[0054] The open-field vegetable row identification method provided by this invention uses YOLOv8s as the basic framework for crop segmentation. EMA modules are introduced into the C2f modules of layers 3 and 5 of the backbone network, which helps improve the feature extraction capability of the target area and the identification capability of small targets. A CBAM module is introduced into layer 10 of the backbone network to enhance the model's ability to extract features in channels and space, while reducing computational parameters and making the model more lightweight. A BiFPN structure is introduced into the neck network, enabling the model to have good inference performance for targets at different time periods, thus enhancing the model's robustness.
[0055] In some embodiments, the CBS module in the YOLOv8s model uses the SiLU function.
[0056] Specifically, the CBS module is the main feature extraction module, which includes convolutional layers, batch normalization layers, and activation function layers. The activation function in the activation function layer uses the SiLU function.
[0057] In some embodiments, training images of open-field vegetables under different lighting conditions, weather environments, and growth stages are collected, for example, a total of 1200 images. The open-field vegetable training images are then augmented using offline enhancement methods to obtain augmented open-field vegetable training images. Offline enhancement methods include image rotation and cropping, image scaling, hue enhancement, saturation enhancement, brightness enhancement, and adding motion blur. The augmented open-field vegetable training images and the original open-field vegetable training images are combined to form an augmented dataset. The augmented dataset is then divided into a training set, a test set, and a validation set according to a preset ratio (e.g., 8:1:1). The crop segmentation model is trained, tested, and validated based on the training set, test set, and validation set, respectively.
[0058] The training data was expanded using offline enhancement methods such as cropping and rotating, increasing brightness, enhancing contrast, and adding motion blur. The dataset was expanded to five times its original size, which improved the robustness of the model and made it suitable for accurate recognition and segmentation under different lighting conditions, weather conditions, and growth stages.
[0059] During training, the image resolution input to the model is automatically adjusted to 640 pixels × 640 pixels. Eight images are input per batch, and a total of 300 training iterations are performed. To accelerate the convergence of the model's loss function and improve training efficiency, the Adam optimizer (Adaptive Moment Estimation) is selected in this invention, with the following parameters: initial learning rate of 0.001, weight decay coefficient of 0.0005, and momentum factor of 0.9. After each training iteration, the model weights are updated. This weight file is then used to test on the test set, and the model weights with the highest average accuracy are saved as the final weight model file.
[0060] The trained crop segmentation model achieved an accuracy of 95.9% in identifying open-field vegetables and 96.0% in segmenting them, with a single-frame image processing time of 20.2ms, demonstrating good recognition and segmentation performance.
[0061] The crop segmentation model was deployed on the NVIDIA Jetson Xavier NX developer kit using the PyTorch 1.13 framework. To improve image analysis speed, the trained model weight files were compiled using the TensorRT environment, and the generated .engine file effectively accelerated inference speed. The development board was connected to a camera via a USB 3.0 interface for image transmission, and OpenCV 4.0 library functions were used to read real-time images captured by the camera. Simultaneously, the Deepstream toolkit was used for video stream analysis acceleration, better meeting real-time requirements and improving inference accuracy.
[0062] By deploying the crop segmentation model on the NVIDIA Jetson Xavier NX development board, a 15W edge computing device, it can perform long-term recognition and detection tasks without an external power supply, demonstrating long battery life.
[0063] The open-field vegetable row identification device provided by the present invention is described below. The open-field vegetable row identification device described below can be referred to in correspondence with the open-field vegetable row identification method described above.
[0064] Figure 3 This is a schematic diagram of the structure of the open-field vegetable row identification device provided by the present invention, as shown below. Figure 3 As shown, the present invention provides a field vegetable row identification device, comprising: The acquisition module 310 is used to acquire images of open-field vegetables at different growth stages; The first recognition module 320 is used to, when the open-field vegetables are in the seedling stage and the individual plants are not stuck together, use a crop segmentation model to obtain the individual mask contour of each open-field vegetable in the image of the open-field vegetables, and calculate the center point of each individual mask contour; use a first scan line parallel to the main direction of the crop row to perform a traversal scan of the open-field vegetable image to determine the two side boundaries of all crop rows; and fit all the center points contained in the two side boundaries of each crop row to obtain the crop row center line corresponding to the two side boundaries of each crop row. The second recognition module 330 is used to obtain multiple crop row mask contours in the open-field vegetable image using a crop row recognition model when the open-field vegetables are in the period of stalk sticking together; to perform a traversal scan of the open-field vegetable image using a second scan line perpendicular to the main direction of the crop row, and to obtain the midpoint of each crop row mask contour in each scan step; and to fit all the midpoints contained in each crop row mask contour to obtain the crop row center line corresponding to each crop row mask contour.
[0065] In some embodiments, the first identification module 320 is used for: Starting from one side of the open-field vegetable image, when the first scan line first intersects with two or more individual mask contours, the current position of the first scan line is marked as one side boundary of the first crop row; Continue scanning to the other side of the open-field vegetable image. When the number of intersections between the first scan line and the individual mask contour decreases to one, mark the current position of the first scan line as the other side boundary of the first crop row. Repeat the above steps until you have traversed to the other side of the open-field vegetable image and obtained the two side boundaries of all crop rows.
[0066] In some embodiments, the second identification module 330 is used for: In each scan step, the leftmost and rightmost intersection points of the second scan line with the mask contour of each crop row are obtained; Calculate the center coordinates of the leftmost and rightmost intersection points to obtain the midpoint of each crop row mask contour in each scan step.
[0067] In some embodiments, the apparatus further includes a transmitting module for: A virtual horizontal line is preset in the camera's field of view of the open-field vegetable image; The coordinates of the intersection of the crop row centerline and the virtual horizontal line are used as the location information; The location information is sent to the nozzle of the agricultural machinery through a fixed distance compensation mechanism.
[0068] In some embodiments, the crop segmentation model is constructed based on the YOLOv8s model by introducing an efficient multi-scale attention module in the C2f module of the 3rd and 5th layers of the backbone network, a convolutional block attention module in the 10th layer of the backbone network, and a weighted bidirectional feature pyramid network in the neck network.
[0069] In some embodiments, the CBS module in the YOLOv8s model uses the SiLU function.
[0070] It should be noted that the above-mentioned open-field vegetable row identification device provided by the present invention can realize all the method steps implemented in the above method embodiments and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiments and the beneficial effects will not be described in detail.
[0071] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 4 As shown, the electronic device may include: a processor 410, a communications interface 420, a memory 430, and a communications bus 440, wherein the processor 410, the communications interface 420, and the memory 430 communicate with each other through the communications bus 440. The processor 410 can call logic instructions in the memory 430 to execute a method for recognizing open-field vegetable rows. This method includes: acquiring images of open-field vegetables at different growth stages; when the open-field vegetables are in the seedling stage and the individual plants are not clustered together, using a crop segmentation model to acquire the individual mask contour of each open-field vegetable in the image, and calculating the center point of each individual mask contour; using a first scan line parallel to the main direction of the crop rows to perform a traversal scan of the open-field vegetable image, determining the two side boundaries of all crop rows; fitting all the center points contained in the two side boundaries of each crop row to obtain the crop row centerline corresponding to the two side boundaries of each crop row; when the open-field vegetables are in a clustered growth stage, using a crop row recognition model to acquire multiple crop row mask contours in the image; using a second scan line perpendicular to the main direction of the crop rows to perform a traversal scan of the open-field vegetable image, obtaining the midpoint of each crop row mask contour in each scan step; fitting all the midpoints contained in each crop row mask contour to obtain the crop row centerline corresponding to each crop row mask contour.
[0072] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0073] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the open-field vegetable row recognition method provided by the above methods. The method includes: acquiring images of open-field vegetables at different growth stages; when the open-field vegetables are in the seedling stage and the individual plants are not stuck together, using a crop segmentation model to acquire the individual mask contour of each open-field vegetable in the image, and calculating the center point of each individual mask contour; and using a first scan line parallel to the main direction of the crop row to scan the open-field vegetable image. A traversal scan is performed to determine the two side boundaries of all crop rows; all the center points contained in the two side boundaries of each crop row are fitted to obtain the crop row centerline corresponding to the two side boundaries of each crop row; when the open-field vegetables are in the plant adhesion growth period, multiple crop row mask contours in the open-field vegetable image are obtained using a crop row recognition model; the open-field vegetable image is traversed using a second scan line perpendicular to the main direction of the crop rows to obtain the midpoint of each crop row mask contour in each scan step; all the midpoints contained in each crop row mask contour are fitted to obtain the crop row centerline corresponding to each crop row mask contour.
[0074] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the open-field vegetable row recognition method provided by the above methods. The method includes: acquiring images of open-field vegetables at different growth stages; when the open-field vegetables are in the seedling stage and the individual plants are not adhered together, using a crop segmentation model to acquire the individual mask contour of each open-field vegetable in the image, and calculating the center point of each individual mask contour; and using a first scan line parallel to the main direction of the crop rows to perform a traversal scan of the open-field vegetable images to determine the row positions of all crops. Both sides of the crop row are fitted with all the center points contained in the two sides of the crop row to obtain the crop row center line corresponding to the two sides of the crop row; when the open-field vegetables are in the period of plant adhesion growth, multiple crop row mask contours in the open-field vegetable image are obtained using a crop row recognition model; the open-field vegetable image is traversed by a second scan line perpendicular to the main direction of the crop row to obtain the midpoint of each crop row mask contour in each scan step; all the midpoints contained in each crop row mask contour are fitted to obtain the crop row center line corresponding to each crop row mask contour.
[0075] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.
[0076] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.
[0077] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for identifying rows of open-field vegetables, characterized in that, include: Acquire images of open-field vegetables at different growth stages; When the open-field vegetables are in the seedling stage and the individual plants are not stuck together, a crop segmentation model is used to obtain the individual mask contour of each open-field vegetable in the image, and the center point of each individual mask contour is calculated; the open-field vegetable image is traversed by a first scan line parallel to the main direction of the crop row to determine the two side boundaries of all crop rows; all the center points contained in the two side boundaries of each crop row are fitted to obtain the crop row center line corresponding to the two side boundaries of each crop row; When open-field vegetables are in the stage of stalk-adhesive growth, a crop row recognition model is used to obtain multiple crop row mask contours in the open-field vegetable image; the open-field vegetable image is traversed by a second scan line perpendicular to the main direction of the crop row to obtain the midpoint of each crop row mask contour in each scan step; all the midpoints contained in each crop row mask contour are fitted to obtain the crop row centerline corresponding to each crop row mask contour.
2. The method for identifying open-field vegetable rows according to claim 1, characterized in that, The step of using a first scan line parallel to the main direction of the crop rows to perform a traversal scan of the open-field vegetable image to determine the two side boundaries of all crop rows includes: Starting from one side of the open-field vegetable image, when the first scan line first intersects with two or more individual mask contours, the current position of the first scan line is marked as one side boundary of the first crop row; Continue scanning to the other side of the open-field vegetable image. When the number of intersections between the first scan line and the individual mask contour decreases to one, mark the current position of the first scan line as the other side boundary of the first crop row. Repeat the above steps until you have traversed to the other side of the open-field vegetable image and obtained the two side boundaries of all crop rows.
3. The method for identifying open-field vegetable rows according to claim 1, characterized in that, The step of traversing the open-field vegetable image using a second scan line perpendicular to the main direction of the crop rows to obtain the midpoint of the mask contour of each crop row in each scan step includes: In each scan step, the leftmost and rightmost intersection points of the second scan line with the mask contour of each crop row are obtained; Calculate the center coordinates of the leftmost and rightmost intersection points to obtain the midpoint of each crop row mask contour in each scan step.
4. The method for identifying open-field vegetable rows according to claim 1, characterized in that, The method further includes: A virtual horizontal line is preset in the camera's field of view of the open-field vegetable image; The coordinates of the intersection of the crop row centerline and the virtual horizontal line are used as the location information; The location information is sent to the nozzle of the agricultural machinery through a fixed distance compensation mechanism.
5. The method for identifying open-field vegetable rows according to claim 2, characterized in that, The crop segmentation model is constructed based on the YOLOv8s model by introducing an efficient multi-scale attention module in the C2f module of the 3rd and 5th layers of the backbone network, a convolutional block attention module in the 10th layer of the backbone network, and a weighted bidirectional feature pyramid network in the neck network.
6. The method for identifying open-field vegetable rows according to claim 5, characterized in that, The CBS module in the YOLOv8s model uses the SiLU function.
7. A device for identifying rows of open-field vegetables, characterized in that, include: The acquisition module is used to acquire images of open-field vegetables at different growth stages; The first recognition module is used to obtain the individual mask contour of each open-field vegetable in the open-field vegetable image using a crop segmentation model when the open-field vegetables are in the seedling stage and the individual plants are not stuck together, and to calculate the center point of each individual mask contour; to perform a traversal scan of the open-field vegetable image using a first scan line parallel to the main direction of the crop row, and to determine the two side boundaries of all crop rows; and to fit all the center points contained in the two side boundaries of each crop row to obtain the crop row center line corresponding to the two side boundaries of each crop row. The second recognition module is used to obtain multiple crop row mask contours in the open-field vegetable image when the open-field vegetables are in the period of stalk sticking together; to perform a traversal scan of the open-field vegetable image using a second scan line perpendicular to the main direction of the crop row, and to obtain the midpoint of each crop row mask contour in each scan step; and to fit all the midpoints contained in each crop row mask contour to obtain the crop row centerline corresponding to each crop row mask contour.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the open-field vegetable row identification method as described in any one of claims 1 to 6.
9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the open-field vegetable row identification method as described in any one of claims 1 to 6.
10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the open-field vegetable row identification method as described in any one of claims 1 to 6.