An orchard inter-row road edge extraction method

Abstract

The present application belongs to the technical field of agricultural machinery automation, and provides a method for extracting the edge of the inter-row road in an orchard, comprising the following steps: S1, acquiring the inter-row road image in the forward direction of the machine and segmenting the image; S2, pre-processing the segmented image to acquire the inter-row road image of the orchard; S3, extracting the region of interest from the acquired inter-row road image of the orchard; S4, extracting the long-range boundary points and the short-range edge points of the region of interest, and dividing the left image and the right image of the region of interest; S5, fitting the road edge in the left image and the right image of the region of interest respectively to obtain the left road edge fitting straight line and the right road edge fitting straight line; and S6, combining the fitted inter-row road edge straight line of the orchard, the long-range boundary points and the short-range edge points to form the complete inter-row road edge of the orchard on the left and the right sides respectively. The present application solves the problem of large error and the existence of obstacles in the extracted inter-row road edge of the orchard in the prior art, which affects the path extraction.

Description

Technical Field

[0001] This invention belongs to the technical field of agricultural machinery automation, and more specifically, relates to a method for extracting the edges of orchard rows. Background Technology

[0002] The modern fruit industry is a labor-intensive industry. Spraying and harvesting in orchards require a large labor force, resulting in low labor productivity and labor shortages that severely restrict the development of the fruit industry. To promote the healthy development of the fruit industry, intelligent agricultural machinery is an inevitable path for future development. Positioning and autonomous navigation technologies are important components of intelligent agricultural machinery, and their implementation methods often employ GNSS navigation, lidar navigation, and multi-sensor fusion navigation. In recent years, deep learning has developed rapidly, and machine vision, with its rich information content and low cost, has been widely applied in the field of agricultural robots. Road edge extraction is a key focus of machine vision research in orchard autonomous navigation. Domestic research on orchard inter-row road extraction has already been conducted.

[0003] In the prior art, Chinese patent document CN102194233A proposed a method to obtain forest roads by using horizontal and vertical projection of GB color difference images. However, this method cannot effectively distinguish forest roads in orchards with tall grass using GB color difference, thus affecting the extraction of navigation lines in orchards.

[0004] Chinese patent document CN113280820A proposes a method for extracting orchard road condition information based on the SEGNet deep learning neural network model. It directly uses the edge information identified as roads in the extracted orchard road condition images as the edges of the orchard inter-row roads. However, this method does not consider the influence of environmental factors such as light and weeds on the orchard road condition information extracted by the SEGNet deep learning neural network model. Directly using the road edges in the extracted orchard road condition images will lead to a large error in the extracted edges of the orchard inter-row roads.

[0005] Chinese patent documents CN111983637A and CN111539473A, both proposing a method for extracting paths between rows in an orchard based on lidar and a method for extracting navigation lines between rows in an orchard based on 3D lidar, both propose a path extraction method based on lidar by detecting the trunks of fruit trees. However, when this method is applied to a general orchard, there may be situations where weeds and branches obstruct the trunk and fail to detect it. Summary of the Invention

[0006] The present invention aims to overcome at least one of the defects of the prior art and provide a method for extracting the edges of orchard rows.

[0007] The detailed technical solution of this invention is as follows:

[0008] To address the aforementioned technical problems, this invention provides a method for extracting the edges of orchard inter-row roads. This method solves the problems of existing techniques that directly utilize the extracted road edges from road condition images, resulting in large errors in the extracted orchard inter-row road edges and the presence of occlusions affecting path extraction.

[0009] A method for extracting the edges of orchard rows includes the following steps:

[0010] S1. Obtain the inter-row road map in the machine's forward direction and use a deep learning-based semantic segmentation algorithm to segment the inter-row road map in the machine's forward direction.

[0011] The machine's forward-moving path map is segmented into an initial orchard path map and a non-orchard path map. The pixel values ​​of the images segmented into the initial orchard path map are preset pixel values, and the pixel values ​​of the images segmented into the non-orchard path map are 0. The preset pixel values ​​can be set according to the actual situation. The semantic segmentation algorithm includes, but is not limited to, FCN, PSPNet, Deeplabv3, and Deeplabv3plus.

[0012] S2. Preprocess the initial map of the segmented orchard rows to obtain the orchard rows map.

[0013] S3. Extract the region of interest from the acquired orchard road map. The region of interest is the horizontal portion of the orchard road map that contains road images.

[0014] S4. Extract the distant boundary points and near edge points of the region of interest, and divide the region of interest into the left image and the right image.

[0015] The distant boundary point is the intersection of the edges of the roads on both sides of the orchard rows and the top of the region of interest;

[0016] The near-field edge points are the intersections of the left and right edges of the orchard's inter-row roads with the left and right ends of the region of interest.

[0017] S5. Fit the road edges in the left and right images of the region of interest respectively to obtain the fitted straight lines for the left and right road edges.

[0018] S6. Using the fitted straight lines of the left and right road edges, the distant boundary points, and the near edge points, complete orchard road edges are formed on both the left and right sides.

[0019] S3 specifically includes: accumulating pixel values ​​for each row of the orchard road map, and the image region formed by the rows with accumulated values ​​greater than 0 is the region of interest for the orchard road.

[0020] According to a preferred embodiment of the present invention, the extraction of the distant boundary points includes: if the cumulative pixel value of the first row of the region of interest is greater than a preset pixel value, then the left and right distant boundary points do not overlap; the left distant boundary point is the coordinate of the first pixel with a preset pixel value from left to right in the first row of the region of interest, and the right distant boundary point is the coordinate of the first pixel with a preset pixel value from right to left in the first row of the region of interest; if the cumulative pixel value of the first row of the region of interest is equal to the preset pixel value, then the left and right distant boundary points overlap; the left and right distant boundary points are the coordinates of the pixels with preset pixel values ​​in the first row of the region of interest; the coordinates of the left distant boundary point are denoted as (Yhz1, Ylz1), and the coordinates of the right distant boundary point are denoted as (Yhy2, Yly2).

[0021] The extraction of near-field edge points includes: if there is a pixel with a preset pixel value in the first column on the left side of the region of interest, then the left near-field edge point is the first pixel with a preset pixel value from top to bottom; if there is no pixel with a preset pixel value in the first column on the left side of the region of interest, then the left near-field edge point is the first pixel with a preset pixel value from left to right in the lowest row of the region of interest. If there is a pixel with a preset pixel value in the first column on the right side of the region of interest, then the right near-field edge point is the first pixel with a preset pixel value from top to bottom; if there is no pixel with a preset pixel value in the first column on the right side of the region of interest, then the right near-field edge point is the first pixel with a preset pixel value from right to left in the lowest row of the region of interest. The coordinates of the left near-field edge point are (Jhz1, Jlz1), and the coordinates of the right near-field edge point are (Jhy2, Jly2).

[0022] The left image of the region of interest is a square region with points (Yhz1, Jlz1), (Yhz1, Ylz1), (Jhz1, Ylz1), and (Jhz1, Jlz1) as its four vertices; the right image of the region of interest is a square region with points (Yhy2, Yly2), (Yhy2, Jly2), (Jhy2, Jly2), and (Jhy2, Yly2) as its four vertices.

[0023] S5 specifically includes the following steps:

[0024] S51. Divide the left and right images of the region of interest into five segments along the vertical direction, from top to bottom, which are the 1st to the 5th segments. In each segment, fit the road edge straight line using the least squares method. Store the slopes of the five straight lines in the left image of the region of interest into list Ll, and store the slopes of the five straight lines in the right image of the region of interest into list Lr.

[0025] S52. In lists Ll and Lr, select three straight line slopes respectively, and select two road edge coordinate points in the segment corresponding to each selected slope in the left and right images of the region of interest. Store the road edge coordinate points selected in the left image of the region of interest in list Lz, and store the road edge coordinate points selected in the right image of the region of interest in list Ly.

[0026] According to a preferred embodiment of the present invention, the method for selecting the slopes of the three lines is as follows: three sets of standard deviations are calculated in Ll and Lr respectively. The first set is calculated by removing the largest and smallest data and then calculating the standard deviation of the remaining three data. The second set is calculated by removing the two smallest numbers and then calculating the standard deviation of the remaining three numbers. The third set is calculated by removing the two largest numbers and then calculating the standard deviation of the remaining three numbers. When removing data, the slope of which segment is removed should be recorded. The smallest standard deviation is selected from the three sets of standard deviations. The three numbers corresponding to the smallest standard deviation are the three slopes of the lines to be selected.

[0027] S53. Based on the coordinate points in Lz and Ly, the least squares method is used to fit the straight lines of the orchard road edges. The fitted straight line of the left road edge is denoted as Y1, and the fitted straight line of the right road edge is denoted as Y2.

[0028] According to a preferred embodiment of the present invention, the selection of road edge coordinate points specifically includes: dividing the selected segment into three equal parts based on the number of rows; in the left image of the region of interest, the edge coordinate point is selected as the first pixel value from left to right of the row containing the road edge point, which is a preset pixel value; in the right image of the region of interest, the edge coordinate point is selected as the first pixel value from right to left of the row containing the road edge point, which is a preset pixel value.

[0029] S6 specifically includes: the coordinate system in the extraction of the orchard inter-row road edges is based on the top left corner vertex of the inter-row road image as the origin, the positive x-axis is the direction from top to bottom along the left side of the image, and the positive y-axis is the direction from left to right along the top of the image; the coordinates of the top left corner vertex of the inter-row road image are (0, 0), and the coordinates of the bottom left corner vertex are (T...). h The coordinates of the upper right corner vertex are (0, T). l The coordinates of the lower right vertex are (T) h T l );

[0030] Substitute the row coordinates Yhz1 of the left distant boundary point into line Y1 to calculate the intersection point (Yhz1, Yjhz1); substitute the row coordinates Yhy2 of the right distant boundary point into line Y2 to calculate the intersection point (Yhy2, Yjhy2).

[0031] If Yjhy2 is greater than or equal to Yjhz1, then (Yhz1, Yjhz1) is denoted as point A, and (Yhy2, Yjhy2) is denoted as point B; if Yjhy2 is less than Yjhz1, then the intersection of lines Y1 and Y2 is calculated, and the intersection points are denoted as points A and B. At this time, points A and B coincide, and their coordinates are denoted as (Zh, Zl); if Jhz1 is less than Th, then Jlz1 is substituted into line Y1, and the coordinates of the intersection point (Jjlz1, Jlz1) are calculated. If Jjlz1 is less than or equal to Jhz1, then the intersection point (Jjlz1, Jlz1) is denoted as point C; if If Jjlz1 is greater than Jhz1, then substitute the left near edge point's row coordinates Jlz1 into line Y1 to calculate the intersection coordinates (Jhz1, Jjhz1), and denote the intersection point (Jhz1, Jjhz1) as point C; once point C is determined, denote the left near edge point (Jhz1, Jlz1) as point D, and denote the lower left corner vertex (Th, 0) of the inter-row road image as point E; if Jhz1 is equal to Th, then substitute Jhz1 into line Y1 to calculate the intersection coordinates (Jhz1, Jjhz1), and denote the intersection points as points C, D, and E. At this time, points C, D, and E coincide with point E;

[0032] If Jhy2 is less than Th, substitute Jly2 into line Y2 to calculate the intersection coordinates (Jjly2, Jly2). If Jjly2 is less than or equal to Jhy2, record the intersection (Jjly2, Jly2) as point F. If Jjly2 is greater than Jhy2, substitute the right near edge point row coordinates Jhy2 into line Y2 to calculate the intersection coordinates (Jhy2, Jjhy2), and record the intersection (Jhy2, Jjhy2) as point F. Once point F is determined, record the right near edge point (Jhy2, Jly2) as point G, and the lower right corner vertex (Th, Tl) of the inter-row road image as point H. If Jhy2 is equal to Th, substitute Jhy2 into line Y2 to calculate the intersection coordinates (Jhy2, Jjhy2), and record the intersection as points F, G, and H. At this point, points F, G, and H coincide.

[0033] Connecting points A, C, D, and E in sequence forms the complete left path edge; connecting points B, F, G, and H in sequence forms the complete right path edge.

[0034] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0035] (1) The present invention provides a method for extracting the edge of orchard inter-row roads. When extracting the edge of orchard inter-row roads, the problem of “inner / outer convex” curvature of the extracted orchard inter-row road edge is caused by interference from external factors such as light and weeds. Using the idea of ​​filtering algorithm, the region of interest of the extracted path edge is divided into five segments. The standard deviation of the slope of the remaining three segments is calculated after removing the two segments with the largest and smallest slopes, the two largest and the two smallest slopes. The three segments with the smallest standard deviation are selected, and nine key points are extracted from the selected three segments. The road edge line is fitted using the least squares method, thereby increasing the accuracy of extracting the edge of orchard inter-row roads and eliminating interference from external factors such as light and weeds to a certain extent.

[0036] (2) The present invention provides a method for extracting the edges of orchard inter-row roads. It utilizes the idea that small standard deviations represent data that are closer to the average value and that the data are more accurate. By removing the maximum, minimum, two maximum, and two minimum values ​​from the slopes of the five road segments, the standard deviations of the three sets of data are calculated. Six road edge points are selected from the road edge corresponding to the set with the smallest standard deviation for road edge fitting, which can greatly reduce the error of the extracted orchard inter-row road edges. Attached Figure Description

[0037] Figure 1 This is a flowchart of the orchard row road edge extraction process described in this invention.

[0038] Figure 2 This is a schematic diagram of the region of interest, distant boundary points, and near edge points of the orchard road in Embodiment 1 of the present invention.

[0039] Figure 3 These are schematic diagrams of the region of interest (ROI) between rows of orchard roads in Embodiment 1 of the present invention, shown on the left and right.

[0040] Figure 4 This is a flowchart of the road edge fitting method for the region of interest (left image) in Embodiment 1 of the present invention.

[0041] Figure 5 This is a flowchart of the road edge fitting method for the region of interest in Embodiment 1 of the present invention.

[0042] Figure 6 This is a flowchart of the method for fitting the complete path of the left side of the road between rows in the orchard in Embodiment 1 of the present invention.

[0043] Figure 7 This is a flowchart of the method for fitting the complete path of the right-side road edge between rows in an orchard in Embodiment 1 of the present invention.

[0044] Figure 8 This is a schematic diagram of the complete path fitting between the rows of roads in the orchard in Embodiment 1 of the present invention.

[0045] Figure 2 In the middle: 1 is the non-orchard road area, 2 is the orchard road area, 3 is the extracted region of interest, 4 is the distant boundary point, 5 is the right near edge point, and 6 is the left distant edge point.

[0046] Figure 3 In the image: 7 represents the region of interest (left image), and 8 represents the region of interest (right image).

[0047] Figure 8 In the middle: 9 represents the fitted complete left road edge, and 10 represents the fitted complete right road edge. Detailed Implementation

[0048] The present disclosure will be further described below with reference to the accompanying drawings and embodiments.

[0049] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of this disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains.

[0050] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this disclosure. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms “comprising” and / or “including” are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0051] Where there is no conflict, the embodiments and features described herein can be combined with each other.

[0052] Example 1

[0053] This embodiment provides a method for extracting the edges of orchard rows, such as... Figure 1 As shown, it includes the following steps:

[0054] S1. Obtain the road map between rows in the direction the robot is moving and perform road segmentation: Obtain the road image between rows in the orchard in the direction the orchard robot is moving through the camera, and use a semantic segmentation algorithm based on deep learning to segment the road map between rows in the direction the robot is moving.

[0055] The pixel value of the image segmented into orchard inter-row roads is a preset pixel value. Preferably, in this embodiment, the preset pixel value is 255, and the pixel value of the image segmented into non-orchard inter-row roads is 0. Figure 2 Image 1 is an image of a road outside the orchard rows, and image 2 is an image of a road between orchard rows.

[0056] The semantic segmentation algorithms include, but are not limited to, FCN, PSPNet, Deeplabv3, and Deeplabv3plus.

[0057] S2. Preprocess the segmented image: perform grayscale conversion and filtering on the segmented image to obtain the orchard road map;

[0058] The filtering methods include, but are not limited to, mean filtering, median filtering, and Gaussian filtering.

[0059] S3. Extract Region of Interest: Extract the region of interest from the acquired orchard road map, such as... Figure 2 As shown;

[0060] The region of interest is the horizontal portion of the orchard road map that contains road images, such as... Figure 2 The region shown in Figure 3; the method for extracting the region of interest for orchard inter-row roads is to accumulate the pixel values ​​of the orchard inter-row road map row by row, and the image region composed of the rows with accumulated values ​​greater than 0 is the region of interest for orchard inter-row roads.

[0061] S4. Extract distant boundary points and near edge points, and divide the region of interest into left and right images, as shown. Figure 2 , Figure 3 As shown: In the extracted region of interest, distant boundary points and near edge points are obtained, and the region of interest is divided according to the distant boundary points and near edge points. (Left and right images) Figure 2 4 is the boundary point of the distant view, 5 is the right edge point of the near view, and 6 is the left edge point of the distant view; Figure 3 In the middle, 7 is the region of interest in the left image, and 8 is the region of interest in the right image;

[0062] The coordinate system for extracting the edges of the orchard rows is based on the top-left corner of the road image as the origin, with the positive x-axis running from top to bottom along the left side of the image and the positive y-axis running from left to right along the top. The coordinates of the top-left corner of the road image are (0, 0), the bottom-left corner is (Th, 0), the top-right corner is (0, Tl), and the bottom-right corner is (Th, Tl). The distant boundary points are the intersections of the left and right edges of the orchard rows with the top of the region of interest. The near-field edge points are the intersections of the left and right edges of the orchard rows with the left and right ends of the region of interest.

[0063] Furthermore, the method for obtaining the distant boundary points is as follows: if the cumulative pixel value of the first row of the region of interest is greater than 255, then the left and right distant boundary points do not overlap. The left distant boundary point is the coordinate of the first pixel with a pixel value of 255 from left to right in the first row of the region of interest, and the right distant boundary point is the coordinate of the first pixel with a pixel value of 255 from right to left in the first row of the region of interest. If the cumulative pixel value of the first row of the region of interest is equal to 255, then the left and right distant boundary points overlap. The left and right distant boundary points are the coordinates of the pixels with a pixel value of 255 in the first row of the region of interest. The coordinates of the left distant boundary point are denoted as (Yhz1, Ylz1), and the coordinates of the right distant boundary point are denoted as (Yhy2, Yly2).

[0064] Further, the method for obtaining the near-field edge points is as follows: if there is a pixel with a pixel value of 255 in the first column on the left side of the region of interest, then the left near-field edge point is the first pixel with a pixel value of 255 from top to bottom; if there is no pixel with a pixel value of 255 in the first column on the left side of the region of interest, then the left near-field edge point is the first pixel with a pixel value of 255 from left to right in the lowest row of the region of interest. If there is a pixel with a pixel value of 255 in the first column on the right side of the region of interest, then the right near-field edge point is the first pixel with a pixel value of 255 from top to bottom; if there is no pixel with a pixel value of 255 in the first column on the right side of the region of interest, then the right near-field edge point is the first pixel with a pixel value of 255 from right to left in the lowest row of the region of interest. The coordinates of the left near-field edge point are (Jhz1, Jlz1), and the coordinates of the right near-field edge point are (Jhy2, Jly2).

[0065] The left image of the region of interest is a square region with points (Yhz1, Jlz1), (Yhz1, Ylz1), (Jhz1, Ylz1), and (Jhz1, Jlz1) as its four vertices; the right image of the region of interest is a square region with points (Yhy2, Yly2), (Yhy2, Jly2), (Jhy2, Jly2), and (Jhy2, Yly2) as its four vertices.

[0066] S5. Fit the road edges in the left and right images of the region of interest respectively to obtain the fitted straight lines for the left and right road edges:

[0067] Furthermore, S5 specifically includes the following steps, such as... Figure 4 , Figure 5 As shown:

[0068] S51. Divide the left and right images of the region of interest into five segments along the vertical direction, from top to bottom, which are the 1st to the 5th segments. In each segment, fit the road edge straight line using the least squares method. Store the slopes of the five straight lines in the left image of the region of interest into list Ll, and store the slopes of the five straight lines in the right image of the region of interest into list Lr.

[0069] S52. In lists Ll and Lr, select three straight line slopes respectively, and select 6 coordinate edge points in the segments corresponding to the three straight line slopes selected in the left and right images of the region of interest. Store the road edge coordinate points selected in the left image of the region of interest in list Lz, and store the road edge coordinate points selected in the right image of the region of interest in list Ly.

[0070] Furthermore, the method for selecting the slopes of the three lines is as follows: calculate three sets of standard deviations in Ll and Lr respectively. The first set is calculated by removing the largest and smallest data and then calculating the standard deviation of the remaining three data. The second set is calculated by removing the two smallest data and then calculating the standard deviation of the remaining three data. The third set is calculated by removing the two largest data and then calculating the standard deviation of the remaining three data. When removing data, record which segment's slope is being removed. Select the smallest standard deviation from the three sets of standard deviations. The three data corresponding to the smallest standard deviation are the three slopes of the lines to be selected.

[0071] Specifically, taking the road edge fitting method in the left image of the region of interest as an example, such as... Figure 4 As shown: Read the 5 slope values ​​in L1; remove the largest and smallest data, record the position of the removed value, and calculate the standard deviation σ1 of the remaining data; remove the two smallest data, record the position of the removed value, and calculate the standard deviation σ2 of the remaining data; remove the two largest data, record the position of the removed value, and calculate the standard deviation σ3 of the remaining data; select the smallest standard deviation from the three sets of standard deviations, and the three numbers corresponding to the selected smallest standard deviation are the three slopes of the line to be selected.

[0072] Furthermore, the method of randomly selecting two road edge coordinate points from the segment corresponding to the selected slope is as follows: randomly select two points from each of the three segments corresponding to the group with the smallest standard deviation. The method of selecting points includes, but is not limited to, selecting at fixed intervals within the coordinate range of the segment, or generating random numbers for random selection. In the left image of the region of interest, the edge coordinate point is selected as the first pixel with a value of 255 from left to right in the row where the road edge point is located. In the right image of the region of interest, the edge coordinate point is selected as the first pixel with a value of 255 from right to left in the row where the road edge point is located.

[0073] Furthermore, the left road edge line is fitted using the least squares method with the selected six coordinate edge points; the road edge fitting method for the right image of the region of interest is the same as that for the right image of the region of interest, such as... Figure 5 As shown, no further details will be provided.

[0074] S53. Based on the coordinate points in Lz and Ly, the least squares method is used to fit the straight lines of the orchard road edges. The fitted straight line of the left road edge is denoted as Y1, and the fitted straight line of the right road edge is denoted as Y2.

[0075] S6. Fit the left and right edges of the orchard rows; using the fitted straight lines of the orchard rows, distant boundary points, and near edge points, form the complete edges of the orchard rows on both the left and right sides, respectively. Figure 8 As shown in the figure, 9 represents the fitted complete left road edge, and 10 represents the fitted complete right road edge;

[0076] Furthermore, the complete orchard row road edge fitting method is as follows: Figure 6 , Figure 7 As shown, the intersection point (Yhz1, Yjhz1) is calculated by substituting the row coordinates of the left distant boundary point Yhz1 into the line Y1; the intersection point (Yhy2, Yjhy2) is calculated by substituting the row coordinates of the right distant boundary point Yhy2 into the line Y2.

[0077] If Yjhy2 is greater than or equal to Yjhz1, then (Yhz1, Yjhz1) is denoted as point A, and (Yhy2, Yjhy2) is denoted as point B; if Yjhy2 is less than Yjhz1, then the intersection of lines Y1 and Y2 is calculated, and the intersection is denoted as point A and point B. At this time, point A and point B coincide, and their coordinates are denoted as (Zh, Zl). Figure 8 As shown.

[0078] When Jhz1 is less than Th, substitute Jlz1 into line Y1 to calculate the intersection coordinates (Jjlz1, Jlz1). If Jjlz1 is less than or equal to Jhz1, then record the intersection (Jjlz1, Jlz1) as point C. If Jjlz1 is greater than Jhz1, substitute the row coordinates Jlz1 of the left near edge point into line Y1 to calculate the intersection coordinates (Jhz1, Jjhz1), and record the intersection (Jhz1, Jjhz1) as point C. Once point C is determined, record the left near edge point (Jhz1, Jlz1) as point D, and the lower left vertex (Th, 0) of the inter-row road image as point E.

[0079] If Jhz1 is not less than Th, then substitute Jhz1 into line Y1 and calculate the coordinates of the intersection point (Jhz1, Jjhz1). Record the intersection points as points C, D, and E. At this time, points C, D and E coincide with point E.

[0080] Connecting points A, C, D, and E in sequence forms the complete left path edge;

[0081] When Jhy2 is less than Th, substitute Jly2 into line Y2 to calculate the intersection coordinates (Jjly2, Jly2). If Jjly2 is less than or equal to Jhy2, then record the intersection (Jjly2, Jly2) as point F. If Jjly2 is greater than Jhy2, substitute the right near edge point row coordinates Jhy2 into line Y2 to calculate the intersection coordinates (Jhy2, Jjhy2), and record the intersection (Jhy2, Jjhy2) as point F. Once point F is determined, record the right near edge point (Jhy2, Jly2) as point G, and the lower right corner vertex (Th, Tl) of the inter-row road image as point H.

[0082] If Jhy2 is not less than Th, then substitute Jhy2 into line Y2 and calculate the coordinates of the intersection point (Jhy2, Jjhy2). Record the intersection points as points F, G, and H. At this time, points F, G, and H coincide.

[0083] Connecting points B, F, G, and H in sequence forms the complete right path edge.

[0084] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the technical solutions of the present invention, and are not intended to limit the specific implementation of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the claims of the present invention should be included within the protection scope of the claims of the present invention.

Claims

1. A method for extracting the edges of orchard rows, characterized in that, include; S1. Obtain the inter-row road map in the machine's forward direction and use a deep learning-based semantic segmentation algorithm to segment the inter-row road map in the machine's forward direction. S2. Preprocess the segmented image to obtain the orchard road map; S3. Extract the region of interest from the obtained orchard road map; S4. Extract the distant boundary points and near edge points of the region of interest, and divide the region of interest into left and right images; S5. Fit the road edges in the left and right images of the region of interest respectively to obtain the fitted straight lines for the left and right road edges. The S5 specifically Includes the following steps: S51. Divide the left and right images of the region of interest into five segments along the vertical direction, from top to bottom, which are the 1st to the 5th segments. In each segment, fit the road edge straight line using the least squares method. Store the slopes of the five straight lines in the left image of the region of interest into list Ll, and store the slopes of the five straight lines in the right image of the region of interest into list Lr. S52. In lists Ll and Lr, select three straight line slopes respectively, and select two road edge coordinate points in the segment corresponding to each selected slope in the left and right images of the region of interest. Store the road edge coordinate points selected in the left image of the region of interest in list Lz, and store the road edge coordinate points selected in the right image of the region of interest in list Ly. S53. Based on the coordinate points in Lz and Ly, the least squares method is used to fit the straight lines of the orchard road edges. The fitted straight line of the left road edge is denoted as Y1, and the fitted straight line of the right road edge is denoted as Y2. The method for selecting the slopes of the three lines is as follows: Calculate three sets of standard deviations in Ll and Lr respectively: the first set is the standard deviation of the slopes of the remaining three lines after removing the largest and smallest data; the second set is the standard deviation of the slopes of the remaining three lines after removing the two smallest data; the third set is the standard deviation of the slopes of the remaining three lines after removing the two largest data. Select the smallest standard deviation from each of the three sets of standard deviations. The slopes of the three lines corresponding to the smallest standard deviation are the three slopes of the lines to be selected in the left and right plots of the region of interest, respectively. The method for selecting two road edge coordinate points in the segment corresponding to each selected slope in the left and right images of the region of interest is as follows: In each of the three segments corresponding to the group with the smallest standard deviation, two points are randomly selected; the edge coordinate points in the left image of the region of interest are selected as the first pixel value from left to right of the row containing the road edge points, which is the road edge coordinate point to be selected; the edge coordinate points in the right image of the region of interest are selected as the first pixel value from right to left of the row containing the road edge points, which is the road edge coordinate point to be selected. S6. Using the fitted orchard row road edge lines, distant boundary points, and near edge points, complete orchard row road edges are formed on the left and right sides respectively.

2. The method for extracting the edges of orchard rows according to claim 1, characterized in that, S3 specifically includes: accumulating pixel values ​​for each row of the orchard road map, and the image region formed by the rows with accumulated values ​​greater than 0 is the region of interest for the orchard road.

3. The method for extracting the edges of orchard rows according to claim 2, characterized in that, The extraction of distant boundary points includes: if the cumulative pixel value of the first row of the region of interest is greater than a preset pixel value, then the left and right distant boundary points do not overlap. The left distant boundary point is the coordinate of the first pixel in the first row of the region of interest with a preset pixel value from left to right, and the right distant boundary point is the coordinate of the first pixel in the first row of the region of interest with a preset pixel value from right to left. If the cumulative pixel value of the first row of the region of interest is equal to the preset pixel value, then the left and right distant boundary points overlap. The left and right distant boundary points are the coordinates of the pixels in the first row of the region of interest with the preset pixel value. The coordinates of the left distant boundary point are (Yhz1, Ylz1), and the coordinates of the right distant boundary point are (Yhy2, Yly2). The extraction of near-field edge points includes: if there is a pixel with a preset pixel value in the first column on the left side of the region of interest, then the left near-field edge point is the first pixel with a preset pixel value from top to bottom; if there is no pixel with a preset pixel value in the first column on the left side of the region of interest, then the left near-field edge point is the first pixel with a preset pixel value from left to right in the lowest row of the region of interest. If there is a pixel with a preset pixel value in the first column on the right side of the region of interest, then the right near-field edge point is the first pixel with a preset pixel value from top to bottom; if there is no pixel with a preset pixel value in the first column on the right side of the region of interest, then the right near-field edge point is the first pixel with a preset pixel value from right to left in the lowest row of the region of interest. The coordinates of the left near-field edge point are (Jhz1, Jlz1), and the coordinates of the right near-field edge point are (Jhy2, Jly2).

4. The method for extracting the edges of orchard rows according to claim 3, characterized in that, The left image of the region of interest is a square region with points (Yhz1, Jlz1), (Yhz1, Ylz1), (Jhz1, Ylz1), and (Jhz1, Jlz1) as its four vertices; the right image of the region of interest is a square region with points (Yhy2, Yly2), (Yhy2, Jly2), (Jhy2, Jly2), and (Jhy2, Yly2) as its four vertices.

Images