Semantic segmentation-based method for positioning and navigating robot for closed orchard operation

Abstract

The invention discloses a method for positioning and navigating a closed orchard operation robot based on semantic segmentation, and the method is characterized in that the method comprises the following steps: installing a camera on the operation robot, and carrying out the continuous collection of a current fruit tree inter-row navigation path image when the operation robot runs; establishing a semantic segmentation model based on the improved Fast-Unet network, and solving a path center line equation, namely an inter-row navigation line equation; obtaining fruit tree row endpoints based on a normal method; when the vertical distance D of the fruit tree row end point straight line is smaller than 1 / 2 of the wheelbase of the operation robot, current inter-row navigation is ended; during line changing, the course of the operation robot is controlled to rotate by 90 degrees firstly, then the operation robot is controlled to linearly run by the distance of the diameter of a crown, then the operation robot is controlled to rotate by 90 degrees to enter fruit tree lines, and finally the navigation operation of the operation robot is controlled by repeating the line-to-line navigation method.

Description

Technical Field

[0001] The invention relates to the technical field of artificial intelligence, and in particular to a positioning and navigation method for a robot operating in a closed orchard based on semantic segmentation. Background Art

[0002] In recent years, the country has attached great importance to the development of agricultural robots, invested a lot of personnel and funds in its research and development, and actively promoted the use of automated agricultural machinery throughout the society to achieve agricultural modernization. With the implementation of policies, various intelligent agricultural robots have emerged one after another, among which most of the robot navigation and positioning technology uses satellite navigation and positioning systems. However, due to the particularity and complexity of the operating environment, these agricultural robots face various interferences and obstructions when navigating and positioning in the field, which can easily cause the satellite positioning signal to be lost or inaccurate, and also greatly reduce the reliability and practicality of the robot.

[0003] When orchard robots walk between rows of fruit trees, they are often blocked or covered by lush branches and leaves. At this time, the GNSS signal is blocked by the fruit trees during transmission, which reduces the accuracy of the positioning signal. At the same time, the inaccurate position signal of the robot will affect its navigation control and path planning. At present, the mainstream methods to deal with the problem of inaccurate robot positioning signals include: GNSS combined with visual positioning, GNSS combined with lidar positioning, and map-based positioning. Although the positioning effect has been improved, it is still impossible to guarantee the positioning accuracy when the GNSS is blocked. Moreover, these fusion positioning methods have certain requirements for the application environment and cannot work well under load conditions. Therefore, it is urgent to study a method for inter-row positioning and navigation of closed orchard operation robots based on semantic segmentation. Summary of the invention

[0004] In view of this, the present invention proposes an inter-row positioning and navigation method for a working robot in a closed orchard based on semantic segmentation, which realizes the effective identification and segmentation of navigation paths between rows of fruit trees when the road is blocked by lush branches and leaves or covered by the road. The Canny operator is used to extract the boundary of the maximum connected area, the equation of the inter-row navigation line is calculated, and the endpoints of the fruit tree row are obtained based on the normal method, which can accurately control the path tracking of the inter-row navigation line L of the working robot.

[0005] On the one hand, the present invention proposes a method for positioning and navigating between rows of a robot operating in a closed orchard based on semantic segmentation, comprising the following steps:

[0006] S1: A camera is installed on the working robot to continuously collect images of the current navigation path between fruit tree rows when the working robot is driving;

[0007] S2: A semantic segmentation model is established based on the improved Fast-Unet network. After model training and transfer learning, the semantic segmentation model is used to identify and segment the navigation path between fruit tree rows, and a binary image consisting of the navigation path between fruit tree rows and the background is generated. The largest connected area in the binary image is selected as the navigation path between rows, and the Canny operator is used to extract the boundary of the largest connected area. The points on the boundary are medianized to generate a scatter plot of the center line of the path. The center line scatter plot is used to perform the least squares regression of a linear equation with the length and width of the image as the coordinate axis, and the path center line equation is obtained, which is the navigation line equation between rows.

[0008] S3: Obtain the endpoint of the fruit tree row based on the normal method; the gradient vector of a point on the boundary contour curve of the navigation path between rows is the vertical line of the curve segment passing through the point. A small line segment near the point is used to replace the curve segment, and the vertical line of the line segment is calculated as the gradient vector. If the absolute value of the direction angle difference of the gradient vectors of adjacent points is greater than 2π, the point is considered to be the endpoint of the fruit tree row;

[0009] S4: Set the camera coordinate system (O C X C Y C Z C ), image coordinate system (xO1y) and world coordinate system (O W X W Y W Z W ) to obtain the posture and position of the working robot and calculate the next control instruction; the world coordinate system (O W X W Y W Z W ) is located on the chassis of the operating robot, Y W The axis is perpendicular to the ground and points upward, and the intersection with the ground is the origin O W , X W The two tracks of the robot chassis are vertical, Z W The axis direction is the forward direction of the chassis of the working robot, parallel to the wheels on both sides and passing through the center line of the chassis; the camera coordinate system (O C X C Y C Z C )Z C The axis is consistent with the optical axis, and the origin of the coordinate system is O C is the optical center of the camera, the origin height of the camera is h, the installation angle is α, X C With X W Consistent, Y C Axis perpendicular to Z C O C X Cplane; the image plane coordinate system (xO1y) is the imaging plane in the camera; the origin of the world coordinate system is O W The vertical distance to the inter-row navigation line L is the offset distance λ of the chassis, and the angle between the center line of the chassis and the inter-row navigation line L is the offset angle θ. The offset distance λ and the offset angle θ of the chassis can be converted according to the relationship between the three coordinate systems, so as to accurately control the path tracking of the inter-row navigation line L of the working robot;

[0010] S5: Use the normal method to obtain the endpoint of the fruit tree row. When the vertical distance D of the straight line to the endpoint of the fruit tree row is less than 1 / 2 of the wheelbase of the working robot, end the current inter-row navigation. When changing rows, first control the working robot to rotate 90°, then control the working robot to travel in a straight line the distance of the crown diameter, and then control the working robot to rotate 90° to enter the fruit tree rows. Finally, repeat the inter-row navigation method to control the working robot's navigation operation.

[0011] In a further technical solution, the improved Fast-Unet network structure optimizes the Unet network structure. Since the targets in the classification task of the navigation path between fruit tree rows are large and the classification task is relatively simple, the number of convolution kernels is reduced to 1 / 4 of the Unet network structure. The reduction of semantic segmentation model parameters provides a guarantee for the real-time performance of path recognition; at the same time, the deconvolution upsampling operation of the Unet network structure is replaced by the bilinear interpolation method, which reduces the computational complexity of the semantic segmentation model while keeping the accuracy level basically unchanged; the convolution layer after the last downsampling of the semantic segmentation model is replaced by the ASPP module (porous spatial pyramid pooling), and multiple feature maps with receptive fields of different scales are generated through the hole convolutions with different expansion rates in the ASPP module (porous spatial pyramid pooling), so as to capture multi-scale features and retain the path edge features to the maximum extent; the ASPP module (porous spatial pyramid pooling) consists of 3 3×3 hole convolutions, 1 1×1 convolution and 1 global average pooling.

[0012] In a further technical solution, the collected navigation path image data set between fruit tree rows is divided into a training set and a validation set, and the navigation path image data between fruit tree rows is annotated using EasyDL software, and the moving path of the operating robot between fruit tree rows is marked in blue, and the rest is used as the background;

[0013] During the model training process, the original input image is randomly rotated 90°, flipped horizontally or vertically, and the image hue, saturation, value (HSV) is adjusted to perform random data enhancement to improve the robustness and accuracy of the model; when reading the image data in each iterative training, the original image executes the above steps in sequence; each step has a 50% probability of triggering a change; the HSV random transformation is limited to a certain range, the hue (H) transformation range is less than 20°, the saturation (S) transformation range is less than 30%, and the value (V) transformation range is less than 20%; the randomly rotated and flipped images provide guarantees for the direction confirmation of the navigation path between the rows of various fruit trees in the data set, and the random adjustment of the image HSV provides a guarantee for the model to adapt to the uneven natural lighting environment;

[0014] The hyperparameters used in the model training process are: the number of iterations is 100 times, the minimum batch size is 32 pictures; the BCE Loss-Dice Loss combination is used as the loss function of the training; the loss function is calculated by weighting the binary cross entropy loss (BCE Loss) and Dice Loss. The specific formula is as follows:

[0015] BCELoss-DiceLoss=BCE Loss+μ×Dice Loss

[0016]

[0017] Where μ is the weight of Dice Loss, y is the image pixel value in the label map, is the image pixel value in the prediction image, m is the number of pixels in each image; Y is the label image, For the prediction graph, Smooth is used to avoid the situation where the denominator is 0, and its value is 1×10 -5 ; BCE Loss judges the classification effect of each pixel from the details, and Dice Loss judges the classification effect of the prediction map from the global perspective; The recognition of the navigation path between fruit tree rows requires the accuracy of the path direction to be judged from the global perspective, while taking into account the accuracy of the recognition of the path boundary details, so the weight μ=0.5 is given to Dice Loss in the calculation of BCE Loss-Dice Loss;

[0018] In the process of transfer learning, the semantic segmentation model learned in the feature space of the source domain is applied to the feature space of the target domain, and then a new semantic segmentation model is learned based on the target domain.

[0019] In a further technical solution, in step S5, the optical center of the camera is taken as the intersection point, the distance between the endpoints of the fruit tree row being photographed is L, the vertical distance between the optical center of the camera and the straight line between the endpoints of the fruit tree row is D, the straight line size of the endpoints of the fruit tree row being photographed on the imaging plane is m, and the focal length of the camera is F, wherein

[0020] Compared with the prior art, the inter-row positioning and navigation method of a closed orchard operation robot based on semantic segmentation of the present invention has the following beneficial effects:

[0021] 1. Based on the improved Fast-Unet network, a semantic segmentation model is established to realize the effective identification and segmentation of the navigation path between fruit tree rows when the road is blocked by lush branches and leaves or covered by roads. The Canny operator is used to extract the boundary of the maximum connected area, the equation of the navigation line between rows is calculated, and the endpoints of the fruit tree row are obtained based on the normal method, which can accurately control the path tracking of the navigation line L between rows of the operating robot.

[0022] 2. The improved Fast-Unet network structure uses the ASPP structure and bilinear interpolation method, which can maintain high accuracy while reducing the number of convolution parameters. Before upsampling, the input image is extracted by using multiple hole convolutions in ASPP for parallel sampling. The semantic information is enriched by expanding the receptive field, and the relationship between the context is considered to avoid segmentation errors caused by being trapped in local features, thereby improving the segmentation accuracy of the target. And by using bilinear interpolation for upsampling, the spatial resolution of the image can be enhanced based on neighboring pixels, enriching the detailed information of the image and making the edge features of the image clearer. The combination of ASPP and bilinear interpolation method improves the segmentation accuracy of the target from both global and pixel aspects, and further improves the accuracy of edge pixel classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the accompanying drawings:

[0024] Figure 1 It is a flow chart of the positioning and navigation method of a robot operating in a closed orchard based on semantic segmentation of the present invention;

[0025] Figure 2 This is a schematic diagram of the image acquisition of the navigation path between fruit tree rows of the present invention;

[0026] Figure 3 This is a schematic diagram of the data enhancement process of the present invention;

[0027] Figure 4 This is a schematic diagram of the improved Fast-Unet network structure of the present invention;

[0028] Figure 5 Schematic diagram of the coordinate system of the working robot of the present invention;

[0029] Figure 6 Schematic diagram of the offset distance and the offset angle in the image coordinate system and the world coordinate system of the present invention;

[0030] Figure 7 Schematic diagram for calculating the vertical distance D between the optical center of the camera and the straight line of the end points of the fruit tree row of the present invention;

[0031] Figure 8 Schematic diagram of the learning rate adjustment strategy of the present invention. DETAILED DESCRIPTION

[0032] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0033] like Figure 1 As shown, the present invention preferably adopts a closed orchard operation robot positioning and navigation method based on semantic segmentation, which is implemented based on a field camera image acquisition device. It includes a camera, a video acquisition card, a computer, and corresponding image acquisition, transmission and storage algorithms and software. The camera is used to collect, transmit and store video images between orchard rows in the computer, and then the software system performs corresponding algorithm processing and recognition, and the computer outputs the positioning information of the operation robot.

[0034] 1. Collection of fruit tree row navigation path image dataset.

[0035] like Figure 2 As shown in the figure, the camera is installed on the working robot, and the current navigation path image between fruit trees is continuously collected while the working robot is driving. The computer for image storage is Inter Core i5-7300HQ CPU, main frequency 2.5GHz, memory 16GB. Starting from the complex scene, three kinds of fruit trees are selected based on the height and crown shape of the fruit trees to represent the common navigation scenes between orchard rows. The relatively tall upright fruit trees are represented by peach trees; the medium-height common fruit trees are represented by orange trees; and the trellis-shaped fruit trees are represented by kiwis. There are shade and weeds on the navigation paths between the selected fruit trees, which increases the complexity of the navigation path scene.

[0036] 2. Dataset annotation.

[0037] The collected fruit tree row navigation path image dataset was divided into a training set and a validation set. The fruit tree row navigation path image data was annotated using the EasyDL software. The guided path of the operating robot between the fruit tree rows was marked in blue, and the rest was used as the background. Considering the computational complexity of the semantic segmentation algorithm and the occupation of memory resources, the original image was scaled to 512×512.

[0038] 3. Dataset enhancement.

[0039] like Figure 3 As shown in the figure, in order to expand the data set, the original input image is randomly rotated 90°, flipped horizontally or vertically, and the image hue, saturation, value (HSV) is adjusted to perform random data enhancement during the model training process to improve the robustness and accuracy of the model; when reading the image data in each iterative training, the original image executes the above steps in sequence; each step has a 50% probability of triggering a change; the HSV random transformation is limited to a certain range, the hue (H) transformation range is less than 20°, the saturation (S) transformation range is less than 30%, and the value (V) transformation range is less than 20%; the randomly rotated and flipped images provide a guarantee for the direction confirmation of the navigation path between the rows of various fruit trees in the data set, and the random adjustment of the image HSV provides a guarantee for the model to adapt to the uneven natural lighting environment.

[0040] 4. Based on the improved Fast-Unet network structure, a semantic segmentation model is established to identify and segment the navigation path between fruit tree rows.

[0041] like Figure 4 As shown in the figure, the traditional FCN has limited connections between features of different scales in image semantic segmentation. The Unet network structure solves this problem, but the operation efficiency of the Unet network structure is not high. Therefore, the present invention proposes an improved Fast-Unet network structure based on the Unet network structure. The Unet network structure integrates the high-resolution information during network downsampling and the resolution information during upsampling by means of feature superposition through skip connections to enrich the sampling information and improve the segmentation accuracy.

[0042] The improved Fast-Unet is a lightweight semantic segmentation algorithm based on the Unet network structure with high accuracy, fewer model parameters and fast inference speed. In order to meet the real-time requirements of navigation path recognition between fruit tree rows of the operating robot, the Unet network structure is optimized. Considering that the targets in the classification task of navigation path between fruit tree rows are large and the classification task is relatively simple, the number of convolution kernels is reduced to 1 / 4 of the Unet network structure. The reduction of model parameters provides a guarantee for the real-time performance of path recognition. At the same time, the deconvolution upsampling operation of the Unet network structure is replaced by the bilinear interpolation method, which reduces the amount of model calculation while keeping the accuracy level basically unchanged. In order to meet the high-precision requirements of navigation path recognition between fruit tree rows of the operating robot, the context information of the image is captured at multiple scales, and the convolution layer after the last downsampling of the model is replaced by ASPP. After the hole convolution with different expansion rates in ASPP, multiple feature maps with receptive fields of different scales are generated to capture multi-scale features and retain the edge features of the path to the maximum extent. The ASPP module (Atrous Spatial Pyramid Pooling) consists of three 3×3 dilated convolutions, one 1×1 convolution, and one global average pooling. The reason why the improved Fast-Unet network structure can reduce the number of convolution parameters and still maintain high accuracy is because it uses the ASPP structure and bilinear interpolation method. Before upsampling, the input image is extracted by using multiple dilated convolutions in ASPP in parallel sampling. By expanding the receptive field to enrich the semantic information, the relationship between the context is considered, and the segmentation error caused by being trapped in local features is avoided, thereby improving the segmentation accuracy of the target. And by using bilinear interpolation for upsampling, the spatial resolution of the image can be enhanced based on neighboring pixels, enriching the detailed information of the image, and making the edge features of the image clearer. The combination of ASPP and bilinear interpolation method improves the segmentation accuracy of the target from both global and pixel aspects, and further improves the accuracy of edge pixel classification.

[0043] 5. Hyperparameter setting and model training

[0044] The hyper parameters used in the training of fruit tree row navigation path recognition are: 100 epochs and 32 images for the minimum batch size. The learning rate adjustment strategy in the learning rate decay strategy is adopted, and the learning rate distribution is as follows: Figure 8 shown.

[0045] The task of fruit tree row navigation path recognition is clear and quite different from the conventional semantic segmentation object recognition task. In addition, there is sufficient peach tree navigation path data, so transfer learning was not used in the peach tree dataset training. When performing transfer learning training on the orange and kiwi datasets, the best model parameters obtained by direct training on the peach dataset were used as the initial training parameters of the new model, and the remaining training parameters and learning strategies remained the same as the original training.

[0046] The BCE Loss-Dice Loss combination is used as the loss function for training. The loss function is calculated by weighting the binary cross entropy loss (BCE Loss) and Dice Loss. The specific formula is as follows:

[0047] BCE Loss-Dice Loss=BCE Loss+μ×Dice Loss

[0048]

[0049] Where μ is the weight of Dice Loss, y is the image pixel value in the label map, is the image pixel value in the prediction image, m is the number of pixels in each image; Y is the label image, For the prediction graph, Smooth is used to avoid the situation where the denominator is 0, and its value is 1×10 -5 .

[0050] BCE Loss judges the classification effect of each pixel from the details, and Dice Loss judges the classification effect of the prediction map from the global perspective. In this paper, the recognition of the navigation path between fruit tree rows requires the accuracy of the path direction to be judged from the global perspective, while taking into account the accuracy of the recognition of the path boundary details. Therefore, the weight μ=0.5 of Dice Loss is given in the calculation of BCE Loss-Dice Loss.

[0051] 6. The navigation path model between peach trees was transferred to orange trees and kiwis

[0052] Transfer learning is the process of deep learning models using existing knowledge to learn new knowledge. Its core is to find similarities between existing knowledge and new knowledge. In transfer learning, existing knowledge is called the source domain. The source domain is the feature space of the peach tree row navigation path recognition dataset, and the target domain is the feature space of the orange and kiwi fruit row navigation path dataset. The source domain and the target domain are different but have a certain relationship. Based on model transfer, it is assumed that the source domain and the target domain can share some model parameters, and the model learned in the feature space of the source domain is applied to the feature space of the target domain, and then a new model is obtained based on the target domain learning.

[0053] 7. Extract the navigation path contour and navigation line between fruit tree rows based on Canny operator

[0054] First, the deep learning semantic segmentation model segments the inter-row navigation path and generates a binary image consisting of the inter-row navigation path and the background. The largest connected area in the binary image is selected as the inter-row navigation path, and the Canny operator is used to extract the boundary of the largest connected area. Then, the points on the boundary are medianized to generate a scatter plot of the path centerline. Finally, the centerline scatter plot is subjected to the least squares regression of a linear equation with the length and width of the image as the coordinate axis. The path centerline equation is obtained as the navigation line equation. The calculation formula is as follows:

[0055] y=kx+b

[0056] 8. Get the endpoints of the fruit tree row based on the normal method

[0057] The gradient vector of a point on the contour curve of the boundary of the navigation path between lines is the vertical line of the curve segment passing through the point. A small line segment near the point is used to replace the curve segment, and the vertical line of the line segment is calculated as the gradient vector. The greater the change in the direction angle of the gradient vector of adjacent points, the more likely it is that there is an endpoint near the point. A threshold is set here. When the absolute value of the adjacent angle difference is greater than This point is considered the end point of the fruit tree row.

[0058] Let P n ={p1,…,p n} is an ordered set of points on the contour line of the fruit tree row path. n ={l1,…,l n} is a small line segment near an ordered point on the contour line, l i (i=1,…,n) represents point p i As the center, the domain length is d, which is the connection point p i-d and p i+d The line segment K n ={k1,…,k n} Represents line segment l i The set of normal slopes of A. n ={a1,…,a n} represents point p i Nearby line segment l i (that is, the connection point p i-d and p i+d The angle set of the normal line of the line segment) is a i The range is [0,2π]. Passing point p i (x i ,y i ) i (that is, the connection point p i-d (x i-d ,yi-d ) and point p i+d (x i+d ,y i+d) ) has a slope of:

[0059] g i =(y i+d -y i-d ) / (x i+d -x i-d )

[0060] Line segment l i The slope of the normal is

[0061] k i =-1×(x i+d -x i-d ) / (y i+d -y i-d )

[0062] By point (p i-d ,p i-d+1 ,…,p i ,…,p i+d-1 ,p i+d ) can be either convex or concave, passing through point p i The straight line l i Although the normal is the same, the direction of the normal may point upward or downward. The normal slope a is defined as i The calculation method is shown in Table 1.

[0063] Table 1a i Calculation method

[0064]

[0065] 9. Vision-based navigation methods

[0066] To implement navigation line tracking, the robot must first determine its own posture parameters, convert the camera coordinate system, image coordinate system and world coordinate system to obtain the posture and position of the robot, and calculate the next control instruction. Figure 5 Set the world coordinate system as shown (O W X W Y W Z W ) is located on the chassis of the agricultural machinery, Y W The axis is perpendicular to the ground and points upward, and the intersection with the ground is the origin O W , X W The two tracks of the vertical agricultural machinery chassis, Z W The axis direction is the forward direction of the agricultural machinery chassis, parallel to the wheels on both sides and passing through the center line of the chassis; the camera coordinate system (O C XC Y C Z C )Z C The axis is consistent with the optical axis, and the origin of the coordinate system is O C is the optical center of the camera, the origin height of the camera is h, the installation angle is α, X C With X W Consistent, Y C Axis perpendicular to Z C O C X C Plane; the image plane coordinate system (xO1y) is the imaging plane within the camera.

[0067] In the image plane, L is the line between rows (i.e., the line between rows obtained based on the Canny operator), and the origin of the world coordinate system is O. W The vertical distance to the inter-row guide line L is the chassis offset distance λ, and the angle between the chassis centerline and the inter-row guide line L is the offset angle θ, as shown in Figure 6 As shown in the figure, the offset distance λ and the offset angle θ of the chassis can be converted according to the relationship between the three coordinate systems, so as to accurately control the inter-row navigation path tracking of the working robot. At the same time, the normal method is used to obtain the end point of the orchard row, and the current inter-row navigation ends when driving to the end point of the fruit tree row.

[0068] 10. Determine the end of single-line navigation and line break

[0069] At the same time, the normal method is used to obtain the endpoints of the orchard row. When the vertical distance to the straight line of the fruit tree row endpoint is less than 1 / 2 of the wheelbase of the working robot, the current row navigation is terminated.

[0070] According to the principle of pinhole imaging, the camera imaging follows, such as Figure 7 As shown, with the optical center of the camera lens as the intersection point, the distance between the endpoints of the fruit tree row being photographed is L, the vertical distance between the optical center and the endpoint line is D, the distance between the optical center and the imaging plane is d, and the dimension of the endpoint line of the fruit tree row being photographed on the imaging plane is m, which forms a similar triangle relationship. The imaging plane is located near the focus of the camera, and D is much larger than d, so d can be approximately equal to the focal length F. Therefore, the vertical distance D between the optical center and the endpoint line is expressed as:

[0071]

[0072] When D is less than 1 of the wheelbase of the robot / The current navigation between fruit tree rows ends at 2 o'clock. When changing rows, the encoder control method is used. First, the heading of the working robot is controlled to rotate 90°, and then the working robot is controlled to drive in a straight line the distance of the crown diameter, and then the working robot is controlled to rotate 90° to enter the fruit tree rows. Finally, the navigation method between rows is repeated to control the navigation operation of the working robot.

[0073] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A closed orchard operation robot positioning and navigation method based on semantic segmentation, characterized in that: The following steps are involved: S1: A camera is installed on the working robot to continuously collect images of the current navigation path between fruit tree rows when the working robot is driving; S2: A semantic segmentation model is established based on the improved Fast-Unet network. After model training and transfer learning, the semantic segmentation model is used to identify and segment the navigation path between fruit tree rows, and a binary image consisting of the navigation path between fruit tree rows and the background is generated. The largest connected area in the binary image is selected as the navigation path between rows, and the Canny operator is used to extract the boundary of the largest connected area. The points on the boundary are medianized to generate a scatter plot of the center line of the path. The center line scatter plot is used to perform the least squares regression of a linear equation with the length and width of the image as the coordinate axis, and the path center line equation is obtained, which is the navigation line equation between rows. S3: Obtain the endpoints of the fruit tree row based on the normal method; the gradient vector of a point on the boundary contour curve of the navigation path between rows is the vertical line of the curve segment passing through the point. A small line segment near the point is used to replace the curve segment, and the vertical line of the line segment is calculated as the gradient vector. The absolute value of the gradient vector direction angle difference of adjacent points is greater than This point is considered the end point of the fruit tree row; S4: Set the camera coordinate system (O C X C Y C Z C ), image coordinate system (xO1y) and world coordinate system (O W X W Y W Z W ) to obtain the posture and position of the working robot and calculate the next control instruction; the world coordinate system (O W X W Y W Z W ) is located on the chassis of the operating robot, Y W The axis is perpendicular to the ground and points upward, and the intersection with the ground is the origin O W , X W The two tracks of the robot chassis are vertical, Z W The axis direction is the forward direction of the chassis of the working robot, parallel to the wheels on both sides and passing through the center line of the chassis; the camera coordinate system (O C X C Y C Z C )Z C The axis is consistent with the optical axis, and the origin of the coordinate system is O C is the optical center of the camera, the origin height of the camera is h, the installation angle is α, X C With X W Consistent, Y C Axis perpendicular to Z C O C X C plane; the image plane coordinate system (xO1y) is the imaging plane in the camera; the origin of the world coordinate system is O W The vertical distance to the inter-row navigation line L is the offset distance λ of the chassis, and the angle between the center line of the chassis and the inter-row navigation line L is the offset angle θ. The offset distance λ and the offset angle θ of the chassis can be converted according to the relationship between the three coordinate systems, so as to accurately control the path tracking of the inter-row navigation line L of the working robot; S5: Use the normal method to obtain the endpoint of the fruit tree row. When the vertical distance D of the straight line to the endpoint of the fruit tree row is less than 1 / 2 of the wheelbase of the working robot, end the current inter-row navigation. When changing rows, first control the working robot to rotate 90°, then control the working robot to travel in a straight line the distance of the crown diameter, and then control the working robot to rotate 90° to enter the fruit tree rows. Finally, repeat the inter-row navigation method to control the working robot's navigation operation.

2. The closed orchard operation robot positioning and navigation method based on semantic segmentation according to claim 1 is characterized by: The improved Fast-Unet network structure optimizes the Unet network structure. Since the targets in the classification task of the navigation path between fruit trees are large and the classification task is relatively simple, the number of convolution kernels is reduced to 1 / 4 of the Unet network structure. The reduction of semantic segmentation model parameters provides a guarantee for the real-time performance of path recognition. At the same time, the deconvolution upsampling operation of the Unet network structure is replaced by the bilinear interpolation method, which reduces the computational complexity of the semantic segmentation model while keeping the accuracy level basically unchanged. The convolution layer after the last downsampling of the semantic segmentation model is replaced by the ASPP module (porous spatial pyramid pooling). After the hole convolutions with different expansion rates in the ASPP module (porous spatial pyramid pooling), multiple feature maps with receptive fields of different scales are generated to capture multi-scale features and retain the path edge features to the maximum extent. The ASPP module (Atrous Spatial Pyramid Pooling) consists of three 3×3 dilated convolutions, one 1×1 convolution, and one global average pooling.

3. The closed orchard operation robot positioning and navigation method based on semantic segmentation according to claim 2 is characterized by: The collected fruit tree row navigation path image dataset was divided into a training set and a validation set. The fruit tree row navigation path image data was annotated using the EasyDL software. The moving path of the operating robot between the fruit tree rows was marked in blue, and the rest was used as the background. During the model training process, the original input image is randomly rotated 90°, flipped horizontally or vertically, and the image hue, saturation, value (HSV) is adjusted to perform random data enhancement to improve the robustness and accuracy of the model; when reading the image data in each iterative training, the original image executes the above steps in sequence; each step has a 50% probability of triggering a change; the HSV random transformation is limited to a certain range, the hue (H) transformation range is less than 20°, the saturation (S) transformation range is less than 30%, and the value (V) transformation range is less than 20%; the randomly rotated and flipped images provide guarantees for the direction confirmation of the navigation path between the rows of various fruit trees in the data set, and the random adjustment of the image HSV provides a guarantee for the model to adapt to the uneven natural lighting environment; The hyperparameters used in the model training process are: the number of iterations is 100 times, the minimum batch size is 32 pictures; the BCE Loss-Dice Loss combination is used as the loss function of the training; the loss function is calculated by weighting the binary cross entropy loss (BCE Loss) and Dice Loss. The specific formula is as follows: BCELoss-DiceLoss=BCE Loss+μ×Dice Loss Where μ is the weight of Dice Loss, y is the image pixel value in the label map, is the image pixel value in the prediction image, m is the number of pixels in each image; Y is the label image, For the prediction graph, Smooth is used to avoid the situation where the denominator is 0, and its value is 1×10 -5 ; BCE Loss judges the classification effect of each pixel from the details, and Dice Loss judges the classification effect of the prediction map from the global perspective; The recognition of the navigation path between fruit tree rows requires the accuracy of the path direction to be judged from the global perspective, while taking into account the accuracy of the recognition of the path boundary details, so the weight μ=0.5 is given to Dice Loss in the calculation of BCE Loss-Dice Loss; In the process of transfer learning, the semantic segmentation model learned in the feature space of the source domain is applied to the feature space of the target domain, and then a new semantic segmentation model is learned based on the target domain.

4. The closed orchard operation robot positioning and navigation method based on semantic segmentation according to any one of claims 1 to 3, characterized in that: In step S5, the optical center of the camera is taken as the intersection point, the distance between the endpoints of the fruit tree row being photographed is L, the vertical distance between the optical center of the camera and the straight line between the endpoints of the fruit tree row is D, the straight line size of the endpoints of the fruit tree row being photographed on the imaging plane is m, and the focal length of the camera is F, where

Images