Road navigation recognition method for pitaya orchard based on machine vision

By combining the improved DeepLabv3+ network and the REALSENSE D455 camera, the interference and real-time problems of navigation path recognition in the Dragon Fruit Orchard are solved, and more accurate and fast navigation line fit is achieved, which improves the development of orchard machinery intelligence.

CN118333825BInactive Publication Date: 2024-11-15ZHONGKAI UNIV OF AGRI & ENG
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202410482192.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-04-22
Publication Date
2024-11-15
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

In the Dragon Fruit Orchard, traditional machine vision algorithms are difficult to deal with interference from factors such as light and shadow changes, occlusion and weeds, resulting in inaccurate division of navigation roads, limiting the development of orchard machinery intelligence, and the existing navigation path recognition algorithm is complex to deploy. Large parameters and poor real-time performance lead to problems with path recognition lag and offset.

Method used

The improved DeepLabv3+ image semantic segmentation network is adopted, and the lightweight MobileNetv2 is used as feature extraction backbone network, and the ECA-Net module and pyramid split self-attention module are inserted into its shallow features to improve segmentation effect and boundary clarity. Combined with the REALSENSE D455 depth camera, the road edge points are acquired through binarized images and the navigation line is fitted, and the accuracy of the navigation line is further judged by manual fitting of the center line.

Benefits of technology

It effectively avoids interference from external environmental factors, improves the accuracy of road edge information extraction and real-time response of navigation line fitting, ensures the accuracy and real-time nature of navigation line, and avoids the problems of path offset and inaccurate positioning.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118333825B_ABST
    Figure CN118333825B_ABST
Patent Text Reader

Abstract

The present invention provides a method for navigation and recognition of pitaya orchard roads based on machine vision, which relates to the field of road planning and includes the following steps: step A, data collection; step B, image segmentation; step C, navigation line fitting; step D, navigation line accuracy judgment; step E, navigation recognition algorithm output. The method is based on improved DeepLabv3+ The network can effectively solve the problems faced by visual navigation in the road environment of pitaya orchards, such as many interference factors, complex image background, huge model deployment, and certain lag in planning algorithm.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the technical field of road planning, and in particular to a pitaya orchard road navigation and recognition method based on machine vision. Background Art

[0002] Among my country's agricultural planting industries, fruit planting ranks among the top three; however, the degree of mechanized fruit picking is still low, and most of it is still mainly manual picking, which is significantly different from that of developed countries. Since the orchard environment is mostly semi-structured, large-scale mechanical operations are limited, and traditional machine vision algorithms are difficult to handle interference from factors such as light and shadow changes, occlusion, and weeds, resulting in inaccurate navigation road segmentation, which greatly limits the development of intelligent orchard machinery.

[0003] So far, with the continuous development of deep learning technology, many researchers have applied deep learning technology to the research of navigation path recognition and have proposed many different semantic segmentation algorithms and combinations, such as threshold segmentation ( Otsu ), histogram-based bundling, region growing ( Nock&Nielsen )、k-means clustering( Dhanachandra etc.) and watershed ( Najman However, the navigation path detection and recognition task of pitaya orchard roads is heavy. The orchard roads are affected by weeds and fallen branches, which makes the boundary information of the roads blurred, thereby increasing the difficulty of navigation path recognition. At the same time, most existing navigation path recognition algorithms pay less attention to the problem of model deployment, resulting in complex network structure, large number of model parameters, and high requirements for computing resources, which in turn affects the real-time performance of path recognition. That is, the navigation path recognition algorithm cannot provide feedback and fit the path in time, and has a certain lag. The interference of weeds and fallen branches on the mixed roads is prone to problems such as orchard road navigation path deviation and inaccurate positioning. Summary of the invention

[0004] In view of the problems existing in the above prior art, the object of the present invention is to provide a pitaya orchard road navigation and recognition method based on machine vision, which is based on an improved DeepLabv3+ The network can effectively solve the problems faced by visual navigation in the road environment of pitaya orchards, such as many interference factors, complex image background, huge model deployment, and certain lag in planning algorithm.

[0005] The purpose of the present invention is achieved through the following technical solutions:

[0006] A method for navigation and recognition of pitaya orchard roads based on machine vision comprises the following steps:

[0007] Step A, data collection: collect a large number of pitaya orchard road images to form a training data set, a test data set and a validation data set for the full convolutional network;

[0008] Step B, Image Segmentation: First, design an improved DeepLabv3+ Image semantic segmentation network; then, through the improved DeepLabv3+ The image semantic segmentation network segments the orchard road image to obtain a road segmentation map;

[0009] Step C, navigation line fitting: First, the mask area of ​​the road segmentation map obtained in step B is binarized to obtain a binarized image, and the left and right edge points of the road are obtained through the binarized image; then, the left and right edge points of the road are fitted to obtain the navigation line;

[0010] Step D, determining the accuracy of the navigation line: manually fitting the center line of the orchard road, and comparing the manually fitted center line of the orchard road with the navigation line obtained in step C to determine whether the navigation line is accurate;

[0011] Step E: Navigation recognition algorithm output: Use the precise navigation line fitting method output after judgment in step D to perform pitaya orchard road recognition and navigation.

[0012] Based on further optimization of the above scheme, the REALSENSE D455 depth camera is used in step A to collect road images of the pitaya orchard; during the data collection process, the REALSENSE D455 depth camera is tilted downward at an angle of 10 to 15° and at a height of 0.5 to 0.8 m, and the shooting direction is directly in front of the road.

[0013] Based on further optimization of the above scheme, the pitaya orchard road images collected in the step A include images under three environments: sunny day with direct light, sunny day with backlight and cloudy day; after collecting the images, Labelme annotation software is used to annotate the pitaya orchard road images, annotate the orchard roads, and divide the annotated pitaya orchard road images into a training data set, a test data set and a validation data set according to the ratio of 8:1:1.

[0014] Based on the further optimization of the above scheme, the improved DeepLabv3+ The image semantic segmentation network is specifically: using a lightweight MobileNetv2 As DeepLabv3+ The feature extraction backbone network is MobileNetv2 Insert into the shallow features ECA-Net Modules ( Efficient channel attention network , channel attention mechanism) to improve the segmentation effect without destroying the network structure, reduce the computational complexity and improve the clarity of the target boundary; at the same time, ASPPThe pyramid split self-attention module is introduced after the module ( PSA, Polarized Self- Attention mechanism ) to extract the multi-scale spatial information of road images at a more detailed level and establish a long-range channel dependency, thereby improving the spatial feature representation of the feature map.

[0015] Based on the further optimization of the above scheme, the specific method of obtaining the left and right edge points of the road by binarizing the image in step C is as follows: ensure that the pixel value of the road part in the mask area is 255 and the pixel value of the background area is 0 by binarizing the image; then, regard the mask area of ​​the road part as a trapezoidal structure extending away from the shooting point and with a gradually decreasing width, and determine the vertex position of the mask area of ​​the road part (i.e., the left and right end points of the trapezoidal structure close to the shooting point); then, take the left vertex of the image as the scanning starting point, adopt the method of scanning pixels row by row, scan from left to right in sequence, detect whether the values ​​of the two adjacent columns of pixels corresponding to the scanned pixel point change, and obtain the left boundary point; similarly, take the right vertex of the image as the scanning starting point, adopt the direction of scanning pixels row by row, and scan from right to left in sequence, detect whether the values ​​of the two adjacent columns of pixels corresponding to the scanned pixel point change, and obtain the right boundary point;

[0016] The specific method of detecting whether the values ​​of the two adjacent columns of pixels corresponding to the scanned pixel point have changed and obtaining the left and right boundary points is as follows:

[0017] For scanning pixels (x 0 ,y 0 ) , respectively detect the pixels in the adjacent columns on the left (x 0 ,y 0 -1) The pixel in the adjacent column to the right (x 0 ,y 0 +1) Is the pixel value the same as the scanned pixel point? If the pixel value is the same, the scanned pixel point is not a left boundary point or a right boundary point, and the scan continues;

[0018] If different, continue to judge, if When (x 0 ,y 0 ) is the left boundary point;

[0019] like When (x 0 ,y 0 ) is the right boundary point.

[0020] Based on the further optimization of the above scheme, the specific method of fitting the left and right edge points of the road in step C to obtain the navigation line is:

[0021] Step C1: First, according to the left boundary point set Fit left boundary line , where n represents the number of left boundary points; according to the right boundary point set Fit left boundary line ,in, m Indicates the number of right boundary points; a l 、a r are the slopes of the left and right boundary lines, respectively. b l 、b r are the intercepts of the left and right boundary lines respectively;

[0022] Step C2: Combine the linear equations of the left boundary line and the right boundary line to obtain the coordinates of their intersection. (x i , y i ) :

[0023] ;

[0024] Step C3: For each intersection coordinate (x i ,y i ) , the distance from the navigation line to the intersection d i for:

[0025] ;

[0026] Where: x d Indicates the horizontal coordinate of the point on the navigation line, a n 、b n They represent the slope and intercept of the navigation line respectively;

[0027] Get all guide lines to the intersection point (x i ,y i ) distance d i The sum of squares, i.e. the loss function L :

[0028] ;

[0029] Where: N represents the number of intersection coordinates;

[0030] The optimal solution is obtained by solving the partial derivative of the loss function to zero. a n and b n , thereby obtaining the navigation line;

[0031] Step C4, obtain the correlation coefficient R of the navigation line and determine the fitting accuracy of the navigation line:

[0032] ;

[0033] Where: (x di ,y di ) Indicates fitting any fitting point on the navigation line; , They represent the average values ​​of the abscissa of the fitting points and the average values ​​of the ordinate of the intersection points respectively;

[0034] R The closer it is to 1, the higher the accuracy of the navigation line.

[0035] Based on the further optimization of the above scheme, the step D is specifically as follows:

[0036] First, the center points of each road were manually marked on the image of the orchard road using the annotation tool of the image processing software. (P 1 ,P 2 ,…,P n ) , thus forming an artificial fitting center line;

[0037] Then, extract the points on the navigation line fitted in step C ( Q 1 ,Q 2 ,…,Q m ) , and then calculate the direction vectors of the artificial fitting center line and the navigation line respectively V 1 and V 2:

[0038] ;

[0039] ;

[0040] After that, get the angle between the manually fitted center line and the navigation line θ :

[0041]

[0042] Where: , Represents the direction vector V 1. V 2 modulus;

[0043] Then, H pixels are selected from top to bottom in the segmented image of the artificial fitting center line at equal distances, and the horizontal coordinates of the navigation line and the artificial fitting center line in the same vertical coordinate direction are compared to obtain the navigation pixel error e x :

[0044] ;

[0045] Where: X DCi 、X HCi They represent the horizontal coordinates of the pixels on the navigation line and the horizontal coordinates of the pixels corresponding to the artificial fitting center line respectively;

[0046] Finally, set the angle threshold ,like , then output this method for actual operation;

[0047] like , then the navigation pixel error e x Compensate the navigation line obtained in step C to obtain a new navigation line, and then compare the new navigation line with the manually fitted center line until .

[0048] Based on further optimization of the above scheme, the is 5°.

[0049] Based on the further optimization of the above scheme, the specific method of compensating the navigation line through the navigation pixel error is:

[0050] ;

[0051] Where: represents an adjustment parameter used to control the magnitude of compensation (obtained based on a large amount of experimental data); Indicates the first i The horizontal pixel coordinates of the detection points; represents the first navigation line obtained by fitting in step C. i The horizontal pixel coordinates of the detection points.

[0052] The present invention has the following technical effects:

[0053] This application is improved DeepLabv3+ Image semantic segmentation network, that is, using lightweight MobileNetv2 As DeepLabv3+The feature extraction backbone network is DeepLabv3+ Insert ECA-Net The module and the pyramid split self-attention module, on the basis of destroying the original backbone network, improve the network's perception ability and segmentation accuracy, thereby giving greater weight to key information, extracting multi-scale spatial information of road images from a more detailed level, and obtaining spatial information of different scales, thereby effectively avoiding interference from external environmental factors (such as roadside weeds, fallen branches, etc.) in the process of extracting road edge information, and problems such as insufficient smoothness of road edge information fitting, ensuring that the extracted road edge information is richer and more complete, thereby ensuring the accuracy of navigation line fitting. In addition, this application further ensures the accuracy of the navigation line by comparing the navigation line fitted by the left boundary line and the right boundary line with the manually fitted center line, improving the real-time response of the fitted navigation line, and avoiding problems such as path deviation and inaccurate positioning during the operation of the robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 Improved in the embodiment of the present application DeepLabv3+ The structure diagram of the image semantic segmentation network.

[0055] Figure 2 Improved in the embodiment of the present application DeepLabv3+ Pyramid splitting self-attention module and image semantic segmentation network ECA-Net The structural diagram of the module; Figure 2 (a) is the SPC module structure diagram, Figure 2 (b) is the structure diagram of SE module. Figure 2 (c) is the structure diagram of the pyramid split self-attention module. Figure 2 (d) ECA-Net Module structure diagram.

[0056] Figure 3 This is a flowchart of fitting a navigation line in an embodiment of the present application.

[0057] Figure 4 This is a schematic diagram of the structure of the fitted navigation line in the embodiment of the present application; wherein, Figure 4 (a) is the orchard road map to be detected. Figure 4 (b) is the segmentation map of the drivable area. Figure 4 (c) is the road binary map, Figure 4 (d) is the left and right edge point extraction diagram. Figure 4 (e) is the navigation line fitting diagram. DETAILED DESCRIPTION

[0058] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0059] Embodiment 1:

[0060] A method for navigation and recognition of pitaya orchard roads based on machine vision comprises the following steps:

[0061] Step A, data collection: Collect a large number of pitaya orchard road images to form a training data set, a test data set and a validation data set for the full convolutional network, specifically:

[0062] A REALSENSE D455 depth camera is used to collect images of pitaya orchard roads. During data collection, the REALSENSE D455 depth camera is tilted downward at an angle of 10 to 15° (preferably 12°) and at a height of 0.5 to 0.8 m (preferably 0.6 m). The shooting direction is directly in front of the road, and the RGB image and the depth image are saved in JPEG and PNG formats, respectively. In this embodiment, a total of 2,000 pitaya orchard road images are collected, wherein the pitaya orchard road images include images under three environments: sunny day front light, sunny day back light, and cloudy day. After collecting the images, the collected original images are manually screened to obtain 1,120 original pitaya orchard road images, and then the pitaya orchard road images are annotated using Labelme annotation software to annotate the pitaya orchard road, and the annotated pitaya orchard road images are divided into a training data set, a test data set, and a validation data set according to the ratio of 8:1:1.

[0063] Step B, Image Segmentation: First, design an improved DeepLabv3+ Image semantic segmentation network, improved DeepLabv3+ Image semantic segmentation network Figure 1 As shown, specifically: using lightweight MobileNetv2 As DeepLabv3+ The feature extraction backbone network is MobileNetv2 Insert into the shallow features ECA-Net Modules ( Efficient channel attention network , channel attention mechanism) to improve the segmentation effect without destroying the network structure, reduce the computational complexity and improve the clarity of the target boundary; at the same time, ASPP The pyramid split self-attention module is introduced after the module ( PSA, Polarized Self-Attention mechanism ) to extract the multi-scale spatial information of road images at a more detailed level and establish a long-range channel dependency, thereby improving the spatial feature representation of the feature map.

[0064] The pyramid split self-attention module consists of SPC ( split and concat ) module and SE module, among which the SPC module is as follows Figure 2As shown in (a), the SPC module can realize the splitting and merging of features. The input channel number is C. The feature X is evenly divided into n parts. Each group adopts the convolution with different convolution kernels to obtain the receptive fields of different scales, that is, to obtain the target information of different scales. Figure 2 As shown in (b), it includes three stages: first, the feature compression stage, in which the feature map of size HxWxC is compressed into global features with a channel size of 1x1xC through global average pooling; then, these global features are recalibrated using the sigmoid activation function to learn the weight relationship between each channel, thereby obtaining a weight of 1x1xC; finally, these weights are multiplied with the feature map, which effectively improves the effect of orchard road segmentation. Figure 2 As shown in (c), first, the feature vector of HxWxC is transformed using SPC ( Spatial Pyramid Channel ) module is decomposed into S blocks, and the multi-scale features of the channel are obtained through the pyramid structure; then, the merged feature map U is passed through SE ( Squeeze-and-Excitation ) module calculates the multi-scale feature map channel attention weight Z i , this method can make PSA ( pyramid split attention ) module fuses contextual information of different scales to generate better pixel-level attention for high-level feature maps; then, in order to achieve the interaction between local and global channel attention, softmax is used to recalibrate the multi-scale feature map channel attention weights, and the final result ATTi contains both spatial location information and attention weights in the channel; finally, the recalibrated weights are multiplied by the feature map to obtain the final result. In image semantic segmentation, the PSA module can fuse multi-scale spatial information with cross-channel attention, so that the final result can extract multi-scale spatial information at a finer level and establish longer-range inter-channel dependencies.

[0065] ECA-Net modules such as Figure 2 As shown in (d), the ECA-Net module introduces an attention mechanism structure on the feature channel, and significant performance gains can be achieved by introducing a small number of parameters. The ECA module replaces the original fully connected layer of the SE module with a one-dimensional convolution that can adaptively transform the convolution kernel size. Its forward propagation process includes the input feature matrix being average pooled to generate a feature matrix (1x1xC), and then learning the output feature matrix (1x1xC) through a one-dimensional convolution sliding, and finally using the Sigmoid function to nonlinearize the one-dimensional feature matrix; the relationship between the channel dimension C and the convolution kernel size k is as follows:

[0066] ;

[0067] Use ECA-Net to embed the model at different depths and adaptively select convolution kernels of different sizes according to the channel dimension C; insert the ECA-Net module into the shallow features of MobileNetV2 to improve the segmentation effect without destroying the network structure.

[0068] Then, through the improved DeepLabv3+ The image semantic segmentation network segments the orchard road image and obtains the road segmentation map, such as Figure 4 (b) as shown;

[0069] Step C: Navigation line fitting: refer to Figure 3 As shown, first, the mask area of ​​the road segmentation map obtained in step B is binarized to obtain a binary image (such as Figure 4 (c)), the left and right edge points of the road are obtained by binarizing the image (as shown in Figure 4 (d)), specifically:

[0070] The image is binarized to ensure that the pixel value of the road portion in the mask area is 255 and the pixel value of the background area is 0; then, the mask area of ​​the road portion is regarded as a trapezoidal structure extending away from the shooting point and with a gradually decreasing width, and the vertex position of the mask area of ​​the road portion is determined (i.e., the left and right endpoints of the trapezoidal structure close to the shooting point); then, the left vertex of the image is used as the scanning starting point, and the pixels are scanned row by row from left to right in sequence, and the pixel values ​​of the scanned pixel point and the corresponding two adjacent columns are detected to obtain the left boundary point; similarly, the right vertex of the image is used as the scanning starting point, and the pixels are scanned row by row from right to left in sequence, and the pixel values ​​of the scanned pixel point and the corresponding two adjacent columns are detected to obtain the right boundary point;

[0071] The specific method of detecting whether the values ​​of the two adjacent columns of pixels corresponding to the scanned pixel point have changed and obtaining the left and right boundary points is as follows:

[0072] For scanning pixels (x 0 ,y 0 ) , respectively detect the pixels in the adjacent columns on the left (x 0 ,y 0 -1) The pixel in the adjacent column to the right (x 0 ,y 0 +1) Is the pixel value the same as the scanned pixel point? If the pixel value is the same, the scanned pixel point is not a left boundary point or a right boundary point, and the scan continues;

[0073] If different, continue to judge, if When (x 0 ,y 0) is the left boundary point;

[0074] like When (x 0 ,y 0 ) is the right boundary point.

[0075] Then, the left and right edge points of the road are fitted to obtain the navigation line (such as Figure 4 e), specifically:

[0076] Step C1: First, according to the left boundary point set Fit left boundary line , where n represents the number of left boundary points; according to the right boundary point set Fit left boundary line ,in, m Indicates the number of right boundary points; a l 、a r are the slopes of the left and right boundary lines, respectively. b l 、b r are the intercepts of the left and right boundary lines respectively;

[0077] Step C2: Combine the linear equations of the left boundary line and the right boundary line to obtain the coordinates of their intersection. (x i , y i ) :

[0078] ;

[0079] Step C3: For each intersection coordinate (x i ,y i ) , the distance from the navigation line to the intersection d i for:

[0080] ;

[0081] Where: x d Indicates the horizontal coordinate of the point on the navigation line, a n 、b n They represent the slope and intercept of the navigation line respectively;

[0082] Get all guide lines to the intersection point (x i ,y i ) distance d i The sum of squares, i.e. the loss function L :

[0083] ;

[0084] Where: N represents the number of intersection coordinates;

[0085] The optimal solution is obtained by solving the partial derivative of the loss function to zero. a n and b n , thereby obtaining the navigation line;

[0086] Step C4, obtain the correlation coefficient R of the navigation line and determine the fitting accuracy of the navigation line:

[0087] ;

[0088] Where: (x di ,y di ) Indicates fitting any fitting point on the navigation line; , They represent the average values ​​of the abscissa of the fitting points and the average values ​​of the ordinate of the intersection points respectively;

[0089] R The closer it is to 1, the higher the accuracy of the navigation line.

[0090] Step D: Determine the accuracy of the navigation line: manually fit the center line of the orchard road, and compare the manually fitted center line of the orchard road with the navigation line obtained in step C to determine whether the navigation line is accurate, specifically:

[0091] First, the center points of each road were manually marked on the image of the orchard road using the annotation tool of the image processing software. (P 1 ,P 2 ,…,P n ) , thus forming an artificial fitting center line;

[0092] Then, extract the points on the navigation line fitted in step C ( Q 1 ,Q 2 ,…,Q m ) , and then calculate the direction vectors of the artificial fitting center line and the navigation line respectivelyV 1 and V 2:

[0093] ;

[0094] ;

[0095] After that, get the angle between the manually fitted center line and the navigation line θ :

[0096]

[0097] Where: , Represents the direction vector V 1. V 2 modulus;

[0098] Then, H pixels are selected from top to bottom in the segmented image of the artificial fitting center line at equal distances, and the horizontal coordinates of the navigation line and the artificial fitting center line in the same vertical coordinate direction are compared to obtain the navigation pixel error e x :

[0099] ;

[0100] Where: X DCi 、X HCi They represent the horizontal coordinates of the pixels on the navigation line and the horizontal coordinates of the pixels corresponding to the artificially fitted center line respectively;

[0101] Finally, set the angle threshold (In this embodiment, is 5°), if , then output this method for actual operation;

[0102] like , then the navigation pixel error e x Compensate the navigation line obtained by fitting in step C and obtain a new navigation line, that is:

[0103] ;

[0104] Where: represents an adjustment parameter used to control the magnitude of compensation (obtained based on a large amount of experimental data); Indicates the first i The horizontal pixel coordinates of the detection points; represents the first navigation line obtained by fitting in step C. i The horizontal pixel coordinates of the detection points;

[0105] Then compare the new navigation line with the manually fitted center line until .

[0106] Step E: Navigation recognition algorithm output: Use the precise navigation line fitting method output after judgment in step D to perform pitaya orchard road recognition and navigation.

Claims

1. A pitaya orchard road navigation and recognition method based on machine vision, characterized in that: The following steps are involved: Step A, data collection: collect a large number of pitaya orchard road images to form a training data set, a test data set, and a validation data set for the full convolutional network; Step B: Image segmentation: Designing an improved DeepLabv3+ Image semantic segmentation network, through improved DeepLabv3+ The image semantic segmentation network segments the orchard road image to obtain a road segmentation map; Step C, navigation line fitting: binarizing the mask area of ​​the road segmentation map obtained in step B to obtain a binary image; The left and right edge points of the road are obtained by binarizing the image. The specific method is as follows: determine the vertex position of the mask area of ​​the road, use the left and right vertices of the image as the scanning starting point, scan the pixels row by row, and detect whether the values ​​of the two adjacent columns of pixels corresponding to the scanned pixel point change, and obtain the left and right boundary points respectively. Specifically: For scanning pixels (x 0 ,y 0 ) , respectively detect the pixels in the adjacent columns on the left (x 0 ,y 0 -1) The pixel in the adjacent column to the right (x 0 , y 0 +1) Is the pixel value the same as the scanned pixel point? If the pixel value is the same, the scanned pixel point is not a left boundary point or a right boundary point, and the scan continues; If different, continue to judge, if When (x 0 ,y 0 ) is the left boundary point; like When (x 0 ,y 0 ) is the right boundary point; Fit the left and right edge points of the road to obtain the navigation line. The specific method is: Step C1: According to the left boundary point set Fit left boundary line , where n represents the number of left boundary points; according to the right boundary point set Fit right boundary line ,in, m Indicates the number of right boundary points; a l 、a r are the slopes of the left and right boundary lines, respectively. b l 、b r are the intercepts of the left and right boundary lines respectively; Step C2: Combine the linear equations of the left boundary line and the right boundary line to obtain the coordinates of their intersection. (x i ,y i ) : Step C3: For each intersection coordinate (x i ,y i ) , the distance from the navigation line to the intersection d i for: Where: x d Indicates the horizontal coordinate of the point on the navigation line, a n 、b n They represent the slope and intercept of the navigation line respectively; Get all guide lines to the intersection point (x i ,y i ) distance d i The sum of squares, i.e. the loss function L : Where: N represents the number of intersection coordinates; The optimal solution is obtained by solving the partial derivative of the loss function to zero. a n and b n , thereby obtaining the navigation line; Step C4, obtain the correlation coefficient R of the navigation line and determine the fitting accuracy of the navigation line: Where: (x di ,y di ) Indicates fitting any fitting point on the navigation line; , They represent the average values ​​of the abscissa of the fitting points and the average values ​​of the ordinate of the intersection points respectively; Step D: Determine the accuracy of the navigation line: manually fit the center line of the orchard road, and compare the manually fitted center line of the orchard road with the navigation line obtained in step C to determine whether the navigation line is accurate; specifically: Manually mark the center points of each road on the image of the orchard road using the annotation tool of the image processing software (P 1 ,P 2 ,…,P n ) , thus forming an artificial fitting center line; Extract the points on the navigation line fitted in step C ( Q 1 ,Q 2 ,…,Q m ) , and then calculate the direction vectors of the artificial fitting center line and the navigation line respectively V 1 and V 2: Get the angle between the manually fitted center line and the navigation line θ : Where: , Represents the direction vector V 1. V 2 modulus; Select H pixels from top to bottom at equal distances in the segmented image of the artificial fitting center line, compare the horizontal coordinates of the navigation line and the artificial fitting center line in the same vertical coordinate direction, and obtain the navigation pixel error. e x : Where: X DCi 、X HCi They represent the horizontal coordinates of the pixels on the navigation line and the horizontal coordinates of the pixels corresponding to the artificially fitted center line respectively; Set angle threshold ,like , then output this method for actual operation; like , then the navigation pixel error e x Compensate the navigation line obtained by fitting in step C, obtain a new navigation line, and then compare the new navigation line with the manually fitted center line until ; Step E: Navigation recognition algorithm output: Use the precise navigation line fitting method output after judgment in step D to perform pitaya orchard road recognition and navigation.

2. A method for navigation and identification of a pitaya orchard road based on machine vision according to claim 1, characterized in that: In step A, a REALSENSE D455 depth camera is used to collect images of the pitaya orchard roads. During the data collection process, the REALSENSE D455 depth camera is tilted downward at an angle of 10 to 15° and at a height of 0.5 to 0.8 m, and the shooting direction is directly in front of the road.

3. A method for navigation and identification of a pitaya orchard road based on machine vision according to claim 1 or 2, characterized in that: The pitaya orchard road images collected in the step A include images under three environments: sunny day with direct light, sunny day with backlight, and cloudy day; after collecting the images, Labelme annotation software is used to annotate the pitaya orchard road images, annotate the orchard roads, and divide the annotated pitaya orchard road images into a training data set, a test data set, and a validation data set according to the ratio of 8:1:

1.

4. A method for navigation and identification of a pitaya orchard road based on machine vision according to claim 3, characterized in that: The improved step B DeepLabv3+ The image semantic segmentation network is specifically: using a lightweight MobileNetv2 As DeepLabv3+ The feature extraction backbone network is MobileNetv2 Insert into the shallow features ECA-Net module; at the same time, in the model ASPP The pyramid split self-attention module is introduced after the module.

Citation Information

Patent Citations

  • Orchard visual navigation path extraction method and system based on neural network

    CN113280820A

  • Orchard complex road segmentation method based on lightweight semantic segmentation algorithm

    CN114463542A

  • Unmanned street view image segmentation implementation method based on DeeplabV3 +

    CN116884002A

  • Orchard environment scene detection method and terminal based on multi-task deep learning

    CN116935296A

  • Orchard vehicle automatic navigation system based on laser radar and machine vision fusion

    CN117774708A