Ridge line navigation method and system based on visual segmentation

Abstract

The invention relates to a ridge line navigation method and system based on visual segmentation. The ridge line navigation method and system based on visual segmentation comprises the following steps: acquiring an original image, preprocessing the original image, and importing a YOLOv8 model for training to obtain a ridge surface segmentation mask; edge detection is carried out on the ridge surface segmentation mask, pixel point information of ridge surface contour coordinates is determined, and central point pixel coordinates are obtained; fitting navigation lines for all the center points by adopting a B spline interpolation method, and drawing reference lines; and calculating the average absolute percentage error of each pixel point of the reference line and the navigation line, confirming a deviation signal, and judging the vehicle body adjustment direction through the deviation signal. According to the scheme provided by the invention, the model can be trained through the complex environment image, the ridge surface segmentation mask can be obtained through the YOLOv8 model, the field ridge contour boundary can be extracted, the navigation line can be fitted by adopting the B-spline interpolation method, the real-time reference line can be set, the average absolute error of the navigation line and the reference line can be calculated, the deflection signal can be given, and more accurate deflection information can be provided.

Description

Technical Field

[0001] The present application relates to the field of visual navigation technology, and in particular to a ridge navigation method and system based on visual segmentation. Background Art

[0002] With the development of agricultural modernization, the intelligence and automation of agricultural machinery have become an important means to improve agricultural production efficiency. The needs of modern agriculture are not limited to traditional mechanized operations, but are developing in a more efficient, more precise and more intelligent direction. Autonomous navigation of agricultural machinery in the field is an important part of realizing agricultural automation, and extracting navigation lines is the key technology to achieve autonomous navigation.

[0003] Compared with existing navigation technologies, GPS satellite positioning is easily affected by clouds and weather, and LiDAR is expensive and not suitable for small agricultural machinery. Vision can provide high-dimensional data at a relatively low cost. In recent years, the development of deep learning technology has further improved the accuracy and efficiency of visual segmentation and fitting. Traditional methods for extracting ridges or navigation lines mostly use simple image processing techniques, such as dilation and corrosion followed by Hough transform for edge detection, or U-Net model segmentation followed by edge detection, and finally the least squares method is used to fit the linear navigation line. The accuracy is low and it is not suitable for complex terrain. Therefore, a method with strong generalization ability, good robustness, and the ability to accurately provide navigation information is needed. Summary of the invention

[0004] In order to solve or partially solve the problems existing in the related art, the present application provides a ridge navigation method and system based on visual segmentation, aiming to solve the problem that traditional navigation methods are not suitable for small agricultural machinery and have low certainty.

[0005] The present application provides a ridge line navigation method based on visual segmentation, comprising:

[0006] Collect ridge video data to extract original images, pre-process the original images, import them into the YOLOv8 model for training, and obtain ridge surface segmentation masks;

[0007] Perform edge detection on the ridge segmentation mask to obtain accurate ridge contour coordinate information, determine the pixel point information of the ridge contour coordinates, average the coordinates of each pixel point, and obtain the pixel coordinates of the center point;

[0008] The B-spline interpolation method is used to fit the navigation line for all center points, and a straight line perpendicular to the bottom interface of the image is drawn at the pixel height where the navigation line is located as a reference line for the vehicle body's forward direction;

[0009] Calculate the average absolute percentage error of each pixel point between the reference line and the navigation line, confirm the deviation signal, and determine the adjustment direction of the vehicle body through the deviation signal.

[0010] Optionally, in some implementations, edge detection is performed on the ridge surface segmentation mask to obtain accurate ridge surface contour coordinate information, including:

[0011] Traverse all the pixels of the segmentation mask and define the foreground pixel value as 1 and the background pixel value as 0;

[0012] Gaussian blur the segmentation mask to reduce noise and details;

[0013] Set two thresholds for Canny edge detection to detect edges;

[0014] Find all connected regions on the edge image, calculate the area of ​​each connected region, and filter out connected regions whose area is smaller than a preset threshold;

[0015] Obtain the edge points of the connected area after filtering, generate a new connected area mask, and obtain the ridge surface contour boundary;

[0016] The ridge surface contour boundary is divided into a left contour and a right contour, and the specific pixel point coordinate information is stored in the corresponding array.

[0017] Optionally, in some implementations, averaging the coordinates of each pixel point to obtain the pixel coordinates of the center point includes:

[0018] Traverse the pixels on the left and calculate the average value at the same pixel height;

[0019] Traverse the pixels on the right and calculate the average value at the same pixel height;

[0020] Take the average value of the left and right pixels of the same pixel height, and use the average value as the pixel coordinate of the center point.

[0021] Optionally, in some implementations, the navigation line is fitted to all center points using a B-spline interpolation method, including:

[0022] Separate the X and Y coordinates of the center point pixel coordinates;

[0023] Use the splrep function to generate the node parameters, coefficient parameters and order required for B-spline interpolation;

[0024] Use the splev function to generate smooth navigation line coordinates;

[0025] Generate a sequence of Y coordinates from 0 to the image height, calculate the X coordinate corresponding to each Y coordinate, and form a smooth navigation line.

[0026] Optionally, in some implementations, drawing a straight line perpendicular to the bottom interface of the image at the pixel height where the navigation line is located includes:

[0027] Draw a straight line perpendicular to the bottom interface of the camera image at the pixel point where the navigation line is located at 0.6 times the pixel height as a reference line for the vehicle's forward direction.

[0028] Optionally, in some implementations, calculating the mean absolute percentage error of each pixel point of the reference line and the navigation line includes:

[0029] Filter the coordinate points and find the coordinate points whose pixel height is 0.4-0.6 times of the navigation line and reference line;

[0030] Calculate the error, calculate the percentage error between the current navigation line X coordinate and the reference line X coordinate, and add it to the total error;

[0031] Update the counters to update the left and right turn counts based on the position of the navigation line's X coordinate relative to the reference line's X coordinate;

[0032] Calculate the mean absolute percentage error.

[0033] Optionally, in some implementations, determining the vehicle body adjustment direction by using the deviation signal includes:

[0034] Determine the deflection signal, and when the average absolute error is less than 1%, it is determined that the vehicle body does not need to adjust the direction;

[0035] When the navigation line is on the left side of the reference line and the mean absolute error is greater than 1%, it is determined that the vehicle body needs to be adjusted to the left;

[0036] When the navigation line is on the right side of the reference line and the mean absolute error is greater than 1%, it is determined that the vehicle body needs to be adjusted to the right;

[0037] The deflection signal is sent to the lower computer through serial communication for direction control.

[0038] Optionally, in some implementations, YOLOv8 model training is imported to obtain a ridge surface segmentation mask, including:

[0039] Configure the deep learning virtual environment and download the YOLOv8 model source code;

[0040] Set training parameters, including epochs, patience, batch, imgsz, and lrf.

[0041] Optionally, in some implementations, the original image is preprocessed, including:

[0042] Shoot multiple sets of videos of field ridge information under different weather conditions, extract frames from the videos, and obtain field ridge images;

[0043] The original field ridge images were adjusted for brightness, added salt and pepper noise, and motion blur to simulate different weather light intensities and camera shake, thus doubling the dataset.

[0044] Labelme was used to annotate the dataset at the pixel level, and the ridge surface was wrapped with polylines to fit the ridge surface tightly, and a dataset of json file type was obtained;

[0045] Convert the json file into a txt file type and divide it into training set: validation set: test set = 8:1:1 ratio.

[0046] In addition, to achieve the above purpose, the present application also provides a ridge navigation system based on visual segmentation, including:

[0047] A memory, a processor, and a ridge line navigation program based on visual segmentation stored in the memory and executable on the processor. The ridge line navigation program based on visual segmentation is executed by the processor to implement a ridge line navigation method based on visual segmentation.

[0048] The technical solution provided by this application may have the following beneficial effects:

[0049] By training the model with complex environment images, the diversity of training data is improved, thereby improving the generalization ability of the model; the ridge surface segmentation mask is obtained through the YOLOv8 model, the ridge contour boundary is extracted, and the accurate ridge contour coordinate information is obtained to ensure the accuracy of the center point fitting; the B-spline interpolation method is used to fit the navigation line, which is suitable for paths of arbitrary complex shapes and has local control characteristics and good data stability; the real-time reference line is set, the average absolute error between the navigation line and the reference line is calculated, and the deflection signal is given, which provides more accurate deflection information and avoids errors caused by inaccurate navigation line fitting due to the field of view being too close or too far.

[0050] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The above and other objects, features and advantages of the present application will become more apparent through a more detailed description of exemplary embodiments of the present application in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments of the present application.

[0052] Figure 1 It is a flowchart of a ridge navigation method based on visual segmentation shown in an embodiment of the present application;

[0053] Figure 2 It is another implementation flow diagram of the ridge line navigation method based on visual segmentation shown in an embodiment of the present application;

[0054] Figure 3 is a field ridge mask map of the ridge line navigation method based on visual segmentation shown in an embodiment of the present application;

[0055] Figure 4 It is a schematic diagram of the architecture of the hardware operating environment of the ridge navigation system based on visual segmentation shown in an embodiment of the present application. DETAILED DESCRIPTION

[0056] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.

[0057] It should be understood that although the terms "first", "second", "third", etc. may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0058] In the description of the present application, it should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0059] Unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0060] Compared with existing navigation technologies, GPS satellite positioning is easily affected by clouds and weather, and LiDAR is expensive and not suitable for small agricultural machinery. Vision can provide high-dimensional data at a relatively low cost. In recent years, the development of deep learning technology has further improved the accuracy and efficiency of visual segmentation and fitting. Traditional methods for extracting ridges or navigation lines mostly use simple image processing techniques, such as dilation and corrosion followed by Hough transform for edge detection, or U-Net model segmentation followed by edge detection, and finally the least squares method is used to fit a linear navigation line. This method has low accuracy and is not suitable for complex terrain.

[0061] In response to the above problems, an embodiment of the present application provides a ridge navigation method based on visual segmentation, which can train the model through complex environment images, improve the diversity of training data, and thus improve the generalization ability of the model; obtain the ridge surface segmentation mask through the YOLOv8 model, extract the ridge contour boundary, obtain accurate ridge contour coordinate information, and ensure the accuracy of center point fitting; use B-spline interpolation method to fit the navigation line, adapt to paths of any complex shape, and have local control characteristics and good data stability; set a real-time reference line, calculate the average absolute error between the navigation line and the reference line, give a deflection signal, and provide more accurate deflection information, avoiding errors caused by inaccurate navigation line fitting when the field of view is too close or too far.

[0062] The technical solution of the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0063] Figure 1 It is a flow chart of the ridge navigation method based on visual segmentation shown in an embodiment of the present application.

[0064] See also Figure 1 , a ridge navigation method based on visual segmentation, comprising:

[0065] S101, collecting ridge video data to extract original images, preprocessing the original images, importing them into YOLOv8 model training, and obtaining ridge surface segmentation masks;

[0066] Specifically, the original image is preprocessed, including:

[0067] Shoot multiple sets of videos of field ridge information under different weather conditions, extract frames from the videos, and obtain field ridge images;

[0068] The original field ridge images were adjusted for brightness, added salt and pepper noise, and motion blur to simulate different weather light intensities and camera shake, thus doubling the dataset.

[0069] Labelme was used to annotate the dataset at the pixel level, and the ridge surface was wrapped with polylines to fit the ridge surface tightly, and a dataset of json file type was obtained;

[0070] Convert the json file into a txt file type and divide it into training set: validation set: test set = 8:1:1 ratio.

[0071] Specifically, import the YOLOv8 model training to obtain the ridge surface segmentation mask, including:

[0072] Configure the deep learning virtual environment and download the YOLOv8 model source code;

[0073] Set training parameters, including epochs, patience, batch, imgsz, and lrf.

[0074] S102, performing edge detection on the ridge surface segmentation mask to obtain accurate ridge surface contour coordinate information, determining pixel point information of the ridge surface contour coordinates, averaging the coordinates of each pixel point, and obtaining the pixel coordinates of the center point;

[0075] Specifically, edge detection is performed on the ridge surface segmentation mask to obtain accurate ridge surface contour coordinate information, including:

[0076] Traverse all the pixels of the segmentation mask and define the foreground pixel value as 1 and the background pixel value as 0;

[0077] Gaussian blur the segmentation mask to reduce noise and details;

[0078] Set two thresholds for Canny edge detection to detect edges;

[0079] Find all connected regions on the edge image, calculate the area of ​​each connected region, and filter out connected regions whose area is smaller than a preset threshold;

[0080] Obtain the edge points of the connected area after filtering, generate a new connected area mask, and obtain the ridge surface contour boundary;

[0081] The ridge surface contour boundary is divided into a left contour and a right contour, and the specific pixel point coordinate information is stored in the corresponding array.

[0082] Specifically, the coordinates of each pixel point are averaged to obtain the pixel coordinates of the center point, including:

[0083] Traverse the pixels on the left and calculate the average value at the same pixel height;

[0084] Traverse the pixels on the right and calculate the average value at the same pixel height;

[0085] Take the average value of the left and right pixels of the same pixel height, and use the average value as the pixel coordinate of the center point.

[0086] S103, fitting a navigation line for all center points using a B-spline interpolation method, and drawing a straight line perpendicular to the bottom interface of the image at the pixel height where the navigation line is located as a reference line for the forward direction of the vehicle body;

[0087] Specifically, the navigation line is fitted using the B-spline interpolation method for all center points, including:

[0088] Separate the X and Y coordinates of the center point pixel coordinates;

[0089] Use the splrep function to generate the node parameters, coefficient parameters and order required for B-spline interpolation;

[0090] Use the splev function to generate smooth navigation line coordinates;

[0091] Generate a sequence of Y coordinates from 0 to the image height, calculate the X coordinate corresponding to each Y coordinate, and form a smooth navigation line.

[0092] Specifically, a straight line perpendicular to the bottom interface of the image is drawn at the pixel height where the navigation line is located, including:

[0093] Draw a straight line perpendicular to the bottom interface of the camera image at the pixel point where the navigation line is located at 0.6 times the pixel height as a reference line for the vehicle's forward direction.

[0094] S104, calculating the average absolute percentage error of each pixel point of the reference line and the navigation line, confirming the deviation signal, and determining the vehicle body adjustment direction through the deviation signal.

[0095] Specifically, the mean absolute percentage error of each pixel point of the reference line and the navigation line is calculated, including:

[0096] Filter the coordinate points and find the coordinate points whose pixel height is 0.4-0.6 times of the navigation line and reference line;

[0097] Calculate the error, calculate the percentage error between the current navigation line X coordinate and the reference line X coordinate, and add it to the total error;

[0098] Update the counters to update the left and right turn counts based on the position of the navigation line's X coordinate relative to the reference line's X coordinate;

[0099] Calculate the mean absolute percentage error.

[0100] Specifically, the vehicle body adjustment direction is determined by the deviation signal, including:

[0101] Determine the deflection signal, and when the average absolute error is less than 1%, it is determined that the vehicle body does not need to adjust the direction;

[0102] When the navigation line is on the left side of the reference line and the mean absolute error is greater than 1%, it is determined that the vehicle body needs to be adjusted to the left;

[0103] When the navigation line is on the right side of the reference line and the mean absolute error is greater than 1%, it is determined that the vehicle body needs to be adjusted to the right;

[0104] The deflection signal is sent to the lower computer through serial communication for direction control.

[0105] Figure 2 It is another implementation flow diagram of the ridge navigation method based on visual segmentation shown in an embodiment of the present application.

[0106] The present application also provides another implementation of a ridge navigation method based on visual segmentation, which specifically includes:

[0107] S201, installing an industrial camera to ensure that the direction of the lens coincides with the central axis of a mobile device that implements a ridge line navigation method based on visual segmentation;

[0108] S202, controlling the mobile device to shoot multiple groups of videos of field ridge information under different weather conditions, extracting frames from the videos, and obtaining 1200 field ridge images;

[0109] S203, data enhancement, improves the generalization and robustness of the model. The original 1200 ridge images are adjusted in brightness, added with salt and pepper noise, and motion blur to simulate different weather light intensities, camera shake, etc., and finally the data set is expanded to 2400 images;

[0110] S204, manually annotating the dataset at the pixel level using Labelme, wrapping the ridge surface with a polyline that fits the ridge surface tightly, and obtaining a dataset in a json file type;

[0111] S205, convert the json file into a txt file type, and divide it into training set: validation set: test set = 8:1:1 ratio;

[0112] S206, import the txt file data set into the YOLOv8 model training, set the training parameters, and select the optimal training weight;

[0113] Specifically, import the txt file data set into the YOLOv8 model training and set the training parameters, including:

[0114] Configure the deep learning virtual environment and download the YOLOv8 model source code;

[0115] Set the training parameters, including epochs=300, patience=50, batch=16, imgsz=640, lrf=0.01.

[0116] S207, calling the optimal training weight to predict the field ridge image, and obtaining the ridge surface segmentation mask, such as Figure 3 As shown;

[0117] S208, using the Canny operator to perform edge detection on the ridge surface segmentation mask, and filtering the connected areas with smaller pixel areas to obtain accurate ridge surface contour coordinate information;

[0118] Specifically, the Canny operator is used to perform edge detection on the ridge segmentation mask, including:

[0119] Traverse all pixels of the segmentation mask, the foreground pixel value is 1, and the background pixel value is 0;

[0120] Gaussian blur is applied to the segmentation mask. Gaussian blur refers to the weighted average of the pixel values ​​in the image and the pixels in the domain through convolution operation to achieve the purpose of smoothing the image and reducing noise. Its weight is determined by a Gaussian function. The center position of the Gaussian function corresponds to the current pixel of the image. The farther away from the center, the smaller the weight of the pixel.

[0121] The standard form of the Gaussian function is:

[0122]

[0123] Here, σ is the standard deviation of the Gaussian function, which determines the degree of blur.

[0124] This application implements Gaussian blur by calling the "cv2.GaussianBlur()" function in OpenCV, specifically blurred_mask = cv2.GaussianBlur(mask, (5,5), 0), where mask represents the input segmentation strict mask, (5,5) represents the use of a 5×5 convolution kernel, and 0 represents the automatic calculation of the standard deviation of the Gaussian function;

[0125] Set two thresholds for Canny edge detection to detect edges;

[0126] Call the "cv2.connectedComponentsWithStats" function in OpenCV to find all connected regions on the edge image, return the number of pixels in each connected region as the area of ​​the connected region, and filter out connected regions whose area is smaller than the preset threshold;

[0127] Get the edge points of the connected region after filtering and generate a new connected region mask.

[0128] S209, dividing the ridge surface contour boundary into a left contour and a right contour, and creating a dictionary to store the x coordinate point on each y coordinate, as follows:

[0129] Let the contour point set be C, where each point p∈(x p ,y p ), where x pand p are the horizontal and vertical coordinates of point p respectively.

[0130] Let the width of the connected region mask be W, and the horizontal coordinate of the mask center be:

[0131]

[0132] The left and right side contour point sets are defined as follows:

[0133] C left ={p=(x p ,y p )∈C|x p <x center} (3)

[0134] C right ={p=(x p ,y p )∈C|x p ≥x center} (4)

[0135] Sort and store by y coordinate:

[0136] Assume D left and D right Respectively represent dictionaries storing the left and right contour points. The key of the dictionary is the y coordinate, and the value is the set of all x coordinates under the y coordinate.

[0137] For the left contour point set C left :

[0138] D left (y) = {x p |(x p ,y)∈C left (y p )} (5)

[0139] For the right contour point set C left :

[0140] d right (y) = {x p |(x p ,y)∈C right (y p )} (6)

[0141] S2010, for pixel points at the same pixel height coordinate, first average the coordinates of the points on each side, and then average to obtain the pixel coordinates of the center point;

[0142] Specifically, they include:

[0143] Traverse the pixels on the left and calculate the average value at the same pixel height;

[0144] Traverse the pixels on the right and calculate the average value at the same pixel height;

[0145] Take the average value of the left and right pixels of the same pixel height, and use the average value as the pixel coordinate of the center point.

[0146] S2011, fitting navigation lines for all center points using B-spline interpolation method;

[0147] Specifically, the navigation line is fitted using the B-spline interpolation method for all center points, including:

[0148] Separate the X and Y coordinates of the center point and set them as (x i ,y i );

[0149] Use the splrep function to generate the parameters (nodes, coefficients, and orders) required for B-spline interpolation. The splrep function uses the cubic B-spline interpolation basis function by default. The cubic B-spline basis function is as follows:

[0150]

[0151] Among them, t i is the node in the knot vector, k=3, indicating the order of the spline.

[0152] The spline interpolation function S(x) is represented by a linear combination of B-spline basis functions, specifically:

[0153]

[0154] Among them, c i are the coefficients of the B-spline basis function, B i,k (x) is the ith B-spline basis function, t={t0,t1,…t n+k} is the node vector, which defines the segment position of the spline curve. The splrep function can automatically select the appropriate node vector.

[0155] In order to determine the spline interpolation function S(x), it is necessary to minimize the objective function to determine the coefficient c of the optimal B-spline basis function i , as follows:

[0156]

[0157] Among them, (x i ,y i ) is the given center point coordinate, S(x) is the spline interpolation function, S″(x) is the second derivative of the spline function, s is the smoothing factor, and the default value of the splrep function is s=3;

[0158] Generate a Y coordinate sequence with a step size of 1, from 0 to the image height, set to y j ,Use the splev function to calculate the X coordinate corresponding to each Y coordinate to form a smooth navigation line, specifically:

[0159] The parameters (knot vector, coefficients and order) obtained from the previous splrep function are calculated as follows: j :

[0160]

[0161] Among them, x j is the interpolation function S(y j ) in y j The value at c i are the B-spline coefficients generated by the splrep function, B i,k (y j ) is the B-spline basis function in y j The value at .

[0162] S2012, draw a straight line perpendicular to the bottom interface of the camera image through the pixel point where the navigation line is located with a height of 0.6 times the pixel as a reference line for the forward direction of the vehicle body;

[0163] S2013, draw two horizontal lines through the points where the navigation line is located at 0.4 times and 0.6 times the pixel height, and calculate the average absolute percentage error of each pixel point between the reference line and the navigation line within this height range;

[0164] Specifically, they include:

[0165] Filter the coordinate points and find the coordinate points whose pixel height is 0.4-0.6 times of the navigation line and reference line;

[0166] Calculate the error, calculate the percentage error between the current navigation line X coordinate and the reference line X coordinate, set to E j , and add it to the total error, set as E total , as follows:

[0167]

[0168]

[0169] Among them, x j_ref is the horizontal coordinate of the reference point at the same pixel height, y 0.4 ≤y j ≤y 0.6 Indicates the vertical coordinate at 0.4-0.6 times the pixel height.

[0170] Update the counters to update the left and right turn counts based on the position of the navigation line's X coordinate relative to the reference line's X coordinate;

[0171] Calculate the mean absolute percentage error and set it to The details are as follows:

[0172]

[0173] Where N is the value that satisfies y 0.4 ≤y j ≤y 0.6 The number of points.

[0174] S2014. Determine the deflection signal. When the average absolute error is less than 1%, determine that the vehicle body does not need to adjust the direction. When the navigation line is on the left side of the reference line and the average absolute error is greater than 1%, determine that the vehicle body needs to be adjusted to the left. When the navigation line is on the right side of the reference line and the average absolute error is greater than 1%, determine that the vehicle body needs to be adjusted to the right.

[0175] S2015. Visually display the ridge navigation results, send a deflection signal to the lower computer through serial port communication, and control the mobile device to adjust the direction.

[0176] As an implementation plan, Figure 4 This is a schematic diagram of the architecture of the hardware operating environment of the ridge navigation system based on visual segmentation involved in the embodiment of the present application.

[0177] like Figure 4 As shown, the ridge navigation system based on visual segmentation may include: a processor 1001, such as a CPU, a memory 1005, a user interface 1003, a network interface 1004, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the user interface 1003 may optionally include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may optionally be a storage device independent of the aforementioned processor 1001.

[0178] Those skilled in the art will understand that Figure 4 The architecture of the ridge line navigation system based on visual segmentation shown in the figure does not constitute a limitation of the ridge line navigation system based on visual segmentation, and may include more or less components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0179] like Figure 4As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and a ridge navigation program based on visual segmentation. The operating system is a program that manages and controls the hardware and software resources of the ridge navigation system based on visual segmentation, and the operation of the ridge navigation program based on visual segmentation and other software or programs.

[0180] exist Figure 4 In the ridge navigation system based on visual segmentation shown, the user interface 1003 is mainly used to connect to the terminal and communicate data with the terminal; the network interface 1004 is mainly used for the background server and communicates data with the background server; the processor 1001 can be used to call the ridge navigation program based on visual segmentation stored in the memory 1005.

[0181] The embodiments of the present application have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A ridge navigation method based on visual segmentation, characterized in that: include: Collecting field ridge video data to extract original images, preprocessing the original images, importing them into YOLOv8 model training, and obtaining ridge surface segmentation masks; Perform edge detection on the ridge segmentation mask to obtain accurate ridge contour coordinate information, determine the pixel point information of the ridge contour coordinates, average the coordinates of each pixel point, and obtain the pixel coordinates of the center point; The B-spline interpolation method is used to fit the navigation line for all center points, and a straight line perpendicular to the bottom interface of the image is drawn at the pixel height where the navigation line is located as a reference line for the vehicle body's forward direction; The average absolute percentage error of each pixel point of the reference line and the navigation line is calculated, a deviation signal is confirmed, and the adjustment direction of the vehicle body is determined by the deviation signal.

2. The ridge navigation method based on visual segmentation according to claim 1, characterized in that: The edge detection of the ridge surface segmentation mask is performed to obtain accurate ridge surface contour coordinate information, including: Traverse all the pixels of the segmentation mask and define the foreground pixel value as 1 and the background pixel value as 0; Gaussian blur the segmentation mask to reduce noise and details; Set two thresholds for Canny edge detection to detect edges; Find all connected regions on the edge image, calculate the area of ​​each connected region, and filter out connected regions whose area is smaller than a preset threshold; Obtain the edge points of the connected area after filtering, generate a new connected area mask, and obtain the ridge surface contour boundary; The ridge surface contour boundary is divided into a left contour and a right contour, and the specific pixel point coordinate information is stored in the corresponding array.

3. The ridge navigation method based on visual segmentation according to claim 1, characterized in that: The averaging of the pixel coordinates to obtain the center pixel coordinates includes: Traverse the pixels on the left and calculate the average value at the same pixel height; Traverse the pixels on the right and calculate the average value at the same pixel height; Take the average value of the left and right pixels of the same pixel height, and use the average value as the pixel coordinate of the center point.

4. The ridge navigation method based on visual segmentation according to claim 1, characterized in that: The method of fitting the navigation line to all the center points using the B-spline interpolation method includes: Separate the X and Y coordinates of the center point pixel coordinates; Use the splrep function to generate the node parameters, coefficient parameters and order required for B-spline interpolation; Use the splev function to generate smooth navigation line coordinates; Generate a sequence of Y coordinates from 0 to the image height, calculate the X coordinate corresponding to each Y coordinate, and form a smooth navigation line.

5. The ridge navigation method based on visual segmentation according to claim 1, characterized in that: Drawing a straight line perpendicular to the bottom interface of the image at the pixel height where the navigation line is located includes: Draw a straight line perpendicular to the bottom interface of the camera image at the pixel point where the navigation line is located at 0.6 times the pixel height as a reference line for the vehicle's forward direction.

6. The ridge navigation method based on visual segmentation according to claim 1, characterized in that: The calculating the mean absolute percentage error of each pixel point of the reference line and the navigation line includes: Filter the coordinate points and find the coordinate points whose pixel height is 0.4-0.6 times of the navigation line and reference line; Calculate the error, calculate the percentage error between the current navigation line X coordinate and the reference line X coordinate, and add it to the total error; Update the counters to update the left and right turn counts based on the position of the navigation line's X coordinate relative to the reference line's X coordinate; Calculate the mean absolute percentage error.

7. The ridge navigation method based on visual segmentation according to claim 1, characterized in that: The determining the vehicle body adjustment direction by using the deviation signal comprises: Determine the deflection signal, and when the average absolute error is less than 1%, it is determined that the vehicle body does not need to adjust the direction; When the navigation line is on the left side of the reference line and the mean absolute error is greater than 1%, it is determined that the vehicle body needs to be adjusted to the left; When the navigation line is on the right side of the reference line and the mean absolute error is greater than 1%, it is determined that the vehicle body needs to be adjusted to the right; The deflection signal is sent to the lower computer through serial communication for direction control.

8. The ridge navigation method based on visual segmentation according to claim 1, characterized in that: The YOLOv8 model training is imported to obtain the ridge surface segmentation mask, including: Configure the deep learning virtual environment and download the YOLOv8 model source code; Set training parameters, including epochs, patience, batch, imgsz, and lrf.

9. The ridge navigation method based on visual segmentation according to claim 1, characterized in that: The preprocessing of the original image comprises: Shoot multiple sets of videos of field ridge information under different weather conditions, extract frames from the videos, and obtain field ridge images; The original field ridge images were adjusted for brightness, added salt and pepper noise, and motion blur to simulate different weather light intensities and camera shake, thus doubling the dataset. Labelme was used to annotate the dataset at the pixel level, and the ridge surface was wrapped with polylines to fit the ridge surface tightly, and a dataset of json file type was obtained; Convert the json file into a txt file type and divide it into training set: validation set: test set = 8:1:1 ratio.

10. A ridgeline navigation system based on visual segmentation, characterized in that: The ridge line navigation system based on visual segmentation includes: a memory, a processor, and a ridge line navigation program based on visual segmentation stored in the memory and executable on the processor. When the ridge line navigation program based on visual segmentation is executed by the processor, the steps of the ridge line navigation method based on visual segmentation are implemented as described in any one of claims 1 to 9.

Images