Shrub stumping and sawing method based on machine vision and image segmentation technology

Abstract

The invention relates to an intelligent sawing system, in particular to a shrub stumping sawing method based on a machine vision and image segmentation technology, which comprises the following steps: (S1) collecting a salix psammophila image; (S2) preprocessing the collected salix psammophila image; (S3) extracting salix psammophila root features from the preprocessed image; (S4) segmenting the extracted feature image through an image segmentation module, and separating the soil hill contour of the root of the salix psammophila from the background; and (S5) calculating the length, width, contour radius and circle center point coordinates of the contour, generating a control signal, and driving a saw cutting device of the salix psammophila stumping machine to carry out precise profiling cutting. According to the shrub stumping saw cutting method based on the machine vision and the image segmentation technology, the saw cutting efficiency and the saw cutting effect can be improved.

Description

Technical Field

[0001] The invention relates to an intelligent sawing system, and in particular to a shrub stubble sawing method based on machine vision and image segmentation technology. Background Art

[0002] Desert shrubs are one of the representative tree species widely planted for ecological protection in arid areas. Some species of desert shrubs (Salix psammophila) have the growth characteristics of pruning and rejuvenation. Pruning and rejuvenation means that after 2 to 3 years of transplantation, all branches above the root neck of the shrub need to be cut off so that it can grow straight and strong trunks again. Therefore, Salix psammophila needs to be pruned periodically, and the commonly used equipment for pruning Salix psammophila is a pruning machine.

[0003] At present, the research on stubble leveling machines in China is mostly focused on the improvement of hardware, and the stubble leveling effect of the machine is improved by improving the structure. The utility model patent with the publication number CN220422540U discloses a stubble leveling mechanism of a Eucommia ulmoides twig stubble leveling machine, which realizes the convenient adjustment of the cutting angle by controlling the rotation speed of the circular saw and the feed roller separately. The utility model patent with the publication number CN217850301U discloses a shrub stubble leveling machine with a hydraulic lifting function, which transmits power to the main transmission box through the transmission shaft, connects the right-angle transmission boxes on both sides through the coupling, realizes the rotation of the right and left feeders, so as to achieve the function of cutting materials, and the rear end realizes the lifting and lowering of the feeder through a pair of hydraulic cylinders, and the two ground wheels can be lifted and lowered at the same time by the hydraulic cylinder, which overcomes the problem that the material cannot be harvested due to the uneven terrain, is safe to operate, and solves the problem of difficult manual harvesting.

[0004] It can be seen that although the current cutting machines have simple profiling functions, this profiling function is mostly concentrated in the actuator, and the low degree of automation of the stubble equipment makes it difficult to meet the requirements of modern agriculture for high efficiency and high precision. Summary of the invention

[0005] In order to overcome the deficiencies in the prior art, the present invention provides a shrub stubble sawing method based on machine vision and image segmentation technology, which is used to improve the intelligence of a Salix psammophila stubble sawing machine.

[0006] To achieve the above purpose, the present invention discloses a shrub stubble sawing method based on machine vision and image segmentation, comprising the following steps:

[0007] (S1) acquiring an image of the Salix psammophila in real time through an image acquisition module, and transmitting the acquired image of the Salix psammophila to an image processing module;

[0008] (S2) applying preprocessing technology to the original image collected by the image processing module to denoise, enhance and normalize the image;

[0009] (S3) analyzing the processed image through a feature extraction module, extracting a feature image of the root of Salix psammophila, and saving the root image to a designated folder to form a new data set, and inputting it into an image segmentation module;

[0010] (S4) segmenting the extracted feature image through an image segmentation module, separating the contour of the mound at the root of the Salix psammophila from the background, and generating a binary image;

[0011] (S5) The control execution module calculates the length, width, radius and center point coordinates of the contour according to the segmentation results, and generates a control signal to drive the sawing device of the Salix psammophila trimming machine to perform precise contour cutting.

[0012] Preferably, in step (S1), the collected Salix psammophila images are converted into a tensor form of (H, W, C), and the Salix psammophila images are divided into a training set and a validation set.

[0013] Preferably, the specific operation method of step (S4) is:

[0014] (A1) Generate target box through algorithm;

[0015] (A2) The input image is cropped based on the acquired target frame coordinates of the Salix psammophila root and the length and width of the target frame, the Salix psammophila root area is retained, and redundant non-background information is removed;

[0016] (A3) The cropped Salix psammophila root images are combined into a tensor of size (H, W, C), where H and W are the image height and width, and C is the number of image channels. The images are then divided into a training set and a validation set according to the ratio.

[0017] (A4) The soil mound outline at the root of Salix psammophila is separated from the background to generate a binary segmentation map.

[0018] Preferably, the specific operation method of step (S5) is:

[0019] (B1) Approximate the segmented mound contour as a circular arc with a chord, extract the coordinates of the start and end points of the contour, find all the boundary points of the contour, the minimum coordinate of the x pixel and the minimum coordinate of the y pixel of all the boundary points are the starting point coordinates, and the maximum coordinate of the x pixel and the maximum coordinate of the y pixel of all the boundary points are the end point coordinates;

[0020] (B2) Convert the coordinate system of each pixel point in the image to the world coordinate system. The difference between the minimum and maximum coordinates of the x pixel is the contour width L, and the difference between the maximum and minimum coordinates of the y pixel is the contour height H. Then the contour radius Radius is The center angle of the contour is θ. The coordinates of the center point can be calculated by the coordinates of the starting point, the end point and the radius. Create a new variable circle_x to store the x coordinate of the center point, and create a new variable circle_y to store the y coordinate of the center point. Then circle_x = (x_coords.min() + x_coords.max()) / 2, circle_x = y_coords.max() - Radius;

[0021] (B3) Through the camera calibration experiment, the camera's internal and external parameters are calculated; the pixel coordinates are converted into world coordinates through coordinate system conversion, and the calculated parameters are converted into control instructions to drive the trimming machine to perform contour sawing.

[0022] The present invention has the following technical effects:

[0023] The image positioning cutting technology is introduced into the sawing machine. The root contour of the cutting object is extracted and cut through the intelligent control system, and the edge is automatically cut, which greatly improves the cutting efficiency. This technology not only saves labor costs, but also reduces the error rate in the cutting process, thereby improving the overall sawing efficiency; image positioning cutting technology ensures the accuracy of cutting through high-precision image recognition and positioning, thereby improving the quality of sawing. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 This is the effect diagram of the identification and contour segmentation of Salix psammophila and its roots according to the present invention. DETAILED DESCRIPTION

[0025] The principles and features of the present invention are described below in conjunction with embodiments; the examples given are only used to explain the present invention; they are not used to limit the scope of the present invention.

[0026] A shrub stubble sawing method based on machine vision and image segmentation comprises the following steps:

[0027] (S1) acquiring an image of the Salix psammophila in real time through an image acquisition module, and transmitting the acquired image of the Salix psammophila to an image processing module;

[0028] (S2) applying preprocessing technology to the original image collected by the image processing module to denoise, enhance and normalize the image;

[0029] (S3) analyzing the processed image through a feature extraction module, extracting a feature image of the root of Salix psammophila, and saving the root image to a designated folder to form a new data set, and inputting it into an image segmentation module;

[0030] (S4) segmenting the extracted feature image through an image segmentation module, separating the contour of the mound at the root of the Salix psammophila from the background, and generating a binary image;

[0031] (S5) The control execution module calculates the length, width, radius and center point coordinates of the contour according to the segmentation results, and generates a control signal to drive the sawing device of the Salix psammophila trimming machine to perform precise contour cutting.

[0032] Specifically, steps (S1) to (S5) are explained in detail below.

[0033] In step (S1), the image acquisition module includes a high-definition camera, an image transmission interface and an image storage unit, which can record and store multiple images of the Salix psammophila growth environment in real time. After the image is acquired, the Salix psammophila image is converted into a tensor form of (H, W, C). At the same time, in order to meet the needs of algorithm learning, the acquired Salix psammophila images are divided into a training set and a validation set in a ratio of 9:1.

[0034] In step (S2), the image processing module may use a variety of image processing techniques, including but not limited to image filtering, histogram equalization, edge detection, etc., to ensure the clarity and analyzability of the image.

[0035] In step (S3), the feature image of the root of Salix psammophila is extracted through the YOLOv5 network structure. The YOLOv5 network structure includes a feature extraction backbone network (Backbone), a feature fusion network (Neck) and a detection head network (Head). The backbone network (Backbone) processes the input image through a series of convolutional layers and pooling layers, gradually reducing the size of the feature map while increasing the number of channels. The feature fusion network (Neck) further performs feature fusion and upsampling operations based on the features extracted by the backbone network to provide more advanced semantic information and the ability to adapt to images of different scales. The detection head network (Head) further processes the extracted features and generates the final output result.

[0036] In step (S4), the extracted feature image is segmented by the Unet network, and the outline of the mound at the root of the Salix psammophila is separated from the background. The Unet network mainly includes two parts: an encoder and a decoder, and the low-level features in the encoder and the high-level features in the decoder are fused through a skip connection. The encoder includes multiple program blocks, each of which usually includes a 3x3 convolution (using a Relu activation function) and a 2x2 pooling layer (downsampling) with a step size of 2. After being processed by each program block, the size of the feature map will gradually decrease. The decoder includes multiple program blocks, each of which increases the size of the feature map through a 2x2 upsampling operation with a step size of 2, and then concatenates the feature map with the feature map of the corresponding level of the encoder part. Finally, it is further processed by two 3x3 convolutions (Relu). The skip connection is located between the encoder and the decoder, and usually includes two 3x3 convolution layers to further extract and fuse features and provide more useful information for the decoder.

[0037] The original input image is cropped by the target frame position coordinates of the root of Salix psammophila obtained from the YOLOv5 network, the root area of ​​Salix psammophila is retained, and the redundant irrelevant background information is removed; the cropped Salix psammophila root image is combined into a tensor of size (H, W, C), where H and W are the image height and width, and C is the number of image channels, which is specifically converted into a tensor of size (527, 527, 3), and divided into a training set and a validation set according to a ratio of 9:1. The converted tensor is sent to the Unet network to segment the extracted feature image, separate the mound outline of the root of Salix psammophila from the background, and processed by OpenCV to generate a binary segmentation map. The OpenCV (open source computer vision library) is a cross-platform computer vision library released under the BSD license (open source). The binarization is the process of setting the grayscale value of the pixel on the image to 0 or 255, that is, the entire image presents an obvious black and white effect. The present invention sets the background in the image to 0 and the root area of ​​the psammophila tamarisk to 255 through the numpy.where(image>0,1,0).astype(numpy.uint8) function.

[0038] In step (S5), the segmented mound contour is approximated as a circular arc with a chord, the coordinates of the start and end points of the contour are extracted, and all boundary points of the contour are found. The minimum coordinate of the x pixel and the minimum coordinate of the y pixel of all boundary points are the starting coordinates, and the maximum coordinate of the x pixel and the maximum coordinate of the y pixel of all boundary points are the end coordinates. The coordinate system of each pixel point in the image is converted into the world coordinate system, and the difference between the minimum coordinate and the maximum coordinate of the x pixel is calculated as the contour width L, and the difference between the maximum coordinate and the minimum coordinate of the y pixel is calculated as the contour height H, then the contour radius Radius is The center angle of the contour is θ. The coordinates of the center point can be calculated from the coordinates of the starting point, the end point and the radius. Create a new variable circle_x to store the x coordinate of the center point, and create a new variable circle_y to store the y coordinate of the center point. Then circle_x(x_coords.min()+x_coords.max()) / 2, circle_x=y_coords.max()-Radius. Through the camera calibration experiment, calculate the intrinsic and extrinsic parameters of the camera; transform the pixel coordinates into world coordinates through coordinate system conversion, and transform the calculated parameters into control instructions to drive the trimming machine to perform contour sawing.

[0039] In order to verify that the present invention describes the detection effect of the current model detection by predicting the accuracy (Precision), the calculation method is specifically as follows:

[0040] Precision = (TP + TN) / (TP + TN + FP + FN), where TP means that the positive sample is predicted as a positive sample, TN means that the negative sample is predicted as a negative sample, FP means that the negative sample is predicted as a positive sample, and FN means that the positive sample is predicted as a negative sample. Salix Psammophila means that the current prediction target is Salix Psammophila, and its value is the prediction accuracy; Root means that when the prediction target is Salix Psammophila, the target area is further predicted as the root characteristics of Salix Psammophila, and its value is the prediction accuracy.

[0041] The specific indicators are shown in the following table, which are the relevant indicators for the training and testing of the model of the present invention for predicting the characteristics of Salix psammophila and its roots, including Salix psammophila and Root in the training stage and the testing stage. Figure 1 As shown (left), the left side is a diagram showing the effect of detecting a Salix psammophila image, where the letters in the upper left corner of the target box represent the corresponding category, and the numbers next to them represent the confidence of the corresponding category; Figure 1 Shown (right) is the image segmentation effect of Unet on the contour of the root of Salix psammophila. It can be seen that the present invention can accurately identify the Salix psammophila target in the image and can effectively extract the characteristic contour of the root of Salix psammophila.

[0042] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A shrub stubble sawing method based on machine vision and image segmentation, characterized in that: The following steps are involved: (S1) acquiring an image of the Salix psammophila in real time through an image acquisition module, and transmitting the acquired image of the Salix psammophila to an image processing module; (S2) applying preprocessing technology to the original image collected by the image processing module to denoise, enhance and normalize the image; (S3) analyzing the processed image through a feature extraction module, extracting a feature image of the root of Salix psammophila, and saving the root image to a designated folder to form a new data set, and inputting it into an image segmentation module; (S4) segmenting the extracted feature image through an image segmentation module, separating the contour of the mound at the root of the Salix psammophila from the background, and generating a binary image; (S5) The control execution module calculates the length, width, radius and center point coordinates of the contour according to the segmentation results, and generates a control signal to drive the sawing device of the Salix psammophila trimming machine to perform precise contour cutting.

2. The method for stubble sawing based on machine vision and image segmentation according to claim 1 is characterized in that: In step (S1), the collected Salix psammophila images are converted into a tensor form of (H, W, C), where H and W are the image height and width, and C is the number of image channels. The Salix psammophila images are divided into a training set and a validation set.

3. The method for stubble sawing based on machine vision and image segmentation according to claim 1 is characterized in that: The specific operation method of step (S4) is: (A1) Generate target box through algorithm; (A2) The input image is cropped based on the acquired target frame coordinates of the Salix psammophila root and the length and width of the target frame, the Salix psammophila root area is retained, and redundant non-background information is removed; (A3) The cropped Salix psammophila root images are combined into a tensor of size (H, W, C), where H and W are the image height and width, and C is the number of image channels. The images are then divided into a training set and a validation set according to the ratio. (A4) The soil mound outline at the root of Salix psammophila is separated from the background to generate a binary segmentation map.

4. The method for stubble sawing based on machine vision and image segmentation according to claim 1, characterized in that: The specific operation method of step (S5) is: (B1) Approximate the segmented mound contour as a circular arc with a chord, extract the coordinates of the start and end points of the contour, find all the boundary points of the contour, the minimum coordinate of the x pixel and the minimum coordinate of the y pixel of all the boundary points are the starting point coordinates, and the maximum coordinate of the x pixel and the maximum coordinate of the y pixel of all the boundary points are the end point coordinates; (B2) Convert the coordinate system of each pixel point in the image to the world coordinate system. The difference between the minimum and maximum coordinates of the x pixel is the contour width L, and the difference between the maximum and minimum coordinates of the y pixel is the contour height H. Then the contour radius Radius is The center angle of the contour is θ. The coordinates of the center point can be calculated by the coordinates of the starting point, the end point and the radius. Create a new variable center to store the coordinates of the center point, create a new variable circle x to store the x coordinates of the center point, and create a new variable circle_y to store the y coordinates of the center point. Then circle_x = (x_coords.min() + x_coordsmax()) / 2, circle_x = 3_coords.max() - Radius; (B3) Through the camera calibration experiment, the camera's internal and external parameters are calculated; the pixel coordinates are converted into world coordinates through coordinate system conversion, and the calculated parameters are converted into control instructions to drive the trimming machine to perform contour sawing.

Images