Farmland feature semantic segmentation method based on laser radar and SegNet

Abstract

The invention provides a farmland terrain semantic segmentation method based on a laser radar and SegNet, and the method comprises the steps: scanning a farmland through employing a vehicle-mounted or unmanned plane-carried laser radar device, obtaining original point cloud data, carrying out the data preprocessing of the original point cloud data, obtaining the point cloud data of a target farmland, and carrying out the segmentation of the target farmland. Voxelization is carried out on the point cloud data to obtain a training data set, a segmentation model is trained based on the training data set, semantic segmentation prediction is carried out on the target farmland through the segmentation model, and according to a training result obtained by training the segmentation model and a segmentation result obtained by semantic segmentation prediction, the target farmland is obtained. And obtaining a final semantic segmentation result of the target farmland, and visually displaying the semantic segmentation result. According to the method, end-to-end farmland feature semantic segmentation is realized, visual display is provided, and the effect on complex farmland scene semantic segmentation is remarkable through the laser radar point cloud.

Description

Technical Field

[0001] The present invention belongs to the technical field of farmland segmentation, and in particular relates to a farmland feature semantic segmentation method based on laser radar SegNet. Background Art

[0002] In the field of farmland features, it is often necessary to classify the features within the farmland, that is, to classify the farmland feature data information map. Currently, there are few relevant classification and segmentation schemes, and there are few archives that use deep learning models as segmentation models for accurate segmentation. The existing schemes mainly have the following problems:

[0003] (1) The collected farmland point cloud data contains a large number of discrete noise points and redundant ground points, and the semantic segmentation accuracy of the farmland lidar point cloud is low;

[0004] (2) The inherent three-dimensional spatial information and multi-scale features of farmland point clouds were not effectively utilized, resulting in poor segmentation results;

[0005] (3) The classification effects of different land object categories in farmland are unbalanced. In particular, small objects are easily ignored, resulting in inaccurate segmentation results.

[0006] Therefore, we need to develop a semantic segmentation method for farmland objects based on lidar and SegNet, which can achieve end-to-end high-precision semantic segmentation of farmland objects and provide intuitive visualization, providing important and reliable information support for smart agriculture applications. Summary of the invention

[0007] The purpose of the present invention is to provide a semantic segmentation method for farmland objects based on lidar and SegNet, so as to solve the problems of low segmentation accuracy and inaccurate segmentation results in the existing farmland object category segmentation schemes mentioned in the above background technology.

[0008] To achieve the above object, the present invention provides a semantic segmentation method of farmland objects based on laser radar and SegNet, and the method is specifically as follows:

[0009] Use a laser radar device mounted on a vehicle or a drone to scan the target farmland to be segmented, obtain the original point cloud data of the target farmland, and perform data preprocessing on the original point cloud data to obtain the point cloud data of the target farmland;

[0010] voxelize the point cloud data to obtain a training data set, train a segmentation model based on the training data set, and perform semantic segmentation prediction on the target farmland through the segmentation model;

[0011] Obtaining a final semantic segmentation result of the target farmland according to the training result of the segmentation model and the segmentation result of the semantic segmentation prediction, and visually displaying the semantic segmentation result;

[0012] Based on the above scheme, the laser radar independently emits laser pulses and captures the returned signals to generate the original point cloud data of each reflection point. The information in the original point cloud data includes but is not limited to the three-dimensional coordinates of each point. and reflection intensity ;

[0013] Based on the above solution, the data preprocessing is specifically as follows:

[0014] Using a statistical outlier removal filter in an open source C++ library to mark and remove noise points in the original point cloud data, to obtain denoised point cloud data, recorded as first point cloud data;

[0015] Performing plane fitting on the first point cloud data using a least squares algorithm to obtain a ground plane equation, calculating the distance from each point to the ground based on the ground plane equation, and marking points whose distance to the ground is less than a threshold value A as ground points;

[0016] The ground points are removed to obtain non-ground point cloud data, which is recorded as second point cloud data, mainly including ground object data information such as crops and farmland facilities, and the second point cloud data is divided into a training set, a validation set and a test set;

[0017] Based on the above scheme, the voxelization specifically includes:

[0018] The feature extraction of the second point cloud data is performed to obtain a first point cloud feature vector, wherein each point cloud sample is composed of a series of three-dimensional points, each three-dimensional point contains coordinate information and reflection intensity information, and the three-dimensional coordinates of each three-dimensional point are extracted. and reflection intensity Form a four-dimensional feature vector ;

[0019] The extracted first point cloud feature vector is divided into regular three-dimensional grids according to the preset voxel size, which are denoted as voxel grids. The feature data of each voxel grid is represented by the average value of the feature vectors of all points inside it. Assuming that the voxel size is N, the point cloud sample is divided into regular three-dimensional grids of size The voxel grid of , , ; , , and , , are the minimum and maximum coordinate values ​​of the point cloud samples, respectively. Indicates a round-up operation.

[0020] For each point cloud sample in the training set and the validation set, it is necessary to extract the corresponding point-by-point annotation information as the Ground Truth, that is, the annotation data of each three-dimensional point in the point cloud sample. The annotation data is represented in the form of One-Hot encoding. Each three-dimensional point corresponds to a C-dimensional binary vector, where C is the total number of farmland feature categories. If the point belongs to the i-th category of features, the i-th element of the corresponding binary vector is 1, and the remaining elements are 0.

[0021] Based on the above scheme, after the annotation data is extracted, each point cloud sample includes the feature data of the voxel grid and the annotation data, wherein the feature data is , representing the four-dimensional feature vector of each voxel in the voxel grid, and the annotation data is A four-dimensional array representing the type of terrain feature to which each voxel corresponds.

[0022] Based on the above scheme, it is necessary to perform data enhancement on the point cloud samples of the training set, including random rotation, random translation, random scaling, Gaussian noise, and random downsampling, and by combining multiple data enhancement methods, a large number of point cloud samples with different perspectives, positions, scales, and noise levels are generated. The point cloud samples after data enhancement and the original point cloud samples together constitute the final training data set.

[0023] Based on the above scheme, the segmentation model uses a deep learning model, and preferably a SegNet model, and adopts an end-to-end supervised learning paradigm to train the segmentation model, taking the feature data of the voxel grid as input and the labeled data as a supervisory signal, and learning the parameters of the SegNet model through a back propagation algorithm, including: forward propagation calculation, loss function calculation and segmentation model parameter optimization, as follows:

[0024] The forward propagation calculation includes multi-scale feature fusion and object category prediction, specifically:

[0025] The feature data and the annotation data of the training data set in the voxel grid are packaged into training sample pairs ,in For the The feature data of point cloud samples, The corresponding labeled data; randomly divide the training sample pairs into several of small batches, each of which contains The training sample pairs ; Input each mini-batch into the SegNet model for forward propagation calculation;

[0026] The multi-scale feature fusion and the prediction of the object category are to extract the multi-scale features of the feature data through the encoder, perform feature fusion on the multi-scale features through the ASPP module, and gradually upsample them through the decoder to obtain the prediction output of the same size as the feature data. ,in represents the forward propagation function of the SegNet model, are the parameters of the SegNet model; the predicted output With real annotation The same shape indicates the predicted probability of the ground object category to which each voxel belongs by the SegNet model, that is, the category prediction result;

[0027] The loss function calculation is specifically as follows: using a multi-scale weighted cross entropy loss function to measure the predicted output With the real annotation The difference between The loss of a point cloud sample is: , where S is the feature map scale number, is the weight of the sth scale, N is the total number of voxel grids, C is the total number of ground object categories, is the probability of the cth class that is truly labeled for the i-th voxel grid, is the i-th voxel grid predicted by the model at the s-th scale Class probability; take the average of the losses of all samples in the mini-batch to obtain the mini-batch loss: ;

[0028] The segmentation model parameter optimization includes back propagation calculation, as follows: the parameters of the SegNet model are optimized according to the mini-batch loss L Computing Gradients , and use the Adam optimization algorithm to update the parameters: ,in is the learning rate, and are the first-order moment estimate and the second-order moment estimate of the gradient, respectively. is a smoothing term to prevent the denominator from being zero.

[0029] Based on the above scheme, forward propagation, loss calculation and back propagation are performed sequentially on all small batches in the training data set to complete an Epoch training. When each Epoch training is completed, the performance of the SegNet model is evaluated on the validation set, including but not limited to calculating indicators such as mIoU and Accuracy. When there is no improvement in the validation set indicators of an Epoch, the Early Stopping mechanism is triggered to stop training and save the current optimal parameters of the SegNet model.

[0030] Based on the above scheme, the voxel grid after data preprocessing is input into the trained SegNet model. The SegNet model extracts the multi-scale features of the voxel grid through the encoder and performs feature fusion through the ASPP module. The fused features are gradually upsampled through the decoder to restore the resolution consistent with the original point cloud data. The output of the decoder is the probability vector that each voxel grid belongs to each land object category. The Argmax operation is performed on the probability vector to obtain the predicted category label of each voxel grid, that is, the semantic segmentation result of the target farmland.

[0031] Based on the above scheme, the visual display can intuitively show the spatial distribution and attribute information of the target farmland features;

[0032] According to the semantic segmentation result, each point of the original point cloud data is assigned a different color; commonly used farmland feature categories and their corresponding colors include: crops: green; soil: brown; farmland roads: gray; farmland facilities such as greenhouses, irrigation systems, etc.: blue;

[0033] The point cloud visualization library is used to display the colored point cloud and generate a three-dimensional distribution map of the target farmland.

[0034] Compared with the prior art, the present invention has the following advantages and effects:

[0035] (1) Preprocess the original point cloud data of farmland to remove noise and redundant ground points, significantly improving the data quality;

[0036] (2) By converting farmland point cloud data into regular three-dimensional voxel grids and using a variety of data enhancement methods to expand training samples, the improved SegNet network structure that introduces the ASPP module to extract multi-scale features can fully mine and utilize the three-dimensional spatial information of the point cloud and increase the accuracy of semantic segmentation of farmland objects;

[0037] (3) A multi-scale weighted cross entropy loss function is adopted to balance the losses at different scales, which effectively improves the segmentation effect of small targets in farmland objects. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for the specific embodiments or the description of the prior art. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn according to the actual scale.

[0039] Figure 1 This is a flow chart of a method for semantic segmentation of farmland objects based on lidar and SegNet provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0040] In order to more clearly explain the purpose, technical solutions and advantages of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. The example implementation methods can be implemented in various forms and should not be understood as being limited to the examples described herein. On the contrary, these implementation methods are provided to make the present invention more comprehensive and complete, and to fully convey the concepts of the example implementation methods to those skilled in the art.

[0041] In addition, the described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present invention. However, those skilled in the art will appreciate that the technical solution of the present invention can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring various aspects of the present invention.

[0042] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0043] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0044] The present invention is described in detail below in conjunction with specific embodiments:

[0045] As attached Figure 1 As shown, the embodiment of the present invention provides a semantic segmentation method of farmland objects based on laser radar and SegNet, and the specific steps of the method are as follows:

[0046] Step S1, using a laser radar device mounted on a vehicle or an unmanned aerial vehicle to scan the target farmland to be segmented, obtaining the original point cloud data of the target farmland, and performing data preprocessing on the original point cloud data to obtain the point cloud data of the target farmland;

[0047] Step S2, voxelizing the point cloud data to obtain a training data set, training a segmentation model based on the training data set, and performing semantic segmentation prediction on the target farmland through the segmentation model;

[0048] Step S3, obtaining a final semantic segmentation result of the target farmland according to the training result of the segmentation model and the segmentation result of the semantic segmentation prediction, and visually displaying the semantic segmentation result;

[0049] Preferably, in step S1, the laser radar independently emits laser pulses and captures the returned signals to generate the original point cloud data of each reflection point, and the information in the original point cloud data includes but is not limited to the three-dimensional coordinates of each point. and reflection intensity ;

[0050] Preferably, the data preprocessing in step S1 is specifically:

[0051] Step S101: Use open source A statistical outlier removal filter in a CL library (C++ library) is used to mark and remove noise points in the original point cloud data to obtain denoised point cloud data, which is recorded as first point cloud data;

[0052] Step S102: Use the least squares algorithm to perform plane fitting on the first point cloud data to obtain a ground plane equation, calculate the distance from each point to the ground based on the ground plane equation, and mark the points whose distance to the ground is less than a threshold A as ground points;

[0053] Step S103: remove the ground points to obtain non-ground point cloud data, recorded as second point cloud data, mainly including ground object data information such as crops and farmland facilities, and divide the second point cloud data into a training set, a validation set and a test set;

[0054] Specifically, the random division is preferably 70% for the training set, 10% for the validation set and 20% for the test set.

[0055] Preferably, in step S2, the voxelization specifically includes:

[0056] Step S201: extract the features of the point cloud from the second point cloud data to obtain a first point cloud feature vector, wherein each point cloud sample is composed of a series of three-dimensional points, each three-dimensional point contains coordinate information and reflection intensity information, and the three-dimensional coordinates of each three-dimensional point are extracted. and reflection intensity Form a four-dimensional feature vector ;

[0057] Step S202: Divide the extracted first point cloud feature vector into regular three-dimensional grids according to a preset voxel size, denoted as voxel grids. The feature data of each voxel grid is represented by the average value of the feature vectors of all points inside it. Assuming that the voxel size is N, the point cloud sample is divided into regular three-dimensional grids of size The voxel grid of , , ; , , and , , are the minimum and maximum coordinate values ​​of the point cloud samples, respectively. Indicates rounding up operation;

[0058] Step S203: For each point cloud sample in the training set and the validation set, it is necessary to extract the corresponding point-by-point annotation information as the Ground Truth, that is, the annotation data of each 3D point in the point cloud sample, the annotation data is represented in the form of One-Hot coding, each 3D point corresponds to a C-dimensional binary vector, C is the total number of farmland feature categories, if the point belongs to the i-th category of features, then the i-th element of the corresponding binary vector is 1, and the other elements are 0;

[0059] Preferably, after the annotation data extraction in step S203 is completed, each point cloud sample includes the feature data of the voxel grid and the annotation data, wherein the feature data is , representing the four-dimensional feature vector of each voxel in the voxel grid, and the annotation data is A four-dimensional array representing the type of terrain feature to which each voxel corresponds.

[0060] Preferably, in step S2, it is necessary to perform data enhancement on the point cloud samples of the training set, including random rotation, random translation, random scaling, Gaussian noise, and random downsampling, and by combining a variety of data enhancement methods, a large number of point cloud samples with different perspectives, positions, scales, and noise levels are generated. The point cloud samples after data enhancement and the original point cloud samples together constitute the final training data set.

[0061] Preferably, in step S2, the segmentation model uses a deep learning model, and preferably a SegNet model, and adopts an end-to-end supervised learning paradigm to train the segmentation model, taking the feature data of the voxel grid as input and the labeled data as a supervisory signal, and learning the parameters of the SegNet model through a back propagation algorithm, including: forward propagation calculation, loss function calculation, and segmentation model parameter optimization, as follows:

[0062] Step S204, the forward propagation calculation includes multi-scale feature fusion and feature category prediction, specifically:

[0063] Step S2041: Pack the feature data and the annotation data of the training data set in the voxel grid into a training sample pair. ,in For the The feature data of point cloud samples, The corresponding labeled data; randomly divide the training sample pairs into several of small batches, each of which contains The training sample pairs ; Input each mini-batch into the SegNet model for forward propagation calculation;

[0064] Step S2042, the multi-scale feature fusion and the prediction of the object category are to extract the multi-scale features of the feature data through the encoder, the multi-scale features are fused through the ASPP module, and the multi-scale features are gradually up-sampled through the decoder to obtain the prediction output of the same size as the feature data. ,in represents the forward propagation function of the SegNet model, are the parameters of the SegNet model; the predicted output With real annotation The same shape indicates the predicted probability of the ground object category to which each voxel belongs by the SegNet model, that is, the category prediction result;

[0065] Step S205: The loss function calculation is specifically as follows: using a multi-scale weighted cross entropy loss function to measure the predicted output With the real annotation The difference between The loss of a point cloud sample is: , where S is the feature map scale number, is the weight of the sth scale, N is the total number of voxel grids, C is the total number of ground object categories, is the probability of the cth class that is truly labeled for the i-th voxel grid, is the i-th voxel grid predicted by the model at the s-th scale Class probability; take the average of the losses of all samples in the mini-batch to obtain the mini-batch loss: ;

[0066] Step S206, the segmentation model parameter optimization includes back propagation calculation, as follows: the parameters of the SegNet model are optimized according to the mini-batch loss L Computing Gradients , and use the Adam optimization algorithm to update the parameters: ,in is the learning rate, and are the first-order moment estimate and the second-order moment estimate of the gradient, respectively. is a smoothing term to prevent the denominator from being zero.

[0067] Preferably, after step S206, forward propagation, loss calculation and back propagation are sequentially performed on all small batches in the training data set to complete an Epoch training. When each Epoch training is completed, the performance of the SegNet model is evaluated on the validation set, including but not limited to calculating indicators such as mIoU and Accuracy. When there is no improvement in the validation set indicators of an Epoch, the Early Stopping mechanism is triggered to stop training and save the current optimal parameters of the SegNet model.

[0068] Preferably, the trained segmentation model is used to perform semantic segmentation prediction on the test set, specifically: each point cloud sample of the test set is input into the trained SegNet model, the SegNet model outputs a predicted probability vector of the category to which each voxel grid belongs through forward propagation calculation, and an Argmax operation is performed on the predicted probability vector to obtain a predicted category label of each voxel grid, and the output dimension is consistent with the input voxel grid;

[0069] Specifically, the performance of the segmentation model on the test set is evaluated to verify its generalization ability, analyze the shortcomings of the segmentation model and make optimization suggestions. The evaluation indicators involved include:

[0070] mIoU (mean intersection over union): mIoU , calculate the IoU of each category and then find the average;

[0071] Accuracy: ;

[0072] Precision: ;

[0073] Recall: ;

[0074] The calculation process includes: traversing each point in the test set, counting the matching between the true category and the predicted category; calculating the above indicators for each category separately, and counting the overall performance.

[0075] For example, if the test set size is 2,000,000 points, the results of verifying the segmentation model are as follows: mIoU: 0.78; Accuracy: 89%; Precision and Recall:

[0076] Crop: Precision=91%, Recall=85%;

[0077] Soil: Precision=87%, Recall=89%.

[0078] Preferably, in step S2, the voxel grid after data preprocessing is input into the trained SegNet model, the SegNet model extracts the multi-scale features of the voxel grid through the encoder, and performs feature fusion through the ASPP module, and the fused features are gradually upsampled through the decoder to restore the resolution consistent with the original point cloud data. The output of the decoder is a probability vector that each voxel grid belongs to each land feature category, and the probability vector is subjected to an Argmax operation to obtain the predicted category label of each voxel grid, that is, the semantic segmentation result of the target farmland.

[0079] Preferably, in step S3, the visual display can intuitively show the spatial distribution and attribute information of the target farmland features;

[0080] According to the predicted category label, each point of the original point cloud data is assigned a different color; commonly used farmland feature categories and their corresponding colors include: crops: green; soil: brown; farmland roads: gray; farmland facilities such as greenhouses, irrigation systems, etc.: blue;

[0081] Use a point cloud visualization library such as PCL, Open3D, etc. to display the colored point cloud and generate a three-dimensional distribution map of the target farmland.

[0082] For example, in this embodiment, specific data is taken as an example to show the detection effect of the test set of the solution of the present invention on the segmentation model, as follows:

[0083] (1) Assume that the target farmland point cloud data is:

[0084] Farmland area: 1 hectare;

[0085] Number of original point cloud points: 10,000,000 (10 million points);

[0086] Land feature categories: crops, soil, farm roads, farm facilities;

[0087] The proportion of land feature categories: crops (50%), soil (30%), roads (10%), and facilities (10%).

[0088] (2) Training data set division:

[0089] The training set accounts for 70%, the validation set accounts for 10%, and the test set accounts for 20%, of which the test set data contains 2,000,000 (2 million) point cloud points.

[0090] (3) Use the point cloud samples in the test set to verify the performance of the segmentation model on unseen data. The specific steps are as follows:

[0091] Model input: voxel grid feature data of the test set point cloud, with a dimension of 32×32×32×4 (assuming that each sample voxel is divided into a 32×32×32 grid, and each voxel has 4-dimensional features: 3-dimensional coordinates + reflection intensity);

[0092] Model output: voxel grid class predictions with dimensions of 32×32×32×4, where the 4-dimensional vector corresponding to each voxel represents the predicted probability of each class.

[0093] (4) Prediction results:

[0094] The first voxel grid is predicted as [0.1, 0.7, 0.1, 0.1], and the corresponding One-Hot encoding is [0, 1, 0, 0], which is predicted as "soil" (category 2).

[0095] The second voxel grid is predicted as [0.2, 0.1, 0.6, 0.1], and the corresponding One-Hot encoding is [0, 0, 1, 0], which is predicted as “farm road” (category 3).

[0096] (5) Calculate the test set evaluation index (assuming that the prediction results of the test set are as follows):

[0097] mIoU (mean intersection over union):

[0098] Class "crop": IoU=0.85; Class "soil": IoU=0.80

[0099] Category "road": IoU=0.70; Category "facility": IoU=0.75: ;

[0100] Accuracy: Assuming that the total number of points in the test set is 2,000,000 and the number of correctly predicted points is 1,800,000, the overall prediction accuracy of farmland object classification is: .

[0101] (6) Analyze the prediction results:

[0102] If the mIoU and Accuracy of the test set reach the preset threshold (such as mIoU>0.75 and Accuracy>85%), the segmentation model is considered to have good generalization ability.

[0103] If the test set performs poorly (for example, mIoU is significantly lower than the validation set), the model structure or training process needs to be adjusted.

[0104] In this embodiment, the point cloud data of farmland objects is collected, and pre-processed to remove noise and redundant ground points to improve data quality; the point cloud is converted into a regular three-dimensional voxel grid, and the ASPP module is introduced to extract multi-scale features to improve the SegNet network structure, fully mining and utilizing the three-dimensional spatial information of the point cloud; a multi-scale weighted cross entropy loss function is used to balance the losses at different scales, effectively improving the segmentation effect of small targets. This embodiment provides a complete and highly automated farmland object segmentation and verification solution, which can effectively improve the accuracy of semantic segmentation of farmland objects and perform visual display, providing reliable data support for the field of farmland object category segmentation.

[0105] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. This application is intended to cover any modification, use or adaptation of the present invention, which follows the general principles of the present invention and includes common knowledge or customary techniques in the art that are not disclosed by the present invention. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present invention are indicated by the claims. It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A semantic segmentation method for farmland objects based on LiDAR and SegNet, characterized in that: include: Use a laser radar device mounted on a vehicle or a drone to scan the target farmland to be segmented, obtain the original point cloud data of the target farmland, and perform data preprocessing on the original point cloud data to obtain the point cloud data of the target farmland; voxelize the point cloud data to obtain a training data set, train a segmentation model based on the training data set, and perform semantic segmentation prediction on the target farmland through the segmentation model; Obtaining a final semantic segmentation result of the target farmland according to the training result of the segmentation model and the segmentation result of the semantic segmentation prediction, and visually displaying the semantic segmentation result; The laser radar independently emits laser pulses and captures the returned signals to generate the original point cloud data of each reflection point. The information in the original point cloud data includes but is not limited to the three-dimensional coordinates of each point. and reflection intensity ; The data preprocessing specifically includes: Using a statistical outlier removal filter in an open source C++ library to mark and remove noise points in the original point cloud data, to obtain denoised point cloud data, recorded as first point cloud data; Performing plane fitting on the first point cloud data using a least squares algorithm to obtain a ground plane equation, calculating the distance from each point to the ground based on the ground plane equation, and marking points whose distance to the ground is less than a threshold value A as ground points; The ground points are removed to obtain non-ground point cloud data, which is recorded as the second point cloud data, mainly including ground object data information such as crops and farmland facilities. The second point cloud data is divided into a training set, a verification set and a test set.

2. The method for semantic segmentation of farmland objects based on laser radar and SegNet according to claim 1, characterized in that: The voxelization specifically includes: The feature extraction of the second point cloud data is performed to obtain a first point cloud feature vector, wherein each point cloud sample is composed of a series of three-dimensional points, each three-dimensional point contains coordinate information and reflection intensity information, and the three-dimensional coordinates of each three-dimensional point are extracted. and reflection intensity Form a four-dimensional feature vector ; The extracted first point cloud feature vector is divided into regular three-dimensional grids according to the preset voxel size, which are denoted as voxel grids. The feature data of each voxel grid is represented by the average value of the feature vectors of all points inside it. Assuming that the voxel size is N, the point cloud sample is divided into regular three-dimensional grids of size The voxel grid of , , ; , , and , , are the minimum and maximum coordinate values ​​of the point cloud samples, respectively. Indicates a round-up operation.

3. The method for semantic segmentation of farmland objects based on laser radar and SegNet according to claim 2, characterized in that: For each point cloud sample in the training set and the validation set, it is necessary to extract its corresponding point-by-point annotation information as the Ground Truth, that is, the annotation data of each 3D point in the point cloud sample, the annotation data is represented in the form of One-Hot encoding, each 3D point corresponds to a C-dimensional binary vector, C is the total number of farmland feature categories, if the point belongs to the i-th category of features, then the i-th element of the corresponding binary vector is 1, and the remaining elements are 0; After the annotation data is extracted, each point cloud sample includes the feature data of the voxel grid and the annotation data, wherein the feature data is , representing the four-dimensional feature vector of each voxel in the voxel grid, and the annotation data is A four-dimensional array representing the type of terrain feature to which each voxel corresponds.

4. The method for semantic segmentation of farmland objects based on laser radar and SegNet according to claim 3, characterized in that: It is necessary to perform data enhancement on the point cloud samples of the training set, including random rotation, random translation, random scaling, Gaussian noise, and random downsampling, and by combining multiple data enhancement methods, a large number of point cloud samples with different perspectives, positions, scales, and noise levels are generated. The point cloud samples after data enhancement and the original point cloud samples together constitute the final training data set.

5. The method for semantic segmentation of farmland objects based on laser radar and SegNet according to claim 4, characterized in that: The segmentation model uses a deep learning model, and preferably a SegNet model, and adopts an end-to-end supervised learning paradigm to train the segmentation model, using the feature data of the voxel grid as input and the labeled data as a supervisory signal, and learning the parameters of the SegNet model through a back-propagation algorithm, including: forward propagation calculation, loss function calculation, and segmentation model parameter optimization.

6. The method for semantic segmentation of farmland objects based on laser radar and SegNet according to claim 5, characterized in that: The forward propagation calculation specifically includes: The feature data and the annotation data of the training data set in the voxel grid are packaged into training sample pairs ,in For the The feature data of point cloud samples, The corresponding labeled data; randomly divide the training sample pairs into several Each mini-batch contains The training sample pairs ; Input each mini-batch into the SegNet model for forward propagation calculation.

7. The method for semantic segmentation of farmland objects based on laser radar and SegNet according to claim 6, characterized in that: The forward propagation calculation also includes multi-scale feature fusion and prediction of the category of the object. Specifically, the multi-scale feature fusion and the category of the object are to extract the multi-scale features of the feature data through the encoder, and the multi-scale features are fused through the ASPP module, and gradually up-sampled through the decoder to obtain the predicted output of the same size as the feature data. ,in represents the forward propagation function of the SegNet model, are the parameters of the SegNet model; the predicted output With real annotation The shapes are the same, indicating the predicted probability of the ground object category to which each voxel belongs by the SegNet model, that is, the category prediction result.

8. The method for semantic segmentation of farmland objects based on laser radar and SegNet according to claim 7, characterized in that: The loss function calculation is specifically as follows: using a multi-scale weighted cross entropy loss function to measure the predicted output With the real annotation The difference between The loss of a point cloud sample is: , where S is the feature map scale number, is the weight of the sth scale, N is the total number of voxel grids, C is the total number of ground object categories, is the probability of the cth class that is truly labeled for the i-th voxel grid, is the i-th voxel grid predicted by the model at the s-th scale Class probability; take the average of the losses of all samples in the mini-batch to obtain the mini-batch loss: .

9. The method for semantic segmentation of farmland objects based on laser radar and SegNet according to claim 8, characterized in that: The segmentation model parameter optimization includes back propagation calculation, as follows: the parameters of the SegNet model are optimized according to the mini-batch loss L Computing Gradients , and use the Adam optimization algorithm to update the parameters: ,in is the learning rate, and are the first-order moment estimate and the second-order moment estimate of the gradient, respectively. is a smoothing term to prevent the denominator from being zero.

10. The method for semantic segmentation of farmland objects based on laser radar and SegNet according to claim 9, characterized in that: The voxel grid after data preprocessing is input into the trained SegNet model. The SegNet model extracts the multi-scale features of the voxel grid through the encoder and performs feature fusion through the ASPP module. The fused features are gradually upsampled through the decoder to restore the resolution consistent with the original point cloud data. The output of the decoder is the probability vector that each voxel grid belongs to each ground object category. The Argmax operation is performed on the probability vector to obtain the predicted category label of each voxel grid, that is, the semantic segmentation result of the target farmland.

Images