Cultivated land shelter cross measurement method fusing unmanned aerial vehicle laser point cloud and image

Abstract

The invention belongs to the technical field of remote sensing measurement, and discloses a farmland shelter cross measurement method fusing unmanned aerial vehicle laser point cloud and images. The method comprises the steps that cultivated land point cloud data and cultivated land image data are collected and fused, and fused point cloud data are generated; processing the fused point cloud data, and identifying first green land data in the fused point cloud data; calculating a visible light vegetation index, and identifying second green land data in the fused point cloud data; integrating the first green land data and the second green land data to generate green land point cloud data, and performing regionalization processing to form a discrete green land region; performing vertical structure analysis on the green land area, identifying and deleting a suspension area in the green land area, and obtaining cultivated land area data; according to the cultivated land area data, generating a cultivated land area orthoimage; according to the invention, high-precision identification and removal of vegetation shelters in cultivated land can be realized, and misjudgment in topographic relief or vegetation dense areas can be avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing measurement, and more specifically, to a cross-measurement method for cultivated land shelters by integrating unmanned aerial vehicle (UAV) lidar point cloud and imagery. Background Art

[0002] With the rapid development of UAV technology, especially its wide application in the agricultural field, the combination of UAV lidar and imagery data provides new technical means for farmland management, cultivated land monitoring, and precise analysis of agricultural resources; lidar can provide accurate three-dimensional information of terrain and ground objects, while imagery data can capture the detailed appearance and texture features of ground objects; combining the two can not only improve the spatial accuracy of data but also make up for the deficiencies of a single data source; in the process of detecting and removing cultivated land shelters, traditional remote sensing imagery often has difficulty in accurately identifying the shelter effects caused by vegetation, especially in complex terrains and areas with high vegetation coverage; lidar point cloud data can provide higher ground resolution, accurately measure the height information of ground objects, and can distinguish between the ground and non-ground objects, but it is weak in the ability to identify details (such as vegetation types, precise boundaries of shelters); therefore, there is an urgent need for an intelligent method for measuring and removing cultivated land shelters that can effectively combine lidar point cloud and high-resolution imagery data to improve the recognition accuracy and precision of cultivated land shelters.

[0003] The patent with the publication number CN111487643A discloses a building detection method based on lidar point cloud and near-infrared imagery; it includes: obtaining an orthoimage with a near-infrared band and lidar point cloud, and registering and fusing them; calculating the normalized difference vegetation index (NDVI) of each lidar point after fusion, and completing vegetation recognition based on NDVI and a support vector machine classifier; for non-vegetation lidar points, complete the recognition of buildings through a nearest neighbor search algorithm and a height threshold; for building lidar points, extract roof seed points and candidate facade points, and obtain a roof point group of a single building based on the roof seed points; based on the roof point group of each building, estimate the vertical facade of the building; based on the candidate facade points and the estimated vertical facade, perform fine extraction of facade points; complete the detection of three-dimensional buildings through the roof points and the finely extracted facade points; this invention can effectively improve the detection accuracy of buildings and ensure a high level of detail in the building model.

[0004] However, although the above technology realizes vegetation recognition through the fusion of lidar point cloud and imagery, it only uses NDVI and a support vector machine classifier; in the cultivated land scenario, a single NDVI cannot fully distinguish vegetation from the ground, and the support vector machine classifier has insufficient generalization ability in complex terrains, with a risk of misjudgment; especially when the spectral characteristics of vegetation and the ground are close, it is easy to cause false detection or missed detection, reducing the vegetation recognition accuracy.

[0005] In view of this, the present invention proposes a cross-measurement method for cultivated land shelters that integrates UAV lidar point clouds and images to solve the above problems. Summary of the Invention

[0006] To overcome the above-mentioned defects of the prior art and achieve the above object, the present invention provides the following technical solutions: A cross-measurement method for cultivated land shelters that integrates UAV lidar point clouds and images, including: S1: Collect cultivated land point cloud data and cultivated land image data; S2: Use spatial registration technology and attribute mapping technology to fuse the cultivated land point cloud data and the cultivated land image data to generate fused point cloud data; S3: Use radar feature fusion technology to perform ground object classification on the fused point cloud data to identify the first green land data in the fused point cloud data; calculate the visible light vegetation index, and identify the second green land data in the fused point cloud data by threshold segmentation method; S4: Integrate the first green land data and the second green land data to generate green land point cloud data, and perform regionalization on the green land point cloud data based on the spatial clustering algorithm to form discrete green land regions; S5: Use suspension discrimination technology to perform vertical structure analysis on the green land regions, identify the suspended regions in the green land regions, and delete the suspended regions from the green land regions to obtain cultivated land region data; S6: According to the cultivated land region data, use point cloud reverse modeling technology to generate an orthophoto of the cultivated land region.

[0007] Further, the cultivated land point cloud data includes point cloud points, where is an integer greater than 1; each point cloud point includes coordinate data and echo data, and the coordinate data includes coordinate values, coordinate values, and coordinate values, and the echo data includes echo intensity and echo time; The cultivated land image data includes pixel points, where is an integer greater than 1; each pixel point includes planar coordinate data and color data; the planar coordinate data includes coordinate values and coordinate values, and the color data includes , , surface reflectance of three bands; The method for generating the fused point cloud data includes: Using point cloud processing software and aerial survey software, convert the cultivated land point cloud data and cultivated land image data into the same coordinate system; adopt a feature point matching algorithm to roughly register the cultivated land point cloud data and cultivated land image data in the same coordinate system to obtain a preliminary transformation matrix; adopt an iterative closest point algorithm to iteratively optimize the preliminary transformation matrix to obtain a fine transformation matrix; according to the fine transformation matrix, project each point cloud point in the cultivated land point cloud data onto the cultivated land image data to establish a spatial correspondence between the point cloud points and pixel points; according to the spatial correspondence, use a high-order interpolation algorithm to map the color data of each pixel point to the corresponding point cloud point to obtain mapping points containing coordinate data, echo data, and color data; fuse all mapping points to generate fused point cloud data.

[0008] Further, the steps for identifying the first green land data include: Step S101: Construct multiple fuzzy sets for the echo intensity and echo time respectively; Step S102: Convert the echo data of each mapping point into the membership degrees of the corresponding fuzzy sets respectively through a fuzzification technique; Step S103: Define fuzzy rules; Step S104: Match each group of fuzzified echo data with the fuzzy rules respectively, and use a fuzzy inference method for fuzzy inference to obtain the fuzzy inference results corresponding to each mapping point. The fuzzy inference results are the membership degrees corresponding to each type of ground object, and the types of ground objects include ground points and vegetation points; Step S105: Compare each membership degree in each fuzzy inference result, and use the type of ground object corresponding to the membership degree with the largest value as the type of ground object of the corresponding mapping point; Step S106: Use the mapping points with the ground object type of vegetation points in the fused point cloud data as the first green land data.

[0009] Further, the method for identifying the second green land data includes: Multiply the surface reflectance of each mapping point corresponding to the band by 2, and subtract the surface reflectance of the corresponding band, and then subtract the surface reflectance of the corresponding band to obtain the green differential reflectance of each mapping point; multiply the surface reflectance of each mapping point corresponding to the band by 2, and add the surface reflectance of the corresponding band, and then add the surface reflectance of the corresponding The surface reflectance of the band is obtained, and the composite reflectance of each mapping point is acquired; the green differential reflectance of each mapping point is divided by the corresponding composite reflectance respectively to obtain the visible light vegetation index of each mapping point; a vegetation threshold is preset, and the visible light vegetation index of each mapping point is compared with the vegetation threshold respectively; if the visible light vegetation index is greater than the vegetation threshold, the corresponding mapping point is marked as a green object point; if the visible light vegetation index is less than or equal to the vegetation threshold, the corresponding mapping point is not marked; all the green object points in the fused point cloud data are used as the second green land data.

[0010] Further, the method for generating the green point cloud data includes: Each vegetation point in the first green land data and each green object point in the second green land data are taken as a set of point collections respectively, that is, each set of point collections includes a vegetation point and a green object point; each set of point collections is analyzed in turn. If the vegetation point and the green object point in the point collection are the same mapping point, the corresponding point collection is marked as a reserved collection. If the vegetation point and the green object point in the point collection are not the same mapping point, the corresponding point collection is not marked; the vegetation points in each set of reserved collections are obtained and all are marked as green plant points; according to all the green plant points, the green point cloud data is generated.

[0011] Further, the steps for forming discrete green areas include: Step S201: According to the coordinate data of each green plant point, the Euclidean distance between every two green plant points is calculated in turn and marked as the node distance; Step S202: Randomly select a green plant point that has not been marked as a selected point as the current point; Step S203: According to the node distance corresponding to the current point, the adjacent points corresponding to the current point are obtained; Step S204: Count the number of adjacent points corresponding to the current point and mark it as the adjacent number; judge whether the current point is marked as a core point or an edge point according to the adjacent number; Step S205: If the current point is marked as a core point, a new point cluster is created, and the current point and the corresponding adjacent points are all added to the new point cluster, and then go to Step S206; if the current point is marked as an edge point, a new point cluster is not created, and the current point is marked as a selected point, and jump back to Step S202; Step S206: Expand the new point cluster and mark all the green plant points in the new point cluster as selected points; Step S207: Loop Steps S202 to S206 until all green plant points are marked as selected points, the loop ends, and the green plant points in each point cluster are obtained; Step S208: Filter the green plant points in each point cluster, and use the green plant points in each filtered point cluster as a discrete green area, that is, the green areas correspond one by one to the point clusters.

[0012] Further, in the step S203, the method for obtaining adjacent points corresponding to the current point is: compare the node distance corresponding to the current point with a preset maximum distance in sequence, mark the green plant points with node distances less than or equal to the maximum distance as adjacent points, and do not mark the green plant points with node distances greater than the maximum distance; In the step S204, the method for determining whether to mark the current point as a core point or an edge point is: add one to the adjacent number and compare it with a preset minimum number of points; if the adjacent number is greater than or equal to the minimum number of points, mark the current point as a core point; if the adjacent number is less than the minimum number of points, mark the current point as an edge point; In the step S206, the steps for expanding a new point cluster include: Step S301: Obtain adjacent points corresponding to all green plant points in the new point cluster and mark them as adjacent points; sequentially determine whether each adjacent point is marked as a core point, and mark the adjacent points marked as core points as adjacent core points; obtain adjacent points corresponding to each adjacent core point and mark them as neighborhood points; add each adjacent core point and the corresponding neighborhood points to the new point cluster as well; Step S302: Loop step S301 until no new green plant points are added to the new point cluster, then the loop ends and the expansion of the new point cluster is completed; In the step S208, the method for filtering the green plant points in each point cluster is: sequentially calculate the node distances between every two green plant points in each point cluster, compare the node distances corresponding to each green plant point, and use the smallest node distance as the minimum distance corresponding to the green plant point; compare the minimum distance of each green plant point with a preset distance threshold respectively, delete the green plant points with minimum distances greater than or equal to the distance threshold from the corresponding point cluster, and retain the green plant points with minimum distances less than the distance threshold in the corresponding point cluster.

[0013] Further, the method for identifying the suspended area includes: Regarding the vegetation points not marked as green plant points in the first green land data as ground points as well; generate cultivated land ground data based on all the ground points; generate a ground elevation grid by using a weighted interpolation algorithm according to the cultivated land ground data, where each grid unit contains a ground elevation value, and the ground elevation value is the vertical height of the ground point relative to the mean sea level; use a basic interpolation algorithm to calculate the ground elevation value corresponding to each green plant point from the ground elevation grid; for each green plant point The coordinate values are respectively subtracted by the corresponding ground elevation values to obtain the relative height differences of each green plant point; according to the ground elevation grid, a window analysis algorithm is adopted to generate a ground slope grid, where each grid cell includes a slope value, and the slope value is the inclination angle of the ground point corresponding to the ground surface and the horizontal plane; a basic interpolation algorithm is used to calculate the slope value corresponding to each green plant point from the ground slope grid; Both the relative height difference and the slope value of each green plant point are used as a set of green plant data, and the green plant data corresponds one-to-one with the green plant point; each set of green plant data is respectively input into the trained suspension recognition model, and the corresponding recognition label is output. The suspension recognition model is a deep neural network model; the recognition label is the digital label corresponding to the recognition result, and different recognition results correspond to different digital labels. The recognition results include suspension and fixation; according to the recognition label corresponding to each set of green plant data, the recognition result corresponding to each set of green plant data is obtained; if the recognition result is suspension, the green plant point corresponding to the corresponding green plant data is marked as a suspension point, and if the recognition result is fixation, the green plant point corresponding to the corresponding green plant data is not marked; all suspension points are regionalized to form discrete suspension regions; the method of forming discrete suspension regions is the same as the method of forming discrete green regions.

[0014] Furthermore, the training process of the suspension recognition model includes: Pre-collect sets of different green plant data, and set corresponding recognition labels for sets of green plant data. is an integer greater than 1. The green plant data and the corresponding recognition labels are converted into a corresponding set of feature vectors; each set of feature vectors is used as the input of the suspension recognition model. The suspension recognition model outputs a set of predicted recognition labels corresponding to each set of green plant data, and uses the actual recognition label corresponding to each set of green plant data as the prediction target. The actual recognition label is the recognition label preset corresponding to the green plant data; minimizing the sum of the prediction errors of all green plant data is used as the training target; the suspension recognition model is trained until the sum of the prediction errors converges and then the training stops; The method for obtaining the cultivated land area data includes: Delete the suspension regions from the green regions, and obtain the cultivated land area data according to the ground points and the remaining green plant points in the green regions.

[0015] Furthermore, the method for generating the orthophoto of the cultivated land area includes: Based on the cultivated land image data, photogrammetric modeling is carried out using the structure from motion reconstruction technology and the multi-view stereo vision technology to generate the original oblique model; according to the mapping points in the cultivated land area data, a triangular mesh is constructed using the triangulation method; using the texture mapping method, the image texture information in the original oblique model is mapped to the triangular mesh to reconstruct the oblique three-dimensional model; according to the oblique three-dimensional model, an orthophoto image of the cultivated land area without vegetation occlusion is generated using the orthographic projection method.

[0016] The technical effects and advantages of the cross-measurement method for cultivated land occlusion objects by integrating UAV lidar and images of the present invention: By integrating the cultivated land point cloud data and the cultivated land image data, the respective advantages of the lidar and high-resolution image data can be fully utilized to improve the spatial accuracy and detail description ability of the data; moreover, by combining the dual vegetation recognition strategies of radar feature fusion technology, fuzzy inference method, and visible light index calculation, and using the suspension discrimination technology to perform vertical structure on the green area, the ground objects in the fused point cloud data can be effectively classified, overcoming the limitation that single use of NDVI and support vector machine classifier cannot fully distinguish vegetation and ground, and realizing high-precision recognition and removal of vegetation occlusion objects in cultivated land; finally, using the point cloud reverse modeling technology to generate an orthophoto image of the cultivated land area without vegetation occlusion effectively solves the technical problems of inaccurate vegetation recognition and incomplete occlusion removal in the traditional method in a complex cultivated land environment, avoids misjudgment in areas with uneven illumination, undulating terrain, and dense vegetation, and provides a more accurate, clear, and reliable cultivated land image data basis for agricultural precision monitoring, land resource assessment, crop growth analysis, and production decision-making. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flowchart of the cross-measurement method for cultivated land occlusion objects by integrating UAV lidar and images in Embodiment 1 of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0018] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0019] Embodiment 1 Please refer to Figure 1 As shown, the cross-measurement method for cultivated land occlusion objects by integrating UAV lidar and images in this embodiment includes: S1: Collect cultivated land point cloud data and cultivated land image data.

[0020] The cultivated land point cloud data is three-dimensional spatial data, which is obtained by a lidar device mounted on a drone; the cultivated land point cloud data includes point cloud points, where is an integer greater than 1; each point cloud point includes coordinate data and echo data, and the coordinate data includes coordinate values, coordinate values, and coordinate values, and the echo data includes echo intensity and echo time; the echo intensity is the signal intensity when the laser pulse is reflected back to the lidar device, and the echo time is the time experienced by the laser pulse from emission to return to the lidar device; the cultivated land point cloud data can accurately reflect the three-dimensional structure of the ground surface and ground objects, and can detail the spatial structure of cultivated land and vegetation; it should be noted that the cultivated land point cloud data is point cloud data for the ground surface elevation, so for each planar position there is only one corresponding coordinate value, that is, at the same planar position there are no multiple point cloud points stacked at different heights. The cultivated land image data is two-dimensional image data, usually with a spatial resolution of centimeter level, which is obtained by an optical camera mounted on a drone; the cultivated land image data includes pixel points, where is an integer greater than 1; each pixel point includes planar coordinate data and color data; the planar coordinate data includes coordinate values and coordinate values, and the color data includes the surface reflectance of three bands, that is, the surface reflectance of the band, the surface reflectance of the

[0021] S2: Using spatial registration technology and attribute mapping technology, fuse the cultivated land point cloud data and the cultivated land image data to generate fused point cloud data.

[0022] The method for generating the fused point cloud data includes: Using point cloud processing software (such as CloudCompare, TerraScan, etc.) and aerial survey software (such as Pix4D, Agisoft Metashape, etc.), the cultivated land point cloud data and the cultivated land image data are converted into the same coordinate system; using feature point matching algorithms (such as SIFT, SURF, etc.), the cultivated land point cloud data and the cultivated land image data in the same coordinate system are roughly registered to obtain a preliminary transformation matrix; using the iterative closest point algorithm, the preliminary transformation matrix is iteratively optimized to obtain a fine transformation matrix; according to the fine transformation matrix, each point cloud point in the cultivated land point cloud data is projected onto the cultivated land image data to establish a spatial correspondence between the point cloud points and the pixel points, that is, the pixel points corresponding to each point cloud point in the two-dimensional space; according to the spatial correspondence, using a high-order interpolation algorithm (such as spline interpolation algorithm, bicubic interpolation algorithm, etc.) to map the color data of each pixel point into the corresponding point cloud point to obtain a mapped point containing coordinate data, echo data, and color data; fusing all the mapped points to generate fused point cloud data; the iterative closest point algorithm is a prior art, and the specific process will not be elaborated here.

[0023] S3: Using radar feature fusion technology to perform ground object classification processing on the fused point cloud data to identify the first green land data in the fused point cloud data; calculating the visible light vegetation index and identifying the second green land data in the fused point cloud data by threshold segmentation method.

[0024] The steps for identifying the first green land data include: Step S101: Constructing multiple fuzzy sets for the echo intensity and the echo time respectively; for example: the fuzzy sets corresponding to the echo intensity are low intensity, medium intensity, high intensity, etc., and the fuzzy sets corresponding to the echo time are long time, medium time, short time, etc.; Step S102: Converting the echo data of each mapped point into the membership degree of the corresponding each fuzzy set respectively through the fuzzification technology; the fuzzification technology is the process of converting accurate numerical values into the membership degrees corresponding to fuzzy sets, and the fuzzification technology is, for example, triangular membership function, trapezoidal membership function, etc.; for example, if the numerical value of the echo intensity is relatively low, it is inferred that the membership degree of low intensity is 0.9, the membership degree of medium intensity is 0.1, and the membership degree of high intensity is 0; Step S103: Define fuzzy rules, which are defined based on expert knowledge or relevant literature. For example, if the echo intensity is high and the echo time is long, it is inferred that the probability of the corresponding mapping point being a ground point is high, that is, the corresponding mapping point is on the cultivated land ground; if the echo intensity is low and the echo time is short, it is inferred that the probability of the corresponding mapping point being a vegetation point is high, that is, the corresponding mapping point is on the cultivated land vegetation. It should be understood that since the UAV usually carries a short-wave radar with weak penetration, the laser wave is usually difficult to penetrate the vegetation layer. Also, in the cultivated land scenario, the height of the vegetation is generally higher than the ground. Therefore, when the laser echo comes from the vegetation surface, the corresponding echo time is shorter than that from the ground surface. Moreover, due to the complex structure and irregular shape of the vegetation, the laser is prone to multiple scattering and energy attenuation on the vegetation surface, resulting in a relatively low echo intensity. While the ground is relatively flat and usually shows specular or near-specular reflection, and the laser energy returns concentratedly. Therefore, the laser echo from the ground surface has a higher echo intensity. Step S104: Match each group of fuzzified echo data with the fuzzy rules respectively, and perform fuzzy inference using a fuzzy inference method (such as Mamdani fuzzy inference model, Sugeno fuzzy inference model, etc.) to obtain the fuzzy inference result corresponding to each mapping point. The fuzzy inference result is the membership degree corresponding to each ground object type, and the ground object types include ground points and vegetation points. For example, the membership degree of the ground point is 0.2 and the membership degree of the vegetation point is 0.8 in the fuzzy inference result. Step S105: Compare each membership degree in each fuzzy inference result, and take the ground object type corresponding to the membership degree with the largest value as the ground object type of the corresponding mapping point. Step S106: Take all the mapping points with the ground object type of vegetation points in the fused point cloud data as the first green land data.

[0025] The method for identifying the second green land data includes: Multiply the surface reflectance of each mapping point corresponding to each band by 2, and subtract the surface reflectance of the corresponding band, and then subtract the surface reflectance of the corresponding band to obtain the green differential reflectance of each mapping point; Multiply the surface reflectance of each mapping point corresponding to each band by 2, and add the surface reflectance of the corresponding band, and then add the surface reflectance of the corresponding The surface reflectance of the band is obtained, and the composite reflectance of each mapping point is acquired; the composite reflectance of each mapping point is obtained; the green differential reflectance of each mapping point is divided by the corresponding composite reflectance respectively to obtain the visible light vegetation index (i.e., the visible light band differential vegetation index) of each mapping point; a vegetation threshold is preset, and the vegetation threshold is preset by those skilled in the art according to the actual situation. The visible light vegetation index of each mapping point is compared with the vegetation threshold respectively; if the visible light vegetation index is greater than the vegetation threshold, the corresponding mapping point is marked as a green object point; if the visible light vegetation index is less than or equal to the vegetation threshold, the corresponding mapping point is not marked; all the green object points in the fused point cloud data are used as the second green land data.

[0026] S4: Integrate the first green land data and the second green land data to generate green land point cloud data, and perform regional processing on the green land point cloud data based on the spatial clustering algorithm to form discrete green land areas.

[0027] The method for generating green land point cloud data includes: Each vegetation point in the first green land data and each green object point in the second green land data are respectively used as a set of point collections, that is, each set of point collections includes a vegetation point and a green object point; each set of point collections is analyzed in turn. If the vegetation point and the green object point in the point collection are the same mapping point, the corresponding point collection is marked as a reserved collection. If the vegetation point and the green object point in the point collection are not the same mapping point, the corresponding point collection is not marked; the vegetation points in each set of reserved collections are obtained and all are marked as green plant points; according to all the green plant points, green land point cloud data is generated.

[0028] The steps for forming discrete green land areas include: Step S201: According to the coordinate data of each green plant point, calculate the Euclidean distance between every two green plant points in turn and mark it as the node distance; the calculation method of the Euclidean distance is the prior art and will not be elaborated here. Step S202: Randomly select a green plant point that has not been marked as a selected point as the current point. Step S203: According to the node distance corresponding to the current point, obtain the adjacent points corresponding to the current point. Step S204: Count the number of adjacent points corresponding to the current point and mark it as the adjacent number; judge whether to mark the current point as a core point or an edge point according to the adjacent number. Step S205: If the current point is marked as a core point, create a new point cluster, add the current point and the corresponding adjacent points to the new point cluster, and enter step S206; if the current point is marked as an edge point, do not create a new point cluster, mark the current point as a selected point, and jump back to step S202. Step S206: Expand the new point cluster, and mark all green plant points in the new point cluster as selected points; Step S207: Loop through steps S202 to S206 until all green plant points are marked as selected points. When the loop ends, obtain the green plant points in each point cluster; Step S208: Filter the green plant points in each point cluster, and use the green plant points in each filtered point cluster as a discrete green area, that is, the green area corresponds one-to-one with the point cluster.

[0029] In the above step S203, the method for obtaining adjacent points corresponding to the current point is as follows: Compare the node distance corresponding to the current point with the preset maximum distance in sequence, mark the green plant points with a node distance less than or equal to the maximum distance as adjacent points, and do not mark the green plant points with a node distance greater than the maximum distance; the maximum distance is preset by those skilled in the art according to the actual situation.

[0030] In the above step S204, the method for determining whether to mark the current point as a core point or an edge point is as follows: Add one to the number of adjacent points and compare it with the preset minimum number of points; if the number of adjacent points is greater than or equal to the minimum number of points, mark the current point as a core point; if the number of adjacent points is less than the minimum number of points, mark the current point as an edge point; the minimum number of points is preset by those skilled in the art according to the actual situation.

[0031] In the above step S206, the steps for expanding the new point cluster include: Step S301: Obtain the adjacent points corresponding to all green plant points in the new point cluster and mark them as adjacent points; sequentially determine whether each adjacent point is marked as a core point, and mark the adjacent points marked as core points as adjacent core points; obtain the adjacent points corresponding to each adjacent core point and mark them as neighborhood points; add each adjacent core point and the corresponding neighborhood point to the new point cluster as well; Step S302: Loop through step S301 until no new green plant points are added to the new point cluster. When the loop ends, the expansion of the new point cluster is completed.

[0032] In the above step S208, the method for filtering the green plant points in each point cluster is as follows: Calculate the node distance between every two green plant points in each point cluster in sequence, compare the node distances corresponding to each green plant point, and use the smallest node distance as the minimum distance corresponding to the green plant point; compare the minimum distance of each green plant point with the preset distance threshold respectively, delete the green plant points with a minimum distance greater than or equal to the distance threshold from the corresponding point cluster, and retain the green plant points with a minimum distance less than the distance threshold in the corresponding point cluster; the distance threshold is preset by those skilled in the art according to the actual situation.

[0033] S5: Use the suspension discrimination technology to analyze the vertical structure of the green area, identify the suspended areas in the green area, delete the suspended areas from the green area, and obtain the cultivated land area data.

[0034] The methods for identifying the suspended areas include: In the first green land data, the vegetation points not marked as green plant points are also regarded as ground points; based on all the ground points, the cultivated land ground data is generated; according to the cultivated land ground data, using weighted interpolation algorithms (such as IDW algorithm, TIN algorithm, Kriging algorithm, etc.), a ground elevation grid is generated, where each grid cell contains a ground elevation value, and the ground elevation value is the vertical height of the ground point relative to the mean sea level; using basic interpolation algorithms (such as nearest neighbor interpolation algorithm, bilinear interpolation algorithm, etc.), calculate the ground elevation value corresponding to each green plant point from the ground elevation grid; Subtract the corresponding ground elevation value from the coordinate value of each green plant point to obtain the relative height difference of each green plant point; according to the ground elevation grid, using window analysis algorithms (such as Horn algorithm, Zevenbergen & Thorne algorithm, etc.), generate a ground slope grid, where each grid cell includes a slope value, and the slope value is the inclination angle of the ground point corresponding surface to the horizontal plane; using basic interpolation algorithms (such as nearest neighbor interpolation algorithm, bilinear interpolation algorithm, etc.), calculate the slope value corresponding to each green plant point from the ground slope grid; Regard the relative height difference and slope value of each green plant point as a set of green plant data, and the green plant data corresponds one-to-one with the green plant points; input each set of green plant data into the trained suspension recognition model respectively, and output the corresponding recognition label. The suspension recognition model is a deep neural network model; the recognition label is the digital label corresponding to the recognition result, and different recognition results correspond to different digital labels. The recognition results include suspension and fixed; according to the recognition label corresponding to each set of green plant data, obtain the recognition result corresponding to each set of green plant data; if the recognition result is suspension, mark the green plant points corresponding to the corresponding green plant data as suspended points, and if the recognition result is fixed, do not mark the green plant points corresponding to the corresponding green plant data; perform regionalization processing on all the suspended points to form discrete suspended areas; the method for forming discrete suspended areas is the same as the method for forming discrete green areas.

[0035] The training process of the suspension recognition model includes: Pre-collect sets of different green plant data, and set corresponding recognition labels for sets of green plant data. is an integer greater than 1. Convert the green plant data and the corresponding recognition label into a corresponding set of feature vectors; the recognition label corresponding to the green plant data is collected by those skilled in the art during the process of historically identifying suspended areas. A set of different green plant data is analyzed for each green plant point in each set of green plant data in turn according to the actual situation to determine whether each green plant point is a floating point, that is, to determine whether the vegetation corresponding to the green plant point will block the cultivated land. Different recognition labels are set for different recognition results, and corresponding recognition labels are set for each set of different green plant data in turn; Each set of feature vectors is used as the input of the floating recognition model. The floating recognition model outputs a set of predicted recognition labels corresponding to each set of green plant data, and uses the actual recognition label corresponding to each set of green plant data as the prediction target. The actual recognition label is the recognition label preset corresponding to the green plant data; the training target is to minimize the sum of the prediction errors of all green plant data. Among them, the calculation formula of the prediction error is where is the prediction error, is the group number of the feature vector corresponding to the green plant data, is the predicted recognition label corresponding to the th group of green plant data, is the actual recognition label corresponding to the

[0036] The method for obtaining the cultivated land area data includes: Delete the floating area from the green area, and obtain the cultivated land area data according to the ground points and the remaining green plant points in the green area.

[0037] S6: Generate an orthophoto of the cultivated land area using the point cloud reverse modeling technology according to the cultivated land area data.

[0038] The method for generating an orthophoto of the cultivated land area includes: Based on the cultivated land image data, photogrammetric modeling is carried out using the Structure from Motion (SfM) technology and the Multi-View Stereo (MVS) technology to generate an original oblique model, which includes the three-dimensional geometric structure and image texture information of the ground, vegetation, and other features within the cultivated land; both the Structure from Motion technology and the Multi-View Stereo technology are existing technologies, and the specific process will not be elaborated here; according to the mapping points in the cultivated land area data, a triangular mesh is constructed using triangulation methods (such as constrained Delaunay triangulation method, Bowyer-Watson algorithm, etc.); using texture mapping methods (such as multi-band fusion texture mapping method, spherical projection texture mapping method, Poisson texture fusion method, etc.), the image texture information in the original oblique model is mapped to the triangular mesh to reconstruct an oblique three-dimensional model, and the reconstructed oblique three-dimensional model is a cultivated land surface model without vegetation occlusion; based on the oblique three-dimensional model, an orthophoto image of the cultivated land area without vegetation occlusion is generated using orthographic projection methods (such as Z-buffer orthographic projection method, differential orthorectification method, etc.).

[0039] In this embodiment, by fusing the cultivated land point cloud data and the cultivated land image data, the respective advantages of the laser point cloud and the high-resolution image data can be fully utilized to improve the spatial accuracy and detail description ability of the data; moreover, by combining the radar feature fusion technology, the fuzzy inference method, and the dual vegetation recognition strategy of visible light index calculation, and using the suspension discrimination technology to perform vertical structure on the green area, the ground objects in the fused point cloud data can be effectively classified, overcoming the limitation that using only NDVI and the support vector machine classifier cannot fully distinguish vegetation and the ground, and realizing high-precision recognition and removal of vegetation occlusions in the cultivated land; finally, using the point cloud reverse modeling technology to generate an orthophoto image of the cultivated land area without vegetation occlusion effectively solves the technical problems of inaccurate vegetation recognition and incomplete occlusion removal in the traditional method in a complex cultivated land environment, avoids misjudgment in areas with terrain undulation or dense vegetation, and provides a more accurate, clear, and reliable cultivated land image data basis for agricultural precision monitoring, land resource assessment, crop growth analysis, and production decision-making.

[0040] Embodiment 2 This application also provides an electronic device. The electronic device may include one or more processors and one or more memories. Among them, the memory stores computer-readable code, and when the computer-readable code is run by one or more processors, it can execute the cross-measurement method for cultivated land occlusions by fusing drone laser point clouds and images as described above.

[0041] The method or system according to the embodiments of the present application can also be implemented by means of the architecture of the electronic device shown in the present application. The electronic device may include a bus, one or more CPUs, a ROM, a RAM, a communication port connected to a network, an input / output, a hard disk, etc. The storage device in the electronic device, such as a ROM or a hard disk, can store the method for cross-measuring cultivated land shelters by fusing UAV laser point clouds and images provided by the present application. Further, the electronic device may also include a user interface. Of course, the architecture shown in the present application is only exemplary. When implementing different devices, one or more components shown in the electronic device of the present application can be omitted according to actual needs.

[0042] Embodiment 3 As shown in the figure, an embodiment of the present application discloses a computer-readable storage medium. A computer-readable instruction is stored on the computer-readable storage medium. When the computer-readable instruction is run by a processor, it can execute the method for cross-measuring cultivated land shelters by fusing UAV laser point clouds and images according to the embodiments of the present application described with reference to the above figures. The storage medium includes, but is not limited to, for example, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and cache memory, etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc.

[0043] In addition, according to the embodiments of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the present application provides a non-transitory machine-readable storage medium storing machine-readable instructions that can be run by a processor to execute instructions corresponding to the method steps provided by the present application, such as: the method for cross-measuring cultivated land shelters by fusing UAV laser point clouds and images. When the computer program is executed by a central processing unit (CPU), the above functions defined in the method of the present application are executed.

[0044] The above is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, and all should be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

[0045] Finally: The above is only the preferred embodiment of the present invention and is not used to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A cross-measurement method for cultivated land shelters by fusing UAV lidar point clouds and images, characterized in that, Including: S1: Collect cultivated land point cloud data and cultivated land image data; S2: Use spatial registration technology and attribute mapping technology to fuse the cultivated land point cloud data and the cultivated land image data to generate fused point cloud data; S3: Use radar feature fusion technology to perform ground object classification on the fused point cloud data to identify the first green land data in the fused point cloud data; Calculate the visible light vegetation index and identify the second green land data in the fused point cloud data through threshold segmentation; S4: Integrate the first green land data and the second green land data to generate green land point cloud data, and perform regionalization on the green land point cloud data based on the spatial clustering algorithm to form discrete green land areas; S5: Use suspension discrimination technology to perform vertical structure analysis on the green land area, identify the suspension areas in the green land area, and delete the suspension areas from the green land area to obtain cultivated land area data; S6: Generate an orthophoto of the cultivated land area based on the cultivated land area data using point cloud reverse modeling technology.

2. The cross-measurement method of cultivated land shelters by fusing UAV laser point cloud and image according to claim 1, characterized in that, The plowing point cloud data includes point cloud points, where is an integer greater than 1; each point cloud point includes coordinate data and echo data, and the coordinate data includes coordinate values, coordinate values, and coordinate values, and the echo data includes echo intensity and echo time; The cultivated land image data includes pixel points, being an integer greater than 1; each pixel point includes planar coordinate data and color data; the planar coordinate data includes coordinate values and coordinate values, and the color data includes , , surface reflectance of three bands; The method for generating the fused point cloud data includes: Use point cloud processing software and aerial survey software to convert the cultivated land point cloud data and the cultivated land image data into the same coordinate system; use the feature point matching algorithm to perform rough registration on the cultivated land point cloud data and the cultivated land image data in the same coordinate system to obtain a preliminary transformation matrix; use the iterative closest point algorithm to iteratively optimize the preliminary transformation matrix to obtain a fine transformation matrix; project each point cloud point in the cultivated land point cloud data into the cultivated land image data according to the fine transformation matrix to establish a spatial correspondence relationship between the point cloud points and the pixel points; according to the spatial correspondence relationship, use a high-order interpolation algorithm to map the color data of each pixel point to the corresponding point cloud point to obtain mapped points containing coordinate data, echo data, and color data; fuse all the mapped points to generate fused point cloud data.

3. The cross-measurement method of cultivated land shelters by fusing UAV laser point cloud and images according to claim 2, wherein The steps for identifying the first green land data include: Step S101: Construct multiple fuzzy sets for the echo intensity and echo time respectively; Step S102: Convert the echo data of each mapped point into the membership degrees of the corresponding fuzzy sets through fuzzy technology respectively; Step S103: Define fuzzy rules; Step S104: Match the fuzzy echo data with the fuzzy rules respectively and perform fuzzy inference using the fuzzy inference method to obtain the fuzzy inference results corresponding to each mapped point. The fuzzy inference results are the membership degrees corresponding to each ground object type, and the ground object types include ground points and vegetation points; Step S105: Compare each membership degree in each fuzzy inference result, and take the ground object type corresponding to the membership degree with the largest value as the ground object type of the corresponding mapped point; Step S106: Take the mapped points with the ground object type of vegetation points in the fused point cloud data as the first green land data.

4. The cross-measurement method for cultivated land shelters by fusing UAV laser point clouds and images according to claim 3, wherein The method for identifying the second green land data includes: Correspond each mapping point Multiply the surface reflectance of each band by 2 and subtract the corresponding surface reflectance of the band, and then subtract the corresponding surface reflectance of the band to obtain the green differential reflectance of each mapping point; correspond each mapping point Multiply the surface reflectance of each band by 2 and add the corresponding surface reflectance of the band, and then add the corresponding surface reflectance of the band to obtain the composite reflectance of each mapping point; divide the green differential reflectance of each mapping point by the corresponding composite reflectance to obtain the visible light vegetation index of each mapping point; preset a vegetation threshold and compare the visible light vegetation index of each mapping point with the vegetation threshold respectively; if the visible light vegetation index is greater than the vegetation threshold, mark the corresponding mapping point as a green object point; if the visible light vegetation index is less than or equal to the vegetation threshold, do not mark the corresponding mapping point; take all the green object points in the fused point cloud data as the second green land data.

5. The cross-measurement method of cultivated land shelters by fusing UAV laser point cloud and images according to claim 4, characterized in that The method for generating the green land point cloud data includes: For each vegetation point in the first green space data, pair it with each green object point in the second green space data to form a set of point pairs, that is, each set of point pairs includes one vegetation point and one green object point; analyze each set of point pairs in sequence. If the vegetation point and the green object point in the point pair are the same mapping point, mark the corresponding set of point pairs as a reserved set. If the vegetation point and the green object point in the point pair are not the same mapping point, do not mark the corresponding set of point pairs; obtain the vegetation points in each reserved set and mark them all as green plant points; generate green point cloud data based on all the green plant points.

6. The cross-measurement method for cultivated land shelters by fusing UAV laser point cloud and images according to claim 5, wherein The steps of forming discrete green areas include: Step S201: According to the coordinate data of each green plant point, calculate the Euclidean distance between every two green plant points in sequence and mark it as the node distance; Step S202: Randomly select a green plant point that has not been marked as a selected point as the current point; Step S203: According to the node distance corresponding to the current point, obtain the adjacent points corresponding to the current point; Step S204: Count the number of adjacent points corresponding to the current point and mark it as the adjacent quantity; judge whether to mark the current point as a core point or an edge point according to the adjacent quantity; Step S205: If the current point is marked as a core point, create a new point cluster, add the current point and the corresponding adjacent points to the new point cluster, and proceed to Step S206; if the current point is marked as an edge point, do not create a new point cluster, mark the current point as a selected point, and jump back to Step S202; Step S206: Expand the new point cluster and mark all the green plant points in the new point cluster as selected points; Step S207: Loop Steps S202 to S206 until all green plant points are marked as selected points, the loop ends, and obtain the green plant points in each point cluster; Step S208: Filter the green plant points in each point cluster, and use the green plant points in each filtered point cluster as a discrete green area, that is, the green areas correspond one-to-one with the point clusters.

7. The cross-measurement method of cultivated land shelters by fusing UAV laser point cloud and image according to claim 6, characterized in that, In Step S203, the method of obtaining the adjacent points corresponding to the current point is: compare the node distance corresponding to the current point with the preset maximum distance in sequence, mark the green plant points with a node distance less than or equal to the maximum distance as adjacent points, and do not mark the green plant points with a node distance greater than the maximum distance; In Step S204, the method of judging whether to mark the current point as a core point or an edge point is: add one to the adjacent quantity and compare it with the preset minimum number of points; If the adjacent quantity is greater than or equal to the minimum number of points, mark the current point as a core point; if the adjacent quantity is less than the minimum number of points, mark the current point as an edge point; In Step S206, the steps of expanding the new point cluster include: Step S301: Obtain the adjacent points corresponding to all the green plant points in the new point cluster and mark them as adjacent points; Judge whether each adjacent point is marked as a core point in sequence, mark the adjacent points marked as core points as adjacent core points; obtain the adjacent points corresponding to each adjacent core point and mark them as neighborhood points; add each adjacent core point and the corresponding neighborhood points to the new point cluster as well; Step S302: Loop step S301 until no new green plant points are added to the new point clusters, then the loop ends and the expansion of the new point clusters is completed. In step S208, the method of filtering the green plant points in each point cluster is as follows: Calculate the node distances between every two green plant points in each point cluster in sequence, compare the node distances corresponding to each green plant point, and take the node distance with the smallest value as the minimum distance corresponding to the green plant point; Compare the minimum distance of each green plant point with the preset distance threshold respectively, delete the green plant points whose minimum distance is greater than or equal to the distance threshold from the corresponding point cluster, and retain the green plant points whose minimum distance is less than the distance threshold in the corresponding point cluster.

8. The cross-measurement method for cultivated land shelters by fusing UAV laser point clouds and images according to claim 7, characterized in that, The method for identifying the suspended area includes: In the first green space data, vegetation points not marked as green plant points are also regarded as ground points; based on all ground points, cultivated land ground data is generated; based on the cultivated land ground data, a weighted interpolation algorithm is used to generate a ground elevation grid, where each grid cell contains a ground elevation value, and the ground elevation value is the vertical height of the ground point relative to the mean sea level; using a basic interpolation algorithm, the ground elevation value corresponding to each green plant point is calculated from the ground elevation grid; for each green plant point, the coordinate values are respectively subtracted by the corresponding ground elevation values to obtain the relative height difference of each green plant point; based on the ground elevation grid, a window analysis algorithm is used to generate a ground slope grid, where each grid cell includes a slope value, and the slope value is the inclination angle of the ground point corresponding surface to the horizontal plane; using a basic interpolation algorithm, the slope value corresponding to each green plant point is calculated from the ground slope grid; Take the relative height difference and slope value of each green plant point as a set of green plant data, and the green plant data corresponds to the green plant point one by one; Input each set of green plant data into the trained suspended area recognition model respectively, and output the corresponding recognition label. The suspended area recognition model is a deep neural network model; The recognition label is the digital label corresponding to the recognition result, and different recognition results correspond to different digital labels. The recognition results include suspended and fixed; Obtain the recognition result corresponding to each set of green plant data according to the recognition label corresponding to each set of green plant data; If the recognition result is suspended, mark the green plant point corresponding to the corresponding green plant data as a suspended point. If the recognition result is fixed, do not mark the green plant point corresponding to the corresponding green plant data; Perform regional processing on all suspended points to form discrete suspended areas; The method for forming discrete suspended areas is the same as the method for forming discrete green areas.

9. The cross-measurement method for cultivated land shelters by fusing UAV laser point clouds and images according to claim 8, characterized in that, The training process of the suspended area recognition model includes: Pre-collect a set of different green plant data, for each set of green plant data, set a corresponding recognition label, where \(n\) is an integer greater than 1, convert the green plant data and the corresponding recognition label into a corresponding set of feature vectors; use each set of feature vectors as the input of the floating recognition model, and the floating recognition model outputs a corresponding set of predicted recognition labels for each set of green plant data, use the actual recognition label corresponding to each set of green plant data as the prediction target, and the actual recognition label is the recognition label preset corresponding to the green plant data; use minimizing the sum of prediction errors of all green plant data as the training target; train the floating recognition model until the sum of prediction errors reaches convergence and then stop training; The method for obtaining the cultivated land area data includes: Delete the suspended area from the green area, and obtain the cultivated land area data according to the ground points and the remaining green plant points in the green area.

10. The cross-measurement method of cultivated land shelters by fusing UAV laser point clouds and images according to claim 9, characterized in that The method for generating the orthophoto of the cultivated land area includes: According to the cultivated land image data, use the structure from motion reconstruction technology and multi-view stereo vision technology for photogrammetric modeling to generate the original oblique model; According to the mapping points in the cultivated land area data, use the triangulation method to construct a triangular mesh; Use the texture mapping method to map the image texture information in the original oblique model to the triangular mesh to reconstruct the oblique three-dimensional model; According to the oblique three-dimensional model, use the orthographic projection method to generate the orthophoto of the cultivated land area without vegetation occlusion.

Images