Cross-measurement method of farmland shelterbelt by fusing unmanned aerial vehicle laser point cloud and image

Abstract

The application belongs to the technical field of remote sensing measurement, and discloses a farmland shelter cross measurement method fusing unmanned aerial vehicle laser point cloud and image; the method comprises the following steps: collecting farmland point cloud data and farmland image data and fusing to generate fused point cloud data; processing the fused point cloud data, and identifying first green land data in the fused point cloud data; calculating a visible light vegetation index, and identifying second green land data in the fused point cloud data; integrating the first green land data and the second green land data to generate green land point cloud data, and performing regionalization processing to form discrete green land regions; performing vertical structure analysis on the green land regions, identifying and deleting a suspended region in the green land regions, and obtaining farmland region data; generating a farmland region orthographic image according to the farmland region data; the application can realize high-precision identification and removal of vegetation shelters in farmland, and avoid misjudgment in terrain undulation or dense vegetation regions.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of remote sensing measurement, more particularly, the present application relates to a cultivated land shelter cross measurement method fusing unmanned aerial vehicle laser point cloud and image. BACKGROUND

[0002] With the rapid development of unmanned aerial vehicle technology, especially in the wide application in the field of agriculture, the combination of unmanned aerial vehicle laser radar and image data provides a new technical means for farmland management, cultivated land monitoring and precise analysis of agricultural resources; laser radar can provide accurate terrain and object three-dimensional information, while image data can capture the detailed appearance and texture characteristics of the object; the combination of the two not only improves the spatial accuracy of the data, but also makes up for the shortcomings of a single data source; in the process of cultivated land shelter detection and removal, traditional remote sensing images often have difficulty in accurately identifying the sheltering effect caused by vegetation, especially in complex terrain and high vegetation coverage areas; laser point cloud data can provide higher ground resolution and accurately measure object height information, and can distinguish between ground and non-ground objects, but its ability to identify details (such as vegetation type and accurate boundary of shelter) is weak; therefore, an intelligent cultivated land shelter measurement and removal method is urgently needed, which can effectively combine laser point cloud and high-resolution image data to improve the identification accuracy and accuracy of cultivated land shelter.

[0003] The patent with publication number CN111487643A discloses a building detection method based on laser radar point cloud and near-infrared image; including: obtaining orthophoto with near-infrared band, laser radar point cloud, and registering and fusing them; calculating the normalized difference vegetation index NDVI of each laser point after fusion, and completing vegetation identification based on NDVI and support vector machine classifier; for non-vegetation laser points, completing building identification through nearest neighbor search algorithm and height threshold; for building laser points, extracting roof seed points and candidate facade points, and obtaining roof point group based on individual building based on roof seed points; based on the roof point group of each building, estimating the vertical facade of the building; based on the candidate facade points and the estimated vertical facade, performing precise extraction of the facade points; completing three-dimensional building detection through the roof points and the precisely extracted facade points; this invention can effectively improve the detection accuracy of buildings and ensure a high level of detail of the building model.

[0004] However, the above-mentioned technology, although through the fusion of laser point cloud and image, realizes vegetation identification, but only uses NDVI and support vector machine classifier; in the cultivated land scene, single NDVI cannot fully distinguish vegetation and ground, and the generalization ability of the support vector machine classifier is insufficient in complex terrain, which has the risk of misjudgment; especially when the spectral characteristics of vegetation and ground are close, it is easy to cause false detection or missed detection, reducing the vegetation identification accuracy.

[0005] In view of this, the present application proposes a farmland shelter cross-measurement method fusing unmanned aerial vehicle laser point cloud and image to solve the above problems. SUMMARY

[0006] In order to overcome the above-mentioned defects of the prior art and achieve the above-mentioned purposes, the present application provides the following technical scheme: a farmland shelter cross-measurement method fusing unmanned aerial vehicle laser point cloud and image, comprising:

[0007] S1: collecting farmland point cloud data and farmland image data;

[0008] S2: using spatial registration technology and attribute mapping technology to fuse the farmland point cloud data and the farmland image data to generate fused point cloud data;

[0009] S3: using radar feature fusion technology to perform feature classification processing on the fused point cloud data to identify first green land data in the fused point cloud data; calculating a visible light vegetation index and identifying second green land data in the fused point cloud data through threshold segmentation method;

[0010] S4: integrating the first green land data and the second green land data to generate green land point cloud data, and performing regionalization processing on the green land point cloud data based on a spatial clustering algorithm to form discrete green land regions;

[0011] S5: using a suspension discrimination technology to perform vertical structure analysis on the green land regions to identify a suspension region in the green land regions, and deleting the suspension region from the green land regions to obtain farmland region data;

[0012] S6: generating a farmland region orthographic image using point cloud reverse modeling technology according to the farmland region data.

[0013] Further, the farmland point cloud data comprises a point cloud point, an integer greater than 1; each point cloud point comprises coordinate data and echo data, the coordinate data comprising a coordinate value, a coordinate value, and a coordinate value, the echo data comprising echo intensity and echo time;

[0014] The farmland image data comprises a pixel point, an integer greater than 1; each pixel point comprises planar coordinate data and color data; the planar coordinate data comprising a coordinate value, and a coordinate value, the color data comprising , , three-band ground reflectance;

[0015] The method for generating the fusion point cloud data comprises:

[0016] The point cloud data and the image data of the farmland are converted into the same coordinate system by using point cloud processing software and aerial survey software; the point cloud data and the image data of the farmland in the same coordinate system are coarsely registered by using a feature point matching algorithm to obtain a preliminary transformation matrix; the preliminary transformation matrix is iteratively optimized by using an iterative closest point algorithm to obtain a fine transformation matrix; each point cloud point in the point cloud data of the farmland is projected into the image data of the farmland according to the fine transformation matrix to establish a spatial correspondence between the point cloud points and the pixel points; the color data of each pixel point is mapped into the corresponding point cloud point by using a high-order interpolation algorithm according to the spatial correspondence to obtain a mapping point containing coordinate data, echo data and color data; and all the mapping points are fused to generate the fusion point cloud data.

[0017] Further, the step of identifying the first green land data comprises:

[0018] Step S101: constructing a plurality of fuzzy sets for echo intensity and echo time respectively;

[0019] Step S102: converting the echo data of each mapping point into the membership degrees of the corresponding fuzzy sets by using a fuzzification technique respectively;

[0020] Step S103: defining fuzzy rules;

[0021] Step S104: matching each set of the fuzzified echo data with the fuzzy rules respectively, and performing fuzzy reasoning by using a fuzzy reasoning method to obtain the fuzzy reasoning result corresponding to each mapping point, wherein the fuzzy reasoning result is the membership degree corresponding to each ground object type, and the ground object types include ground points and vegetation points;

[0022] Step S105: comparing each membership degree in each fuzzy reasoning result, and taking the ground object type corresponding to the membership degree with the largest value as the ground object type of the corresponding mapping point;

[0023] Step S106: taking all the mapping points with the vegetation point as the first green land data in the fusion point cloud data.

[0024] Further, the method for identifying the second green land data comprises:

[0025] The ground surface reflectivity of each mapping point corresponding to the waveband is multiplied by 2, and the ground surface reflectivity of the corresponding waveband is subtracted, and the ground surface reflectivity of the corresponding waveband is subtracted again to obtain the green difference reflectivity of each mapping point; the ground surface reflectivity of each mapping point corresponding to the waveband is multiplied by 2, and the ground surface reflectivity of the corresponding The surface reflectance of the band, plus the corresponding The surface reflectance of the band is used to obtain the synthetic reflectance of each mapping point; the green difference reflectance of each mapping point is divided by the corresponding synthetic reflectance to obtain the visible light vegetation index of each mapping point; a preset vegetation threshold is set, and the visible light vegetation index of each mapping point is compared with the vegetation threshold; if the visible light vegetation index is greater than the vegetation threshold, the corresponding mapping point is marked as a green object point; if the visible light vegetation index is less than or equal to the vegetation threshold, the corresponding mapping point is not marked; all green objects points in the fused point cloud data are used as the second green area data.

[0026] Furthermore, the method for generating green location cloud data includes:

[0027] Each vegetation point in the first green space data and each green object point in the second green space data are grouped into a point set, i.e., each point set includes one vegetation point and one green object point. Each point set is analyzed sequentially. If the vegetation point and the green object point in the point set are the same mapping point, the corresponding point set is marked as a reserved set. If the vegetation point and the green object point in the point set are not the same mapping point, the corresponding point set is not marked. The vegetation points in each reserved set are obtained and marked as green vegetation points. Based on all the green vegetation points, green space point cloud data is generated.

[0028] Furthermore, the step of forming discrete green areas includes:

[0029] Step S201: Based on the coordinate data of each greening point, calculate the Euclidean distance between every two greening points in sequence, and mark it as the node distance;

[0030] Step S202: Randomly select a green plant spot that has not been marked as a selected point as the current point;

[0031] Step S203: Based on the node distance corresponding to the current point, obtain the neighboring points corresponding to the current point;

[0032] Step S204: Count the number of adjacent points corresponding to the current point and mark them as adjacent points; determine whether the current point is marked as a core point or an edge point based on the adjacent points;

[0033] Step S205: If the current point is marked as a core point, a new point cluster is created, and the current point and its corresponding neighboring points are added to the new point cluster, and then proceed to step S206; if the current point is marked as an edge point, no new point cluster is created, and the current point is marked as a selected point, and the process jumps back to step S202.

[0034] Step S206: Expand the new point cluster and mark all green plant points in the new point cluster as selected points;

[0035] Step S207: Circulating step S202 to step S206 until all greenery points are marked as selected points, and the circulation ends, and the greenery points in each point cluster are obtained;

[0036] Step S208: Filtering the greenery points in each point cluster, and taking the greenery points in each filtered point cluster as a discrete green region, that is, the green region corresponds to the point cluster one by one.

[0037] Further, in the step S203, the method for obtaining the adjacent points corresponding to the current point is: comparing the node distance corresponding to the current point with the preset maximum distance in turn, marking the greenery points corresponding to the node distance less than or equal to the maximum distance as adjacent points, and not marking the greenery points corresponding to the node distance greater than the maximum distance;

[0038] In the step S204, the method for judging whether to mark the current point as a core point or an edge point is: comparing the adjacent number plus one with the preset minimum point number; if the adjacent number is greater than or equal to the minimum point number, the current point is marked as a core point; if the adjacent number is less than the minimum point number, the current point is marked as an edge point;

[0039] In the step S206, the step of expanding the new point cluster includes:

[0040] Step S301: Obtaining all adjacent points corresponding to the greenery points in the new point cluster, and marking them as adjacent points; judging whether each adjacent point is marked as a core point in turn, marking the adjacent points marked as core points as adjacent core points; obtaining the adjacent points corresponding to each adjacent core point, and marking them as adjacent domain points; and adding each adjacent core point and the corresponding adjacent domain points to the new point cluster;

[0041] Step S302: Circulating step S301 until there is no new greenery point added to the new point cluster, and the circulation ends, and the expansion of the new point cluster is completed;

[0042] In the step S208, the method for filtering the greenery points in each point cluster is: calculating the node distance between each two greenery points in each point cluster in turn, comparing the node distance corresponding to each greenery point, taking the minimum node distance as the minimum distance of the corresponding greenery point; comparing the minimum distance of each greenery point with the preset distance threshold value respectively, deleting the greenery points with the minimum distance greater than or equal to the distance threshold value from the corresponding point cluster, and retaining the greenery points with the minimum distance less than the distance threshold value in the corresponding point cluster.

[0043] Further, the method for identifying the floating region includes:

[0044] In the first green space data, vegetation points not marked as greening points are also treated as ground points; based on all ground points, farmland ground data is generated; based on the farmland ground data, a weighted interpolation algorithm is used to generate a ground elevation grid, where each grid cell contains a ground elevation value, which is the vertical height of the ground point relative to mean sea level; a basic interpolation algorithm is used to calculate the ground elevation value corresponding to each greening point from the ground elevation grid; and each greening point... The coordinate values ​​are subtracted from the corresponding ground elevation values ​​to obtain the relative height difference of each greening point; based on the ground elevation grid, a window analysis algorithm is used to generate a ground slope grid, where each grid cell includes a slope value, which is the angle of inclination between the ground surface and the horizontal plane corresponding to the ground point; a basic interpolation algorithm is used to calculate the slope value corresponding to each greening point from the ground slope grid.

[0045] The relative height difference and slope value of each greening point are used as a set of greening data, with a one-to-one correspondence between the greening data and the greening points. Each set of greening data is input into a trained floating recognition model, which outputs a corresponding recognition label. The floating recognition model is a deep neural network model. The recognition label is a numerical label corresponding to the recognition result. Different recognition results correspond to different numerical labels, and the recognition results include floating and fixed. Based on the recognition label corresponding to each set of greening data, the recognition result corresponding to each set of greening data is obtained. If the recognition result is floating, the greening point corresponding to the corresponding greening data is marked as a floating point. If the recognition result is fixed, the greening point corresponding to the corresponding greening data is not marked. All floating points are regionalized to form discrete floating regions. The method for forming discrete floating regions is the same as the method for forming discrete green areas.

[0046] Furthermore, the training process of the levitation recognition model includes:

[0047] Pre-collection Data on different types of green plants Each group of green plant data was assigned a corresponding identification tag. For integers greater than 1, convert the green plant data and corresponding identification labels into a set of feature vectors; use each set of feature vectors as input to the floating identification model, the floating identification model takes a set of predicted identification labels corresponding to each set of green plant data as output, and takes the actual identification labels corresponding to each set of green plant data as the prediction target, the actual identification labels are the pre-set identification labels corresponding to the green plant data; the training objective is to minimize the sum of prediction errors of all green plant data; train the floating identification model until the sum of prediction errors converges and then stop training;

[0048] The method for obtaining cultivated land area data includes:

[0049] The suspended area is deleted from the green area, and the cultivated area data is obtained according to the ground points and the remaining green points in the green area.

[0050] Further, the method for generating the cultivated area orthographic image comprises:

[0051] According to the cultivated land image data, the photogrammetry modeling is performed by using the motion structure reconstruction technology and the multi-view stereo vision technology to generate the original tilt model; according to the mapping points in the cultivated area data, the triangular mesh is constructed by using the triangular subdivision method; the image texture information in the original tilt model is mapped to the triangular mesh by using the texture mapping method to reconstruct the tilt three-dimensional model; and the cultivated area orthographic image without vegetation shelter is generated by using the orthographic projection method according to the tilt three-dimensional model.

[0052] The technical effects and advantages of the cultivated land shelter cross measurement method fusing the unmanned aerial vehicle laser point cloud and the image are as follows:

[0053] By fusing the cultivated point cloud data and the cultivated image data, the respective advantages of the laser point cloud and the high-resolution image data can be fully utilized to improve the spatial accuracy and the detail description capability of the data; by combining the radar feature fusion technology, the fuzzy inference method and the double vegetation identification strategy of the visible light index calculation, and by using the suspended discrimination technology to vertically structure the green area, the fused point cloud data can be effectively classified, the limitation that the single use of the NDVI and the support vector machine classifier cannot fully distinguish the vegetation and the ground is overcome, and the high-precision identification and removal of the vegetation shelter in the cultivated land are realized; finally, the point cloud reverse modeling technology is used to generate the cultivated area orthographic image without vegetation shelter, the technical problems that the vegetation identification is inaccurate and the shelter removal is incomplete in the traditional method in the complex cultivated land environment are effectively solved, the misjudgment in the uneven light, the terrain undulation and the dense vegetation area is avoided, and more accurate, clear and reliable cultivated land image data basis is provided for the agricultural precision monitoring, the land resource evaluation, the crop growth analysis and the production decision. BRIEF DESCRIPTION OF DRAWINGS

[0054] Figure 1 The fusion unmanned aerial vehicle laser point cloud and image cultivated land shelter cross measurement method flowchart of the embodiment 1 of the present application. DETAILED DESCRIPTION

[0055] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the protection scope of the present application.

[0056] Embodiment 1

[0057] Referring to Figure 1 The embodiment fuses the cultivated land shelter cross-measurement method of the unmanned aerial vehicle laser point cloud and the image, and the method comprises the following steps:

[0058] S1: Collecting cultivated land point cloud data and cultivated land image data.

[0059] The cultivated land point cloud data is three-dimensional space data, which is obtained by a laser radar device carried on the unmanned aerial vehicle; the cultivated land point cloud data comprises a point cloud point, which is an integer greater than 1; each point cloud point comprises coordinate data and echo data, the coordinate data comprises a coordinate value, a coordinate value and a coordinate value, and the echo data comprises echo intensity and echo time; the echo intensity is the signal intensity when the laser pulse is reflected back to the laser radar device, and the echo time is the time experienced by the laser pulse from emission to return to the laser radar device; the cultivated land point cloud data can accurately reflect the three-dimensional structure of the ground and the ground object, and can also describe the spatial structure of the cultivated land and the vegetation in detail; it should be noted that the cultivated land point cloud data is the point cloud data for the ground elevation, and therefore for each planar position , there is only one corresponding coordinate value, that is, there is no stacking of multiple point cloud points at different heights at the same planar position .

[0060] The cultivated land image data is two-dimensional image data, which usually has a spatial resolution of centimeters, and is obtained by an optical camera carried on the unmanned aerial vehicle; the cultivated land image data comprises a pixel point, which is an integer greater than 1; each pixel point comprises planar coordinate data and color data; the planar coordinate data comprises a coordinate value and a coordinate value, and the color data comprises , , three-band ground reflectance, i.e. two-band ground reflectance, one-band ground reflectance and one-band ground reflectance; the cultivated land image data can provide rich ground visual information and effectively reflect the visual color characteristics of the ground object, and can be used for ground object identification and vegetation analysis.

[0061] S2: Using spatial registration technology and attribute mapping technology to fuse the cultivated land point cloud data and the cultivated land image data to generate fused point cloud data.

[0062] The method for generating the fused point cloud data comprises:

[0063] The point cloud data and the image data of the farmland are converted into the same coordinate system by using point cloud processing software (such as CloudCompare, TerraScan, etc.) and aerial survey software (such as Pix4D, Agisoft Metashape, etc.); the point cloud data and the image data of the farmland in the same coordinate system are coarsely registered by using a feature point matching algorithm (such as SIFT, SURF, etc.), to obtain a preliminary transformation matrix; the preliminary transformation matrix is iteratively optimized by using an iterative closest point algorithm, to obtain a fine transformation matrix; each point cloud point in the point cloud data of the farmland is projected into the image data of the farmland according to the fine transformation matrix, to establish a spatial correspondence relationship between the point cloud points and the pixel points, that is, a corresponding pixel point of each point cloud point in the two-dimensional space; the color data of each pixel point is mapped into the corresponding point cloud point according to the spatial correspondence relationship by using a high-order interpolation algorithm (such as a spline interpolation algorithm, a bicubic interpolation algorithm, etc.), to obtain a mapping point containing coordinate data, echo data and color data; all the mapping points are fused to generate fused point cloud data; the iterative closest point algorithm is prior art, and the specific process is not described in detail here.

[0064] S3: performing ground object classification processing on the fused point cloud data by using a radar feature fusion technology, to identify first green land data in the fused point cloud data; calculating a visible light vegetation index, and identifying second green land data in the fused point cloud data by threshold segmentation.

[0065] The step of identifying the first green land data comprises:

[0066] Step S101: constructing a plurality of fuzzy sets for echo intensity and echo time respectively; for example, the fuzzy set corresponding to echo intensity is low intensity, medium intensity and high intensity, and the fuzzy set corresponding to echo time is long time, medium time and short time.

[0067] Step S102: converting the echo data of each mapping point into the membership degree of each corresponding fuzzy set by using a fuzzification technology; the fuzzification technology is a process of converting an accurate value into the membership degree corresponding to the fuzzy set, and the fuzzification technology is, for example, a triangular membership function, a trapezoidal membership function, etc.; for example, if the value of echo intensity is low, the membership degree of low intensity is 0.9, the membership degree of medium intensity is 0.1, and the membership degree of high intensity is 0.

[0068] Step S103: define the fuzzy rule, the fuzzy rule is defined according to expert knowledge or related literature; for example, if the echo intensity is high and the echo time is long, it is inferred that the corresponding mapping point has a high probability of being a ground point, that is, the corresponding mapping point is on the ground surface of the cultivated land; if the echo intensity is low and the echo time is short, it is inferred that the corresponding mapping point has a high probability of being a vegetation point, that is, the corresponding mapping point is on the vegetation of the cultivated land; it should be understood that since the unmanned aerial vehicle usually carries a short-wave radar, its penetration is weak, and the laser wave is usually difficult to penetrate the vegetation layer; and since the height of the vegetation in the cultivated land scene is generally higher than the ground, when the laser echo comes from the vegetation surface, the corresponding echo time is shorter than that of the echo from the ground surface; and since the vegetation structure is complex and irregular, the laser is easy to scatter and attenuate on the vegetation surface multiple times, resulting in a relatively low echo intensity; and the ground is relatively flat, usually exhibiting mirror or near-mirror reflection, and the laser energy is concentrated and returned, so the laser echo from the ground surface has a higher echo intensity;

[0069] Step S104: match each set of fuzzy echo data with the fuzzy rule respectively, and use a fuzzy reasoning method (such as a Mamdani fuzzy reasoning model, a Sugeno fuzzy reasoning model, etc.) to perform fuzzy reasoning to obtain a fuzzy reasoning result corresponding to each mapping point, the fuzzy reasoning result being a membership degree corresponding to each ground object type, the ground object type including a ground point and a vegetation point; for example, the membership degree of the ground point is 0.2 and the membership degree of the vegetation point is 0.8;

[0070] Step S105: compare each membership degree in each fuzzy reasoning result, and take the ground object type corresponding to the membership degree with the largest value as the ground object type of the corresponding mapping point.

[0071] Step S106: take all mapping points of the vegetation point in the fused point cloud data as the first green land data.

[0072] The method for identifying the second green land data comprises:

[0073] The ground surface reflectivity of each mapping point corresponding to the waveband is multiplied by 2, and the ground surface reflectivity of the corresponding waveband is subtracted, and the ground surface reflectivity of the corresponding waveband is subtracted, to obtain the green difference reflectivity of each mapping point; the ground surface reflectivity of each mapping point corresponding to the waveband is multiplied by 2, and the ground surface reflectivity of the corresponding waveband is added, and the ground surface reflectivity of the corresponding The surface reflectivity of the wave band is obtained, the synthetic reflectivity of each mapping point is obtained, the green difference reflectivity of each mapping point is divided by the corresponding synthetic reflectivity respectively to obtain the visible light vegetation index (i.e. the visible light wave band difference vegetation index) of each mapping point, a vegetation threshold is preset, the visible light vegetation index of each mapping point is compared with the vegetation threshold, if the visible light vegetation index is greater than the vegetation threshold, the corresponding mapping point is marked as a green point, if the visible light vegetation index is less than or equal to the vegetation threshold, the corresponding mapping point is not marked, and all green points in the fused point cloud data are taken as the second green land data.

[0074] S4: The first green land data and the second green land data are integrated to generate green point cloud data, and the green point cloud data is regionized based on a spatial clustering algorithm to form discrete green regions.

[0075] The method for generating the green point cloud data comprises:

[0076] Each vegetation point in the first green land data is taken as a point set together with each green point in the second green land data, i.e. each point set comprises one vegetation point and one green point, each point set is analyzed in sequence, if the vegetation point and the green point in the point set are the same mapping point, the corresponding point set is marked as a reserved set, if the vegetation point and the green point in the point set are not the same mapping point, the corresponding point set is not marked, the vegetation points in each reserved set are obtained and are all marked as green plant points, and the green point cloud data is generated according to all the green plant points.

[0077] The step of forming the discrete green regions comprises:

[0078] Step S201: The Euclidean distance between each two green plant points is calculated in sequence according to the coordinate data of each green plant point and is marked as a node distance; the calculation method of the Euclidean distance is a prior art and is not described in detail here;

[0079] Step S202: A green plant point that is not marked as a selected point is randomly selected as a current point;

[0080] Step S203: The adjacent points corresponding to the current point are obtained according to the node distance corresponding to the current point;

[0081] Step S204: The number of the adjacent points corresponding to the current point is counted and is marked as an adjacent number; the current point is marked as a core point or an edge point according to the adjacent number;

[0082] Step S205: If the current point is marked as a core point, a new point cluster is created, the current point and the corresponding adjacent points are added to the new point cluster, and step S206 is entered; if the current point is marked as an edge point, a new point cluster is not created, and the current point is marked as a selected point, and step S202 is returned;

[0083] Step S206: The new point cluster is expanded, and all green points in the new point cluster are marked as selected points.

[0084] Step S207: Steps S202 to S206 are cycled until all green points are marked as selected points, and the cycle is ended, and the green points in each point cluster are obtained.

[0085] Step S208: Each green point in each point cluster is filtered, and the green points in each filtered point cluster are taken as a discrete green region, that is, the green region and the point cluster are one-to-one corresponding.

[0086] In the above step S203, the method for obtaining the adjacent points corresponding to the current point is: the node distance corresponding to the current point is compared with the preset maximum distance in sequence, the green points corresponding to the node distance less than or equal to the maximum distance are marked as adjacent points, and the green points corresponding to the node distance greater than the maximum distance are not marked; the maximum distance is pre-set by the person skilled in the art according to the actual situation.

[0087] In the above step S204, the method for judging whether the current point is marked as a core point or an edge point is: the adjacent number is added by one and compared with the preset minimum point number; if the adjacent number is greater than or equal to the minimum point number, the current point is marked as a core point; if the adjacent number is less than the minimum point number, the current point is marked as an edge point; the minimum point number is pre-set by the person skilled in the art according to the actual situation.

[0088] In the above step S206, the step of expanding the new point cluster includes:

[0089] Step S301: Obtain all green points corresponding to the adjacent points in the new point cluster, and mark them as adjacent points; judge whether each adjacent point is marked as a core point in sequence, mark the adjacent points marked as core points as adjacent core points; obtain the adjacent points corresponding to each adjacent core point, and mark them as adjacent domain points; each adjacent core point and the corresponding adjacent domain points are also added to the new point cluster;

[0090] Step S302: Step S301 is cycled until there is no new green point added to the new point cluster, the cycle is ended, and the expansion of the new point cluster is completed.

[0091] In step S208 above, the method for filtering the green plant points in each point cluster is as follows: calculate the node distance between every two green plant points in each point cluster in turn, compare the node distances corresponding to each green plant point, and take the node distance with the smallest value as the minimum distance of the corresponding green plant point; compare the minimum distance of each green plant point with a preset distance threshold, delete the green plant points whose minimum distance is greater than or equal to the distance threshold from the corresponding point cluster, and retain the green plant points whose minimum distance is less than the distance threshold in the corresponding point cluster; the distance threshold is preset by those skilled in the art according to the actual situation.

[0092] S5: Use suspension discrimination technology to perform vertical structure analysis on green areas, identify suspended areas in green areas, delete suspended areas from green areas, and obtain farmland area data.

[0093] Methods for identifying floating regions include:

[0094] In the first green space data, vegetation points not marked as greening points are also treated as ground points. Based on all ground points, farmland ground data is generated. Using the farmland ground data, a weighted interpolation algorithm (such as IDW, TIN, Kriging, etc.) is used to generate a ground elevation grid, where each grid cell contains a ground elevation value, which is the vertical height of the ground point relative to mean sea level. A basic interpolation algorithm (such as nearest neighbor interpolation, bilinear interpolation, etc.) is used to calculate the ground elevation value corresponding to each greening point from the ground elevation grid. The ground elevation value for each greening point is then calculated. The coordinate values ​​are subtracted from the corresponding ground elevation values ​​to obtain the relative height difference of each greening point. Based on the ground elevation grid, a ground slope grid is generated using window analysis algorithms (such as the Horn algorithm, Zevenbergen & Thorne algorithm, etc.), where each grid cell includes a slope value, which is the angle of inclination between the ground surface and the horizontal plane corresponding to the ground point. Basic interpolation algorithms (such as nearest neighbor interpolation algorithm, bilinear interpolation algorithm, etc.) are used to calculate the slope value corresponding to each greening point from the ground slope grid.

[0095] The relative height difference of each green plant point and the slope value are taken as a set of green plant data, and the green plant data correspond one-to-one to the green plant points; each set of green plant data is input into the trained suspension identification model, and a corresponding identification label is output, the suspension identification model being a deep neural network model; the identification label is a digital label corresponding to the identification result, and different digital labels correspond to different identification results, the identification result including suspension and fixation; the identification result corresponding to each set of green plant data is obtained according to the identification label corresponding to each set of green plant data; if the identification result is suspension, the green plant point corresponding to the corresponding green plant data is marked as a suspension point, and if the identification result is fixation, the green plant point corresponding to the corresponding green plant data is not marked; all the suspension points are subjected to regionalization processing to form discrete suspension regions; and the method of forming the discrete suspension regions is the same as the method of forming the discrete green regions.

[0096] The training process of the suspension identification model includes:

[0097] Pre-collecting different sets of green plant data, analyzing each green plant point in each set of green plant data in turn, and determining whether each green plant point is a suspension point, i.e., determining whether the vegetation corresponding to the green plant point will shield the farmland, setting different identification labels for different identification results, and setting the identification labels for each set of green plant data in turn; being an integer greater than 1, converting the green plant data and the corresponding identification label into a corresponding set of feature vectors; the identification label corresponding to the green plant data is collected by a person skilled in the art in the process of historically identifying the suspension region different sets of green plant data, analyzing each green plant point in each set of green plant data in turn according to the actual situation, determining whether each green plant point is a suspension point, i.e., determining whether the vegetation corresponding to the green plant point will shield the farmland, setting different identification labels for different identification results, and setting the identification labels for each set of green plant data in turn;

[0098] Each set of feature vectors is taken as the input of the suspension identification model, a set of predicted identification labels corresponding to each set of green plant data is taken as the output of the suspension identification model, the actual identification label corresponding to each set of green plant data is taken as the prediction target, and the actual identification label is the identification label set in advance corresponding to the green plant data; the training target is to minimize the sum of the prediction errors of all green plant data; wherein the calculation formula of the prediction error is wherein is the prediction error, is the group number of the feature vector corresponding to the green plant data, is the predicted identification label corresponding to the th set of green plant data, is the actual identification label corresponding to the th set of green plant data; the suspension identification model is trained until the sum of the prediction errors converges.

[0099] The method for obtaining the farmland region data includes:

[0100] The suspended area is deleted from the green area, and the cultivated area data is obtained according to the ground points and the remaining green points in the green area.

[0101] S6: According to the cultivated area data, a cultivated area orthographic image is generated by using a point cloud reverse modeling technology.

[0102] The method for generating the cultivated area orthographic image comprises:

[0103] According to the cultivated land image data, a photogrammetry modeling is performed by using a motion structure reconstruction technology (SfM) and a multi-view stereo vision technology (MVS) to generate an original tilt model, wherein the original tilt model includes the three-dimensional geometric structure and image texture information of the ground, vegetation and other ground objects in the cultivated land; the motion structure reconstruction technology and the multi-view stereo vision technology are prior art, and the specific process is not described in detail here; according to the mapping points in the cultivated area data, a triangular mesh is constructed by using a triangular subdivision method (such as a constrained Delaunay triangular subdivision method, a Bowyer-Watson algorithm, etc.); the image texture information in the original tilt model is mapped to the triangular mesh by using a texture mapping method (such as a multi-band fusion texture mapping method, a spherical projection texture mapping method, a Poisson texture fusion method, etc.), a tilt three-dimensional model is reconstructed, and the reconstructed tilt three-dimensional model is a cultivated land surface model without vegetation shielding; according to the tilt three-dimensional model, a no-vegetation-shielding cultivated area orthographic image is generated by using an orthographic projection method (such as a Z-buffer orthographic projection method, a differential orthographic correction method, etc.).

[0104] The embodiment can fully exert the respective advantages of the laser point cloud and the high-resolution image data by fusing the cultivated land point cloud data and the cultivated land image data, and improve the spatial accuracy and the detail description capability of the data; and by combining the radar feature fusion technology, the fuzzy inference method and the double vegetation identification strategy of the visible light index calculation, and using the suspended discrimination technology to vertically structure the green area, the fused point cloud data can be effectively classified into ground objects, the limitations that the single use of the NDVI and the support vector machine classifier cannot fully distinguish the vegetation and the ground are overcome, and the high-precision identification and removal of the vegetation shielding object in the cultivated land are realized; finally, the point cloud reverse modeling technology is used to generate the no-vegetation-shielding cultivated area orthographic image, the technical problems that the vegetation identification is inaccurate and the shielding removal is incomplete in the traditional method in the complex cultivated land environment are effectively solved, the misjudgment in the terrain undulation or the dense vegetation area is avoided, and more accurate, clear and reliable cultivated land image data basis is provided for the agricultural precision monitoring, the land resource evaluation, the crop growth analysis and the production decision.

[0105] Embodiment 2

[0106] The application also provides an electronic device. The electronic device can include one or more processors and one or more memories. The memories can store computer readable code that, when executed by the one or more processors, can perform the method of cross-measuring farmland shelter by fusing unmanned aerial vehicle laser point cloud and image as described above.

[0107] The method or system according to the embodiments of the application can also be implemented by means of the architecture of the electronic device shown in the application. The electronic device can include a bus, one or more CPUs, ROM, RAM, a communication port connected to a network, input / output, a hard disk, etc. The storage device in the electronic device, such as ROM or hard disk, can store the method of cross-measuring farmland shelter by fusing unmanned aerial vehicle laser point cloud and image provided by the application. Further, the electronic device can also include a user interface. Of course, the architecture shown in the application is only exemplary, and when implementing different devices, one or more components of the electronic device shown in the application can be omitted according to actual needs.

[0108] Embodiment 3

[0109] Referring to the drawings, one embodiment of the application discloses a computer readable storage medium. The computer readable storage medium stores computer readable instructions. When the computer readable instructions are executed by a processor, the method of cross-measuring farmland shelter by fusing unmanned aerial vehicle laser point cloud and image according to the embodiments of the application described with reference to the above drawings can be performed. The storage medium includes but is not limited to, for example, volatile memory and / or non-volatile memory. The volatile memory may, for example, include random access memory (RAM) and cache memory, etc. The non-volatile memory may, for example, include read-only memory (ROM), hard disk, flash memory, etc.

[0110] In addition, according to the embodiments of the application, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, the application provides a non-transitory machine readable storage medium storing machine readable instructions executable by a processor to perform instructions corresponding to the method steps provided by the application, for example: the method of cross-measuring farmland shelter by fusing unmanned aerial vehicle laser point cloud and image. When the computer program is executed by a central processing unit (CPU), the above functions defined in the method of the application are performed.

[0111] The above description is only a specific embodiment of the application, but the protection scope of the application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by the application, which should be covered within the protection scope of the application. Therefore, the protection scope of the application should be subject to the protection scope of the claims.

[0112] Finally: the above only for the preferred embodiments of the present application, and not for limiting the present application, any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application, should be included in the scope of protection of the present application.

Claims

1. A method for cross-measurement of farmland shading by integrating UAV laser point clouds and imagery, characterized in that, include: S1: Collect cloud data and image data of cultivated land; S2: Using spatial registration and attribute mapping techniques, farmland point cloud data and farmland image data are fused to generate fused point cloud data; S3: Use radar feature fusion technology to classify ground features in the fused point cloud data and identify the first green area data in the fused point cloud data. Calculate the visible light vegetation index and identify the second green space data in the fused point cloud data using a threshold segmentation method; S4: Integrate the first green space data and the second green space data to generate green space location cloud data, and perform regionalization processing on the green space location cloud data based on spatial clustering algorithm to form discrete green space areas; S5: Use suspension discrimination technology to perform vertical structure analysis on green areas, identify suspended areas in green areas, delete suspended areas from green areas, and obtain farmland area data; Methods for identifying floating regions include: In the first green space data, vegetation points not marked as greening points are also treated as ground points. Based on all ground points, farmland ground data is generated. Using the farmland ground data, a weighted interpolation algorithm is used to generate a ground elevation grid, where each grid cell contains a ground elevation value, which is the vertical height of the ground point relative to mean sea level. A basic interpolation algorithm is used to calculate the ground elevation value corresponding to each greening point from the ground elevation grid. The relative height difference of each greening point is obtained by subtracting its corresponding ground elevation value from its Z-coordinate value. Based on the ground elevation grid, a window analysis algorithm is used to generate a ground slope grid, where each grid cell contains a slope value, which is the angle of inclination between the ground surface and the horizontal plane corresponding to the ground point. A basic interpolation algorithm is used to calculate the slope value corresponding to each greening point from the ground slope grid. The relative height difference and slope value of each greening point are used as a set of greening data, with a one-to-one correspondence between the greening data and the greening points. Each set of greening data is input into a trained floating recognition model, which outputs a corresponding recognition label. The floating recognition model is a deep neural network model. The recognition label is a numerical label corresponding to the recognition result. Different recognition results correspond to different numerical labels, and the recognition results include floating and fixed. Based on the recognition label corresponding to each set of greening data, the recognition result corresponding to each set of greening data is obtained. If the recognition result is floating, the greening point corresponding to the corresponding greening data is marked as a floating point. If the recognition result is fixed, the greening point corresponding to the corresponding greening data is not marked. All floating points are regionalized to form discrete floating regions. The method for forming discrete floating regions is the same as the method for forming discrete green areas. S6: Based on the farmland area data, generate orthophotos of the farmland area using point cloud reverse modeling technology.

2. The method for cross-measurement of farmland cover by fusing UAV laser point clouds and imagery according to claim 1, characterized in that, The cultivated land point cloud data includes a point cloud points, where a is an integer greater than 1; each point cloud point includes coordinate data and echo data, the coordinate data includes X coordinate value, Y coordinate value and Z coordinate value, and the echo data includes echo intensity and echo time; The farmland image data includes b pixels, where b is an integer greater than 1; each pixel includes planar coordinate data and color data; the planar coordinate data includes X coordinate value and Y coordinate value, and the color data includes the surface reflectance of the R, G, and B bands; The method for generating fused point cloud data includes: Point cloud processing software and aerial surveying software were used to transform farmland point cloud data and farmland image data into the same coordinate system. A feature point matching algorithm was employed to perform coarse registration of the farmland point cloud data and farmland image data in the same coordinate system, obtaining a preliminary transformation matrix. An iterative nearest-point algorithm was used to iteratively optimize the preliminary transformation matrix, obtaining a fine transformation matrix. Based on the fine transformation matrix, each point cloud point in the farmland point cloud data was projected onto the farmland image data, establishing a spatial correspondence between point cloud points and pixels. Based on this spatial correspondence, a high-order interpolation algorithm was used to map the color data of each pixel to the corresponding point cloud point, obtaining a mapping point containing coordinate data, echo data, and color data. All mapping points were then fused to generate fused point cloud data.

3. The method for cross-measurement of farmland cover by fusing UAV laser point clouds and imagery according to claim 2, characterized in that, The steps for identifying the first green space data include: Step S101: Construct multiple fuzzy sets for echo intensity and echo time respectively; Step S102: Convert the echo data of each mapping point into the membership degree of each corresponding fuzzy set using fuzzification technology; Step S103: Define fuzzy rules; Step S104: Match each set of fuzzy echo data with fuzzy rules and perform fuzzy inference using the fuzzy inference method to obtain the fuzzy inference result corresponding to each mapping point. The fuzzy inference result is the membership degree corresponding to each land cover type. Land cover types include ground points and vegetation points. Step S105: Compare each membership degree in each fuzzy inference result, and take the land cover type corresponding to the membership degree with the largest value as the land cover type of the corresponding mapping point; Step S106: Take all the mapping points of vegetation points in the fused point cloud data as the first green space data.

4. The method for cross-measurement of farmland cover by fusing UAV laser point clouds and imagery according to claim 3, characterized in that, Methods for identifying secondary green space data include: The surface reflectance of each mapping point is multiplied by 2 for the G-band, and then subtracted from the surface reflectance of the corresponding R-band and B-band to obtain the green differential reflectance of each mapping point. The surface reflectance of each mapping point is then multiplied by 2 for the G-band, and then added to the surface reflectance of the corresponding R-band and B-band to obtain the composite reflectance of each mapping point. The green differential reflectance of each mapping point is divided by the corresponding composite reflectance to obtain the visible light vegetation index of each mapping point. A preset vegetation threshold is set, and the visible light vegetation index of each mapping point is compared with the vegetation threshold. If the visible light vegetation index is greater than the vegetation threshold, the corresponding mapping point is marked as a green point; if the visible light vegetation index is less than or equal to the vegetation threshold, the corresponding mapping point is not marked. All green points in the fused point cloud data are used as the second green area data.

5. The method for cross-measurement of farmland cover by fusing UAV laser point clouds and imagery according to claim 4, characterized in that, The method for generating green location cloud data includes: Each vegetation point in the first green space data and each green object point in the second green space data are grouped into a point set, i.e., each point set includes one vegetation point and one green object point. Each point set is analyzed sequentially. If the vegetation point and the green object point in the point set are the same mapping point, the corresponding point set is marked as a reserved set. If the vegetation point and the green object point in the point set are not the same mapping point, the corresponding point set is not marked. The vegetation points in each reserved set are obtained and marked as green vegetation points. Based on all the green vegetation points, green space point cloud data is generated.

6. The method for cross-measurement of farmland cover by fusing UAV laser point clouds and imagery according to claim 5, characterized in that, The steps for forming discrete green areas include: Step S201: Based on the coordinate data of each greening point, calculate the Euclidean distance between every two greening points in sequence, and mark it as the node distance; Step S202: Randomly select a green plant spot that has not been marked as a selected point as the current point; Step S203: Based on the node distance corresponding to the current point, obtain the neighboring points corresponding to the current point; Step S204: Count the number of adjacent points corresponding to the current point and mark them as adjacent points; determine whether the current point is marked as a core point or an edge point based on the adjacent points; Step S205: If the current point is marked as a core point, a new point cluster is created, and the current point and its corresponding neighboring points are added to the new point cluster, and then proceed to step S206; if the current point is marked as an edge point, no new point cluster is created, and the current point is marked as a selected point, and the process jumps back to step S202. Step S206: Expand the new point cluster and mark all green plant points in the new point cluster as selected points; Step S207: Repeat steps S202 to S206 until all green plant points are marked as selected points, then the loop ends and the green plant points in each point cluster are obtained; Step S208: Filter the green plant points in each point cluster, and treat the green plant points in each filtered point cluster as a discrete green area, that is, the green area corresponds one-to-one with the point cluster.

7. The method for cross-measurement of farmland cover by fusing UAV laser point clouds and imagery according to claim 6, characterized in that, In step S203, the method for obtaining the adjacent points corresponding to the current point is as follows: the node distance corresponding to the current point is compared with the preset maximum distance in turn; the green plant points whose node distance is less than or equal to the maximum distance are marked as adjacent points; and the green plant points whose node distance is greater than the maximum distance are not marked. In step S204, the method for determining whether the current point is marked as a core point or an edge point is: increment the number of adjacent points by one and compare it with the preset minimum number of points; If the number of adjacent points is greater than or equal to the minimum number of points, the current point is marked as a core point; if the number of adjacent points is less than the minimum number of points, the current point is marked as an edge point. In step S206, the step of expanding the new point cluster includes: Step S301: Obtain the adjacent points corresponding to all green plant points in the new point cluster and mark them as adjacent points; For each adjacent point, determine whether it has been marked as a core point. Mark the adjacent points of the core points as neighboring core points. Obtain the adjacent points corresponding to each neighboring core point and mark them as neighborhood points. Add each neighboring core point and its corresponding neighborhood points to a new point cluster. Step S302: Repeat step S301 until no new green plant points are added to the new cluster, then the loop ends, completing the expansion of the new cluster; In step S208, the method for filtering the green plant points in each point cluster is as follows: calculate the node distance between every two green plant points in each point cluster in turn, compare the node distances corresponding to each green plant point, and take the node distance with the smallest value as the minimum distance of the corresponding green plant point; compare the minimum distance of each green plant point with a preset distance threshold, delete the green plant points with a minimum distance greater than or equal to the distance threshold from the corresponding point cluster, and retain the green plant points with a minimum distance less than the distance threshold in the corresponding point cluster.

8. The method for cross-measurement of farmland cover by fusing UAV laser point clouds and imagery according to claim 7, characterized in that, The training process of the levitation recognition model includes: Pre-collect r sets of different green plant data, and assign corresponding identification labels to each of the r sets of green plant data, where r is an integer greater than 1. Convert the green plant data and their corresponding identification labels into a set of feature vectors. Use each set of feature vectors as input to the floating identification model. The floating identification model takes a set of predicted identification labels corresponding to each set of green plant data as output and takes the actual identification labels corresponding to each set of green plant data as the prediction target. The actual identification labels are the pre-set identification labels corresponding to the green plant data. The training objective is to minimize the sum of prediction errors of all green plant data. Train the floating identification model until the sum of prediction errors converges, at which point training stops. The method for obtaining cultivated land area data includes: Remove the floating areas from the green areas, and obtain the farmland area data based on the ground points and the remaining green vegetation points in the green areas.

9. The method for cross-measurement of farmland cover by fusing UAV laser point clouds and imagery according to claim 8, characterized in that, Methods for generating orthophotos of cultivated land areas include: Based on farmland image data, photogrammetric modeling was performed using motion structure reconstruction technology and multi-view stereo vision technology to generate an original tilted model. Based on the mapping points in the farmland area data, a triangulation method was used to construct a triangular mesh. The texture mapping method was used to map the image texture information in the original tilted model to the triangular mesh to reconstruct the tilted 3D model. Based on the tilted 3D model, an orthophoto of the farmland area without vegetation obstruction was generated using the orthophoto projection method.

Images