Cotton canopy aggregation degree index calculation method based on unmanned aerial vehicle optical image

Abstract

The invention discloses a cotton canopy aggregation degree index calculation method based on an unmanned aerial vehicle optical image. The method comprises the following steps: acquiring an unmanned aerial vehicle image of a cotton canopy in a ground target grid by using an optical sensor carried by an unmanned aerial vehicle; segmenting the unmanned aerial vehicle image into a plurality of image grid units according to unmanned aerial vehicle flight parameters and sensor key parameters; determining the porosity of each image grid unit by adopting a U-Net network method; and for different cotton periods corresponding to the unmanned aerial vehicle image, a matched aggregation degree calculation method such as a finite length method or a macropore removal method is adopted to calculate a cotton observation layer aggregation degree index CI of the ground target grid. According to the method, multi-angle observation and deep learning algorithms are introduced, so that the monitoring precision and efficiency of the cotton canopy aggregation degree index are greatly improved. The actual growth condition of the cotton canopy is comprehensively captured through multi-angle observation, and the limitation of single-angle observation is overcome; the application of the deep learning algorithm significantly improves the efficiency and accuracy of image data processing.

Description

Technical Field

[0001] The present invention relates to the technical field of agriculture, and relates to but is not limited to a method for calculating the canopy aggregation index of cotton based on unmanned aerial vehicle (UAV) optical images. Background Art

[0002] As an important cash crop, the growth status of cotton directly affects the income of agricultural production. Monitoring the canopy aggregation index (CI) of cotton is an important means for evaluating the growth status of cotton, early warning of pests and diseases, and water and fertilizer management. Traditional monitoring methods mainly rely on manual on-site investigations, which are not only time-consuming and laborious, but also have limited coverage, making it difficult to achieve large-scale and refined agricultural monitoring.

[0003] Due to its advantages such as high efficiency, flexibility, and low cost, UAVs have gradually become an important tool for agricultural monitoring. The multi-angle image data obtained by UAVs can more comprehensively reflect the actual growth of the cotton canopy. However, existing UAV image monitoring methods based on a single angle have problems such as perspective limitations, which affect the accuracy and efficiency of monitoring results.

[0004] Currently, there are some methods for monitoring the canopy aggregation index of vegetation based on UAV images, but there are also some problems. UAV images from a single angle are difficult to comprehensively capture the growth status of the cotton canopy. Especially in areas with dense cotton planting, image occlusion is likely to occur, affecting the accuracy of monitoring results. Using an active emission lidar sensor on UAVs to capture the canopy aggregation of vegetation, however, the high cost of its sensors limits its popularization and utilization in agriculture. Summary of the Invention

[0005] In view of this, the embodiments of the present invention provide a method for calculating the canopy aggregation index of cotton based on UAV optical images, which at least solves the problems of low measurement efficiency and high hardware cost in the prior art.

[0006] The technical solution of the embodiments of the present invention is implemented as follows:

[0007] The embodiments of the present invention provide a method for calculating the canopy aggregation index of cotton based on UAV images, and the method includes:

[0008] Use a drone equipped with an optical sensor to collect drone images of the cotton canopy within the ground target grid; segment the drone images into multiple image grid units according to the drone flight parameters and the key sensor parameters; wherein, different image grid units correspond to different observation angles; use a deep learning method to determine the porosity of each of the image grid units; in the cotton seedling or budding stage, for each of the image network units, recalculate the pore size distribution curve after removing the largest-sized pores caused by non-random distribution and calculate the cotton canopy aggregation index CI(θ) at the corresponding observation angle, and finally average the CI(θ) at all observation angles to obtain the cotton canopy aggregation index CI of the ground target grid; in the cotton flowering and boll stage, determine the cotton canopy aggregation index CI of the ground target grid based on the natural logarithm average of the porosities of the multiple image grid units respectively.

[0009] In some possible embodiments, the key sensor parameters include the diagonal field of view angle, the horizontal field of view angle, and the vertical field of view angle; the drone flight parameters include the drone flight altitude, the forward overlap rate, and the side overlap rate.

[0010] In some possible embodiments, both the forward overlap rate and the side overlap rate are set to 85%, and the drone flight altitude is set to 10 to 30 meters.

[0011] In some possible embodiments, the segmenting the drone images into multiple image grid units according to the drone flight parameters and the key sensor parameters includes: determining the actual image length of the drone images according to the flight altitude of the drone and the horizontal field of view angle of the sensor; determining the actual image width of the drone images according to the flight altitude of the drone and the vertical field of view angle of the sensor; determining the number of pixel units N in the horizontal / vertical direction according to the forward overlap rate and the side overlap rate of the drone; equally dividing the actual image length and the actual image width by N respectively, and segmenting the drone images into N*N image grid units.

[0012] In some possible embodiments, the using a deep learning method to determine the canopy porosity of each of the image grid units includes: using a cotton canopy image sample dataset with pore labels, and performing pore detection on each of the image grid units through a pore detection model trained by combining with a deep learning U-Net network, and outputting a detection result image; calculating the canopy porosity of each of the image grid units according to the pore area and the unit image area in the detection result image.

[0013] In some possible embodiments, during the cotton seedling or budding stage, for each of the image network units, after removing the pores with the largest size, recalculate the canopy cumulative porosity - pore size distribution curve and calculate the cotton canopy aggregation index CI(θ) at the corresponding observation angle, including: for the image grid units at each observation angle, count the actual canopy cumulative porosity - pore size distribution curve F m (λ, θ), where λ represents the pore size and θ is the observation angle corresponding to the image grid unit; determine the canopy cumulative porosity - pore size distribution curve F r (λ, θ) in the random state; iteratively calculate the error between the actual canopy cumulative porosity - pore size distribution curve recalculated after removing the largest pore in the image network unit and the canopy cumulative porosity - pore size distribution curve in the random state until the error no longer increases and stop the operation; take the actual canopy cumulative porosity - pore size distribution curve corresponding to the minimum error as the finally iterated canopy cumulative porosity - pore size distribution curve F mr (λ, θ); based on the above F m (λ, θ), F r (λ, θ), F mr (λ, θ), let λ = 0, calculate F m (0, θ), F r (0, θ), F mr (0, θ) and substitute them into the aggregation formula to obtain the cotton canopy aggregation index CI(θ) of the image grid unit.

[0014] In some possible embodiments, the determination of the canopy cumulative porosity - pore size distribution curve F r (λ, θ) in the random state includes: assuming that the pore sizes in the actual plot are randomly distributed, let λ = 0 and take the negative logarithm of F m (0, θ) to obtain the leaf projection area L p (θ); based on the leaf projection area L p (θ), determine the canopy cumulative porosity - pore size distribution curve F r (λ, θ) in the random state.

[0015] In some possible embodiments, the iterative calculation of the error between the actual canopy cumulative porosity - pore size distribution curve recalculated after removing the largest pore in the image network unit and the canopy cumulative porosity - pore size distribution curve in the random state until the error no longer increases and stop the operation includes: calculating the initially determined F m (λ, θ) and F rThe first error RMS1_mr between (λ,θ); when RMS1_mr is greater than a fixed threshold, after removing the largest pore in the image network unit, re-determine the actual canopy cumulative porosity-pore size distribution curve F corresponding to the second round m-1 The canopy cumulative porosity-pore size distribution curve F under (λ,θ) and random conditions r-1 (λ,θ) and calculate F corresponding to the second round m-1 (λ,θ) and F r-1 The second error RMS2_mr between (λ,θ); when RMS2_mr is greater than the fixed threshold, after removing the largest pore in the remaining pores, re-determine the actual canopy cumulative porosity-pore size distribution curve F corresponding to the third round again m-2 The canopy cumulative porosity-pore size distribution curve F under (λ,θ) and random conditions r-2 (λ,θ) and determine F corresponding to the third round m-2 (λ,θ) and F r-2 The third error RMS3_mr between (λ,θ); stop the operation when RMS3_mr - RMS2_mr > 0 and RMS2_mr - RMS1_mr < 0.

[0016] In some possible embodiments, the aggregation degree calculation formula is as follows:

[0017]

[0018] Wherein, CI(θ) is the cotton canopy aggregation index of the image grid unit corresponding to the observation angle θ, F m F(0,θ) is the cumulative porosity of the pores with a pore size greater than 0 on the image grid unit, F r F(0,θ) is the cumulative porosity of the pores with a pore size greater than 0 under random conditions, F mr F(0,θ) is the cumulative porosity of the pores with a pore size greater than 0 after removing large pores.

[0019] In some possible embodiments, the canopy cumulative porosity-pore distribution curve F under random conditions λ The expression of F(λ,θ) is as follows:

[0020]

[0021] In the formula, L p L(θ) is the projected leaf area at the observation angle θ, λ is the pore size, W p W(θ) is the leaf width, which is determined by drawing a sample line on the image grid unit corresponding to the observation angle θ and measuring the projected width of the leaf on the sample line.

[0022] In an embodiment of the present invention, first, an unmanned aerial vehicle (UAV) is used to carry an optical sensor to collect UAV images of the cotton canopy within the ground target grid; second, the UAV images are segmented into multiple image grid units according to the UAV flight parameters and the key parameters of the sensor; then, a deep learning method is adopted to determine the porosity of each image grid unit; finally, different cotton growth periods when the UAV images are captured are collected, and a corresponding aggregation degree calculation method such as the finite length method or the large pore removal method is used to calculate the cotton canopy aggregation degree index CI of the ground target grid. By introducing multi-angle observation and deep learning algorithms, this method significantly improves the monitoring accuracy and efficiency of the cotton canopy aggregation degree index. Multi-angle observation comprehensively captures the actual growth of the cotton canopy, overcoming the limitations of single-angle observation; the application of the deep learning algorithm significantly improves the efficiency and accuracy of image data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained according to these drawings, where:

[0024] Figure 1 It is a schematic flowchart of a method for calculating the cotton canopy aggregation degree index based on UAV images provided by an embodiment of the present invention;

[0025] Figure 2 It is a schematic diagram of the sensor principle provided by an embodiment of the present invention;

[0026] Figure 3 It is a schematic diagram of the imaging principle of the camera provided by an embodiment of the present invention;

[0027] Figure 4 It is a schematic diagram of the flight scene and activity trajectory of the UAV provided by an embodiment of the present invention;

[0028] Figure 5 It is a schematic diagram of the forward overlap rate during the UAV flight provided by an embodiment of the present invention;

[0029] Figure 6 It is a schematic diagram of the side overlap rate during the UAV flight provided by an embodiment of the present invention;

[0030] Figure 7 It is a schematic diagram of the segmentation effect of multiple observation angle units of the UAV images provided by an embodiment of the present invention;

[0031] Figure 8Schematic diagrams of crop pores under different conditions provided by the embodiments of the present invention; part (a) represents a schematic diagram of actual large pores + random pores, part (b) represents a schematic diagram of pores randomly distributed after removing large pores, and part (c) represents a schematic diagram of pores randomly distributed theoretically. Detailed implementation manners

[0032] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. The following embodiments are used to illustrate the present invention, but are not used to limit the scope of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0033] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments. However, it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0034] It should be noted that the terms "first / second / third" involved in the embodiments of the present invention are only used to distinguish similar objects and do not represent a specific order for the objects. It can be understood that "first / second / third" can be interchanged with a specific order or sequence when permitted, so that the embodiments of the present invention described here can be implemented in an order other than that illustrated or described here.

[0035] Those skilled in the art of the present technology can understand that, unless otherwise defined, all terms (including technical terms and scientific terms) used here have the same meaning as the general understanding of those of ordinary skill in the art in the field to which the embodiments of the present invention belong. It should also be understood that terms such as those defined in a general dictionary should be understood to have a meaning consistent with the meaning in the context of the prior art, and will not be interpreted with an idealized or overly formal meaning unless specifically defined as here.

[0036] The clumping index CI (Clumping Index), also known as the aggregation index, is an important vegetation structure parameter, which characterizes the spatial distribution aggregation characteristics of canopy leaves and is an index for quantifying the clumping level of vegetation canopy leaves relative to random distribution. When the spatial distribution of vegetation canopy leaves is randomly distributed, CI is 1; when the canopy leaves are clumped (non-randomly distributed), CI is less than 1, and the smaller CI is, the more clumped the leaves are.

[0037] The Cotton Canopy Aggregation Index (CI) is an indicator for measuring the aggregation degree of cotton crops, which is calculated by analyzing multi-angle images captured by drones. This method is particularly applicable to specific growth stages of crops, such as the seedling stage or the budding stage, when the row spacing of the crops is more obvious, facilitating drone image capture and subsequent analysis.

[0038] An embodiment of the present invention provides a method for calculating the cotton canopy aggregation index based on drone images, as Figure 1 shown, the method at least includes the following steps:

[0039] Step S110, using a drone equipped with an optical sensor to collect drone images of the cotton canopy within the ground target grid.

[0040] Here, the drone can quickly cover a large area of cotton fields and obtain high-resolution drone image data in real time. Compared with traditional ground measurements, the monitoring efficiency is greatly improved. At the same time, the drone has a low flight altitude and is less affected by climate conditions. Compared with satellite remote sensing, it can stably obtain data under various meteorological conditions. The optical sensor is a passive receiving sensor. The ground target grid is the grid area on the ground where the cotton canopy aggregation index is to be studied.

[0041] Step S120, dividing the drone images into multiple image grid units according to the drone flight parameters and the key sensor parameters.

[0042] Here, the key drone flight parameters include flight altitude, forward overlap rate, and side overlap rate, and these parameters affect the coverage range and stitching accuracy of the images.

[0043] The key sensor parameter, namely the Field of View (FOV), is the viewing angle range that the sensor can capture, including the diagonal field of view (DFOV), horizontal field of view (HFOV), and vertical field of view (VFOV). The field of view is determined by the focal length (F), width (W), and height (H) of the sensor.

[0044] According to the flight altitude and the field of view, the ground resolution corresponding to each pixel can be calculated. According to the forward and side overlap rates, the drone images can be divided into multiple image grid units, and each image grid unit corresponds to an observation angle.

[0045] In other embodiments, preprocessing such as denoising and contrast enhancement can also be performed on the drone images before segmentation to improve the accuracy of subsequent analysis.

[0046] Step S130, using a deep learning method to determine the canopy porosity of each of the image grid units.

[0047] Here, the canopy porosity refers to the proportion of the void part in the canopy to the total area. According to multiple image grid units, the cotton canopy pores in the UAV images are segmented, and the canopy porosity corresponding to the image grid units observed at each observation angle can be calculated. The present invention uses multi-angle observations, combined with a deep learning model and advanced image processing algorithms, to accurately calculate the accurate data of the cotton canopy pores in the image grid units, and improve the accuracy of calculating the cotton canopy aggregation index.

[0048] Certain research and applications of deep learning methods have been carried out in the determination of porosity, including the use of neural network models such as U-Net. These methods improve the accuracy and efficiency of porosity prediction by extracting pore structure features, optimizing model parameters, using specific loss functions and optimization algorithms.

[0049] Step S140, in the cotton seedling or budding stage, for each of the image network units, after removing the pores with the largest size caused by non-random distribution, recalculate the canopy cumulative porosity - pore size distribution curve and calculate the cotton canopy aggregation index CI(θ) at the corresponding observation angle. Finally, average the CI(θ) at all observation angles to obtain the cotton canopy aggregation index CI of the ground target grid.

[0050] Here, the cotton canopy aggregation index (CI) is an index that quantitatively characterizes the deviation degree of the actual spatial distribution of canopy leaves from the random distribution. It is used to correct the estimated leaf area index (LAI) results to more accurately reflect the actual condition of the canopy.

[0051] In implementation, first, devices such as high-resolution cameras or UAVs are used to obtain UAV images of the ground target grid in the cotton seedling or budding stage and segment them into multiple image grid units. After preprocessing the image grid units, image processing techniques are used to identify the pores (i.e., the voids or uncovered areas in the canopy).

[0052] Analyze the distribution characteristics of the pores, identify and remove the pores with the largest size caused by non-random factors (such as terrain, obstacles, etc.). This usually involves a comprehensive analysis of the pore shape, size and position. After removing the non-randomly distributed pores, recalculate the canopy cumulative porosity - pore size distribution curve of each image network unit to reflect the true structural characteristics of the canopy.

[0053] Among them, the cumulative porosity - pore size distribution curve of the canopy is a graphical representation method for describing the size distribution of pores (i.e., the part of the canopy not covered by leaves) in the plant canopy and its cumulative porosity. The vertical axis is the cumulative canopy porosity, and the horizontal axis is the pore size information. Such a curve helps to deeply understand the structure and function of the canopy and is particularly significant in fields such as agriculture, ecology, and environmental science. The cumulative canopy porosity refers to the cumulative value of the probability that light, air, etc. directly reach the ground without being intercepted by leaves when passing through the canopy. Specifically in this invention, it refers to the porosity of the image grid unit. It reflects the sparseness or light transmittance of the canopy. The cumulative porosity can be obtained by classifying the pores in the canopy by size, calculating the total area or total volume of the pores in each size category, and then accumulating them. The pore size distribution refers to the proportion or quantity of pores of different sizes in the canopy. This can be achieved by performing high - resolution imaging on the canopy and then using image - processing techniques to identify and measure the pore sizes. The pore sizes can vary from tiny to relatively large, and the classification criteria are usually determined according to the research purpose and requirements.

[0054] The process of drawing the cumulative canopy porosity - pore size distribution curve can be implemented using any existing technology, and the present invention does not limit this. A possible implementation method is as follows: First, use image - processing techniques to identify and measure the pore sizes in each image grid unit. At the same time, it is also necessary to calculate the total area or total volume of the pores in each size category. Then, organize the collected data, including the pore sizes, quantities, or areas, etc. Finally, with the pore size as the abscissa and the cumulative porosity as the ordinate, draw the cumulative canopy porosity - pore size distribution curve. This usually requires using professional drawing software or tools to complete.

[0055] For each observation angle, using the cumulative canopy porosity - pore size distribution curve and other relevant parameters (such as canopy height, leaf area index, etc.), calculate the cotton canopy aggregation index (CI) using an appropriate mathematical model. Perform an arithmetic average on the CI values at all observation angles to obtain the cotton canopy aggregation index of the target ground grid. This helps to eliminate the bias that may be brought by a single observation angle and improve the accuracy of the evaluation.

[0056] Through the above process, the canopy structure of cotton at the seedling or budding stage can be accurately evaluated, providing a scientific basis for agricultural management. At the same time, this method can also be applied to the analysis of the canopy structure of other crops, providing support for the development of precision agriculture.

[0057] Step S150, in the cotton boll - setting stage, based on the natural logarithm average value of the canopy porosity of each of the multiple image grid units, determine the cotton canopy aggregation index CI of the ground target grid.

[0058] Here, the average value of the natural logarithm of the canopy porosity of all image grid cells is calculated. This average value reflects the average level of the canopy porosity of the entire ground target grid. Based on the average value of the natural logarithm of the canopy porosity, the calculation formula for CI can be derived. The specific calculation formula may vary depending on the research purpose and background knowledge. However, generally, CI is inversely proportional to the average value of the natural logarithm of the canopy porosity, that is, the larger the porosity (the sparser the canopy), the smaller CI; the smaller the porosity (the denser the canopy), the larger CI.

[0059] For example, the cotton canopy aggregation index CI of the ground target grid is calculated by the following formula:

[0060]

[0061] In the formula, Ω lx is the cotton canopy aggregation index CI of a certain ground grid cell A, is the canopy porosity corresponding to a ground target grid, is the natural logarithm value of the average of the canopy porosities of multiple divided image grid cells, is the mean value of the natural logarithm values of the canopy porosities of multiple divided image grid cells.

[0062] In other embodiments, after steps S140 and S150, the aggregation characteristics of the cotton canopy, such as the density of the canopy, the distribution of leaves, etc., can also be analyzed according to the calculated cotton canopy aggregation index CI value. According to the analysis results, targeted agricultural management suggestions, such as adjusting irrigation, fertilization, pruning and other measures, can be put forward to optimize the cotton growth environment and improve the yield and quality.

[0063] In the embodiment of the present invention, first, a drone is used to carry an optical sensor to obtain high-resolution image data in real time. Compared with traditional ground measurements, the monitoring efficiency is greatly improved; at the same time, compared with satellite remote sensing, the influence of climate is reduced; secondly, according to the drone flight parameters and the key parameters of the sensor, the drone images are segmented into multiple image grid cells, so as to accurately calculate the leaf inclination distribution of the cotton canopy from multiple angles and improve the accuracy of the leaf area index calculation. The drone has a low flight altitude and is less affected by climate conditions, and can stably obtain data under various meteorological conditions. This method provides an efficient, accurate and economical method for monitoring the cotton leaf area, can obtain information on cotton fields in real time and with high resolution, is applicable to large-area cotton fields and is not affected by climate, and has broad application prospects and promotion value.

[0064] In some possible embodiments, the key parameters of the sensor include the diagonal field of view angle, the horizontal field of view angle, and the vertical field of view angle; the flight parameters of the drone include the flight altitude of the drone, the forward overlap rate, and the side overlap rate.

[0065] Here, the field of view angle refers to the field of view range that the lens can capture when it matches the CMOS (sensor). Specifically, it is the angle formed by the two edges of the maximum range that the image of the measured target can be seen through the lens with the vertex of the lens of the optical instrument as the vertex, as Figure 2 shown, which is divided into the horizontal field of view angle (HFOV), the vertical field of view angle (VFOV), and the diagonal field of view angle (DFOV). Among them, the diagonal field of view angle is the largest, the horizontal field of view angle is the second largest, and the vertical field of view angle is the smallest. Generally, the field of view angle usually refers to the diagonal field of view angle.

[0066] As Figure 3 shown in the imaging principle of the camera, the imaging width of each camera is fixed. For different focal lengths, the value of the field of view angle α is different, and the corresponding relationship is as follows: the longer the focal length, the smaller the field of view angle; the shorter the focal length, the larger the field of view angle. For the field of view range, the longer the focal length, the farther the field of view range; the shorter the focal length, the shorter the field of view range. The determining factors of the horizontal field of view angle are the focal length f and the width W of the imaging area; the determining factors of the vertical field of view angle are the focal length and the height H of the imaging area; the determining factors of the diagonal field of view angle are the focal length f and the width W and height H of the imaging area. The specific calculation formulas are as follows:

[0067]

[0068] Given that the diagonal size of the CMOS is 1 / 2.3” inches and the diagonal length D is 16 mm (millimeters), through conversion, 16 / 2.3” = 6.95 mm; given that the aspect ratio of the CMOS is 4:3 and using the converted diagonal length σ of 6.95 mm, the values of the imaging height H and the imaging width W can be obtained.

[0069] Given the diagonal length of the CMOS and the diagonal field of view angle of the CMOS, σ represents the diagonal length, and the focal length f at the time of taking a photo can be obtained. Thus, according to formula (1), it can be calculated that: the horizontal field of view angle HFOV = 70.48 degrees; the vertical field of view angle WFOV = 55.8 degrees; the diagonal field of view angle DFOV = 82.8 degrees.

[0070] As Figure 4 shown in the schematic diagram of the flight scenario and activity trajectory of the drone provided by the embodiment of the present invention, the forward overlap rate and the side overlap rate are usually used to describe the overlap degree between adjacent images when the drone or the aircraft is performing photogrammetry. The overlap rate is to ensure that there is enough information for stereo matching and image stitching.

[0071] In some possible embodiments, the heading overlap rate and the lateral overlap rate are both set to 85%, and the flight altitude of the drone is set to 30 meters.

[0072] Here, as Figure 5 shown, the heading overlap rate is expressed as where P x is the overlapping part of the left and right adjacent images 1 and 2, and L x is the length of one image. As Figure 6 shown, the lateral overlap rate is expressed as where P y is the overlapping part of the upper and lower adjacent images I-1 and II-1, and L y is the width of one image.

[0073] In the above embodiments, the heading overlap and the lateral overlap of the drone flight settings are both 85%, which can reduce the blank area between images and improve the accuracy of subsequent image stitching; setting the flight altitude to 30 m (meters) can balance the image resolution and the coverage area.

[0074] In some possible embodiments, the above step 120 "divide the drone image into multiple image grid units according to the drone flight parameters and the key sensor parameters" is further implemented through the following process:

[0075] S121, determine the actual length of the drone image according to the flight altitude of the drone and the horizontal field of view angle of the sensor.

[0076] Here, it is known that the flight altitude d is 30 m (meters), the horizontal field of view angle in the length direction is 70.48 degrees, and the number of pixels in the length direction is 4056 pixels. The actual length l of the image is calculated by the following formula and the result is 42.42 m:

[0077]

[0078] Furthermore, dividing the actual length of the image by the number of pixels in the length direction can obtain a resolution of 0.0105 m, which usually refers to the actual length of the ground represented by each pixel in the image.

[0079] S122, determine the actual width of the drone image according to the flight altitude of the drone and the vertical field of view angle of the sensor.

[0080] Here, it is known that the flight altitude d is 30 m, the vertical field of view angle in the width direction is 55.8 degrees, and the number of pixels in the width direction is 3040 pixels. The actual width w of the image is calculated by the following formula and the result is 31.77 m:

[0081]

[0082] Further, dividing the actual width of the image by the number of pixels in the width direction gives a resolution of 0.0105 m, which generally refers to the actual ground width represented by each pixel in the image.

[0083] S123. Determine the number of pixel units N in the horizontal / vertical direction according to the forward overlap rate and the side overlap rate of the UAV.

[0084] Here, by dividing the number of pixel units N according to the forward overlap rate and the side overlap rate, we can obtain The length of the UAV image is 4056 pixels, divided into 7 equal parts, and the length of each pixel unit is 580 pixels. The width is 3040 pixels, divided into 7 equal parts, and the width of each pixel unit is 434 pixels.

[0085] S124. Divide the actual length and the actual width of the image into N equal parts respectively, and divide the UAV image into N*N image grid units.

[0086] Here, taking the UAV flight altitude of 30 m as an example, the actual length of the UAV image is 42.42 m and the width is 31.77 m. The present invention is not limited thereto.

[0087]

[0088] In the above formula, dividing the actual length l, width w, and N of the image by 2 makes the grid division smaller, making the image of each divided image grid unit clearer. Finally, the entire image area is divided into a grid of 7 rows * 7 columns with a size of 6 m × 4.54 m, that is, 49 observation angle units, as Figure 7 shown.

[0089] In some possible embodiments, the method further includes: determining the observation angle of the corresponding pixel according to the row and column numbers of each pixel from the geometric center in each image grid unit, the pixel width and pixel length of the image, and the UAV flight altitude; taking the average of the observation angles of all pixels in each image grid unit as the observation angle of the corresponding image grid unit.

[0090] Here, according to the following formula, the observation angle of each pixel can be calculated:

[0091]

[0092] Where d is the flight altitude of the UAV, θ is the observation angle of the UAV. x and y are the row and column numbers of the pixel from the geometric center in the image, that is, its position in the image matrix. w and l are the pixel width and pixel length of the image. For a rectangular image, the geometric center is usually the midpoint of the width and height. Using the flight altitude and field of view angle of the UAV, these row and column numbers are converted into observation angles.

[0093] The image is divided into multiple image grid units, and each image grid unit contains a certain number of pixels. For the pixels within each image grid unit, the average value of their observation angles is calculated as the observation angle of the image grid unit. This method can effectively associate each pixel in the image with the actual observation angle, providing a basis for subsequent analysis and processing.

[0094] In some possible embodiments, the above step S130 "determine the canopy porosity of each of the image grid units using a deep learning method" includes:

[0095] S131, using a cotton canopy image sample dataset with porosity labels, combined with a porosity detection model trained by the deep learning U-Net network, perform porosity detection on each of the image grid units, and output a detection result image.

[0096] Here, a set of cotton canopy image samples with porosity labels is collected. These image samples should include cotton canopy images in different growth stages, different lighting conditions, different soil backgrounds, etc., to ensure the generalization ability of the model. First, preprocess the cotton canopy images, such as cropping, scaling, normalization, etc., to meet the input requirements of the model. Then use an image annotation tool (such as Labelme or VGG Image Annotator) to annotate the cotton canopy images, mark the pore areas in the cotton canopy, generate a binary label image, where the pore part is white (1) and the non-pore part is black (0), and finally obtain a set of cotton canopy image sample datasets with porosity labels.

[0097] Then, divide the sample dataset into a training set, a validation set, and a test set according to 8:1:1. Select the deep learning U-Net network and use the training set for model training. Verify the performance of the model on the validation set and optimize the model by adjusting hyperparameters (such as learning rate and batch size, etc.). When evaluating the model on the test set, use metrics such as accuracy, recall rate, and F1 score for evaluation. Finally, obtain a trained porosity detection model.

[0098] Use the porosity detection model to perform porosity detection on each preprocessed image grid unit, and output a detection result image, thereby segmenting the pore areas in the cotton canopy.

[0099] S132. Calculate the canopy porosity of each image grid unit according to the pore area and the unit image area in the detected result image.

[0100] Here, the detected result image obtained by detecting each image grid unit contains multiple pixels. The calculation formula for canopy porosity is the cumulative pore area / unit image area. In other embodiments, the detected result can also be compared with the manually labeled tag image to verify the accuracy and reliability of the model.

[0101] Through deep learning technology, the canopy porosity of cotton canopies can be accurately calculated, thereby improving the accuracy of the subsequent calculation of the cotton canopy aggregation index.

[0102] In some possible embodiments, the above step S140 "In the cotton seedling or budding stage, for each of the image network units, recalculate the canopy cumulative porosity - pore size distribution curve after removing the largest-sized pores and calculate the cotton canopy aggregation index CI(θ) at the corresponding observation angle" is implemented through the following steps:

[0103] S141. For each image grid unit, count the actual canopy cumulative porosity - pore size distribution curve F m (λ, θ), where λ represents the pore size and θ is the observation angle corresponding to the image grid unit.

[0104] Here, the premise of this step is to obtain the pore size distribution. For the image grid unit at each observation angle θ, through the pore identification of the UAV image, count the size of each pore, and calculate the canopy cumulative porosity within each size range. Then count the canopy cumulative porosity - pore size distribution curve F m (0, θ), where λ represents the pore size, and the unit is pixel or mm / cm. The cumulative porosity refers to the proportion of the total area (or number) of all pores smaller than or equal to a certain size to the entire canopy area (or number).

[0105] S142. Determine the canopy cumulative porosity - pore size distribution curve F r (λ, θ) in the random state.

[0106] Here, under the condition of the same leaf area index, the aggregated canopy is more likely to have large-sized pores than the randomly distributed canopy. This method combines the canopy cumulative porosity (vertical axis) and pore size information. First, obtain the pore size distribution curve of the canopy cumulative porosity, and use the large pore removal method to remove the large pores caused by non-random distribution from the total porosity to obtain the canopy cumulative porosity - pore size distribution curve in the approximate random state.

[0107] In some embodiments, S142 is implemented through the following process: Assume that the pore sizes in the ground cotton crops corresponding to the actual sample plots, i.e., the image grid cells, are randomly distributed. To simulate the random state, a set of pores with the same number as the actual pores but randomly assigned sizes can be generated. Based on the generated random pore size data, the canopy cumulative porosity - pore size distribution curve under the random state is calculated.

[0108] In some embodiments, S142 is implemented through the following process: Assume that the pore sizes of the actual sample plots are randomly distributed, let λ = 0 and take the negative logarithm of F m (0, θ) to obtain the leaf projected area L p (θ); Based on the leaf projected area L p (θ), determine the canopy cumulative porosity - pore size distribution curve F r (λ, θ) under the random state.

[0109] Here, F m (0, θ) represents the canopy cumulative porosity of all pores with pore size λ greater than 0.

[0110] If the probability of the pore sizes of the actual sample plots is randomly distributed, in F m (λ, θ), λ can be set to 0 to obtain F m (0, θ), L p (θ) = -ln(F m (0, θ)), and calculate the leaf projected area L p (θ). Substitute it into the following formula (1) to calculate the theoretically canopy cumulative porosity - pore distribution curve F r (λ, θ):

[0111]

[0112] In the formula, L p (θ) is the projected leaf area at the observation angle θ, λ is the pore size, and W p (θ) is the leaf width, which is determined by drawing a sample line on the image grid cell corresponding to the observation angle θ and measuring the projected width of the leaf on the sample line.

[0113] S143, iteratively calculate the error between the actual canopy cumulative porosity - pore size distribution curve recalculated after removing the largest pore in the image network cell and the canopy cumulative porosity - pore size distribution curve under the random state until the error no longer continues to increase and stop the operation.

[0114] Here, in each iteration, find the largest pore in the current image grid cell and remove it, as Figure 8As shown, part (a) represents a schematic diagram of actual macropores + random pores, part (b) represents a schematic diagram of pores randomly distributed after removing macropores, and part (c) represents a schematic diagram of pores randomly distributed theoretically. Recalculate the actual canopy cumulative porosity - pore size distribution curve and the canopy cumulative porosity - pore size distribution curve in the random state, and select a suitable error metric method according to the specific situation and data characteristics, such as area difference, mean square error (MSE), sum of absolute errors (SAE), etc., to compare the differences between the two curves. Preferably, the least squares method is used to calculate the error.

[0115] The present invention uses a preset error metric method to calculate the error between the recalculated actual curve and the random curve. Check whether the current error satisfies the iteration stop condition (the error between two adjacent iterations no longer increases). If the condition is satisfied, stop the iteration; otherwise, continue to the next step. If the iteration does not stop, update the pore data of the image grid unit (the largest pore has been removed) and prepare for the next iteration.

[0116] In some embodiments, step S143 is implemented through the following process: calculate the first error RMS1_mr between the initially determined F m (λ,θ) and F r (λ,θ); in the case where RMS1_mr is greater than a fixed threshold, remove the largest pore in the image network unit and then re - determine the corresponding actual canopy cumulative porosity - pore size distribution curve F m-1 (λ,θ) and the canopy cumulative porosity - pore size distribution curve F r-1 (λ,θ) in the random state and calculate the second error RMS2_mr between the corresponding F m-1 (λ,θ) and F r-1 (λ,θ); in the case where RMS2_mr is greater than the fixed threshold, remove the largest pore in the remaining pores and then re - determine the corresponding actual canopy cumulative porosity - pore size distribution curve F m-2 (λ,θ) and the canopy cumulative porosity - pore size distribution curve F r-2 (λ,θ) in the random state and determine the third error RMS3_mr between the corresponding F m-2 (λ,θ) and F r-2 (λ,θ); stop the operation when RMS3_mr - RMS2_mr > 0 and RMS2_mr - RMS1_mr < 0.

[0117] Here, the fixed threshold is defined according to actual needs, for example, it can be 0.001.

[0118] Exemplarily, in the first round, calculate F m(λ,θ) and F r (λ,θ), and calculate RMS1_mr using the least squares method. If RMS1_mr > 0.001, then remove the largest pore in the current image grid cell;

[0119] In the second round, calculate F respectively after removing the largest pore m-1 (λ,θ) and F r-1 (λ,θ), and calculate RMS2_mr using the least squares method. If RMS2_mr > 0.001, then remove the largest pore among the remaining pores in the current image grid cell;

[0120] In the third round, calculate F respectively after removing the largest pore m-2 (λ,θ) and F r-2 (λ,θ), and calculate RMS2_mr using the least squares method. If RMS2_mr > 0.001, then remove the remaining largest pore in the current image grid cell;

[0121] In the k-th round, calculate F respectively after removing the largest pore m-(k-1) (λ,θ) and F r-(k-1) (λ,θ), and calculate RMS(k - 1)_mr using the least squares method. If RMS(k - 1)_mr > 0.001, then remove the remaining largest pore in the current image grid cell;

[0122] If RMS(k - 3)_mr - RMS(k - 2)_mr > 0 and RMS(k - 2)_mr - RMS(k - 1)_mr < 0, then stop the operation, and let F mr (λ,θ) = F m-(k-2) (λ,θ).

[0123] S144, take the actual canopy cumulative porosity - pore size distribution curve corresponding to the minimum error as the final iterative canopy cumulative porosity - pore size distribution curve F mr (λ,θ).

[0124] Here, when the iteration stops, take the actual canopy cumulative porosity - pore size distribution curve obtained in the last iteration as the final result. Through the iterative process, a canopy cumulative porosity - pore size distribution curve F that can better reflect the true structure of the canopy can be obtained mr (λ,θ).

[0125] S145, based on the above F m (λ,θ), F r (λ,θ), F mr (λ,θ), let λ = 0, and calculate F m (0,θ), F r (0,θ), Fmr (0, θ) is substituted into the aggregation degree calculation formula to obtain the cotton canopy aggregation degree index CI(θ) of the image grid unit.

[0126] Here, the aggregation degree calculation formula is as follows:

[0127]

[0128] Among them, CI(θ) is the cotton canopy aggregation degree index of the image grid unit corresponding to the observation angle θ, and F m (0, θ) is the cumulative porosity of the image grid unit with pore size greater than 0, and F r (0, θ) is the cumulative porosity of pore size greater than 0 in the random state, and F mr (0, θ) is the cumulative porosity of pore size greater than 0 after removing large pores. The cumulative porosity F m (0, θ) and the cumulative porosity F mr (0, θ) after removing large pores on the observation image at all observation angles θ are calculated.

[0129] The cotton canopy aggregation degree CI(θ) of the image grid unit at each observation angle is calculated through the above process, and then the arithmetic mean method is used to calculate the average value of CI(θ) of multiple image grid units as the cotton canopy aggregation degree index CI of the ground target grid.

[0130] The method for calculating the cotton canopy aggregation degree index based on UAV images provided by the embodiments of the present invention obtains and analyzes cotton canopy pore data through deep learning and image processing technologies, and calculates the cotton aggregation degree index CI, which has the following advantages and values:

[0131] (1) Improvement in monitoring efficiency: The UAV can quickly cover large areas of cotton fields and obtain high-resolution image data. This not only saves time and human resources but also reduces errors caused by human factors.

[0132] (2) Improvement in monitoring accuracy: Multi-angle observation can more comprehensively capture the structural characteristics of the cotton canopy. Combining deep learning technology, the canopy porosity of the cotton canopy can be calculated more accurately. This helps to more accurately estimate the cotton aggregation degree index CI.

[0133] (3) Reducing the impact of climate: Due to the low flight altitude of the UAV, compared with satellite remote sensing, it is less affected by climate conditions. This means that the UAV can work stably under various meteorological conditions and obtain reliable data.

[0134] (4) Economy: Compared with traditional ground measurement and satellite remote sensing technologies, UAV technology has advantages in terms of cost and operation difficulty. This makes this method more suitable for large-scale promotion and application.

[0135] (5) Application prospects and promotion value: This method can not only be used for monitoring the cotton aggregation index, but also be extended to monitoring the aggregation indices of other crops, with broad application prospects. Its efficient and accurate monitoring ability is of great significance for aspects such as agricultural production management and crop growth monitoring.

[0136] It should be noted that in the embodiments of the present invention, if the above method for calculating the cotton canopy aggregation index based on UAV images is implemented in the form of software function modules and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiments of the present invention, in essence or the part that contributes to the related technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing an electronic device to execute all or part of the methods described in the various embodiments of the present invention. And the aforementioned storage medium includes: USB flash drives, mobile hard disks, read-only memories (ROMs), magnetic disks, or optical discs, etc., which can store program codes. Thus, the embodiments of the present invention are not limited to any specific combination of hardware and software.

[0137] It should be understood that the term "one embodiment" or "an embodiment" mentioned throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of the present invention. Therefore, the appearances of "in one embodiment" or "in an embodiment" throughout the specification do not necessarily refer to the same embodiment. In addition, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in various embodiments of the present invention, the magnitudes of the sequence numbers of the above processes do not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention. The sequence numbers of the embodiments of the present invention above are only for description and do not represent the advantages or disadvantages of the embodiments.

[0138] It should be noted that in this article, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such a process, method, article or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of another identical element in the process, method, article or device including that element.

[0139] In several embodiments provided by the present invention, it should be understood that the disclosed method can be implemented in other ways. The methods disclosed in several method embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments. The features disclosed in several method embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments.

[0140] As described above, it is only the implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or substitutions, which should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claimed rights.

Claims

1. A method for calculating the aggregation index of cotton canopy based on UAV optical images, characterized in that, The method includes: Using a drone carrying an optical sensor to collect drone images of the cotton canopy within the ground target grid; Segmenting the drone images into multiple image grid units according to the drone flight parameters and the key sensor parameters; wherein, different image grid units correspond to different observation angles; Using a deep learning method to determine the canopy porosity of each of the image grid units; In the cotton seedling stage or bud stage, for each of the image grid units, after removing the largest-sized pores caused by non-random distribution, recalculate the canopy cumulative porosity - pore size distribution curve, and calculate the cotton canopy aggregation index CI(θ) at the corresponding observation angle. Finally, average the CI(θ) at all observation angles to obtain the cotton canopy aggregation index CI of the ground target grid; In the cotton flowering and boll-setting stage, based on the natural logarithm average of the canopy porosities of the multiple image grid units respectively, determine the cotton canopy aggregation index CI of the ground target grid.

2. The method according to claim 1, wherein The key sensor parameters include the diagonal field of view angle, the horizontal field of view angle, and the vertical field of view angle; the drone flight parameters include the drone flight altitude, the forward overlap rate, and the side overlap rate.

3. The method according to claim 2, wherein Both the forward overlap rate and the side overlap rate are set to 85%, and the drone flight altitude is set to 10 to 30 meters.

4. The method according to claim 2, wherein The segmenting the drone images into multiple image grid units according to the drone flight parameters and the key sensor parameters includes: Determining the actual length of the drone image according to the flight altitude of the drone and the horizontal field of view angle of the sensor; Determining the actual width of the drone image according to the flight altitude of the drone and the vertical field of view angle of the sensor; Determining the number of pixel units N in the horizontal / vertical direction according to the forward overlap rate and the side overlap rate of the drone; Dividing the actual length and the actual width of the image into N equal parts respectively, and segmenting the drone image into N*N image grid units.

5. The method according to any one of claims 1 to 4, characterized in that The using a deep learning method to determine the canopy porosity of each of the image grid units includes: Using a pore detection model trained by combining a deep learning U-Net network with a cotton canopy image sample dataset with pore labels to perform pore detection on each of the image grid units, and outputting a detection result image; Calculating the canopy porosity of each image grid unit according to the pore area and the unit image area in the detection result image.

6. The method according to any one of claims 1 to 4, characterized in that The in the cotton seedling stage or bud stage, for each of the image grid units, after removing the largest-sized pores, recalculate the canopy cumulative porosity - pore size distribution curve and calculate the cotton canopy aggregation index CI(θ) at the corresponding observation angle includes: For each image grid unit at each observation angle, the actual canopy cumulative porosity - pore size distribution curve F m (λ,θ) is statistically obtained, where λ represents the pore size and θ is the observation angle corresponding to the image grid unit; Determine the canopy cumulative porosity - pore size distribution curve F under random conditions r (λ,θ); Iteratively calculating the error between the actual canopy cumulative porosity - pore size distribution curve recalculated after removing the largest pore in the image grid unit and the canopy cumulative porosity - pore size distribution curve in the random state until the error no longer increases and stops the operation; Take the actual canopy cumulative porosity - pore size distribution curve corresponding to the minimum error as the final iterative canopy cumulative porosity - pore size distribution curve F mr (λ,θ); Based on the above F m (λ,θ), F r (λ,θ), F mr (λ,θ), let λ = 0, calculate F m (0,θ), F r (0,θ), F mr (0,θ) and substitute it into the aggregation degree calculation formula to obtain the cotton canopy aggregation degree index CI(θ) of the image grid unit.

7. The method according to claim 6, characterized in that The determination of the canopy cumulative porosity-porosity size distribution curve F r (λ,θ), includes: Assume that the pore sizes in the actual plots are randomly distributed, let λ = 0 and take the negative logarithm of F m (0, θ) to obtain the leaf projected area L p (θ); Based on the leaf projected area L p Determine the canopy cumulative porosity - pore size distribution curve F r (λ, θ) in the random state.

8. The method according to claim 6, characterized in that, The iterative calculation removes the error between the actual canopy cumulative porosity - pore size distribution curve recalculated after removing the largest pores in the image network unit and the canopy cumulative porosity - pore size distribution curve in the random state until the error no longer continues to increase and stops the operation, including: Calculate the initially determined F in the first round m (λ,θ) and F r (λ,θ) between the first error RMS1_mr; When RMS1_mr is greater than a fixed threshold, after removing the largest pore in the image network unit, the actual canopy cumulative porosity-pore size distribution curve F corresponding to the second round is re-determined. m-1 (λ,θ) and the canopy cumulative porosity-pore size distribution curve F in a random state r-1 (λ,θ) and calculate F corresponding to the second round m-1 (λ,θ) and F r-1 The second error RMS2_mr between (λ,θ) and F; When RMS2_mr is greater than the fixed threshold, after removing the largest pore in the remaining pores, the actual canopy cumulative porosity - pore size distribution curve F corresponding to the third round is re - determined again. m-2 (λ,θ) and the canopy cumulative porosity - pore size distribution curve F in the random state r-2 (λ,θ) and determine F corresponding to the third round m-2 (λ,θ) and F r-2 The third error RMS3_mr between (λ,θ) and F; Stop the operation when RMS3_mr - RMS2_mr > 0 and RMS2_mr - RMS1_mr < 0 are satisfied.

Images