A non-destructive identification method and system for rice phenological period based on photogrammetry point cloud

Abstract

The present invention discloses a non-destructive identification method and system of rice phenological period based on photogrammetric point cloud, including regularly acquiring multi-view RGB images of rice planting areas during the rice growth period, performing three-dimensional reconstruction processing on the acquired multi-view RGB images, and obtaining rice time-series point cloud data of the whole growth period of the rice planting areas; extracting the time-series plant height curve and the time-series green-red ratio index curve of rice based on the time-series point cloud data; identifying and dividing the rice phenological period according to the key points of the time-series plant height curve and the time-series green-red ratio index curve, combined with agronomic prior knowledge. The present invention uses multi-view RGB image three-dimensional reconstruction to restore the plant height information of crops, and applies it to the identification of rice phenological period, which can well identify the jointing period and the heading period that are not easy to identify using only spectral information.

Description

Technical Field

[0001] The invention belongs to the technical field of agricultural remote sensing, and in particular relates to a method and system for non-destructive identification of rice phenological periods based on photogrammetry point clouds. Background Art

[0002] Seeds are the "chips" of agriculture. Good varieties are an important factor in high grain yields and the key to ensuring food security. The main purpose of rice breeding is to cultivate rice varieties with higher yield potential. The characteristics of key growth and development stages of rice, namely key phenological characteristics, are one of the important factors affecting yield. However, field recording of the phenological periods of a large number of experimental varieties during breeding is time-consuming and labor-intensive, which greatly affects the efficiency of breeding work. Breeding work has put forward an urgent need for large-scale, rapid and non-destructive identification of rice phenological periods.

[0003] Rice phenology can generally be divided into: (1) vegetative growth period, including germination, seedling, tillering and jointing; (2) reproductive growth period, including booting, heading and flowering; (3) maturity period, including milky, waxy and fully mature stages. The heading stage can be further divided into the initial heading stage (10% of the ears have emerged), heading date (50% of the ears have emerged), and full heading stage (80% of the ears have emerged). Among these stages, the tillering stage is the stage for determining the number of ears and the stage for laying the foundation for the number of grains per ear; the booting stage is the stage for consolidating the number of ears, determining the number of grains per ear, and laying the foundation for grain weight; the maturity stage is the stage for determining grain weight.

[0004] The main characteristics of the vegetative growth period are leaf development, increase in the number of tillers, and increase in plant height. The rice plant is compact and has a small leaf inclination angle at this stage. At the end of the vegetative growth period, tillers reach their maximum number and most leaves grow. The reproductive growth period starts from booting and lasts until the end of flowering. At booting, the upper part of the stem swells because the differentiation of the panicle occurs in the leaf sheath of the top sword leaf. At the heading stage, which is marked by the emergence of the panicle, the panicle emerges in a short period of time, and the plant height does not increase significantly after the heading stage. Flowering occurs immediately after heading of rice and ends within 1-2 days. At the end of the reproductive growth period, the elements of the rice canopy change from only leaves to leaves and panicles, while the leaf inclination angle increases and the plant becomes looser. At maturity, the grains appear and fill until hardening, the panicle gradually changes from green to yellow and gradually droops, and the leaves also gradually turn yellow.

[0005] Traditionally, rice phenology is mainly observed and recorded manually at regular intervals. This method is time-consuming, labor-intensive, and highly subjective. Sometimes destructive sampling is required, and it is often only applicable to a small area. However, there are hundreds of rice varieties planted in conventional breeding fields, and manual observation of phenology will greatly affect the efficiency of breeding work. In contrast, remote sensing technology, as a high-throughput non-destructive observation method, has the potential to obtain timely and objective crop physiological and phenological information. In the field of breeding, high-quality varieties that meet specific requirements can be screened based on these physiological and phenological information, greatly reducing the intensity of field survey work and solving the current problem that the acquisition of rice phenological information mainly relies on manual observation, which is inefficient and highly subjective. In recent years, unmanned aerial vehicle (UAV) remote sensing has been widely used in agricultural remote sensing research due to its high resolution, low cost and flexibility, providing a very potential remote sensing platform for extracting crop physiological and phenological information, especially for smaller plots planted for breeding purposes.

[0006] One of the difficulties in using remote sensing technology to study rice is that the physiological and biochemical characteristics of rice canopy structure, pigment composition, biomass distribution, etc. undergo complex changes as rice develops. However, it is these complex changes that provide ideas and basis for the division of phenological periods.

[0007] At present, the research on remote sensing identification of rice phenology is mainly divided into multispectral imagery and RGB imagery. The acquisition cost of multispectral imagery is relatively high, and the width is much narrower than that of RGB imagery. It is troublesome to obtain multi-view images that meet the overlap requirements, so it is not easy to use them to restore the three-dimensional structural information of crops. Existing research based on RGB imagery mainly processes two-dimensional images based on deep learning methods. Although it is possible to identify multiple phenological periods using deep learning methods based on a single RGB image data, it lacks a mechanism explanation and may have a problem of poor generalization. In addition, the distinction between the three adjacent phenological periods of tillering, jointing, and booting with similar texture features is not effective. In addition, the image deep learning-based method does not use the geometric information obtained from multi-view images to more accurately identify the phenological period. Although the key points of the multispectral time series curve (extreme points, zero points, and derivative curve key points) have good generalization performance, they rely on time series data and can identify relatively few rice phenological periods, only two or three.

[0008] In summary, by using the geometric information in the photogrammetric point cloud (which also contains simple spectral information and canopy three-dimensional structure information) constructed using low-altitude UAV RGB data, assisted by RGB color information, and supported by agronomic mechanisms, we explored a set of simple, reliable, low-cost, and easy-to-popularize rice phenological period extraction methods, which can improve the existing methods that are not effective in identifying tillering, jointing, and booting stages, and have important practical significance for accelerating breeding. Summary of the invention

[0009] In view of the deficiencies of the prior art, the present invention provides a method for non-destructive identification of rice phenological period based on photogrammetric point cloud, comprising the following steps:

[0010] Step 1: Use a drone equipped with an RGB sensor to regularly obtain multi-view RGB images of the rice planting area during the rice growing period, perform three-dimensional reconstruction on the obtained multi-view RGB images, and obtain rice time-series point cloud data of the rice planting area throughout the entire growing period;

[0011] Step 2, based on the rice time-series point cloud data of the whole growth period of the rice planting area obtained in step 1, extract the time-series plant height and the time-series green-red ratio index of the rice, and obtain the time-series plant height curve and the time-series green-red ratio index curve of the rice;

[0012] Step 3, extract the date corresponding to the highest value of plant height in the time series, and record it as the heading date;

[0013] Step 4, deriving the time series plant height curve to obtain the daily average growth rate curve of rice, using the binary clustering method to divide the daily average growth rate into two categories, obtaining the daily average growth rate threshold, selecting the first period in the daily average growth rate curve that continuously exceeds the daily average growth rate threshold as the jointing period, and recording the start and end dates of the period as the jointing start date and the jointing end date, respectively;

[0014] Step 5, based on the Douglas-Peucker algorithm, the inflection point of the time series green-red ratio exponential curve is extracted, and the date corresponding to the inflection point is used as the first booting date;

[0015] Step 6, extracting the point on the time series green-red ratio index curve after heading when the time series green-red ratio index is 1 for the first time, and the corresponding date is taken as the late waxy maturity day;

[0016] Step 7: Combine prior knowledge of agronomy to identify and divide rice phenological periods.

[0017] Furthermore, the time series plant height in step 2 is the median of the height from the point cloud to the ground in the rice planting area. The rice seedlings have not emerged before transplanting, so the plant height is 0. The green-red ratio index is the ratio of the DN value of the green band to the DN value of the red band of the point cloud in the rice planting area.

[0018] Furthermore, in step 3, the date corresponding to the maximum value in the time-series plant height data is directly recorded as the heading date, or the time-series plant height data is first interpolated with cubic spline, and then the date corresponding to the maximum value of the plant height after interpolation is taken as the heading date.

[0019] Furthermore, in step 5, the inflection point of the green-red ratio index curve is extracted based on the Douglas-Peucker algorithm. The specific operation is: connecting the two end points of the green-red ratio index curve to form a straight line, calculating the distance from each point on the green-red ratio index curve except the two end points to the straight line, taking the point corresponding to the maximum distance as the inflection point of the green-red ratio index curve, and taking the date corresponding to the inflection point as the first booting date.

[0020] Furthermore, in step 6, it is determined whether each line segment on the green-red ratio index curve after heading intersects with the straight line green-red ratio index equal to 1, and the date corresponding to the first intersection point is taken as the late waxy maturity date.

[0021] Furthermore, in step 7, the average time of the end of jointing and the beginning of booting is taken as the critical day A1 of the booting stage and jointing stage. It is the day of the first ear of corn. The final identification and division results of rice phenological periods are as follows: the period from transplanting to the start of jointing is the greening and tillering period, the period from the start of jointing to the critical day A1 is the jointing period, the critical day A1 to the first heading day is the heading period, the first heading day to the full heading day is the heading period, and the full heading day to the late waxy maturity day is the grain filling maturity period.

[0022] The present invention also provides a non-destructive identification system for rice phenological period based on photogrammetric point cloud, which is used to implement the non-destructive identification method for rice phenological period based on photogrammetric point cloud as described above.

[0023] Furthermore, the method comprises a processor and a memory, wherein the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute the above-mentioned non-destructive identification method of rice phenological period based on photogrammetry point cloud.

[0024] Alternatively, it comprises a readable storage medium having a computer program stored thereon, and when the computer program is executed, the above-mentioned method for non-destructive identification of rice phenological period based on photogrammetric point cloud is implemented.

[0025] Compared with the prior art, the present invention has the following advantages:

[0026] 1) The present invention uses only RGB images to identify rice phenological periods, which is low-cost compared to remote sensing identification of rice phenology using multispectral images; 2) There are significant characteristics in the remote sensing plant height of rice during the jointing stage (the stage with the fastest height growth) and heading stage (the panicle emerges in a short time, and the maximum canopy height in the whole growth period is reached during the heading stage). The present invention uses multi-view RGB image three-dimensional reconstruction to restore the plant height information of the crop, and applies it to the identification of rice phenological periods. It can well identify the jointing stage and heading stage that are difficult to identify using only spectral information; 3) The phenological period identification method proposed in the present invention can be automatically implemented by computer algorithms, which is simple and labor-saving, and is very beneficial for use in breeding a large number of small plots. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0028] Figure 1 It is a flow chart of a method for non-destructive identification of rice phenological period based on photogrammetric point cloud according to an embodiment of the present invention.

[0029] Figure 2 This is a schematic diagram of the extraction results of Qisui Ri in an embodiment of the present invention. Figure 2 (a) The date corresponding to the maximum plant height in the observed data is extracted as the heading day. Figure 2 (b) The date corresponding to the maximum plant height after cubic spline interpolation is the heading date.

[0030] Figure 3 This is a schematic diagram of the extraction results at the jointing stage of an embodiment of the present invention. Figure 3 (a) is the average daily growth rate of rice after transplanting. Figure 3 Middle (b) is the identification result of the jointing stage.

[0031] Figure 4 It is a schematic diagram of the extraction results of the booting day of an embodiment of the present invention.

[0032] Figure 5 It is a schematic diagram of the extraction results of the late wax maturity stage in an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical scheme and advantages of the present invention clearer, the technical scheme of the present invention is further described below in conjunction with the accompanying drawings and embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0034] Example 1

[0035] like Figure 1 As shown, the embodiment of the present invention provides a non-destructive identification method of rice phenological period based on photogrammetric point cloud, comprising the following steps:

[0036] Step 1: Use a drone equipped with an RGB sensor to regularly obtain multi-view RGB images of the rice-growing area during the rice growing period, perform three-dimensional reconstruction on the obtained multi-view RGB images, and obtain the rice time-series point cloud data of the rice-growing area throughout the entire growing period.

[0037] Use a DJI drone equipped with an RGB sensor to regularly collect multi-view RGB images of the rice planting area during the rice growing period. Use pix4d software to perform three-dimensional reconstruction processing on the collected multi-view RGB images, and obtain the time-series point cloud data of the rice planting area based on the unified spatial position of the phased points. The rice variety collected in this embodiment is Fengliangyou No. 4, which was sown on May 10, 2024, transplanted on June 15, 2024, and harvested on September 15, 2024. The planting density is 30 holes / ㎡, the row spacing is 20cm, and the plant spacing is 16.7cm.

[0038] Step 2, based on the time-series point cloud data of the rice planting area throughout the growth period obtained in step 1, extract the time-series plant height and time-series green-red ratio index of rice, and draw the time-series plant height curve and time-series green-red ratio index curve of rice.

[0039] Plant height is the median value of the height from the point cloud to the ground in the rice planting area. Arrange the height data from the point cloud to the ground in the rice planting area in ascending or descending order. If there are n heights, when n is an even number, the median is the average of the n / 2th and (n+2) / 2th digits; if n is an odd number, the median is the height of the (n+1) / 2th digit. The Green Red Ratio Index (GRRI) is the ratio of the green band DN (Digital Number, i.e., the sensor response value) value of the point cloud in the rice planting area to the red band DN value.

[0040] Before transplanting, rice seedlings had not emerged, so the plant height was 0. The plant height and GRRI results extracted in this example are shown in Table 1. The DAT (Days After Transplanting) in Table 1 is obtained by subtracting the transplanting date from the actual date.

[0041] Table 1 Plant height and GRRI during the whole growth period in rice planting areas

[0042]

[0043] Step 3, extract the date corresponding to the maximum plant height and record it as the heading date P2.

[0044] The date corresponding to the maximum value in the time series plant height data can be directly recorded as the heading date, or the time series plant height data can be first interpolated by cubic spline, and then the date corresponding to the maximum value of the plant height after interpolation is taken as the heading date. Figure 2 As shown in (a), the date corresponding to the highest point is August 17, 2024, and the DAT is 63. The cubic spline interpolation results of plant height in the rice planting area are as follows: Figure 2 As shown in (b), the date corresponding to the highest point of the fitting is August 19, 2024, and the DAT is 65. In this embodiment, the result of cubic spline interpolation is taken, and the harvest date P2 is August 19, 2024.

[0045] Step 4, deriving the time-series plant height curve to obtain the daily average growth rate curve of rice, using the binary clustering method to divide the daily average growth rate into two categories, obtaining the daily average growth rate threshold, selecting the first period in the daily average growth rate curve that continuously exceeds the daily average growth rate threshold as the jointing period, and recording the start and end days of this period as the jointing start day T1 and the jointing end day T2, respectively.

[0046] right Figure 2 The daily average growth rate curve of rice can be obtained by taking the derivative of the time series plant height curve shown in (a), as shown in Figure 3 As shown in (a). The dichotomous clustering method is used to divide the daily average growth rate into two categories, and the threshold value of the daily average growth rate is The threshold value of daily average growth rate is This example uses the natural break method to classify the daily average growth rate, and the daily average growth rate threshold is 0.038 m / day. Figure 3 The first period in (a) when the daily average growth rate exceeds the threshold value of 0.038 m / day continuously is the jointing stage. Figure 3 In this embodiment, the DAT of the jointing start date T1 is 21, corresponding to July 6, 2024, and the DAT of the jointing end date T2 is 25, corresponding to July 10, 2024.

[0047] Step 5: Based on the Douglas-Peucker algorithm, the inflection point of the GRRI curve is extracted, and the date corresponding to the inflection point is used as the first booting date B1.

[0048] Based on the Douglas-Peucker algorithm, the inflection point of the GRRI curve is extracted. The specific operation is: connect the two end points of the GRRI curve to form a straight line, calculate the distance from each point on the GRRI curve except the two end points to the straight line, and take the point corresponding to the maximum distance as the inflection point of the GRRI curve. Figure 4 As shown, the date corresponding to the inflection point, July 14, 2024, is taken as the booting date B1, and DAT is 29.

[0049] Step 6, extracting the date corresponding to the first point where the GRRI is 1 on the GRRI curve after heading as the late waxy maturity day R1.

[0050] Determine whether each line segment on the GRRI curve after heading intersects with GRRI=1, and take the date corresponding to the first intersection point as the late waxy maturity date. The heading date extracted in this embodiment is August 19, 2024, so the GRRI line segments of August 17-August 21, August 21-August 26, August 26-September 1, September 1-September 6, September 6-September 11, and September 11-September 15 are traversed in sequence, and it is recognized that the line segment composed of the GRRI data points of the two adjacent dates of September 11-September 15 intersects with GRRI=1 for the first time, such as Figure 5 The date corresponding to the intersection, September 12, 2024, is recorded as the late wax maturity day R1, and the DAT is 89.

[0051] Step 7: Combine prior knowledge of agronomy to identify and divide rice phenological periods.

[0052] The average time of the end of jointing day T2 and the beginning of heading day B1 was taken as the critical day A1 of the heading period and jointing period. Combined with agronomic prior knowledge, 8 days before the heading date P2 was recorded as the heading date P1. The final identification and division results of the rice phenological period are as follows: the period from transplanting to the beginning of jointing day T1 is the greening and tillering period, the beginning of jointing day T1 to the critical day A1 is the jointing period, the critical day A1 to the beginning of heading day P1 is the heading period, the beginning of heading day P1 to the heading day P2 is the heading period, the heading day P2 to the late waxy maturity day R1 is the filling maturity period, and the harvest can be carried out shortly after the late waxy maturity day R1.

[0053] In this embodiment, the jointing end date T2 is July 10, 2024, the booting start date B1 is July 14, 2024, and the average date is July 12, 2024, which is recorded as the critical date A1 of the booting and jointing period. The extracted full-heading date is August 19, 2024, and the booting start date P1 is August 11, 2024. The final identification and division results of rice phenological periods are as follows: transplanting (June 15, 2024) to T1 (July 6, 2024) is the greening and tillering period, T1 (July 6, 2024) to A1 (July 12, 2024) is the jointing period, A1 (July 12, 2024) to P1 (August 11, 2024) is the booting period, P1 (August 11, 2024) to P2 (August 19, 2024) is the heading period, P2 (August 19, 2024) to R1 (September 12, 2024) is the filling and maturity period, and it can be harvested shortly after R1 (September 12, 2024).

[0054] Example 2

[0055] Based on the same inventive concept, the present invention also provides a non-destructive identification system for rice phenological period based on photogrammetric point cloud, including a processor and a memory, the memory is used to store program instructions, and the processor is used to call the program instructions in the memory to execute the above-mentioned non-destructive identification method for rice phenological period based on photogrammetric point cloud.

[0056] Example 3

[0057] Based on the same inventive concept, the present invention also provides a non-destructive identification system for rice phenological period based on photogrammetric point cloud, including a readable storage medium, on which a computer program is stored. When the computer program is executed, a non-destructive identification method for rice phenological period based on photogrammetric point cloud as described above is implemented.

[0058] In specific implementation, the method proposed in the technical solution of the present invention can be implemented by technical personnel in this field using computer software technology to realize automatic operation process. System devices for implementing the method, such as computer-readable storage media storing the corresponding computer program of the technical solution of the present invention and computer equipment running the corresponding computer program, should also be within the scope of protection of the present invention.

[0059] The specific embodiments described herein are merely examples of the spirit of the present invention. Those skilled in the art may make various modifications or additions to the specific implementation cases described, or replace them with similar methods, without departing from the spirit of the present invention or exceeding the scope defined by the appended claims.

Claims

1. A non-destructive identification method of rice phenological period based on photogrammetric point cloud, characterized in that: The following steps are involved: Step 1: Use a drone equipped with an RGB sensor to regularly obtain multi-view RGB images of the rice planting area during the rice growing period, perform three-dimensional reconstruction on the obtained multi-view RGB images, and obtain rice time-series point cloud data of the rice planting area throughout the entire growing period; Step 2, based on the rice time-series point cloud data of the whole growth period of the rice planting area obtained in step 1, extract the time-series plant height and the time-series green-red ratio index of the rice, and obtain the time-series plant height curve and the time-series green-red ratio index curve of the rice; Step 3, extract the date corresponding to the highest value of plant height in the time series, and record it as the heading date; Step 4, deriving the time series plant height curve to obtain the daily average growth rate curve of rice, using the binary clustering method to divide the daily average growth rate into two categories, obtaining the daily average growth rate threshold, selecting the first period in the daily average growth rate curve that continuously exceeds the daily average growth rate threshold as the jointing period, and recording the start and end dates of the period as the jointing start date and the jointing end date, respectively; Step 5, based on the Douglas-Peucker algorithm, the inflection point of the time series green-red ratio exponential curve is extracted, and the date corresponding to the inflection point is used as the first booting date; Step 6, extracting the point on the time series green-red ratio index curve after heading when the time series green-red ratio index is 1 for the first time, and the corresponding date is taken as the late waxy maturity day; Step 7: Combine prior knowledge of agronomy to identify and divide rice phenological periods.

2. The non-destructive identification method of rice phenological period based on photogrammetric point cloud according to claim 1, characterized in that: In step 2, the time-series plant height is the median value of the height from the point cloud to the ground in the rice planting area. The rice seedlings have not emerged before transplanting, so the plant height is 0; the green-red ratio index is the ratio of the DN value of the green band to the DN value of the red band of the point cloud in the rice planting area.

3. The non-destructive identification method of rice phenological period based on photogrammetric point cloud according to claim 1, characterized in that: In step 3, the date corresponding to the maximum value in the time series plant height data is directly recorded as the heading date, or the time series plant height data is first interpolated with cubic spline, and then the date corresponding to the maximum value of the plant height after interpolation is taken as the heading date.

4. The non-destructive identification method of rice phenological period based on photogrammetric point cloud according to claim 1, characterized in that: In step 5, the inflection point of the green-red ratio index curve is extracted based on the Douglas-Peucker algorithm. The specific operation is: connect the two end points of the green-red ratio index curve to form a straight line, calculate the distance from each point on the green-red ratio index curve except the two end points to the straight line, take the point corresponding to the maximum distance as the inflection point of the green-red ratio index curve, and take the date corresponding to the inflection point as the first booting date.

5. The non-destructive identification method of rice phenological period based on photogrammetric point cloud according to claim 1, characterized in that: In step 6, it is determined whether each line segment on the green-red ratio index curve after heading intersects with the straight line green-red ratio index equal to 1, and the date corresponding to the first intersection point is taken as the late waxy maturity date.

6. The non-destructive identification method of rice phenological period based on photogrammetric point cloud according to claim 1, characterized in that: In step 7, the average time of the end of jointing and the beginning of booting is taken as the critical day A1 of the booting period and jointing period; combined with agronomic prior knowledge, the critical day A1 of the booting period is taken as the critical day A2 of the jointing period. It is the day of the first ear of corn. The values ​​are taken according to the actual situation; the final identification and division results of rice phenological periods are as follows: the period from transplanting to the start of jointing is the greening and tillering period, the period from the start of jointing to the critical day A1 is the jointing period, the critical day A1 to the first heading day is the heading period, the first heading day to the full heading day is the heading period, and the full heading day to the late waxy maturity day is the filling maturity period.

7. A non-destructive identification system for rice phenological period based on photogrammetric point cloud, characterized in that: The invention comprises a processor and a memory, wherein the memory is used to store program instructions, and the processor is used to call the program instructions in the memory to execute a non-destructive identification method of rice phenological period based on photogrammetric point cloud as described in any one of claims 1 to 6.

8. A non-destructive identification system for rice phenological period based on photogrammetric point cloud, characterized in that: It comprises a readable storage medium, on which a computer program is stored, and when the computer program is executed, a non-destructive identification method of rice phenological period based on photogrammetric point cloud as described in any one of claims 1 to 6 is implemented.

Images