Three-dimensional variable fertilization device and method thereof

Abstract

The invention relates to a three-dimensional variable fertilization device and method, and the device firstly plans a first flight path, and then controls an unmanned aerial vehicle to fly along the first flight path and shoot an image of a paddy field in the air; after the rice uniformity information of each rice field is obtained based on the image information, the rice growth abnormity type of each rice field is judged based on the rice uniformity information of the rice field and the image information of the rice field, and then the fertilizer supplementing and spraying type needed by each rice field is determined. Determining a fertilizer supplementing spraying amount corresponding to the paddy field based on the image information of the paddy field and the fertilizer supplementing spraying type, and planning a second flight path; and finally, controlling the unmanned aerial vehicle to fly along the second flight path and performing fertilizer supplementing and pesticide spraying operation according to the planning scheme. The application has the advantage of maximizing the fertilization and pesticide spraying effect by adjusting the variety and dosage of fertilization and pesticide spraying according to the growth condition of the paddy field in real time.

Description

Technical Field

[0001] The present application relates to the field of smart agriculture, and in particular to a three-dimensional variable fertilization device and method thereof. Background Art

[0002] In recent years, with the rapid development of drone technology, the application of drones in agriculture has received widespread attention and application. Due to its advantages such as flexible operation, high efficiency and wide coverage, drones have gradually become one of the main tools for rice fertilization and spraying. In traditional agriculture, fertilization and spraying usually require a lot of manpower and time, while drones can complete these tasks quickly and accurately, thereby significantly improving the efficiency and effectiveness of agricultural production.

[0003] The technology of using drones to fertilize and spray rice mainly includes: drones are equipped with spraying equipment, fly according to a preset path and speed, and evenly spray fertilizers or pesticides according to the set dosage. By using high-precision GPS positioning systems and geographic information systems (GIS), drones can achieve precision agriculture, reduce the waste of fertilizers and pesticides, and reduce environmental pollution. In addition, drones can also monitor the growth of rice in real time by carrying different sensors, providing data support for agricultural decision-making.

[0004] However, crop growth is currently affected by many factors, such as climate, soil conditions, and pests and diseases, which change rapidly and interact with each other in a complex manner. In the past, manual judgment was usually used to evaluate the type of pesticides and fertilizers that needed to be sprayed on each rice field. However, it is difficult for drones to adjust the type and dosage of fertilizers and sprays in real time according to the growth conditions of the rice fields, resulting in the inability to maximize the effect of fertilizer spraying. Summary of the invention

[0005] In order to adjust the type and dosage of fertilizers and sprays according to the growth conditions of rice fields in real time to maximize the effect of fertilizers and sprays, the present application provides a three-dimensional variable fertilization device and method thereof.

[0006] In the first aspect, the present application provides a three-dimensional variable fertilization device, which adopts the following technical solution:

[0007] A three-dimensional variable fertilization device, comprising:

[0008] A first path planning module is used to plan a first flight path; wherein the first flight path traverses the area above each rice field;

[0009] An image information acquisition module, used to control the UAV to fly along a first flight path and take images of the rice fields from above;

[0010] A uniformity calculation module is used to obtain rice uniformity information of each rice field based on image information;

[0011] An abnormality judgment module, used for judging the type of abnormal rice growth in each rice field based on the rice uniformity information and the image information of the rice field;

[0012] A fertilizer spraying type determination module is used to determine the type of fertilizer spraying required for each rice field based on the type of abnormal rice growth;

[0013] A fertilizer spraying quantity determination module is used to determine the corresponding fertilizer spraying quantity of the rice field based on the image information of the rice field and the fertilizer spraying type;

[0014] A second path planning module is used to plan a second flight path based on the type and amount of fertilizer spraying required for the rice field;

[0015] The flight and fertilizer spraying control module is used to control the UAV to fly along the second flight path and perform fertilizer spraying operations according to the planned scheme.

[0016] By adopting the above technical solution, the device detects all the rice fields as a whole through the first round of drone flights, and obtains the uniformity information of the rice fields through the images obtained by the detection. Due to the large overall area of ​​all rice fields, if all rice fields are subjected to refined image recognition, such as the recognition granularity is accurate to a single rice plant, the computing power requirements will be too high and the processing time will be long. In fact, there are fewer rice fields with problems, so in this application, the abnormal rice fields are preliminarily screened by the feature of averageness, and the abnormal type is determined after the preliminarily screening. Because when a single rice field is affected by pests and diseases or lacks certain nutrients, the rice inside it does not show symptoms uniformly, but locally and spreads. For example, when a single rice field lacks phosphate fertilizer, the phosphate fertilizer level of the land does not decrease uniformly, but the local phosphorus level is lower than the threshold, and then other locations gradually fall below the threshold. Therefore, the uniformity of the plants in the rice field will change.

[0017] For rice fields with abnormal uniformity, since different types of abnormal rice growth correspond to different rice shapes, this device analyzes the image of the rice field to obtain the type of abnormal rice growth, thereby determining the type of supplementary fertilizer spray required. Since the degree of abnormality of different rice fields is different, the amount of supplementary fertilizer spray required for the rice field can be determined based on the image recognition analysis results. The more serious the abnormality of the rice field, the more pesticides or fertilizers are needed.

[0018] Since the drone carries the material to be sprayed from the starting point to the rice fields that need to be sprayed for spraying operations, different rice fields require different spraying amounts, and different rice fields are at different distances from the starting point. Therefore, it is necessary to plan the flight path based on the flight distance and the weight of rice fertilizer, two different weights, to try to let the heavy and near ones fly first, and the light and far ones fly later, so as to improve work efficiency.

[0019] Therefore, the device can adjust the type and dosage of fertilizer spraying according to the growth conditions of the rice field in real time to maximize the effect of fertilizer spraying.

[0020] Optionally, the abnormality judgment module includes:

[0021] A uniformity threshold judgment unit, used to judge whether the uniformity of the rice field is lower than a preset threshold;

[0022] An image calibration unit, used to calibrate the image of the rice field whose uniformity is lower than a preset threshold, wherein the calibration content is the minimum bounding box, the number of leaves and the tillering level of each rice plant in the image;

[0023] A feature recognition and extraction unit, used for performing instance segmentation of rice plants based on a minimum bounding box, and inputting the segmentation instance into a pre-trained feature extraction model to extract feature information of the rice seedlings, wherein the feature information includes color features, morphological features and texture features;

[0024] The abnormal growth classification unit is used to input the extracted features into the trained abnormal growth classification model to identify the abnormal growth type of each rice plant; wherein the abnormal growth type includes normal type, various fertilizer deficiency types, and various pest and disease types;

[0025] The statistical judgment unit is used to count the abnormal growth types of each rice plant to obtain the abnormal growth type of the rice field.

[0026] By adopting the above technical solution, when the module detects that the uniformity of the rice field is lower than the preset threshold, it means that the rice field is growing abnormally. The image processing method is used to obtain the minimum bounding box, number of leaves, and tillering level of each rice plant in the image. The minimum bounding box, number of leaves, and tillering level are used as prior conditions to perform instance segmentation on a single rice plant. Instance segmentation can eliminate interference from other rice plants and background. The segmented rice plants are then input into the pre-trained feature extraction model to extract the feature information of the rice seedlings, thereby obtaining the growth abnormality class. Since there may be multiple abnormal types of rice in a single rice field, it is necessary to use a statistical method to determine the type of growth abnormality.

[0027] Optionally, the second path planning module includes:

[0028] The rice field variable initialization unit is used to obtain the coordinate set P of all rice fields corresponding to a certain growth abnormality type in all rice fields, and the spraying amount a required for each rice field i and the corresponding spraying time T i , where the drone performs spraying operations over the rice field through a closed path, and the coordinates of the rice field are the starting point and end point of the drone's spraying operation path over the rice field;

[0029] Drone status initialization unit, used to obtain the drone's initial power E max 、Total amount of material to be sprayed A max And the starting point coordinates p0;

[0030] Algorithm initialization unit, used to define D[i][j] to represent point p i To point p j The distance between the ants and the iteration number is defined as m, the pheromone matrix is ​​defined as τ[i][j] and initialized, and the heuristic function is defined as η[i][j] = 1 / D[i][j];

[0031] The path construction unit is used to make each ant start from the starting point and build a path until all points are visited or resources are exhausted; wherein the path connects the interval start point and the interval end point of adjacent points, and the resources include the remaining power and the remaining spraying amount. The probability of each ant selecting the next point from the current point is:

[0032]

[0033] The drone status update unit is used to update the drone's remaining power and remaining spraying amount after the ant selects the next point. If the power is not enough to fly to the next point or the remaining spraying amount is not enough to complete the spraying, it returns to the starting point p0 for resupply;

[0034] The pheromone update unit is used to locally update the pheromone τ each time the ant passes through a path (i, j) during the path construction process. ij ←(1-ρ)τ ij +ρΔτ ij , where ρ is the volatility coefficient, Δτ ij To increase the amount of pheromones;

[0035] The global update unit is used to globally update the pheromone matrix after all ants have completed the path construction:

[0036]

[0037] in, L is the path length, Q is a constant;

[0038] The optimal path output unit is used to terminate the loop after the preset number of iterations t is reached, and output the path with the highest pheromone value as the optimal path.

[0039] By adopting the above technical solution, the ant colony algorithm is used to calculate the optimal path, where the allowed set only contains points that are currently sufficient to reach with the remaining power. This constraint ensures that the drone will not choose an unreachable point, thereby reducing the number of times it returns to recharge. When the global pheromone is updated, higher rewards are given to paths with fewer recharges, which can guide the ants to choose these paths in subsequent iterations.

[0040] Optionally, the uniformity calculation module includes:

[0041] A tillering level calculation unit, used to obtain the number of leaves based on the rice image, and obtain the tillering level in combination with the leaf age;

[0042] An instance segmentation unit is used to perform image recognition and image frame segmentation on the rice image based on the tillering level as a priori condition, and obtain the minimum external bounding box of a single rice plant image;

[0043] A coverage calculation unit, used to calculate the horizontal distance between the external frames according to the minimum external frame of the single rice plant image, and calculate the coverage according to the ratio of the horizontal distance and the width of the adjacent minimum external frame;

[0044] The average degree calculation unit is used to calculate the coefficient of variation of the coverage in the image based on each calculated coverage to express the uniformity.

[0045] By adopting the above technical solution, the coefficient of variation CV = standard deviation / average value, the smaller the CV value, the more uniform the seedling emergence; the larger the CV value, the uneven the seedling emergence.

[0046] Optionally, the instance segmentation unit includes:

[0047] A model preloading subunit, used for preloading pre-trained image recognition models corresponding to different tillering levels, wherein different image recognition models are trained based on different data sets, and each data set corresponds to a rice image of a tillering level;

[0048] A tillering level calculation subunit, used for selecting an image recognition model corresponding to the tillering level of rice in the rice image based on the calculated tillering level of the rice;

[0049] An image frame determination subunit is used to call the selected image recognition model to recognize the rice image to obtain a single rice plant image frame area;

[0050] A semantic segmentation subunit, used for performing semantic segmentation on each individual rice plant in the individual rice plant image frame area to obtain a contour line;

[0051] The minimum bounding box determination subunit is used to generate a target minimum bounding box on the rice image based on the image frame area and contour line of a single rice plant.

[0052] In the second aspect, the present application provides a three-dimensional variable fertilization method, which adopts the following technical solution:

[0053] A three-dimensional variable fertilization method comprises the following steps:

[0054] S1. Planning a first flight path; wherein the first flight path traverses the area above each rice field;

[0055] S2. Control the UAV to fly along the first flight path and take images of the rice fields from above;

[0056] S3. Obtaining rice uniformity information of each rice field based on the image information;

[0057] S4. Determine the type of abnormal rice growth in each rice field based on the rice uniformity information and the image information of the rice field;

[0058] S5. Determine the type of fertilizer spray required for each rice field based on the abnormal rice growth type;

[0059] S6. Determine the amount of fertilizer spraying corresponding to the rice field based on the image information of the rice field and the type of fertilizer spraying;

[0060] S7. Planning a second flight path based on the type and amount of fertilizer spraying required for the rice fields;

[0061] S8. Control the UAV to fly along the second flight path and perform fertilizer and pesticide spraying operations according to the planned scheme.

[0062] By adopting the above technical solution, the method detects all the rice fields as a whole through the first round of drone flights, and obtains the uniformity information of the rice fields through the images obtained by the detection. Due to the large overall area of ​​all rice fields, if all rice fields are subjected to refined image recognition, such as the recognition granularity is accurate to a single rice plant, the computing power requirements will be too high and the processing time will be long. In fact, there are fewer rice fields with problems, so in this application, the abnormal rice fields are preliminarily screened by the feature of averageness, and the abnormal type is determined after the preliminarily screening. Because when a single rice field is affected by pests and diseases or lacks certain nutrients, the rice inside it does not show symptoms uniformly, but occurs locally and spreads. For example, when a single rice field lacks phosphate fertilizer, the phosphate fertilizer level of the land does not decrease uniformly, but the local phosphorus level is lower than the threshold, and then other locations gradually fall below the threshold. Therefore, the uniformity of the plants in the rice field will change.

[0063] For rice fields with abnormal uniformity, since different types of abnormal rice growth correspond to different rice shapes, this method analyzes the image of the rice field to obtain the type of abnormal rice growth, thereby determining the type of supplementary fertilizer spray required. Since the degree of abnormality of different rice fields is different, the amount of supplementary fertilizer spray required for the rice field can be determined based on the image recognition analysis results. The more serious the abnormality of the rice field, the more pesticides or fertilizers are needed.

[0064] Optionally, S4 includes the following sub-steps:

[0065] S41. Determine whether the uniformity of the rice field is lower than a preset threshold;

[0066] S42. Calibrate the image of the rice field whose uniformity is lower than a preset threshold, wherein the calibration content is the minimum bounding box, the number of leaves and the tillering level of each rice plant in the image;

[0067] S43. Performing instance segmentation on the rice plant based on the minimum bounding box, and inputting the segmentation instance into a pre-trained feature extraction model to extract feature information of the rice seedling, wherein the feature information includes color features, morphological features, and texture features;

[0068] S44. Inputting the extracted features into the trained abnormal growth classification model to identify the abnormal growth type of each rice plant; wherein the abnormal growth type includes a normal type, a variety of fertilizer deficiency types, and a variety of pests and diseases types;

[0069] S45. Count the abnormal growth types of each rice plant to obtain the abnormal growth type of the rice field.

[0070] By adopting the above technical solution, when this step detects that the uniformity of the rice field is lower than the preset threshold, it means that the growth of the rice field is abnormal. The image processing method is used to obtain the minimum bounding box, number of leaves, and tillering level of each rice plant in the image. The minimum bounding box, number of leaves, and tillering level are used as prior conditions to perform instance segmentation on a single rice plant. Instance segmentation can eliminate interference from other rice plants and background. The segmented rice plants are then input into a pre-trained feature extraction model to extract the feature information of the rice seedlings, thereby obtaining the growth abnormality class. Since there may be multiple abnormal types of rice in a single rice field, it is necessary to use a statistical method to determine the type of growth abnormality.

[0071] Optionally, the S7 includes the following steps:

[0072] S71. Obtain the coordinate set P of all rice fields corresponding to a certain growth abnormality type in all rice fields, and the spraying amount a required for each rice field i and the corresponding spraying time Ti , where the drone performs spraying operations over the rice field through a closed path, and the coordinates of the rice field are the starting point and end point of the drone's spraying operation path over the rice field;

[0073] S72. Get the drone's initial power E max 、Total amount of material to be sprayed A max And the starting point coordinates p0;

[0074] S73. Define D[i][j] to represent point p i To point p j The distance between the ants and the iteration number is defined as m, the pheromone matrix is ​​defined as τ[i][j] and initialized, and the heuristic function is defined as η[i][j] = 1 / D[i][j];

[0075] S74. Each ant starts from the starting point and builds a path until all points are visited or resources are exhausted; wherein the path connects the interval start point and the interval end point of adjacent points, and the resources include the remaining power and the remaining spraying amount. The probability of each ant selecting the next point from the current point is:

[0076]

[0077] S75. After the ant selects the next point, the remaining power and the remaining amount of spraying material of the drone are updated. If the power is insufficient to fly to the next point or the remaining amount of spraying material is insufficient to complete the spraying, the drone returns to the starting point p0 for resupply;

[0078] S76. During the path construction process, each time the ant passes through a path (i, j), it locally updates the pheromone τ ij ←(1-ρ)τ ij +ρΔτ ij , where ρ is the volatility coefficient, Δτ ij To increase the amount of pheromones;

[0079] S77. After all ants have completed the path construction, the pheromone matrix is ​​globally updated:

[0080]

[0081] in, L is the path length, Q is a constant;

[0082] S78. After the iteration reaches the preset number t, the loop terminates and the path with the highest pheromone value is output as the optimal path.

[0083] Optionally, S3 includes the following steps:

[0084] S31. Obtaining the number of leaves based on the rice image and obtaining the tillering level in combination with the leaf age;

[0085] S32. Perform image recognition and image frame segmentation on the rice image with the tillering level as a priori condition to obtain the minimum bounding box of the single rice plant image;

[0086] S33. Calculate the horizontal distance between the bounding boxes according to the minimum bounding box of the single rice plant image, and calculate the coverage according to the ratio of the horizontal distance and the width of the adjacent minimum bounding box;

[0087] S34. Based on each calculated coverage, calculate the coefficient of variation of the coverage within the image to represent the uniformity.

[0088] Optionally, the S32 includes the following steps:

[0089] S321. Preload pre-trained image recognition models corresponding to different tillering levels, wherein different image recognition models are trained based on different data sets, and each of the data sets corresponds to a rice image of a tillering level;

[0090] S322. Based on the calculated tillering level of rice in the rice image, select an image recognition model corresponding to the tillering level;

[0091] S323. Calling the selected image recognition model to recognize the rice image to obtain a single rice plant image frame area;

[0092] S324. Performing semantic segmentation on each individual rice plant in the individual rice plant image frame region to obtain a contour line;

[0093] S325. Generate a target minimum bounding box on the rice image based on the single rice plant image frame area and contour lines. BRIEF DESCRIPTION OF THE DRAWINGS

[0094] Figure 1 A module connection diagram of a three-dimensional variable fertilization device in one embodiment of the present invention is shown.

[0095] Figure 2 A unit connection diagram of a uniformity calculation module according to an embodiment of the present invention is shown.

[0096] Figure 3 A diagram showing the subunit connections of an example segmentation unit in an embodiment of the present invention

[0097] Figure 4 A unit connection diagram of an abnormality determination module in one embodiment of the present invention is shown.

[0098] Figure 5 A unit connection diagram of the second path planning module according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0099] The present application is further described in detail below in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0100] In the following description, for the purpose of explanation, many specific details are set forth in order to provide a thorough understanding of the inventive concept. As part of this specification, some of the drawings in the drawings of the present disclosure represent structures and devices in the form of block diagrams to avoid making the disclosed principles complicated and difficult to understand. For the sake of clarity, not all features of the actual implementation are necessary to be described. In addition, the language used in this disclosure has been selected primarily for readability and instructional purposes, and may not be selected to delineate or limit the subject matter of the present invention, thereby resorting to the necessary claims to determine such inventive subject matter. References to "a specific implementation" or "specific implementations" in this disclosure mean that the specific features, structures or characteristics described in conjunction with the specific implementation are included in at least one specific implementation, and multiple references to "a specific implementation" or "specific implementations" should not be understood as necessarily referring to the same specific implementation.

[0101] Unless expressly limited, the terms "a", "an", and "the" are not intended to refer to a singular entity, but rather to include general categories of which specific examples may be used for illustration. Thus, the use of the terms "a" or "an" may mean any number of at least one, including "one", "one or more", "at least one", and "one or more than one". The term "or" means any of the alternatives and any combination of the alternatives, including all alternatives, unless the alternatives are expressly indicated to be mutually exclusive. The phrase "at least one of" when combined with a list of items refers to a single item in the list or any combination of the items in the list. The phrase does not require all of the listed items unless expressly limited to that.

[0102] Reference Figure 1 The present embodiment discloses a three-dimensional variable fertilization device, including a first path planning module, an image information acquisition module, a uniformity calculation module, an abnormality judgment module, a fertilizer spraying type determination module, a fertilizer spraying amount determination module, a second path planning module and a flight and fertilizer spraying control module.

[0103] The first path planning module is used to plan a first flight path; wherein the first flight path traverses the area above each rice field.

[0104] Use Geographic Information System (GIS) to obtain the geographic coordinate data of all rice fields. Assume there are four rice fields, and their center coordinates are P1 (x1, y1), P2 (x2, y2), P3 (x3, y3) and P4 (x4, y4).

[0105] Select the starting point P0 (x0, y0) of the drone, which is usually the location of the drone base or charging station. The end point can also be set to the starting point P0 to form a closed path, which is convenient for the drone to return for charging and data transmission.

[0106] The planned path needs to cover the area above all rice fields to ensure that every rice field can be photographed. The purpose of path planning is to collect data rather than optimize the path, so the focus is on coverage rather than path length.

[0107] Determine a path starting from the starting point P0 and passing through each rice field in sequence. For example, the path sequence can be set as: P0→P1→P2→P3→P4→P0.

[0108] The image information acquisition module is used to control the UAV to fly along the first flight path and take images of the rice fields in the air.

[0109] The image information acquisition module controls the drone to fly along the planned first flight path and take images of the rice fields from above. In some cases, such as when the shape of the rice fields is relatively rugged, in order to ensure that the image can cover the entire rice field and capture the details of the rice field clearly, a strategy of multiple low-altitude local flights can be adopted, and the images can be synthesized in the later stage.

[0110] For example, the appropriate flight altitude can be determined based on the resolution and shooting range of the camera carried by the drone. For example, if the camera's shooting width is 20 meters, the flight altitude can be set to 50 meters to ensure clear image details.

[0111] In order to ensure that the entire rice field is fully covered, the rice field needs to be divided into several small areas for shooting. For example, a rice field 200 meters long and 100 meters wide can be divided into 10 small areas of 20 meters x 20 meters, and each small area is photographed separately. The drone flies to the center point of each small area in turn according to the predetermined path and takes images.

[0112] When planning the route, you need to consider the overlapping areas of the shots for later image synthesis. Generally, the overlapping area between images should be 20%-30% to ensure seamless connection when the images are synthesized. For example, when shooting a small area of ​​20m x 20m, the horizontal and vertical intervals between each image are 15-16m to ensure overlap.

[0113] The uniformity calculation module is used to obtain the rice uniformity information of each rice field based on the image information. Figure 2 ,The uniformity calculation module includes a tillering level calculation unit, an ,instance segmentation unit, a coverage calculation unit and an average ,degree calculation unit.

[0114] The tillering level calculation unit is used to obtain the number of leaves based on the rice image and obtain the tillering level in combination with the leaf age.

[0115] Leaves are one of the main characteristics of rice growth. By identifying the number and shape of leaves, the growth status and health of rice can be effectively evaluated. In image recognition, the number of leaves is an important indicator that can be used to characterize the growth status of rice and is also the most significant identification feature of rice.

[0116] Rice fields are usually flooded, and the roots of rice are submerged in water, which makes identification based on leaf count alone inadequate:

[0117] Tillering is one of the growth characteristics of rice. As tillers increase, rice will differentiate into multiple branches from the main stem base, each with leaves. This will significantly increase the number of leaves in the image. If the tillering level is not considered and only the number of leaves is used for identification, a rice plant with a high tillering level may be mistakenly identified as multiple rice plants.

[0118] In a rice field, multiple rice plants may grow very close to each other, or may temporarily be close to each other due to factors such as wind. If identification is performed only by the number of leaves, these multiple rice plants may be mistakenly identified as one rice plant.

[0119] In order to overcome the above problems, it is necessary to calculate the tillering level and use it as a priori condition to improve the accuracy of image recognition. The calculation of tillering level can provide growth information of each rice plant, thus helping to distinguish single and multiple rice plants during the recognition process and improving the recognition effect.

[0120] The tiller-level computing unit is used to perform the following steps 1-5.

[0121] Step 1. Conduct connected region analysis and treat rice in several connected regions within a preset connected region area interval as individual rice plants.

[0122] Connected region analysis is an image processing technique used to identify and mark connected pixel regions in an image. These connected pixel regions are usually called connected components. The basic idea of ​​connected region analysis is to traverse each pixel of the image and mark all pixels connected to the current pixel as the same component, thereby distinguishing different target regions in the image.

[0123] Through connected region analysis, the rice plant area in the image can be automatically separated from the background area. This automatic segmentation can significantly reduce the workload of manual annotation and improve processing efficiency. However, when the number of rice leaves is large and the leaves are long, adjacent rice plants may have leaf crossings. Under wind, this leaf crossing phenomenon is more obvious, resulting in large errors in connected region analysis. The specific analysis is as follows:

[0124] When rice leaves cross, leaves from different plants are connected in the image to form a large connected area. This will cause the connected components of a single rice plant to be unable to be accurately segmented, and the recognition result includes multiple rice plants, increasing the error. When blown by the wind, the leaves will continue to swing, further exacerbating the situation of leaf crossing. Due to the crossing and overlapping of leaves, the area of ​​the connected area will become very large, exceeding the reasonable range of a single rice plant. In this case, the identified connected area cannot accurately reflect the actual situation of a single rice plant, affecting the subsequent growth analysis and uniformity calculation. Therefore, it is necessary to screen the area of ​​the connected area, and use the rice with the area of ​​the connected area within the preset interval as a single rice plant for subsequent analysis.

[0125] Specifically, step 1 includes the following sub-steps ah.

[0126] Step a. Calculate the green vegetation index ExG=2*GRB of each pixel, where G, R, and B are the pixel values ​​of the green, red, and blue channels respectively.

[0127] Step b. Convert the ExG image into a binary image by threshold segmentation method to distinguish vegetation from background.

[0128] For example, the Otsu threshold segmentation method is used to automatically select the optimal threshold, and pixels with ExG values ​​greater than the threshold are marked as white (foreground, i.e., rice plants), and pixels less than or equal to the threshold are marked as black (background).

[0129] Step c: Connect the broken leaf regions based on morphological operations and fill the holes to make the leaf regions more complete.

[0130] Morphological operations are a technique in image processing, which is mainly used to analyze and process the shape structure of images. Common morphological operations include dilation, erosion, opening, and closing.

[0131] For example, first take a binary image after threshold segmentation. The white area (pixel value is 255) in the image represents rice leaves, and the black area (pixel value is 0) represents the background. Then select a suitable structural element for morphological operations. Common structural elements include rectangles, circles, and ellipses. The size and shape of the structural element should be selected according to the characteristics of the image and the processing requirements.

[0132] Use the structural element to perform a dilation operation on the binary image. The dilation operation enlarges the target area in the image (i.e., the rice leaf) so that small gaps are filled and the broken leaf areas are connected. The dilation operation can make the leaf area more coherent and fill small breaks and gaps. Although this operation will make the leaf edge blurred, extracting a clear leaf image is not the purpose of this step, so it has no effect.

[0133] The same structural element is then used to perform an erosion operation on the dilated image. The erosion operation shrinks the target area in the image so that isolated small noise is removed while maintaining the coherence of the leaf area. The erosion operation can remove isolated noise in the image and restore the actual shape of the leaf.

[0134] Step d. Traverse each pixel of the binary image and mark the connected pixels as the same component.

[0135] Connected pixel regions are called connected components. Through recursive or iterative methods, connected pixels can be classified as the same connected region and assigned the same label value for subsequent processing and analysis. Connectivity can be divided into 4-connectivity and 8-connectivity. Among them, 4-connectivity means that each pixel is connected to the four pixels above, below, left and right; 8-connectivity means that each pixel is connected to the eight pixels around it, including the four pixels in the diagonal direction.

[0136] Step e: Use a recursive or iterative method to classify connected pixels into the same connected area and assign the same label value.

[0137] Starting from the upper left corner of the image, traverse each pixel. If the current pixel is a rice leaf and is not marked, use a stack or queue to mark all its connected pixels with the current mark value. Initialize an empty stack or queue and push the current pixel position into the stack or queue. Take a pixel position from the stack or queue, mark the pixel at that position, and push all its connected and unmarked pixel positions into the stack or queue. Repeat this process until the stack or queue is empty.

[0138] Step f. Record the features of each connected region, wherein the features include area, perimeter, and center position.

[0139] In connected region analysis, recording the features of each connected region helps to conduct detailed analysis and identification of targets in the image.

[0140] Area: The total number of pixels in a connected region, indicating the size of the region.

[0141] Perimeter: The total number of pixels on the border of a connected region, indicating the length of the border of the region.

[0142] Center position: The centroid of a connected region, which indicates the geometric center position of the region.

[0143] Step g. setting an upper area threshold and a lower area threshold, and excluding connected areas with too small an area and too large an area based on the upper area threshold and the lower area threshold, so as to filter out noise, non-target areas and overlapping areas of multiple rice seedlings.

[0144] In image processing and connected region analysis, the upper and lower area thresholds are set to exclude noise and non-target areas in the image. Connected regions with too small an area are usually noise or non-target areas, while connected regions with too large an area may be overlapping areas of multiple rice seedlings. By setting an appropriate area threshold, it can be ensured that only connected regions of reasonable size are retained.

[0145] Step h. Mark each remaining connected region using a different color or marker value to visualize the location and range of each individual rice plant in the image.

[0146] Step 2. Use the pre-trained convolutional neural network image recognition model to extract the number of leaves from each individual plant image.

[0147] The single plant image here refers to the regional image with a high probability of being a single rice plant obtained through connected region analysis in step 1, and each regional image contains only one rice plant. The pre-trained CNN model has been trained on a large-scale data set and can effectively extract features from the image. This embodiment uses a pre-trained CNN model to identify the number of leaves in the rice plant image. By processing each single rice plant image, the CNN model can automatically identify and calculate the number of leaves.

[0148] For example, the training steps of this convolutional neural network image recognition model are as follows:

[0149] Collect a large amount of training data containing images of individual rice plants. Make sure the dataset is diverse, covering different lighting conditions, angles, and backgrounds. In addition, each image needs to be annotated to record the number of leaves on each rice plant. The annotated data is used for supervised learning to train the CNN model. In addition, the images can be resized to a fixed size (e.g. 224x224 pixels) and data augmentation can be performed, such as rotation, scaling, flipping, etc., to increase the diversity of the data.

[0150] Choose a pre-trained CNN model, such as ResNet50, VGG16, or InceptionV3. Load the pre-trained model and freeze all layers except the last few layers to preserve the pre-trained weights. Add a new output layer for predicting the number of leaves. The number of neurons in the output layer should be the same as the number of possible leaves.

[0151] Then perform model training and compile the model using an appropriate loss function (such as mean squared error) and optimizer (such as Adam). Train the model on the training data and monitor the model performance using the validation data. Set up early stopping and model checkpoints to prevent overfitting and save the best model.

[0152] Step 3. Calculate the average number of leaves.

[0153] In rice fields, rice plants in a single rice field are usually in a similar growth environment, so the growth conditions of these plants are usually similar. However, due to slight differences in environmental factors in different locations (such as light, soil moisture, nutrient distribution, etc.), the growth conditions of individual plants may be different, resulting in certain differences in the number of leaves. Calculating the average number of leaves for multiple rice plants can balance the differences between individual plants and provide an overall growth condition indicator. The average number of leaves reflects the overall growth of the rice field.

[0154] Step 4. Calculate the leaf age based on the number of leaves and the number of days after transplanting, where growth days = image acquisition time - transplanting time, and leaf age = number of leaves + (growth days / 5).

[0155] Step 5. Determine the tillering level according to the leaf age. When the leaf age is less than 3, the tillering level is 1; when the leaf age is between 3 and 5, the tillering level is 2; when the leaf age is greater than 5, the tillering level is 3.

[0156] Tillering Stage is an indicator that describes the tillering stage of rice growth. Tillers are branches that grow from the base of the main stem during rice growth. The quantity and quality of tillers have an important impact on rice yield. Tillering Stage is determined based on the leaf age of rice, which is calculated by the number of leaves and the number of growing days.

[0157] The instance segmentation unit is used to perform image recognition and image frame segmentation on the rice image based on the tillering level as a priori condition, and obtain the minimum external bounding box of the single rice plant image.

[0158] Specifically, refer to Figure 3 ,The instance segmentation unit includes a model preloading subunit, a tillering level computation subunit, an image box determination subunit, a semantic segmentation subunit, and a minimum bounding box determination subunit.

[0159] The model preloading subunit is used to preload pre-trained image recognition models corresponding to different tillering levels, wherein different image recognition models are trained based on different data sets, and each data set corresponds to a rice image of a tillering level.

[0160] For example, the classification levels are 1, 2, and 3, corresponding to the first image recognition model, the second image recognition model, and the third image recognition model, respectively. The first image recognition model is trained with a data set of single rice plant images with a classification level of 1, the second image recognition model is trained with a data set of single rice plant images with a classification level of 2, and the third image recognition model is trained with a data set of single rice plant images with a classification level of 3. The training method of the image recognition model is similar to the above-mentioned convolutional neural network image recognition model, and other methods can also be used for training, which will not be repeated here.

[0161] The tillering level calculation subunit is used to select an image recognition model corresponding to the tillering level of rice in the rice image based on the calculated tillering level of the rice.

[0162] The images of rice at each tillering level have significant differences in morphology and features, because tillers are branches that grow from the base of the main stem during the growth process of rice. Rice at tillering level 1 may have fewer and sparse leaves, while rice at tillering level 3 has lush and dense leaves. By selecting image recognition models that are specifically trained for different tillering levels, the recognition ability and accuracy of the model can be ensured to be higher. In addition, using models trained for different tillering levels can reduce the complexity and training difficulty of the model. Each model only needs to focus on the features of a specific tillering level, thereby optimizing the model performance and improving the recognition speed and efficiency.

[0163] The image frame determination subunit is used to call the selected image recognition model to recognize the rice image to obtain the image frame area of ​​a single rice plant.

[0164] The image frame determination subunit separates each rice plant from the image through segmentation technology and uses an image frame to enclose the range of each rice plant. This subunit ensures the accurate recognition of each rice plant. During the recognition process, the interval of the minimum external frame reflects the actual interval of the rice. However, it should be noted that since the image frame obtained in this subunit is the image frame area containing a single rice plant, the size of the image frame is not necessarily the smallest, and the calculation of the interval between image frames requires that the edge of the image frame just touches the rice, that is, the minimum image frame, so further processing is required later.

[0165] As an example, the preprocessed rice image is input into the selected image recognition model. The model performs image processing and outputs the image frame area of ​​each rice plant. These image frame areas include the boundary information of the rice plant, which is used to delimit the range of a single rice plant. Specifically, the image recognition model is a convolutional neural network (CNN) structure for rice image recognition, including:

[0166] Input layer: receives the preprocessed rice images.

[0167] Convolutional Layers are used to extract the spatial features of images.

[0168] Pooling Layers are used to reduce data dimensions and retain important features.

[0169] Fully Connected Layers: used to integrate features and output prediction results.

[0170] The output layer is used to generate prediction results for the image box area.

[0171] For example, the input image size can be 224x224 pixels with RGB three channels.

[0172] Convolution and pooling layers:

[0173] Convolutional layer 1: 32 3x3 convolution kernels, ReLU activation function, Same padding.

[0174] Pooling layer 1: 2x2 max pooling.

[0175] Convolutional layer 2: 64 3x3 convolution kernels, ReLU activation function, Same padding.

[0176] Pooling layer 2: 2x2 max pooling.

[0177] Convolutional layer 3: 128 3x3 convolution kernels, ReLU activation function, Same padding.

[0178] Pooling layer 3: 2x2 max pooling.

[0179] Fully connected layer:

[0180] Fully connected layer 1: 512 neurons, ReLU activation function.

[0181] Fully connected layer 2: 256 neurons, ReLU activation function.

[0182] Output layer:

[0183] Output layer: 4 neurons, used to regress the coordinates and size of the output image box.

[0184] The semantic segmentation subunit is used to perform semantic segmentation on each individual rice plant in the individual rice plant image frame area to obtain a contour line.

[0185] Semantic segmentation is performed on individual rice plants in these image frame areas to accurately extract the contours of rice leaves and stems. Secondly, the use of semantic segmentation technology can better handle complex image features, such as overlapping leaves and complex backgrounds, to ensure the accuracy of the segmentation results. In addition, the contours obtained by semantic segmentation can be used to generate the minimum bounding box.

[0186] For example, the image frame regions of individual rice plants are extracted from the results of the image frame determination subunits. These regions contain the boundary information of each rice plant. The image frame regions of individual rice plants are input into the semantic segmentation model. The semantic segmentation model needs to be pre-trained to be able to recognize and segment rice leaves and stems. The semantic segmentation model processes the image frame regions and outputs a segmentation mask. The segmentation mask is a binary image of the same size as the input image, where the white areas represent rice leaves and stems, and the black areas represent the background. Based on the segmentation mask, contours are extracted using an image processing algorithm. The contours represent the boundaries of rice leaves and stems.

[0187] The minimum bounding box determination subunit is used to generate a target minimum bounding box on the rice image based on the image frame area and contour lines of a single rice plant.

[0188] The minimum bounding box determination subunit needs to generate a target minimum bounding box based on these contour lines. The minimum bounding box is the smallest rectangle that can completely surround a single rice plant and is used to determine the actual growth space and position of the rice.

[0189] For example, we obtain the contour data of each rice plant from the output of the semantic segmentation subunit. Based on the contour data, we calculate the minimum bounding box of each rice plant. The minimum bounding box is the smallest rectangle that encloses the contour, ensuring that the boundary just touches the rice. Finally, we record the position (upper left corner coordinates) and size (width and height) of each minimum bounding box.

[0190] The coverage calculation unit is used to calculate the horizontal distance between the external frames according to the minimum external frame of the single rice plant image, and calculate the coverage according to the ratio of the horizontal distance and the width of the adjacent minimum external frame.

[0191] Coverage, in agriculture and ecology, usually refers to the coverage ratio of plants or vegetation in a certain area. It reflects the degree of plant occupation in the area. Specifically in the rice growing environment, coverage can be expressed as the ratio of the coverage area of ​​the leaves and stems of the rice plants to the soil surface. By calculating the coverage, the growth density and uniformity of rice can be evaluated.

[0192] For each pair of adjacent minimum bounding boxes, calculate the horizontal distance between them. The horizontal distance can be determined by calculating the distance between the adjacent and opposite edges of the two bounding boxes on the horizontal axis. Then for each pair of adjacent minimum bounding boxes, calculate the ratio of the horizontal distance to the width of the adjacent bounding boxes.

[0193] The average degree calculation unit is used to calculate the coefficient of variation of the coverage in the image based on each calculated coverage to indicate the uniformity. The coefficient of variation CV = standard deviation / average value. The smaller the CV value, the more uniform the seedling emergence; the larger the CV value, the uneven the seedling emergence.

[0194] The coefficient of variation (CV) is used to measure the uniformity of rice seedling emergence. By calculating and analyzing the CV value, the effectiveness of the current planting strategy can be evaluated and future optimization adjustments can be guided, such as adjusting the planting density, improving irrigation and fertilization strategies, etc.

[0195] Specifically, the coverage data of each pair of adjacent minimum bounding boxes are obtained from the output results of the coverage calculation unit. The average of all coverage values ​​is calculated. The average coverage reflects the overall level of rice coverage in the image. The standard deviation of all coverage values ​​is calculated. The standard deviation reflects the dispersion of the coverage values. Use the formula The coefficient of variation was calculated, and the CV value was used to indicate the uniformity of rice seedling emergence.

[0196] The abnormality judgment module is used to judge the abnormal type of rice growth in each rice field based on the rice uniformity information and the image information of the rice field.

[0197] Specifically, refer to Figure 4 ,The abnormal judgment module includes a uniformity threshold judgment unit, an image calibration unit, a feature recognition and extraction unit, a growth abnormality classification unit and a statistical judgment unit.

[0198] The uniformity threshold judgment unit is used to judge whether the uniformity of the rice field is lower than a preset threshold.

[0199] For example, setting a preset uniformity threshold U th , the threshold is determined based on actual experience and data. For example, the threshold can be set to 0.1, indicating that the deviation of the feature value is within 10%.

[0200] Compare the calculated uniformity CV with the preset threshold U th If CV is greater than U th , it means that the uniformity of the rice field is poor and there may be growth abnormalities; otherwise, it is considered that the growth of the rice field is relatively uniform.

[0201] The image calibration unit is used to calibrate the image of the rice field whose uniformity is lower than a preset threshold, wherein the calibration content is the minimum external frame, the number of leaves and the tillering level of each rice plant in the image.

[0202] From the screened rice field images whose uniformity is lower than the preset threshold, select one image for calibration. Assume that the selected image contains multiple rice seedling areas, and each rice seedling needs to be labeled in detail.

[0203] Identify each rice plant in the image and determine its minimum bounding box. The minimum bounding box is the smallest rectangular area that contains the entire rice seedling, defined as: B i =(x min ,y min ,x max ,y max ), where (x min ,y min ) and (x max ,y max ) are the coordinates of the upper left corner and lower right corner of the rice seedling in the image.

[0204] The minimum bounding box is used to determine the position and range of each rice seedling in the image. This calibration process requires accurate identification of each rice seedling and calculation of its bounding box coordinates. The bounding box can eliminate interference from other rice seedlings and background, providing a clear rice seedling area for subsequent detailed analysis.

[0205] The image area of ​​each rice seedling is analyzed to count the number of leaves. The number of leaves is an important indicator for judging the health of rice seedlings. Each leaf can be identified and counted by the texture and morphological features of the image. Assume that the number of leaves of a rice seedling is L i , the tillering level of a rice seedling is T i .

[0206] The feature recognition and extraction unit is used to perform instance segmentation on the rice plant based on the minimum bounding box, and input the segmentation instance into a pre-trained feature extraction model to extract feature information of the rice seedling, wherein the feature information includes color features, morphological features and texture features.

[0207] Using the obtained minimum bounding box, instance segmentation is performed on each rice seedling area. The goal of instance segmentation is to accurately extract each rice seedling from the image, eliminating interference from other plants and background. A pre-trained feature extraction model is used, which has been trained with a large number of rice images and can accurately extract various features of rice seedlings. The input of the model is the rice seedling image after instance segmentation, and the output is a variety of feature information of the rice seedling.

[0208] Color features include the main color and color distribution of rice seedlings. The proportion of green pixels, RGB values ​​and hue distribution are common color features.

[0209] Morphological characteristics include the height, width, length and shape of the rice seedlings. By analyzing the morphological characteristics of rice seedlings, its growth status and health can be judged. For example, the height and width of rice seedlings can be measured to determine whether they meet normal growth standards. Abnormally short or overly elongated rice seedlings may have growth problems.

[0210] Texture characteristics reflect the details of the rice seedling surface, such as the texture of the leaves, the roughness and consistency of the surface. Healthy rice seedlings usually have uniform texture and consistent surface characteristics. For example, the texture of rice seedling leaves can be analyzed to determine whether there are disease spots or other abnormal texture characteristics. Rice seedlings with abnormal texture may be attacked by pests and diseases.

[0211] The growth abnormality classification unit is used to input the extracted features into the trained growth abnormality classification model to identify the growth abnormality type of each rice plant; wherein the growth abnormality type includes normal type, various fertilizer deficiency types, and various pest and disease types.

[0212] Feature extraction is the basis for the accuracy of the classification model. Color, morphology and texture features fully reflect the health of rice seedlings. Accurate extraction of these features can improve the recognition effect of the classification model.

[0213] Here is an example of the training steps for the growth abnormality classification model:

[0214] Collect a large amount of rice seedling image data from multiple rice fields. These images should cover various growth abnormality types, such as normal, nitrogen deficiency, phosphorus deficiency, potassium deficiency, and pests and diseases. Manually annotate the growth abnormality type for each image and build an annotated dataset containing images and their corresponding abnormality types. For example, one image is annotated as "potassium deficiency" and another is annotated as "pests and diseases".

[0215] Image preprocessing: Standardize all images, including resizing, denoising, contrast enhancement, etc. to ensure image quality and consistency.

[0216] Feature extraction: Extract color features (such as the proportion of green pixels), morphological features (such as the height and width of rice seedlings), and texture features (such as the surface texture of leaves) from the image. These features will be used for model training.

[0217] Feature standardization: Standardize the extracted features and scale the feature values ​​to the same range to avoid the impact of feature value range differences on model training.

[0218] Divide the dataset into training, validation, and test sets. A common ratio is 80% for training, 10% for validation, and 10% for testing. Ensure that the types of growth anomalies in each dataset are evenly distributed to avoid having too many or too few of a particular anomaly type in the training and test sets.

[0219] Select a suitable classification model based on the data characteristics and classification task. For example, use a support vector machine (SVM), decision tree, random forest, or neural network model. Assume that the random forest model is selected. This model builds multiple decision trees and takes the majority vote of their results for classification, which is suitable for processing high-dimensional feature data.

[0220] The features and labels of the training set are input into the random forest model for model training. The model builds classification rules by learning the feature patterns in the training set. Adjust the model parameters (such as the number of decision trees, the maximum depth of the tree, etc.) to optimize the model performance. The cross-validation method can be used to select the best parameter combination.

[0221] Use the validation set to validate the trained model and evaluate the model performance. Common evaluation indicators include accuracy, precision, recall, and F1 value. Analyze the validation results, adjust the model parameters or reselect features to improve the classification performance of the model.

[0222] The statistical judgment unit is used to count the abnormal growth types of each rice plant to obtain the abnormal growth type of the rice field.

[0223] The abnormality judgment module obtains the growth abnormality type of each rice plant. For example, for a rice field, after the classification model is identified, the abnormality type of each rice seedling is obtained. Counting the number of abnormal types of each rice seedling is the basis for judging the growth status of the entire rice field. Accurate statistics can reflect the overall health of the rice field. Calculate the proportion of rice seedlings of each abnormal type to determine the abnormality. The formula used here is Simple and intuitive, it can quickly calculate the proportion of each abnormal type.

[0224] The thresholds for different abnormal types should be different to reflect their different impacts on rice growth. For example, the threshold for potassium deficiency can be set lower because potassium deficiency has a greater impact on rice seedlings.

[0225] The fertilizer and spray type determination module is used to determine the fertilizer and spray type required for each rice field based on the abnormal rice growth type.

[0226] According to the identified abnormal rice growth type, the type of fertilizer required for each rice field is determined. Different abnormal growth types correspond to different nutritional requirements. By prescribing the right medicine, the problems in rice growth can be effectively solved and the healthy growth of rice can be promoted.

[0227] Here is an example:

[0228] 1. Identify the growth anomaly type of a rice field. For example, the rice field has the following anomaly types:

[0229] Abnormal nitrogen deficiency

[0230] Phosphorus deficiency

[0231] Potassium deficiency

[0232] Abnormal pests and diseases

[0233] 2. Determine the type of fertilizer:

[0234] Determine the type of supplementary fertilizer required based on the nutritional requirements corresponding to each abnormality type:

[0235] Abnormal nitrogen deficiency: Nitrogen fertilizer needs to be supplemented. Nitrogen is an important element for plant growth, which can promote the growth of leaves and the formation of green pigment.

[0236] Phosphorus deficiency: Phosphorus fertilizer needs to be supplemented. Phosphorus helps root development and energy transfer, and is also essential for the development of flowers and fruits of plants.

[0237] Potassium deficiency: Potassium fertilizer needs to be supplemented. Potassium can enhance the plant's disease resistance, improve water use efficiency, and promote fruit formation.

[0238] Abnormal diseases and pests: Appropriate pesticides need to be used for prevention and control, and trace element fertilizers should be used to supplement other trace nutrients needed by plants.

[0239] 3. Specific implementation:

[0240] Match the type of supplementary fertilizer with the abnormal type of rice field and formulate a specific fertilization plan. For example:

[0241] For areas deficient in nitrogen, apply urea or another nitrogen-containing fertilizer.

[0242] For areas deficient in phosphorus, apply superphosphate or another phosphorus-containing fertilizer.

[0243] For areas deficient in potassium, apply potassium sulfate or other potassium-containing fertilizers.

[0244] For areas where pests and diseases exist, spray appropriate pesticides and combine them with compound fertilizers or trace element fertilizers.

[0245] The fertilizer spraying amount determination module is used to determine the corresponding fertilizer spraying amount for the rice field based on the image information of the rice field and the fertilizer spraying type.

[0246] Here, the module for determining the amount of fertilizer and spraying determines the size and severity of the abnormal area in the rice field by analyzing the image information, and then determines the specific amount of fertilizer and spraying by comparing the severity of the abnormality according to the pre-set data table. For example, the ratio of the number of abnormal rice plants to the total number of plants in the image is obtained by the previous image processing, and then the severity of each abnormal type is judged according to the ratio of abnormal plants. For example, several severity levels (such as mild, moderate, and severe) can be set here, and the corresponding amount of fertilizer and spraying can be formulated.

[0247] The second path planning module is used to plan the second flight path based on the type and amount of fertilizer spraying required for the rice field.

[0248] It should be noted that, depending on the determined type of fertilizer spraying, the second path here can be one or more.

[0249] Specifically, refer to Figure 5 ,The second path planning module includes a rice field variable initialization unit, a UAV state initialization unit, an algorithm initialization unit, a path construction unit, a UAV state update unit, a pheromone update unit, a global update unit and an optimal path output unit.

[0250] The rice field variable initialization unit is used to obtain the coordinate set P of all rice fields corresponding to a certain growth abnormality type in all rice fields, and the spraying amount a required for each rice field i and the corresponding spraying time T i , where the UAV performs spraying operations over the rice field through a closed path, and the coordinates of the rice field are the starting point and end point of the UAV's spraying operation path over the rice field.

[0251] It should be noted that the closed path here refers to a spraying path that traverses the rice field according to the spraying width. The starting point and the end point of the path are closed, that is, the starting point and the end point coincide with each other and are the same point in the second path, so as to facilitate the path planning of the second path and reduce the amount of calculation. In addition, the spraying time is determined by the flight distance over the rice field, and the spraying amount and power consumption can also be determined accordingly in combination with other states.

[0252] Drone status initialization unit, used to obtain the drone's initial power E max 、Total amount of material to be sprayed A max And the starting point coordinates p0.

[0253] Algorithm initialization unit, used to define D[i][j] to represent point p i To point p j distance, define the number of ants m and the number of iterations t, define the pheromone matrix τ[i][j] and initialize it, and define the heuristic function η[i][j] = 1 / D[i][j].

[0254] Here, the pheromone matrix τ[i][j] has a small initial value, such as 0.1.

[0255] The path construction unit is used to make each ant start from the starting point and build a path until all points are visited or resources are exhausted; the resources include the remaining power and the remaining amount of spraying material. The probability of each ant choosing the next point from the current point is:

[0256]

[0257] The parameters α and β in the ant colony algorithm are the weights that control the relative importance of pheromone and heuristic information. The choice of these parameters has a great impact on the performance of the algorithm. Specifically:

[0258] α: Indicates the importance of pheromone. If the α value is large, the ants are more dependent on pheromones when choosing a path.

[0259] β: Indicates the importance of heuristic information (usually the inverse of distance). If the β value is large, ants are more likely to choose points with shorter distances when choosing a path.

[0260] For example, suppose we are optimizing the flight path of a drone and the initial parameters are set as follows:

[0261] α=1.5: Moderately dependent on pheromones.

[0262] β = 3: Rely more on heuristic information.

[0263] Through experiments, we can gradually adjust the parameter values, observe the performance of the algorithm under different parameter combinations, and finally select the optimal combination.

[0264] In addition, it should be noted that each point here

[0265] The drone status update unit is used to update the drone's remaining power and remaining spraying amount after the ant selects the next point. If the power is not enough to fly to the next point or the remaining spraying amount is not enough to complete the spraying, it returns to the starting point p0 for replenishment.

[0266] The pheromone update unit is used to locally update the pheromone τ each time the ant passes through a path (i, j) during the path construction process. ij ←(1-ρ)τ ij +ρΔτ ij , where ρ is the volatility coefficient, Δτ ij To increase the pheromone.

[0267] The global update unit is used to globally update the pheromone matrix after all ants have completed the path construction:

[0268]

[0269] in, L is the path length and Q is a constant.

[0270] The optimal path output unit is used to terminate the loop after the preset number of iterations t is reached, and output the path with the highest pheromone value as the optimal path.

[0271] The flight and fertilizer spraying control module is used to control the UAV to fly along the second flight path and perform fertilizer spraying operations according to the planned scheme.

[0272] In another embodiment, a three-dimensional variable fertilization method is disclosed, comprising the following steps S1-S8.

[0273] S1. Planning a first flight path; wherein the first flight path traverses the area above each rice field;

[0274] S2. Control the UAV to fly along the first flight path and take images of the rice fields from above;

[0275] S3. Obtaining rice uniformity information of each rice field based on the image information;

[0276] S4. Determine the type of abnormal rice growth in each rice field based on the rice uniformity information and the image information of the rice field;

[0277] S5. Determine the type of fertilizer spray required for each rice field based on the abnormal rice growth type;

[0278] S6. Determine the amount of fertilizer spraying corresponding to the rice field based on the image information of the rice field and the type of fertilizer spraying;

[0279] S7. Planning a second flight path based on the type and amount of fertilizer spraying required for the rice fields;

[0280] S8. Control the UAV to fly along the second flight path and perform fertilizer and pesticide spraying operations according to the planned scheme.

[0281] Optionally, in a certain embodiment, S4 includes the following sub-steps S41-S45.

[0282] S41. Determine whether the uniformity of the rice field is lower than a preset threshold;

[0283] S42. Calibrate the image of the rice field whose uniformity is lower than a preset threshold, wherein the calibration content is the minimum bounding box, the number of leaves and the tillering level of each rice plant in the image;

[0284] S43. Performing instance segmentation on the rice plant based on the minimum bounding box, and inputting the segmentation instance into a pre-trained feature extraction model to extract feature information of the rice seedling, wherein the feature information includes color features, morphological features, and texture features;

[0285] S44. Inputting the extracted features into the trained abnormal growth classification model to identify the abnormal growth type of each rice plant; wherein the abnormal growth type includes a normal type, a variety of fertilizer deficiency types, and a variety of pests and diseases types;

[0286] S45. Count the abnormal growth types of each rice plant to obtain the abnormal growth type of the rice field.

[0287] Optionally, in one embodiment, S7 includes the following steps S71-S78.

[0288] S71. Obtain the coordinate set P of all rice fields corresponding to a certain growth abnormality type in all rice fields, and the spraying amount a required for each rice field i and the corresponding spraying time T i , where the drone performs spraying operations over the rice field through a closed path, and the coordinates of the rice field are the starting point and end point of the drone's spraying operation path over the rice field;

[0289] S72. Get the drone's initial power E max 、Total amount of material to be sprayed A max And the starting point coordinates p0;

[0290] S73. Define D[i][j] to represent point p i To point p j The distance between the ants and the iteration number is defined as m, the pheromone matrix is ​​defined as τ[i][j] and initialized, and the heuristic function is defined as η[i][j] = 1 / D[i][j];

[0291] S74. Each ant starts from the starting point and builds a path until all points are visited or resources are exhausted; wherein the resources include the remaining power and the remaining amount of spraying material, and the probability of each ant selecting the next point from the current point is:

[0292]

[0293] S75. After the ant selects the next point, the remaining power and the remaining amount of spraying material of the drone are updated. If the power is insufficient to fly to the next point or the remaining amount of spraying material is insufficient to complete the spraying, the drone returns to the starting point p0 for resupply;

[0294] S76. During the path construction process, each time the ant passes through a path (i, j), it locally updates the pheromone τ ij ←(1-ρ)τ ij +ρΔτ ij , where ρ is the volatility coefficient, Δτ ij To increase the amount of pheromones;

[0295] S77. After all ants have completed the path construction, the pheromone matrix is ​​globally updated:

[0296]

[0297] in, L is the path length, Q is a constant;

[0298] S78. After the iteration reaches the preset number t, the loop terminates and the path with the highest pheromone value is output as the optimal path.

[0299] Optionally, in one embodiment, S3 includes the following steps S31-S34.

[0300] S31. Obtaining the number of leaves based on the rice image and obtaining the tillering level in combination with the leaf age;

[0301] S32. Perform image recognition and image frame segmentation on the rice image with the tillering level as a priori condition to obtain the minimum bounding box of the single rice plant image;

[0302] S33. Calculate the horizontal distance between the bounding boxes according to the minimum bounding box of the single rice plant image, and calculate the coverage according to the ratio of the horizontal distance and the width of the adjacent minimum bounding box;

[0303] S34. Based on each calculated coverage, calculate the coefficient of variation of the coverage within the image to represent the uniformity.

[0304] Optionally, in a certain embodiment, S32 includes the following steps S321-S325.

[0305] S321. Preload pre-trained image recognition models corresponding to different tillering levels, wherein different image recognition models are trained based on different data sets, and each of the data sets corresponds to a rice image of a tillering level;

[0306] S322. Based on the calculated tillering level of rice in the rice image, select an image recognition model corresponding to the tillering level;

[0307] S323. Calling the selected image recognition model to recognize the rice image to obtain a single rice plant image frame area;

[0308] S324. Performing semantic segmentation on each individual rice plant in the individual rice plant image frame region to obtain a contour line;

[0309] S325. Generate a target minimum bounding box on the rice image based on the single rice plant image frame area and contour lines.

[0310] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0311] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0312] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.

Claims

1. A three-dimensional variable fertilization device, characterized in that: include: A first path planning module is used to plan a first flight path; wherein the first flight path traverses the area above each rice field; An image information acquisition module, used to control the UAV to fly along a first flight path and take images of the rice fields from above; A uniformity calculation module is used to obtain rice uniformity information of each rice field based on image information; An abnormality judgment module, used for judging the abnormal type of rice growth in each rice field based on the rice uniformity information and the image information of the rice field; A fertilizer spraying type determination module is used to determine the type of fertilizer spraying required for each rice field based on the type of abnormal rice growth; A fertilizer spraying quantity determination module is used to determine the corresponding fertilizer spraying quantity of the rice field based on the image information of the rice field and the fertilizer spraying type; A second path planning module is used to plan a second flight path based on the type and amount of fertilizer spraying required for the rice field; The flight and fertilizer spraying control module is used to control the UAV to fly along the second flight path and perform fertilizer spraying operations according to the planned scheme.

2. The three-dimensional variable fertilization device according to claim 1, characterized in that: The abnormality judgment module comprises: A uniformity threshold judgment unit, used to judge whether the uniformity of the rice field is lower than a preset threshold; An image calibration unit, used to calibrate the image of the rice field whose uniformity is lower than a preset threshold, wherein the calibration content is the minimum bounding box, the number of leaves and the tillering level of each rice plant in the image; A feature recognition and extraction unit, used for performing instance segmentation of rice plants based on a minimum bounding box, and inputting the segmentation instance into a pre-trained feature extraction model to extract feature information of the rice seedlings, wherein the feature information includes color features, morphological features and texture features; The abnormal growth classification unit is used to input the extracted features into the trained abnormal growth classification model to identify the abnormal growth type of each rice plant; wherein the abnormal growth type includes normal type, various fertilizer deficiency types, and various pest and disease types; The statistical judgment unit is used to count the abnormal growth types of each rice plant to obtain the abnormal growth type of the rice field.

3. The three-dimensional variable fertilization device according to claim 2, characterized in that: The second path planning module includes: The rice field variable initialization unit is used to obtain the coordinate set P of all rice fields corresponding to a certain growth abnormality type in all rice fields, and the spraying amount a required for each rice field i and the corresponding spraying time T i , where the drone performs spraying operations over the rice field through a closed path, and the coordinates of the rice field are the starting point and end point of the drone's spraying operation path over the rice field; Drone status initialization unit, used to obtain the drone's initial power E max 、Total amount of material to be sprayed A max And the starting point coordinates p0; Algorithm initialization unit, used to define D[i][j] to represent point p i To point p j The distance between the ants and the iteration number is defined as m, the pheromone matrix is ​​defined as τ[i][j] and initialized, and the heuristic function is defined as η[i][j] = 1 / D[i][j]; The path construction unit is used to make each ant start from the starting point and build a path until all points are visited or resources are exhausted; the resources include the remaining power and the remaining amount of spraying material. The probability of each ant choosing the next point from the current point is: The drone status update unit is used to update the drone's remaining power and remaining spraying amount after the ant selects the next point. If the power is not enough to fly to the next point or the remaining spraying amount is not enough to complete the spraying, it returns to the starting point p0 for resupply; The pheromone update unit is used to locally update the pheromone τ each time the ant passes through a path (i, j) during the path construction process. ij ←(1-ρ)τ ij +ρ△τ ij , where ρ is the volatility coefficient, △τ ij To increase the amount of pheromones; The global update unit is used to globally update the pheromone matrix after all ants have completed the path construction: in, L is the path length, Q is a constant; The optimal path output unit is used to terminate the loop after the preset number of iterations t is reached, and output the path with the highest pheromone value as the optimal path.

4. The three-dimensional variable fertilization device according to claim 3, characterized in that: The uniformity calculation module comprises: A tillering level calculation unit, used to obtain the number of leaves based on the rice image, and obtain the tillering level in combination with the leaf age; An instance segmentation unit is used to perform image recognition and image frame segmentation on the rice image based on the tillering level as a priori condition, and obtain the minimum external bounding box of a single rice plant image; A coverage calculation unit, used to calculate the horizontal distance between the external frames according to the minimum external frame of the single rice plant image, and calculate the coverage according to the ratio of the horizontal distance and the width of the adjacent minimum external frame; The average degree calculation unit is used to calculate the coefficient of variation of the coverage in the image based on each calculated coverage to express the uniformity.

5. The three-dimensional variable fertilization device according to claim 4, characterized in that: The instance segmentation unit comprises: A model preloading subunit, used for preloading pre-trained image recognition models corresponding to different tillering levels, wherein different image recognition models are trained based on different data sets, and each data set corresponds to a rice image of a tillering level; A tillering level calculation subunit, used for selecting an image recognition model corresponding to the tillering level of rice in the rice image based on the calculated tillering level of the rice; An image frame determination subunit is used to call the selected image recognition model to recognize the rice image to obtain a single rice plant image frame area; A semantic segmentation subunit, used for performing semantic segmentation on each individual rice plant in the individual rice plant image frame area to obtain a contour line; The minimum bounding box determination subunit is used to generate a target minimum bounding box on the rice image based on the image frame area and contour line of a single rice plant.

6. A three-dimensional variable fertilization method, characterized in that: The following steps are involved: S1. Planning a first flight path; wherein the first flight path traverses the area above each rice field; S2. Control the UAV to fly along the first flight path and take images of the rice fields from above; S3. Obtaining rice uniformity information of each rice field based on the image information; S4. Determine the type of abnormal rice growth in each rice field based on the rice uniformity information and the image information of the rice field; S5. Determine the type of fertilizer spray required for each rice field based on the abnormal rice growth type; S6. Determine the amount of fertilizer spraying corresponding to the rice field based on the image information of the rice field and the type of fertilizer spraying; S7. Planning a second flight path based on the type and amount of fertilizer spraying required for the rice fields; S8. Control the UAV to fly along the second flight path and perform fertilizer and pesticide spraying operations according to the planned scheme.

7. The three-dimensional variable fertilization method according to claim 6, characterized in that: The S4 comprises the following sub-steps: S41. Determine whether the uniformity of the rice field is lower than a preset threshold; S42. Calibrate the image of the rice field whose uniformity is lower than a preset threshold, wherein the calibration content is the minimum bounding box, the number of leaves and the tillering level of each rice plant in the image; S43. Performing instance segmentation of rice plants based on the minimum bounding box, and inputting the segmented instances into a pre-trained feature extraction model to extract feature information of the rice seedlings, wherein the feature information includes color features, morphological features, and texture features; S44. Inputting the extracted features into the trained abnormal growth classification model to identify the abnormal growth type of each rice plant; wherein the abnormal growth type includes a normal type, a variety of fertilizer deficiency types, and a variety of pests and diseases types; S45. Count the abnormal growth types of each rice plant to obtain the abnormal growth type of the rice field.

8. The three-dimensional variable fertilization method according to claim 7, characterized in that: The S7 comprises the following steps: S71. Obtain the coordinate set P of all rice fields corresponding to a certain growth abnormality type in all rice fields, and the spraying amount a required for each rice field i and the corresponding spraying time T i , where the drone performs spraying operations over the rice field through a closed path, and the coordinates of the rice field are the starting point and end point of the drone's spraying operation path over the rice field; S72. Get the drone's initial power E max 、Total amount of material to be sprayed A max And the starting point coordinates p0; S73. Define D[i][j] to represent point p i To point p j The distance between the ants and the iteration number is defined as m, the pheromone matrix is ​​defined as τ[i][j] and initialized, and the heuristic function is defined as η[i][j] = 1 / D[i][j]; S74. Each ant starts from the starting point and builds a path until all points are visited or resources are exhausted; wherein the resources include the remaining power and the remaining amount of spraying material, and the probability of each ant selecting the next point from the current point is: S75. After the ant selects the next point, the remaining power and the remaining amount of spraying material of the drone are updated. If the power is insufficient to fly to the next point or the remaining amount of spraying material is insufficient to complete the spraying, the drone returns to the starting point p0 for resupply; S76. During the path construction process, each time the ant passes through a path (i, j), it locally updates the pheromone τ ij ←(1-ρ)τ ij +ρ△τ ij , where ρ is the volatility coefficient, △τ ij To increase the amount of pheromones; S77. After all ants have completed the path construction, the pheromone matrix is ​​globally updated: in, L is the path length, Q is a constant; S78. After the iteration reaches the preset number t, the loop terminates and the path with the highest pheromone value is output as the optimal path.

9. The three-dimensional variable fertilization method according to claim 8, characterized in that: The S3 comprises the following steps: S31. Obtaining the number of leaves based on the rice image and obtaining the tillering level in combination with the leaf age; S32. Perform image recognition and image frame segmentation on the rice image with the tillering level as a priori condition to obtain the minimum bounding box of the single rice plant image; S33. Calculate the horizontal distance between the bounding boxes according to the minimum bounding box of the single rice plant image, and calculate the coverage according to the ratio of the horizontal distance and the width of the adjacent minimum bounding box; S34. Based on each calculated coverage, calculate the coefficient of variation of the coverage within the image to represent the uniformity.

10. The three-dimensional variable fertilization method according to claim 9, characterized in that: The S32 comprises the following steps: S321. Preload pre-trained image recognition models corresponding to different tillering levels, wherein different image recognition models are trained based on different data sets, and each of the data sets corresponds to a rice image of a tillering level; S322. Based on the calculated tillering level of rice in the rice image, select an image recognition model corresponding to the tillering level; S323. Calling the selected image recognition model to recognize the rice image to obtain a single rice plant image frame area; S324. Performing semantic segmentation on each individual rice plant in the individual rice plant image frame region to obtain a contour line; S325. Generate a target minimum bounding box on the rice image based on the single rice plant image frame area and contour line.

Images