A method and device for calculating the number of missing seedlings in a field crop and a medium

Abstract

The application discloses a method, device, medium and product for calculating the number of missing seedlings of field crops, which comprises the following steps: inputting a crop seedling image into a target detection model to obtain a seedling bounding box and a missing seedling bounding box; obtaining an overlapping area of the seedling bounding box and the missing seedling bounding box; determining each target missing seedling bounding box based on a preset gap, the overlapping area, and the upper and lower boundaries of the seedling bounding box; adjusting the target height of the target missing seedling bounding box based on the preset gap and the seedling bounding box adjacent to the target missing seedling bounding box; determining the number of missing seedlings of any adjusted target missing seedling bounding box based on the target height and the image seedling spacing; and determining the number of missing seedlings of field crops based on the number of missing seedlings of all the adjusted target missing seedling bounding boxes. The application can accurately calculate the number of missing seedlings of field crops.

Description

Technical Field

[0001] This application relates to the field of agricultural testing technology, and in particular to a method, device and medium for calculating the number of missing seedlings in field crops. Background Technology

[0002] Calculating the number of missing seedlings during the seedling stage after crop sowing and providing feedback on the locations of missing seedlings are of great guiding significance for field management. Traditional field seedling shortage statistics rely on manual field inspections and visual estimations, which are often not accurate enough, lack timeliness, and are labor-intensive.

[0003] Combining UAV remote sensing technology with target detection algorithms is an effective alternative for detecting seedlings and missing seedlings in the field. However, the missing seedling bounding boxes obtained through target detection lack prior knowledge, resulting in overlapping areas between seedling and missing seedling bounding boxes, as well as insufficient coverage of the missing seedling bounding boxes. This leads to inaccurate calculations of the number of missing seedlings in the field. Furthermore, most studies often only focus on the results of missing seedling bounding box detection, providing insufficient feedback for replanting in the field. Summary of the Invention

[0004] The purpose of this application is to provide a method, device, and medium for calculating the number of missing seedlings in field crops, which can accurately calculate the number of missing seedlings in field crops, thereby providing detailed guidance and evaluation for calculating the germination rate of field crops and the replanting of missing seedlings.

[0005] To achieve the above objectives, this application provides the following solution:

[0006] Firstly, a method for calculating the number of missing crop seedlings in a field is provided, the method comprising:

[0007] Crop seedling images are input into a target detection model to obtain bounding boxes; the bounding boxes include seedling bounding boxes and missing seedling bounding boxes.

[0008] Obtain the overlapping area between the seedling bounding box and the missing seedling bounding box;

[0009] Based on the preset gap, each target missing seedling boundary frame is determined according to the overlapping area, the upper boundary and the lower boundary of the seedling boundary frame; wherein, the preset gap is a fixed gap between any two adjacent boundary frames;

[0010] Along the y-axis of the Cartesian coordinate system, for each target missing seedling boundary box, the target height of the target missing seedling boundary box is adjusted based on the preset gap and the seedling boundary boxes adjacent to the target missing seedling boundary box; the Cartesian coordinate system is established with the width of the boundary box as the x-axis and the height of the boundary box as the y-axis.

[0011] For any adjusted target missing seedling bounding box, the number of missing seedlings in the adjusted target missing seedling bounding box is determined based on the target height and the seedling spacing in the image.

[0012] The number of missing seedlings in the field is determined based on the number of missing seedlings in all the adjusted target missing seedling bounding boxes.

[0013] In a second aspect, a computer device is provided, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the computer program to implement the method for calculating the number of missing crop seedlings in the field as described in the first aspect.

[0014] Thirdly, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method for calculating the number of missing crop seedlings in the field as described in the first aspect.

[0015] According to the specific embodiments provided in this application, the following technical effects are disclosed:

[0016] Crop seedling images are processed using an object detection model to obtain seedling bounding boxes and missing seedling bounding boxes, allowing for simultaneous detection of both. The overlapping region between the seedling and missing seedling bounding boxes is obtained. Based on a preset gap, and according to the overlapping region and the upper and lower boundaries of the seedling bounding boxes, each target missing seedling bounding box is determined. This targeted processing of the overlapping region between the seedling and missing seedling bounding boxes avoids the problem in existing technologies where overlapping regions are ignored due to a lack of prior knowledge in the missing seedling bounding boxes obtained by the object detection model.

[0017] Along the y-axis, for each target seedling gap boundary box, the target height of the target seedling gap boundary box is determined based on the preset gap and the seedling boundary boxes adjacent to the target seedling gap boundary box, thus avoiding insufficient coverage area of ​​the seedling gap boundary box.

[0018] Finally, based on the target height and the seedling spacing in the image, the number of missing seedlings in each target missing seedling bounding box is accurately determined, thereby obtaining the number of missing seedlings in each target missing seedling bounding box more accurately, i.e. the number of missing seedlings in the field crop, providing fine guidance and evaluation for the calculation of field crop germination rate and seedling replanting at missing seedling points. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic flowchart of a method for calculating the number of missing seedlings in a field crop, provided in Embodiment 1 of this application.

[0021] Figure 2 This is a schematic diagram illustrating the intersection of the missing seedling boundary frame and the seedling boundary frame in Embodiment 1 of this application; wherein, Figure 2 (a) in the diagram shows three overlapping scenarios of the overlapping regions before segmentation; Figure 2 (b) in the diagram shows three possible scenarios for the overlapping regions after segmentation.

[0022] Figure 3 This is a schematic diagram of the collinearity equations provided in Embodiment 1 of this application;

[0023] Figure 4 This is a schematic diagram of the calculation of the number of missing seedlings and the location of the missing seedling points provided in Embodiment 2 of this application. Detailed Implementation

[0024] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] The purpose of this application is to provide a method, device, and medium for calculating the number of missing seedlings in field crops, aiming to accurately calculate the number of missing seedlings in field crops, thereby providing detailed guidance and evaluation for calculating the germination rate of field crops and the replanting of missing seedlings.

[0026] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0027] Example 1

[0028] like Figure 1 As shown in this embodiment, a method for calculating the number of missing crop seedlings in the field is provided, the method comprising:

[0029] Step 101: Input the crop seedling image into the target detection model to obtain bounding boxes; the bounding boxes include seedling bounding boxes and missing seedling bounding boxes; wherein, the crop seedling image is collected along the ridge direction of the field.

[0030] Step 102: Obtain the overlapping area of ​​the seedling boundary box and the missing seedling boundary box.

[0031] Specifically, by traversing all seedling bounding boxes and the missing seedling bounding boxes, it is detected whether they intersect or overlap with any seedling bounding box, i.e., the overlapping area.

[0032] In some embodiments, obtaining the overlapping area of ​​the seedling boundary box and the missing seedling boundary box specifically includes: calculating the intersection-union ratio (IUR) of the seedling boundary box and the missing seedling boundary box; determining whether the IUR is greater than zero to obtain a first result; when the first result is yes, determining that the seedling boundary box and the missing seedling boundary box have the overlapping area; when the first result is no, determining that the seedling boundary box and the missing seedling boundary box do not have the overlapping area.

[0033] Among them, the intersection-union ratio of the missing seedling boundary box and the seedling boundary box is detected. When the intersection-union ratio is greater than 0, there is an intersection or overlap between the missing seedling boundary box and the seedling boundary box, i.e., an overlapping area.

[0034] Step 103: Based on the preset gap, determine each target missing seedling boundary frame according to the overlapping area, the upper boundary and the lower boundary of the seedling boundary frame; wherein, the preset gap is a fixed gap between any two adjacent boundary frames.

[0035] For example, the distance between the upper or lower boundary of the missing seedling bounding box and the lower or lower boundary of the adjacent seedling bounding box is a preset gap, denoted by d, and d is no greater than 10 pixels.

[0036] In some embodiments, determining each target missing seedling boundary frame based on the overlapping region and the upper and lower boundaries of the seedling boundary frame specifically includes: when the overlapping region is located at the upper boundary of the missing seedling boundary frame, segmenting the missing seedling boundary frame using the upper boundary of the missing seedling boundary frame and the lower boundary of the seedling boundary frame to generate a segmented boundary frame, and deleting segmented boundary frames that still overlap between the seedling boundary frame and the missing seedling boundary frame to determine the target missing seedling boundary frame; when the overlapping region is located in the middle region of the missing seedling boundary frame, segmenting the missing seedling boundary frame using the upper and lower boundaries of the seedling boundary frame to generate a segmented boundary frame, and deleting segmented boundary frames that still overlap between the seedling boundary frame and the missing seedling boundary frame to determine the target missing seedling boundary frame; when the overlapping region is located at the lower boundary of the missing seedling boundary frame, segmenting the missing seedling boundary frame using the lower boundary of the missing seedling boundary frame and the upper boundary of the seedling boundary frame to generate a segmented boundary frame, and deleting segmented boundary frames that still overlap between the seedling boundary frame and the missing seedling boundary frame to determine the target missing seedling boundary frame.

[0037] Specifically, when the overlapping area is located at the upper boundary of the seedling-deficient boundary frame, after dividing the seedling-deficient boundary frame using the upper boundary of the seedling-deficient boundary frame and the lower boundary of the seedling boundary frame to generate a segmented boundary frame, the process further includes: the segmented boundary frame comprising a first seedling-deficient frame and a second seedling-deficient frame. The lower boundary of the first seedling-deficient frame coincides with the lower boundary of the seedling boundary frame, and the upper boundary of the second seedling-deficient frame coincides with the lower boundary of the seedling boundary frame; the upper boundary of the second seedling-deficient frame is adjusted downwards until the gap between it and the lower boundary of the seedling boundary frame is a preset gap, and the second seedling-deficient frame is used as the target seedling-deficient boundary frame.

[0038] When the overlapping area is located in the middle region of the seedling-deficient boundary frame, the seedling-deficient boundary frame is divided using the upper and lower boundaries of the seedling boundary frame. After generating the segmented boundary frame, the process further includes: the segmented boundary frame includes a third seedling-deficient frame, a fourth seedling-deficient frame, and a fifth seedling-deficient frame, wherein the lower boundary of the third seedling-deficient frame coincides with the upper boundary of the seedling boundary frame; the fourth seedling-deficient frame coincides with the seedling boundary frame; the upper boundary of the fifth seedling-deficient frame coincides with the lower boundary of the seedling boundary frame; the lower boundary of the third seedling-deficient frame is adjusted upwards until the gap between it and the upper boundary of the seedling boundary frame is a preset gap; the lower boundary of the fifth seedling-deficient frame is adjusted downwards until the gap between it and the lower boundary of the seedling boundary frame is a preset gap; the third seedling-deficient frame and the fifth seedling-deficient frame are used as the target seedling-deficient boundary frame.

[0039] Specifically, when the overlapping area is located at the lower boundary of the missing seedling boundary frame, the missing seedling boundary frame is divided using the lower boundary of the missing seedling boundary frame and the upper boundary of the seedling boundary frame. After generating the segmented boundary frame, the process further includes: the segmented boundary frame includes a sixth missing seedling frame and a seventh missing seedling frame. The lower boundary of the sixth missing seedling frame coincides with the upper boundary of the seedling boundary frame, and the lower boundary of the seventh missing seedling frame is adjusted upwards until the gap between it and the upper boundary of the seedling boundary frame is a preset gap. The seventh missing seedling frame is then determined as the target missing seedling boundary frame.

[0040] It is understandable that when segmenting the seedling gap boundary box, after segmenting according to the upper or lower boundary, the gap between each segmented seedling gap boundary box is adjusted to the preset gap, and then the seedling gap boundary box with overlapping area between the seedling boundary box and the seedling gap boundary box is deleted, that is, the first seedling gap box, the fourth seedling gap box and the sixth seedling gap box mentioned above are deleted, so that the gap between the target seedling gap boundary box and the seedling boundary box is the preset gap.

[0041] It is worth noting that during the subsequent adjustment of the target seedling gap boundary box to the target height, a preset gap is used to determine the gap between the newly acquired seedling boundary box and the target seedling gap boundary box. Because the gap between the newly acquired seedling boundary box and the target seedling gap boundary box is relatively large and there is no overlap, it is easily overlooked. Therefore, this application extends the height of the target seedling gap boundary box to address this situation, preventing insufficient coverage of the seedling gap boundary box and preparing for accurate calculation of the number of missing seedlings in the field.

[0042] Reference Figure 2 ,against Figure 2 (a): ① Represents the first case where the overlapping area is located at the upper boundary of the missing seedling boundary frame. ② Represents the second case where the overlapping area is located in the middle area of ​​the missing seedling boundary frame. ③ Represents the third case where the overlapping area is located at the lower boundary of the missing seedling boundary frame.

[0043] against Figure 2 (b): ① In the first case where the overlapping region is located at the upper boundary of the missing seedling boundary box, after deleting the segmented boundary boxes that still overlap between the seedling boundary box and the missing seedling boundary box, a target missing seedling boundary box is determined, namely the second missing seedling box. ② In the second case where the overlapping region is located in the middle region of the missing seedling boundary box, after deleting the segmented boundary boxes that still overlap between the seedling boundary box and the missing seedling boundary box, two target missing seedling boundary boxes are determined, namely the third missing seedling box and the fifth missing seedling box. ③ In the third case where the overlapping region is located at the lower boundary of the missing seedling boundary box, after deleting the segmented boundary boxes that still overlap between the seedling boundary box and the missing seedling boundary box, a target missing seedling boundary box is determined, namely the seventh missing seedling box.

[0044] In some embodiments, obtaining the target missing seedling border coordinates of the adjusted target missing seedling border specifically includes: presetting the missing seedling border coordinates as the first horizontal coordinate and the first vertical coordinate corresponding to the coordinate point of the upper left corner of the missing seedling border, and the height and width of the missing seedling border.

[0045] For example, the first abscissa of the top left corner of the missing seedling boundary is represented by x1, and the first ordinate is represented by y1; the height of the missing seedling boundary is represented by w1, and the width of the missing seedling boundary is represented by h1. That is, the coordinates of the missing seedling boundary are (x1, y1, w1, h1).

[0046] In some embodiments, the seedling boundary frame coordinates are preset to be the second horizontal coordinate and the second vertical coordinate corresponding to the coordinate point of the upper left corner of the seedling boundary frame, and the height and width of the seedling boundary frame.

[0047] For example, the second abscissa of the top left corner of the seedling boundary is represented by x2, and the second ordinate is represented by y2; the height of the seedling boundary is represented by w2, and the width of the seedling boundary is represented by h2. That is, the coordinates of the target missing seedling boundary are (x2, y2, w2, h2).

[0048] In some embodiments, the target missing seedling border coordinates are determined based on the missing seedling border coordinates, the seedling border coordinates, and the preset gap. The target missing seedling border coordinates are the third abscissa and third ordinate corresponding to the coordinate point of the upper left corner of the target missing seedling border coordinates, and the target height and width of the target missing seedling border coordinates. The third abscissa is determined based on the preset gap, the missing seedling border coordinates, and the seedling border coordinates; the third ordinate is determined based on the preset gap, the missing seedling border coordinates, and the seedling border coordinates; and the target height of the target missing seedling border coordinates is determined based on the preset gap, the missing seedling border coordinates, and the seedling border coordinates.

[0049] For example, the third abscissa of the top left corner of the target missing seedling boundary is represented by x3, and the third ordinate is represented by y3; the height of the target missing seedling boundary is represented by w3, and the width of the target missing seedling boundary is represented by h3. That is, the coordinates of the target missing seedling boundary are (x3, y3, w3, h3).

[0050] Reference Figure 2 The specific process of determining the target missing seedling border coordinates based on the missing seedling border coordinates, the seedling border coordinates, and the preset gap includes three cases:

[0051] ① When the seedling boundary frame is located at the upper boundary of the missing seedling boundary frame, such as Figure 2 As shown, x1 and w1 remain unchanged, and the preset gap between the upper boundary of the target missing seedling bounding box (i.e., the second missing seedling box) and the lower boundary of the seedling bounding box is d. Then y3 is equal to y2+h2+d, and the height h3 is equal to (y1+h1)-(y2+h2+d)=y1+h1-y2-h2-d. Therefore, the adjusted coordinates of the target missing seedling bounding box (x3, y3, w3, h3) are equal to (x1, y2+h2+d, w1, y1+h1-y2-h2-d).

[0052] ② When the seedling boundary frame is located in the middle area of ​​the missing seedling boundary frame, such as Figure 2 As shown, there are two target seedling gap bounding boxes: the upper seedling gap bounding box and the lower seedling gap bounding box, namely the third seedling gap box and the fifth seedling gap box.

[0053] When the target missing seedling frame coordinates are the third missing seedling frame coordinates, the x1, y1 and w1 of the third missing seedling frame coordinates remain unchanged. The preset distance between the lower boundary of the third missing seedling frame and the upper boundary of the seedling frame is d. Then h3 is equal to (y2-d)-y1=y2-d-y1, and the third missing seedling frame coordinates (x3, y3, w3, h3) are equal to (x1, y1, w1, y2-d-y1).

[0054] When the target missing seedling frame coordinates are the fifth missing seedling frame coordinates, the x1 and w1 of the fifth missing seedling frame coordinates remain unchanged. The preset gap between the upper boundary of the fifth missing seedling frame and the lower boundary of the seedling frame is d. Then y3 equals y2+h2+d, and h3 equals y1+h1-(y2+h2+d)=y1+h1-y2-h2-d. Therefore, the fifth missing seedling frame coordinates (x3, y3, w3, h3) equals (x1, y2+h2+d, w1, y1+h1-y2-h2-d).

[0055] ③ When the seedling boundary frame is located at the lower boundary of the missing seedling boundary frame, such as Figure 2 As shown, x1, y1, and w1 remain unchanged. The preset gap between the lower boundary of the target missing seedling bounding box (i.e., the seventh missing seedling box) and the upper boundary of the seedling bounding box is d. Then h3 is equal to y2-y1-d. Therefore, the adjusted target missing seedling bounding box coordinates (x3, y3, w3, h3) are equal to (x1, y1, w1, y2-y1-d).

[0056] Step 104: Along the y-axis of the Cartesian coordinate system, for each target missing seedling boundary frame, adjust the target height of the target missing seedling boundary frame based on the preset gap and the seedling boundary frames adjacent to the target missing seedling boundary frame; the Cartesian coordinate system is established with the width of the boundary frame as the x-axis and the height of the boundary frame as the y-axis.

[0057] In some embodiments, adjusting the target height of the target seedling-deficient boundary frame based on the preset gap and the seedling boundary frames adjacent to the target seedling-deficient boundary frame specifically includes: obtaining a first seedling boundary frame adjacent to the upper boundary of the target seedling-deficient boundary frame, and a second seedling boundary frame adjacent to the lower boundary of the target seedling-deficient boundary frame; extending the height of the target seedling-deficient boundary frame upward to the lower boundary of the first seedling boundary frame, until the gap between the upper boundary of the target seedling-deficient boundary frame and the lower boundary of the first seedling boundary frame is the preset gap; and extending the height of the target seedling-deficient boundary frame downward to the upper boundary of the second seedling boundary frame, until the gap between the lower boundary of the target seedling-deficient boundary frame and the upper boundary of the second seedling boundary frame is the preset gap.

[0058] In order to avoid the problem of insufficient coverage area of ​​the seedling missing boundary box, the gap between the seedling boundary box adjacent to each target seedling missing boundary box and the target seedling missing boundary box is traversed. If the gap is larger than the preset gap, the target height of the target seedling missing boundary box is adjusted to ensure that the seedling missing boundary box covers a sufficient area.

[0059] Step 105: For any adjusted target missing seedling bounding box, determine the number of missing seedlings in the adjusted target missing seedling bounding box based on the target height and the seedling spacing in the image.

[0060] In some embodiments, for any adjusted target missing seedling bounding box, the number of missing seedlings in the adjusted target missing seedling bounding box is determined based on the target height and the seedling spacing in the image. This further includes: obtaining the ground sampling distance and the field seedling spacing; wherein the ground sampling distance is collected by a UAV sensor, and the field seedling spacing is the actual measured value in the field; the ratio of the field seedling spacing to the ground sampling distance is used as the image seedling spacing.

[0061] In practical applications, based on the Ground Sampling Distance (GSD) acquired by the UAV sensor, in remote sensing, Geographic Information Systems (GIS, a technological system used to collect, store, manage, analyze, and display data related to geographic location), and cartography, GSD refers to the actual distance represented by two adjacent pixels on the ground when the sensor acquires an image. The seedling spacing in the image is calculated using GSD and the known field seedling spacing D1. GSD can be obtained from the control software displayed during UAV data acquisition, from the mosaicking software after orthophoto mapping of the UAV image, or calculated based on the UAV sensor parameters and acquisition settings. The field seedling spacing is determined by the specific sowing time or by on-site measurement. Image seedling spacing D2 = D1 / GSD.

[0062] In some embodiments, before determining each target seedling missing bounding box, the method further comprises: acquiring all the segmentation bounding boxes; deleting the segmentation bounding boxes with a height less than the seedling spacing in the image.

[0063] Wherein, a segmentation bounding box with a height less than the seedling spacing in the image is a false detection area where no seedlings should be planted, and removing such false detection areas can ensure the prediction accuracy of the subsequent number of missing seedlings of field crops.

[0064] In practical applications, the height h1 of all seedling missing bounding boxes is traversed, and when h1<D2, the corresponding seedling missing bounding box is deleted.

[0065] Step 4: According to the seedling spacing D2 in the image obtained in step 3, the number of missing seedlings n in each seedling missing bounding box is n=h1 / D2, wherein h1 is rounded down. The number of missing seedlings in each seedling missing bounding box is accumulated to obtain the total number of missing seedlings in the field plot.

[0066] Step 5: Position the coordinates of n missing seedling points according to the number of missing seedlings in the seedling missing bounding box (x1, y1, w1, h1) obtained in step 4, so that the missing seedling points are evenly distributed on the symmetry line of the optimized seedling missing bounding box. Then the image coordinate of the i-th missing seedling point is (x1+w1 / 2, y1+(h1-(n-1)×D2) / 2+(i-1)×D2), wherein 0<i≤n.

[0067] Step 106: Determine the number of missing seedlings of field crops according to the number of missing seedlings of all adjusted target seedling missing bounding boxes.

[0068] In some embodiments, after determining the number of missing seedlings of field crops according to the number of missing seedlings of all adjusted target seedling missing bounding boxes, the method further comprises: for any adjusted target seedling missing bounding box, acquiring target missing seedling bounding box coordinates of the adjusted target seedling missing bounding box; determining field missing seedling points in the adjusted target seedling missing bounding box and image coordinates of the missing seedling points based on the target missing seedling bounding box coordinates and the number of missing seedlings of the adjusted target seedling missing bounding box; wherein the missing seedling points are evenly distributed on the symmetry line of the adjusted target seedling missing bounding box; based on the photogrammetry principle, performing geographic coordinate conversion on the coordinates of the missing seedling points to obtain the geographic position coordinate of each missing seedling point in the total number of missing seedlings in the field, wherein the photogrammetry principle comprises the collinearity equation principle.

[0069] In practical applications, assume that the pitch angle of the camera is ω, the yaw angle and the roll angle is k, according to the attitude parameters of the unmanned aerial vehicle The rotation transformation matrix of the image can be solved (Formula (1)) to realize the transformation from the image space coordinate system xyz to the image space auxiliary coordinate system x'y'z' rectangular coordinate system, so that the rotated image space coordinate system is parallel to the object space coordinate system. The image space auxiliary coordinate system of ground point A is calculated as shown in Formula (2).

[0070] like Figure 3 As shown, in the object space coordinate system XYZ, the ground point A, the image point a in the image space coordinate system x'y'z', and the camera center S (the perspective center of the camera) are on the same straight line. Based on the principle of similar triangles, we can obtain formula (3). Substituting formula (3) into formula (1), and replacing -z' with the focal length f. a The absolute coordinates of ground point A can be obtained, as shown in formula (4). When the surface of the field is relatively flat, the elevation value Z can be ignored. A Based on the image coordinates of the missing seedlings obtained in step 105, these coordinates are used as the coordinates of point a and substituted into formula (4) to convert the image coordinates of the missing seedlings into the geographical coordinates of the missing seedlings.

[0071]

[0072]

[0073] In formula (1), R represents the rotation transformation matrix, and ω represents the pitch angle. Let yaw angle be denoted by κ, roll angle by κ, and a1, a2, a3, b1, b2, b3, c1, c2, c3 represent the corresponding elements in the rotation transformation matrix, respectively. A ,Y A Z A (X) represents the coordinates of ground point A in the object space coordinate system. S ,Y S Z S Let (x) be the coordinates of the photography center S in the object space coordinate system. a ,y a ,z a Let (x) be the coordinates of image point a in the image space coordinate system. a ',y a ',z a ') represents the coordinates of image point a in the auxiliary coordinate system of image space, and f is the focal length of the camera.

[0074] This application employs a five-stage process: missing seedling area segmentation, bounding box expansion, false detection area removal, missing seedling quantity calculation, and missing seedling point localization. Based on existing field drone images and their seedling and missing seedling bounding boxes, crop seedling images collected by the drone along the row direction are used. In the first stage, all missing seedling bounding boxes and seedling bounding boxes are traversed, and overlapping areas are detected. In the second stage, all missing seedling bounding boxes and their adjacent seedling bounding boxes are traversed, and the missing seedling bounding boxes are expanded along their height direction to obtain a target height, ensuring that the distances between the upper and lower boundaries of the missing seedling bounding boxes and the upper and lower boundaries of adjacent seedling bounding boxes are preset gaps. In the third stage, the image seedling spacing is calculated based on the ground sampling distance and the field seedling spacing, and missing seedling bounding boxes with heights smaller than the image seedling spacing are deleted. In the fourth stage, based on the image seedling spacing and the height of the target missing seedling bounding box, the target height of each target missing seedling bounding box is divided by the image seedling spacing, and the integer is obtained by summing the results to determine the number of missing seedlings in the field. In the fifth stage, the image coordinates of each missing seedling point are obtained. Then, based on the principle of collinearity equations, and combining camera intrinsic and extrinsic parameters with acquisition parameters, the image coordinates are converted into geographic coordinates. According to the method proposed in this application, the number of missing crop seedlings can be accurately calculated, and the geographic coordinates of the missing seedling points can be provided, offering detailed guidance and assessment for calculating crop germination rates in the field and for replanting missing seedlings.

[0075] Example 2

[0076] Reference Figure 4 This paper provides a method for calculating the number of missing seedlings in a field crop, which includes the following steps:

[0077] Step 1: Set the UAV's forward repeatability to 80%, lateral repeatability to 80%, flight speed to 1 m / s, and flight altitude to 12 m. Use a DJI Mavic 3E UAV to collect color images of maize at the two-leaf stage along the field ridges, with a resolution of 5280×3956. Input the obtained color images into a trained YOLOv8 object detection model for sliding sampling to predict seedling and missing seedling bounding boxes, resulting in a color image of maize at the two-leaf stage containing seedling and missing seedling bounding box information. The seedling and missing seedling bounding boxes are represented as (c, x, y, w, h), where c is the bounding box category (0 for seedling, 1 for missing seedling), x and y are the coordinates of the top-left corner of the bounding box, and w and h are the width and height of the bounding box, respectively.

[0078] Step 3: Calculate the intersection-union ratio (IUR) of all missing seedling bounding boxes and seedling bounding boxes, and detect whether there is any intersection or overlap between the missing seedling bounding boxes and seedling bounding boxes. When the IUR is > 0, perform bounding box segmentation and adjustment on the corresponding missing seedling bounding box, and make the nearest distance between the upper or lower boundary of the adjusted missing seedling bounding box and the lower or upper boundary of the seedling detection box 5 pixels.

[0079] Step 4: Find the adjacent seedling bounding boxes of all missing seedling bounding boxes in the same row direction (i.e., they intersect or overlap in the x-axis direction), and adjust the height of the corresponding missing seedling bounding box along the y-axis direction to a distance of 5 pixels from the adjacent seedling bounding box.

[0080] Step 5: Based on the drone sensor parameters and acquisition settings, calculate the GSD of the acquired drone image as 0.33 cm / pixel. The seedling spacing in the acquired field is 25 cm, therefore the seedling spacing in the image is 75 pixels. Delete the bounding boxes of missing seedlings with a height less than 75 pixels.

[0081] Step 6: Calculate the number of missing seedlings in each missing seedling boundary box. Figure 3 The number of missing seedlings in each missing seedling boundary box is 5 and 2 respectively. Therefore, the total number of missing seedlings in the field corresponding to this image is 7.

[0082] Step 7: Based on the number of missing seedlings in each missing seedling frame calculated in Step 6, locate the missing seedling points. The coordinates of the missing seedling points in the image are as follows: (141, 135), (141, 210), (141, 285), (141, 360), (141, 435), (373, 365), (373, 440).

[0083] Step 8: Using Python libraries such as osgeo and exifread, read the internal and external parameters of the UAV and convert the image coordinates into geographic coordinates by combining the collinearity equation: (30°19′6.38″N, 119°55′55.82″E), (30°19′6.38″N, 119°55′55.82″E), (30°19′6.38″N, 119°55′55.82″E), (30°19′6.40″N, 119°55′55.82″E), (30°19′6.40″N, 119°55′55.82″E), (30°19′6.40″N, 119°55′55.78″E), (30°19′6.40″N, 119°55′55.78″E).

[0084] Example 3

[0085] A computer device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method for calculating the number of missing crop seedlings in the field as described in Embodiment 1.

[0086] Example 4

[0087] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for calculating the number of missing crop seedlings in the field as described in Embodiment 1.

[0088] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0089] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for calculating the number of missing seedlings in a field crop, characterized in that, The method includes: Crop seedling images are input into a target detection model to obtain bounding boxes; the bounding boxes include seedling bounding boxes and missing seedling bounding boxes. Obtain the overlapping area between the seedling bounding box and the missing seedling bounding box; Based on a preset gap, each target missing seedling boundary frame is determined according to the overlapping area, the upper boundary and the lower boundary of the seedling boundary frame; wherein, the preset gap is a fixed gap between any two adjacent boundary frames; specifically, it includes: when the overlapping area is located at the upper boundary of the missing seedling boundary frame, the missing seedling boundary frame is divided using the upper boundary of the missing seedling boundary frame and the lower boundary of the seedling boundary frame to generate a segmented boundary frame, and the segmented boundary frames that still overlap between the seedling boundary frame and the missing seedling boundary frame are deleted to determine the target missing seedling boundary frame; When the overlapping area is located in the middle area of ​​the seedling boundary box, the seedling boundary box is divided using the upper and lower boundaries of the seedling boundary box to generate a segmented boundary box, and the segmented boundary boxes that still overlap between the seedling boundary box and the seedling boundary box are deleted to determine the target seedling boundary box. When the overlapping area is located at the lower boundary of the seedling-deficient boundary box, the seedling-deficient boundary box is divided using the lower boundary of the seedling-deficient boundary box and the upper boundary of the seedling boundary box to generate a segmented boundary box. Then, the segmented boundary boxes that still overlap with the seedling boundary box are deleted to determine the target seedling-deficient boundary box. Along the y-axis of the Cartesian coordinate system, for each target missing seedling boundary box, the target height of the target missing seedling boundary box is adjusted based on the preset gap and the seedling boundary boxes adjacent to the target missing seedling boundary box; the Cartesian coordinate system is established with the width of the boundary box as the x-axis and the height of the boundary box as the y-axis. For any adjusted target missing seedling bounding box, the number of missing seedlings in the adjusted target missing seedling bounding box is determined based on the target height and the seedling spacing in the image. The number of missing seedlings in the field is determined based on the number of missing seedlings in all the adjusted target missing seedling bounding boxes.

2. The method for calculating the number of missing crop seedlings in the field according to claim 1, characterized in that, Before determining the bounding boxes for each target missing seedling, the following steps are also included: Obtain all the segmentation bounding boxes; Delete the segmentation bounding boxes whose height is smaller than the spacing between the seedlings in the image.

3. The method for calculating the number of missing crop seedlings in the field according to claim 1, characterized in that, Based on the preset gap and the seedling boundary frame adjacent to the target missing seedling boundary frame, the target height of the target missing seedling boundary frame is adjusted, specifically including: Obtain a first seedling boundary box adjacent to the upper boundary of the target seedling gap boundary box, and a second seedling boundary box adjacent to the lower boundary of the target seedling gap boundary box; The height of the target missing seedling boundary box is extended upward to the lower boundary of the first seedling boundary box until the gap between the upper boundary of the target missing seedling boundary box and the lower boundary of the first seedling boundary box is the preset gap. In addition, the height of the target missing seedling boundary box is extended downward to the upper boundary of the second seedling boundary box until the gap between the lower boundary of the target missing seedling boundary box and the upper boundary of the second seedling boundary box is the preset gap.

4. The method for calculating the number of missing crop seedlings in the field according to claim 1, characterized in that, For any adjusted target missing seedling bounding box, based on the target height and the seedling spacing in the image, the number of missing seedlings in the adjusted target missing seedling bounding box is determined, which also includes: The ground sampling distance and the spacing between seedlings in the field are obtained; wherein, the ground sampling distance is collected by the UAV sensor, and the spacing between seedlings in the field is the actual measured value in the field; The ratio of the field seedling spacing to the ground sampling distance is used as the image seedling spacing.

5. The method for calculating the number of missing crop seedlings in the field according to claim 1, characterized in that, Based on the number of missing seedlings in the adjusted target missing seedling bounding box, the number of missing seedlings in the field is determined, followed by: For any adjusted target missing seedling bounding box, obtain the target missing seedling border coordinates of the adjusted target missing seedling bounding box; Based on the coordinates of the target missing seedling border and the number of missing seedlings in the adjusted target missing seedling border, the field missing seedling points in the adjusted target missing seedling border and the image coordinates of the missing seedling points in the field are determined; wherein, the missing seedling points are evenly distributed on the symmetry line of the adjusted target missing seedling border. Based on the principles of photogrammetry, the image coordinates of the missing seedling points are transformed into geographic coordinates to obtain the geographic coordinates of each missing seedling point in the total number of missing seedlings in the field; wherein, the photogrammetric principles include the principle of collinear equations.

6. The method for calculating the number of missing crop seedlings in the field according to claim 5, characterized in that, Obtaining the target missing seedling bounding box coordinates after adjustment specifically includes: The coordinates of the missing seedling boundary are preset as the first horizontal coordinate and the first vertical coordinate corresponding to the coordinate point of the upper left corner of the missing seedling boundary, as well as the height and width of the missing seedling boundary; The seedling boundary frame coordinates are preset to be the second horizontal coordinate and the second vertical coordinate corresponding to the coordinate point of the upper left corner of the seedling boundary frame, as well as the height and width of the seedling boundary frame; Based on the coordinates of the missing seedling border, the coordinates of the seedling border, and the preset gap, the coordinates of the target missing seedling border are determined; the coordinates of the target missing seedling border are the third horizontal coordinate and the third vertical coordinate corresponding to the coordinate point of the upper left corner of the target missing seedling border, as well as the target height and the width of the target missing seedling border.

7. The method for calculating the number of missing crop seedlings in the field according to claim 1, characterized in that, The acquisition of the overlapping region between the seedling boundary box and the missing seedling boundary box specifically includes: Calculate the intersection-union ratio (IUU) of the seedling bounding box and the missing seedling bounding box; Determine whether the intersection-union ratio is greater than zero to obtain the first result; When the first result is yes, it is determined that the seedling boundary box and the missing seedling boundary box have the overlapping area; If the first result is negative, it is determined that the seedling boundary box and the missing seedling boundary box have no overlapping area.

8. A computer device comprising: The memory and processor contain a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the method for calculating the number of missing seedlings in the field as described in any one of claims 1-7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the method for calculating the number of missing crop seedlings in the field as described in any one of claims 1-7.

Images