Seedling missing detection method for operation of rice transplanter, electronic equipment and storage medium

Abstract

The invention provides a seedling missing detection method for rice transplanter operation, electronic equipment and a storage medium, and the method comprises the steps: obtaining a seedling image after the rice transplanter operation through an image sensor installed on the rice transplanter at an oblique view angle; filling a preset range in the seedling image; inputting the filled seedling image into a target detection model to determine first center coordinates of all seedlings in the seedling image; aiming at the first center coordinate of each seedling, correcting the first center coordinate of the seedling to obtain a second center coordinate of the seedling; all the seedlings are grouped based on the corrected second center coordinates of all the seedlings so as to determine a plurality of target seedling groups, and the seedlings in each target seedling group belong to the same operation seedling row of the rice transplanter; for each target seedling group, determining whether a continuous seedling missing condition exists in the group or not; and for two adjacent target seedling groups, determining whether the condition of continuous seedling missing between the groups exists or not.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a method for detecting seedling shortage in a rice transplanter operation, an electronic device and a storage medium. Background Art

[0002] With the continuous development of automation and agricultural machinery, rice transplanting by rice transplanters has gradually replaced manual transplanting. However, in the actual operation process of rice transplanting by rice transplanters, there is a situation where single or multiple rows are continuously missing seedlings.

[0003] At present, there are two main intelligent methods for missing seedlings detection. One is to use traditional image processing methods such as image segmentation, morphology and pattern recognition to detect missing seedlings based on coverage rate of overhead images obtained by aerial cameras or cameras installed on rice transplanters; the other is to use deep learning target detection with positioning equipment such as RTK to convert seedling pixel coordinates into world coordinates and detect missing seedlings based on coverage rate of seedling images. However, the above methods have poor accuracy and high cost for missing seedlings detection. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a method for detecting seedling shortage during operation of a rice transplanter, an electronic device, and a storage medium, so as to improve the accuracy of seedling shortage detection during operation of the rice transplanter.

[0005] In a first aspect, the present invention provides a method for detecting missing seedlings in a rice transplanter operation, the method comprising acquiring a seedling image after the rice transplanter operation through an image sensor installed on the rice transplanter; filling a preset range in the seedling image; inputting the filled seedling image into a target detection model to determine the first center coordinates of all seedlings in the seedling image; correcting the first center coordinate of each seedling to obtain the second center coordinate of the seedling; grouping all seedlings based on the corrected second center coordinates of all seedlings to determine a plurality of target seedling groups, wherein the seedlings in each target seedling group belong to the same operating seedling row of the rice transplanter; determining, for each target seedling group, whether there is a continuous missing seedling situation within the group; and determining, for two adjacent target seedling groups, whether there is a continuous missing seedling situation between groups.

[0006] In an optional embodiment, the first center coordinate of each rice seedling is corrected in the following manner:

[0007] Based on the first center coordinates and the perspective transformation matrix, the first center coordinates are perspective transformed to obtain the intermediate center coordinates; based on the nonlinear transformation of the intermediate center coordinates, the intermediate center coordinates are nonlinearly transformed to obtain the corresponding second center coordinates.

[0008] In an optional implementation, based on the first center coordinates and the perspective transformation matrix, the step of performing perspective transformation on the first center coordinates to obtain the intermediate center coordinates specifically includes:

[0009] Process the two-dimensional first center coordinate (X, Y) into the three-dimensional first center coordinate (X, Y, Z=1); multiply the three-dimensional first center coordinate with the perspective transformation matrix to obtain the perspective transformation coordinate (U, V, W); calculate the ratio between the coordinate value U of the perspective transformation coordinate X-axis and the coordinate value W of the Z-axis as the coordinate value X′ of the X-axis of the two-dimensional intermediate center coordinate; use the coordinate value Y of the Y-axis of the first center coordinate as the coordinate value Y′ of the Y-axis of the two-dimensional intermediate center coordinate.

[0010] In an optional implementation, based on the nonlinear transformation of the intermediate center coordinates, the step of performing a nonlinear transformation on the intermediate center coordinates to obtain the corresponding second center coordinates specifically includes:

[0011] Determine whether the X-axis coordinate value X′ of the intermediate center coordinate is less than the center value; if so, substitute X′ and the nonlinear transformation parameter t into the first calculation formula to obtain the X-axis coordinate value X″ of the second center coordinate; if not, substitute X′ and the nonlinear transformation parameter t into the second calculation formula to obtain the X-axis coordinate value X″ of the second center coordinate; wherein the Y-axis coordinate value Y″=Y′ of the second center coordinate.

[0012] In an optional embodiment, a plurality of rice seedling groups are determined in the following manner;

[0013] Based on the second center coordinates of all the seedlings, all the seedlings are divided into multiple sample seedling groups, wherein the seedlings in each sample seedling group belong to the same operating seedling row of the rice transplanter; and multiple target seedling groups corresponding to the number of seedling rows of the rice transplanter are determined from the multiple sample seedling groups.

[0014] In an optional embodiment, the step of dividing all the seedlings into a plurality of sample seedling groups based on the second center coordinates of all the seedlings specifically includes:

[0015] Arrange the second center coordinates of all seedlings in ascending order according to the coordinate values ​​of the Y axis; use the first second center coordinate after sorting as the first second center coordinate in the first sample seedling group; and perform the following steps in sequence for each of the remaining second center coordinates:

[0016] Determine whether the difference between the X-axis coordinate value of the second center coordinate and the X-axis coordinate value of the last second center coordinate in the current sample seedling group is less than a first threshold; if so, put the second center coordinate into the current sample seedling group; if not, create the next sample seedling group and use the second center coordinate as the first second center coordinate in the next sample seedling group.

[0017] In an optional embodiment, for each target seedling group, the step of determining whether there is a continuous shortage of seedlings in the group specifically includes:

[0018] For all the second center coordinates in the target seedling group, the row spacing value corresponding to the second center coordinate is calculated to determine whether the row spacing value is greater than a second threshold; if so, it is determined that there is a continuous shortage of seedlings in the target seedling group.

[0019] In an optional embodiment, for two adjacent target seedling groups, it is determined whether there is a continuous shortage of seedlings between the groups:

[0020] Based on the X-axis coordinate value of the first second center coordinate in each target seedling group, the group spacing value between two adjacent target seedling groups is calculated; if the group spacing value is greater than the third threshold and less than the fourth threshold, it is determined that there is a single group of continuous missing seedlings; if the group spacing value is greater than the fourth threshold, it is determined that there are multiple groups of continuous missing seedlings; wherein the fourth threshold is greater than the third threshold.

[0021] In a second aspect, the present invention provides an electronic device comprising: a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, when the electronic device is running, the processor and the memory communicate via the bus, and the processor executes the machine-readable instructions to perform the steps of the method for detecting lack of seedlings in a rice transplanter operation as described in any of the aforementioned embodiments.

[0022] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the method for detecting a lack of seedlings in a rice transplanter operation as described in any of the aforementioned embodiments are executed.

[0023] The present application provides a method, electronic device and storage medium for detecting missing seedlings in a rice transplanter operation, wherein the method comprises obtaining a seedling image after the rice transplanter operation through an image sensor installed at an oblique angle on the rice transplanter; filling a preset range in the seedling image; inputting the filled seedling image into a target detection model to determine the first center coordinates of all seedlings in the seedling image; correcting the first center coordinate of each seedling to obtain the second center coordinate of the seedling; grouping all seedlings based on the corrected second center coordinates of all seedlings to determine a plurality of target seedling groups, wherein the seedlings in each target seedling group belong to the same operating seedling row of the rice transplanter; determining, for each target seedling group, whether there is a continuous missing seedling situation within the group; and determining, for two adjacent target seedling groups, whether there is a continuous missing seedling situation between groups. Through seedling image filling, YOLOv10 model training and recognition, perspective transformation, nonlinear transformation, sorting, grouping, group selection, and continuous seedling missing judgment within and between groups of seedling coordinates, the detection of single-row or multiple-row continuous seedling missing is achieved, which not only improves the accuracy of seedling missing detection but also has wide applicability. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0025] Figure 1 A flow chart of a method for detecting seedling shortage in a rice transplanter operation provided in an embodiment of the present application;

[0026] Figure 2 A filled seedling image provided in an embodiment of the present application;

[0027] Figure 3 A rice seedling image after target detection provided in an embodiment of the present application;

[0028] Figure 4 A rice seedling image after coordinate extraction provided in an embodiment of the present application;

[0029] Figure 5 A flowchart of a coordinate correction step provided in an embodiment of the present application;

[0030] Figure 6 A seedling image after coordinate perspective transformation provided in an embodiment of the present application;

[0031] Figure 7A seedling image after nonlinear coordinate transformation provided in an embodiment of the present application;

[0032] Figure 8 A grouped rice seedling image provided in an embodiment of the present application;

[0033] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0034] The present application is applicable to the field of agricultural machinery, and is specifically used for detecting continuous lack of seedlings in a single row or multiple rows in a rice transplanter transplanting scenario.

[0035] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0036] Figure 1 The present invention provides a flow chart of a method for detecting a lack of seedlings in a rice transplanter operation. Figure 1 As shown, a method for detecting seedling shortage in a rice transplanter operation provided in an embodiment of the present application includes:

[0037] S1. Obtaining the image of the rice seedlings after the rice transplanter is operated by using an image sensor installed on the rice transplanter.

[0038] The image sensor here can be a camera, and the camera on the rice transplanter can be installed according to an oblique or horizontal viewing angle.

[0039] S2. Filling a preset range in the rice seedling image.

[0040] Considering that the importance of distant seedlings for transplanting quality assessment is relatively low, the present invention adopts a black filling method to fill the upper part of the image, that is, those areas where the seedlings are severely blocked due to the oblique viewing angle, with black pixels. Since the camera angle is fixed, the severely blocked areas of each image are basically the same, and the image size is 1080p, so the pixel threshold is selected as 320, that is, all the pixels from 0 to 320 in the Y direction of the image are filled with black pixels. The filling effect is as follows: Figure 2 This operation can not only reduce the misidentification and missed identification caused by overlapping seedlings, reduce unnecessary interference from details, improve training efficiency and model generalization ability, but also reduce the workload of manual labeling.

[0041] S3. Input the filled seedling image into the target detection model to determine the first center coordinates of all the seedlings in the seedling image.

[0042] Here, we can train and generate a target detection model based on the YOLOv10 algorithm to identify the rice seedlings in the rice seedling image. The rice seedling detection results can be found in Figure 3 . Extract the normalized center coordinates of the seedling detection frame in the image as the position of the seedling in the image, that is, the first center coordinates of the seedling. The extraction results of the first center coordinates of the seedling can be found in Figure 4 shown.

[0043] S4. With respect to the first center coordinate of each seedling, correct the first center coordinate of the seedling to obtain the second center coordinate of the seedling.

[0044] like Figure 5 As shown, in step S4, the first center coordinate of each seedling can be corrected in the following manner:

[0045] 1. Changes in perspective

[0046] Due to the perspective projection characteristics of the camera, perspective deformation will appear in the image. That is, for the seedlings with the same number of rows, the same row spacing and parallel to each other, the row spacing close to the camera imaging center appears wider, while the row spacing far from the camera imaging center appears narrower. In order to reduce the impact of this problem on subsequent processing, the perspective transformation method can be used here to correct the seedling coordinates.

[0047] Here, based on the first center coordinates and the perspective transformation matrix, the first center coordinates may be perspectively transformed to obtain the intermediate center coordinates, specifically including:

[0048] S400, processing the two-dimensional first center coordinates (X, Y) into the three-dimensional first center coordinates (X, Y, Z=1).

[0049] Since the perspective transformation matrix is ​​a 3*3 matrix, the first center coordinate of each seedling needs to be converted into three dimensions, and the depth information of all seedlings can be set to 1.

[0050] S401, multiply the first three-dimensional center coordinates by the perspective transformation matrix to obtain perspective transformation coordinates (U, V, W).

[0051] Multiply the first center coordinate of the seedling by the perspective transformation matrix. The calculation formula is as follows:

[0052]

[0053] S402 : Calculate the ratio between the X-axis coordinate value U and the Z-axis coordinate value W of the perspective transformation coordinate, and use it as the X-axis coordinate value X′ of the two-dimensional intermediate center coordinate.

[0054] S403 , using the Y-axis coordinate value Y of the first center coordinate as the Y-axis coordinate value Y′ of the two-dimensional intermediate center coordinate.

[0055] In order to keep the vertical coordinates of the seedlings unchanged and facilitate the subsequent calculation of the lack of seedlings in each row, only the horizontal coordinates of the seedlings are corrected, while the vertical coordinates do not need to be corrected. That is, the horizontal coordinates after correction and the vertical coordinates before correction form the optimized central coordinates. The effect after perspective transformation can be seen in Figure 6 shown.

[0056] (II) Nonlinear transformation

[0057] Due to the influence of camera perspective projection and lens distortion, for seedlings with the same row spacing and parallel to each other, the row spacing becomes smaller towards the left and right after imaging. In order to reduce the impact of this problem, the method of adding nonlinear transformation in the x direction can be used for correction.

[0058] Exemplarily, the intermediate center coordinates may be subjected to nonlinear transformation based on nonlinear transformation, and nonlinear transformation may be performed on the intermediate center coordinates to obtain corresponding second center coordinates, specifically including:

[0059] S404: Determine whether the coordinate value X′ of the intermediate center coordinate on the X axis is smaller than the center value.

[0060] The center value here is 0.5.

[0061] S405: If yes, then substitute X′ and the nonlinear transformation parameter t into the first calculation formula to obtain the coordinate value X″ of the X-axis of the second center coordinate.

[0062] The first calculation formula can be expressed as:

[0063] X″=X′-t*(X′-0.5) 2 .

[0064] S406: If not, then substitute X′ and the nonlinear transformation parameter t into the second calculation formula to obtain the X-axis coordinate value X″ of the second center coordinate.

[0065] The second calculation formula can be expressed as:

[0066] X″=X′+t*(X′-0.5) 2 .

[0067] The coordinate value of the Y axis of the second center coordinate is Y″=Y′.

[0068] The nonlinear transformation parameter t here is used to control the degree of nonlinear transformation. The effect after nonlinear transformation can be seen in Figure 7 shown.

[0069] S5. Grouping all the seedlings based on the corrected second center coordinates of all the seedlings to determine a plurality of target seedling groups, wherein the seedlings in each target seedling group belong to the same operating seedling row of the rice transplanter.

[0070] In step S5, multiple rice seedling groups may be determined in the following manner:

[0071] (I) Grouping

[0072] In order to determine the seedling shortage situation in each row, the seedling coordinates need to be grouped, that is, the seedling coordinates of the same operating seedling row are grouped into the same group.

[0073] Based on the second center coordinates of all the seedlings, all the seedlings are divided into a plurality of sample seedling groups, wherein the seedlings in each sample seedling group belong to the same working seedling row of the rice transplanter. Specifically, it includes:

[0074] Arrange the second center coordinates of all seedlings in ascending order according to the coordinate values ​​of the Y axis. Use the first second center coordinate after sorting as the first second center coordinate in the first sample seedling group; for each of the remaining second center coordinates, perform the following steps in sequence:

[0075] Determine whether the difference between the X-axis coordinate value of the second center coordinate and the X-axis coordinate value of the last second center coordinate in the current sample seedling group is less than a first threshold value. If so, put the second center coordinate into the current sample seedling group. If not, create a next sample seedling group, and use the second center coordinate as the first second center coordinate in the next sample seedling group.

[0076] In a specific embodiment, the grouping threshold a can be set in advance, the first second center coordinate after sorting is taken out, and it is placed in the first sample seedling group, and then the second second center coordinate is taken out, and its horizontal coordinate is compared with the horizontal coordinate of the first second center coordinate. If the difference between the coordinates is less than or equal to the threshold a, it is placed in the same sample seedling group as the first seedling and is placed later. If it is greater than the threshold a, it is placed in a new sample seedling group, that is, the second sample seedling group, and the second center coordinates are taken out one by one subsequently, and the difference between it and the horizontal coordinate of the later placed in the existing group is judged. If there is a difference with a certain group that is less than or equal to a, it is placed in the group and is placed later. If the difference with all groups is greater than a, it is placed in a new group until all the seedling coordinates are taken out. The grouping calculation process is as follows:

[0077] (G i ,L i ,m i )=u(G i-1 ,L i-1 ,P i ,g(P i,G i-1 ,L i-1 ,a);

[0078]

[0079] In the formula, G i and L i Respectively represent the group set after processing and the last set of horizontal coordinates, and m i Indicates the total number of groups currently created.

[0080] P={P 1 ,P 2 ,…,P n} is a set of seedling point coordinates arranged in ascending order of x coordinates, where P i =(x i ,y i ),i=1,2,…,n. a is the threshold value set to determine whether two points belong to the same row. G={G 1 ,G 2 ,…,G m} is the final set of seedling point groups, where m is the number of groups formed in the end. Initialize G 1 = {P 1}. j represents the jth group G j The horizontal coordinate of the last element, that is, L j =x jk , where P jk G j Initialize L 1 =x 1 m is the total number of groups currently created, and m is initialized to 1. The grouped results can be found in Figure 8 shown.

[0081] (II) Group Selection

[0082] Since the camera will capture the outer rows that have been planted during the seedling planting process, the seedling groups need to be screened.

[0083] Here, a plurality of target seedling groups corresponding to the number of seedling rows operated by the rice transplanter can be determined from a plurality of sample seedling groups.

[0084] Exemplarily, with x=0.5 as the center line, it is divided into a left group and a right group. Each group on the left is sorted from large to small according to the horizontal coordinate of the first seedling coordinate, and each group on the right is sorted from small to large according to the horizontal coordinate of the first seedling coordinate. Then, the number of groups is selected according to different rice transplanter models. Taking a six-row rice transplanter as an example, the first three groups after sorting are selected on both the left and right sides. If there are less than three groups on one side, all of them are selected.

[0085] Here, taking a six-row rice transplanter as an example, if the number of target seedling groups finally obtained is less than six, it means that there is a continuous shortage of seedlings between groups.

[0086] S6. For each target seedling group, determine whether there is a continuous shortage of seedlings in the group.

[0087] In step S6, for each target seedling group, determining whether there is a continuous shortage of seedlings in the group specifically includes:

[0088] For all the second center coordinates in the target seedling group, the row spacing value corresponding to the second center coordinate is calculated to determine whether the row spacing value is greater than the second threshold value. If so, it is determined that the target seedling group has a continuous shortage of seedlings in the group.

[0089] Exemplarily, there are two cases of continuous seedling shortage. One case is that a row of seedlings is not completely missing, and there are still seedling coordinates in the group, which is continuous seedling shortage within the group. The other case is that a row of seedlings is completely missing, and the group disappears directly, which is continuous seedling shortage between groups.

[0090] For continuous missing seedlings within a group, set the threshold b for continuous missing seedlings within the group in advance, select one of the groups, subtract the absolute value of the ordinate of the first seedling coordinate in the group from 0.3 (the upper filling part of the image y=320 / 1080≈0.3), subtract the absolute value of the ordinate of the second seedling coordinate from the ordinate of the first seedling coordinate, and so on, until the absolute value of the ordinate of the last seedling coordinate is subtracted from 1. If there is a value greater than the threshold b, it is considered that there is continuous missing seedlings within the group. Repeat this operation for all groups to complete the judgment of continuous missing seedlings within all groups. If only one group is continuously missing seedlings, it is a single-row continuous missing seedling. If two or more groups are continuously missing seedlings, it is a multi-row continuous missing seedling.

[0091] S7. For two adjacent target seedling groups, determine whether there is a continuous shortage of seedlings between the groups.

[0092] In step S7, for two adjacent target seedling groups, it is determined whether there is a continuous shortage of seedlings between the groups:

[0093] Based on the coordinate value of the X-axis of the first second center coordinate in each target seedling group, the group spacing value between two adjacent target seedling groups is calculated. If the group spacing value is greater than the third threshold value and less than the fourth threshold value, it is determined that there is a single group of continuous seedling shortages. If the group spacing value is greater than the fourth threshold value, it is determined that there are multiple groups of continuous seedling shortages. Among them, the fourth threshold value is greater than the third threshold value.

[0094] Exemplarily, for continuous missing seedlings between groups, a threshold value c for continuous missing seedlings between groups is set in advance. If the number of groups does not reach the number of rows of the rice transplanter, it is determined to be a continuous missing seedlings between groups. If only one group is less, it is a continuous missing seedlings in a single row. If two or more groups are less, it is a continuous missing seedlings in multiple rows. If the number of groups is the same as the number of rows of the rice transplanter, all groups are sorted from small to large according to the horizontal coordinate of the first seedling coordinate in the group, and the group spacing between all groups is calculated one by one according to the horizontal coordinate. If only one spacing is greater than c and less than 2c, it is determined to be a continuous missing seedlings in a single row. If there is a spacing greater than 2c, it is determined to be a continuous missing seedlings in multiple rows. If more than one spacing is greater than c, it is determined to be a continuous missing seedlings in multiple rows. If there is both continuous missing seedlings within the group and continuous missing seedlings between groups, it is determined to be a continuous missing seedlings in multiple rows.

[0095] The present application provides a method for detecting missing seedlings in rice transplanter operation, which realizes the detection of single-row or multiple-row continuous missing seedlings through seedling image filling, YOLOv10 model training and recognition, perspective transformation, nonlinear transformation, sorting, grouping, group selection, and continuous missing seedling judgment within and between groups of seedling coordinates, which not only improves the accuracy of missing seedling detection but also has wide applicability.

[0096] In one embodiment of the present application, a method for training a target detection model is provided, which specifically includes:

[0097] Dataset creation and model training. Combine images of different time periods (morning, noon, evening) and weather conditions (sunny, cloudy, overcast, rainy) to create a dataset. Remove blurry images and images without targets from the dataset, and use the labelimg tool to annotate the bounding boxes of the rice seedlings in the images. Divide the dataset into training set, validation set, and test set in a ratio of 7:2:1. The training set is used for training the YOLOv10 model, the validation set is used for adjusting parameters and monitoring the overfitting of the model, and the test set is used for performance evaluation of the final model.

[0098] The present invention uses the YOLOv10 algorithm to detect the target of the rice seedlings. The YOLOv10 algorithm is a real-time target detection algorithm. The end-to-end training method of the algorithm can complete the detection of all rice seedlings in the image through one forward propagation. Compared with the two-stage detector, the detection efficiency can be greatly improved. YOLOv10 includes three parts: Backbone (backbone network), Neck (neck structure) and Head (head structure). Backbone uses an enhanced version of CSPNet (cross-stage partial network) to extract the features of the rice seedling image. Neck includes a PAN (path aggregation network) layer to achieve effective multi-scale feature fusion. Head is divided into one-to-many allocation and one-to-one allocation. The model can use the rich supervision signals of the one-to-many allocation during training, and can use the prediction results of the one-to-one allocation during reasoning, thereby achieving efficient reasoning without NMS (non-maximum suppression). Since there are only samples such as seedlings in the dataset, in order to prevent overfitting, random cropping, horizontal flipping, color jittering, scaling, rotation and Mosaic are used to enhance the data when training the model. At the same time, the weight decay coefficient weight_decay is appropriately increased. This coefficient controls overfitting by adding a penalty term proportional to the L2 norm of the weight to the loss function.

[0099] See also Fig. 9 , Fig. 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Fig. 9 As shown in , the electronic device 900 includes a processor 910 , a memory 920 and a bus 930 .

[0100] The memory 920 stores machine-readable instructions executable by the processor 910. When the electronic device 900 is running, the processor 910 communicates with the memory 920 via the bus 930. When the machine-readable instructions are executed by the processor 910, the steps of a method for detecting a lack of seedlings in a rice transplanter operation as described in the above method embodiment can be executed. The specific implementation method can be found in the method embodiment, which will not be repeated here.

[0101] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of a method for detecting a lack of seedlings in a rice transplanter operation as in the above-mentioned method embodiment can be executed. The specific implementation method can be found in the method embodiment and will not be described in detail here.

[0102] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0103] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0104] In addition, the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0105] Furthermore, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0106] It should be noted that if the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can essentially be embodied in the form of a software product, or the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM) random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0107] In this document, relational terms such as first and second, etc. are used merely to distinguish one entity or operation from another entity or operation, but do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0108] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for detecting seedling shortage in rice transplanter operation, characterized in that: The method comprises: The image of the rice seedlings after the rice transplanter is operated is obtained by using an image sensor installed on the rice transplanter; Filling the preset range in the seedling image; The filled seedling image is input into the target detection model to determine the first center coordinates of all the seedlings in the seedling image; With respect to the first center coordinate of each seedling, the first center coordinate of the seedling is corrected to obtain the second center coordinate of the seedling; Based on the corrected second center coordinates of all the seedlings, all the seedlings are grouped to determine a plurality of target seedling groups, wherein the seedlings in each target seedling group belong to the same working seedling row of the rice transplanter; For each target seedling group, determine whether there is a continuous shortage of seedlings in the group; For two adjacent target seedling groups, determine whether there is a continuous shortage of seedlings between the groups.

2. The method according to claim 1, characterized in that The first center coordinate of each seedling is corrected in the following way: Based on the first center coordinates and the perspective transformation matrix, the first center coordinates are perspective transformed to obtain intermediate center coordinates; Based on the nonlinear transformation participating in the intermediate center coordinate, the intermediate center coordinate is nonlinearly transformed to obtain the corresponding second center coordinate.

3. The method according to claim 2, characterized in that The step of performing perspective transformation on the first center coordinates based on the first center coordinates and the perspective transformation matrix to obtain the intermediate center coordinates specifically includes: Process the two-dimensional first center coordinate (X, Y) into the three-dimensional first center coordinate (X, Y, Z=1); Multiply the first center coordinate of the three-dimensional image by the perspective transformation matrix to obtain the perspective transformation coordinates (U, V, W); Calculate the ratio between the coordinate value U of the perspective transformation coordinate X axis and the coordinate value W of the Z axis as the coordinate value X′ of the X axis of the two-dimensional intermediate center coordinate; The Y-axis coordinate value Y of the first center coordinate is used as the Y-axis coordinate value Y′ of the two-dimensional intermediate center coordinate.

4. The method according to claim 3, characterized in that The step of performing nonlinear transformation on the intermediate center coordinates based on the nonlinear transformation to obtain the corresponding second center coordinates specifically includes: Determine whether the coordinate value X′ of the intermediate center coordinate X axis is less than the center value; If yes, then X′ and the nonlinear transformation parameter t are substituted into the first calculation formula to obtain the coordinate value X″ of the X axis of the second center coordinate; If not, then X′ and the nonlinear transformation parameter t are substituted into the second calculation formula to obtain the X-axis coordinate value X″ of the second center coordinate; The coordinate value of the Y axis of the second center coordinate is Y″=Y′.

5. The method according to claim 1, characterized in that A plurality of seedling groups are determined by: Based on the second center coordinates of all the seedlings, all the seedlings are divided into a plurality of sample seedling groups, wherein the seedlings in each sample seedling group belong to the same working seedling row of the rice transplanter; A plurality of target seedling groups corresponding to the number of seedling rows operated by the rice transplanter are determined from the plurality of sample seedling groups.

6. The method according to claim 5, characterized in that The step of dividing all the seedlings into a plurality of sample seedling groups based on the second center coordinates of all the seedlings specifically comprises: Arrange the second center coordinates of all seedlings in ascending order according to the coordinate values ​​of the Y axis; The first second center coordinate after sorting is used as the first second center coordinate in the first sample seedling group; For each remaining second center coordinate, perform the following steps in sequence: Determine whether the difference between the X-axis coordinate value of the second center coordinate and the X-axis coordinate value of the last second center coordinate in the current sample seedling group is less than a first threshold; If yes, the second center coordinate is placed into the current sample seedling group; If not, then create the next sample seedling group, and use the second center coordinate as the first second center coordinate in the next sample seedling group.

7. The method according to claim 1, characterized in that For each target seedling group, the step of determining whether there is a continuous shortage of seedlings in the group specifically includes: For all second center coordinates in the target seedling group, calculate the row spacing value corresponding to the second center coordinate, and determine whether the row spacing value is greater than a second threshold; If so, it is determined that the target seedling group has a continuous shortage of seedlings within the group.

8. The method according to claim 1, characterized in that For two adjacent target seedling groups, determine whether there is a continuous shortage of seedlings between the groups: Based on the coordinate value of the X-axis of the first second center coordinate in each target seedling group, the group spacing value between two adjacent target seedling groups is calculated; If the group spacing value is greater than the third threshold value and less than the fourth threshold value, it is determined that there is a single group of continuous seedling shortages; If the group spacing value is greater than the fourth threshold, it is determined that there are multiple groups of consecutive missing seedlings; The fourth threshold is greater than the third threshold.

9. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the memory via the bus, and the processor executes the machine-readable instructions to perform the steps of the method for detecting missing seedlings in a rice transplanter operation as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for detecting seedling shortage in a rice transplanter operation as claimed in any one of claims 1 to 8 are executed.

Images