Machine vision based positioning and fertilizing method for a fertilizer applicator

Abstract

The present application relates to the field of image processing, more particularly, the present application relates to a machine vision-based fertilizer applicator positioning and fertilizing method, the method comprising: collecting an RGB-D image of a plant to be fertilized; segmenting target plant pixels and ground pixels in the RGB-D image, clustering and analyzing the spatial points corresponding to the target plant pixels to obtain a trunk point cloud cluster, and obtaining the trunk point cloud cluster closest to the ground as a reference point cloud cluster; if the reference point cloud cluster is a rhizome cluster, obtaining a fertilizing position; if the reference point cloud cluster is not a rhizome cluster, calculating a predicted extension direction of the rhizome, and iteratively predicting each extension point according to the extension direction until the ground is reached to obtain a rhizome point, thereby achieving accurate fertilizing positioning.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing. More particularly, the present application relates to a machine vision-based positioning and fertilizing method for a fertilizer applicator. BACKGROUND

[0002] Accurate positioning of crop key parts (such as rootstocks) is a core prerequisite for realizing variable fertilization, intelligent plant protection and other automated operations. With the development of three-dimensional perception technology, crop structure analysis based on point cloud data has become a mainstream means, which obtains three-dimensional point cloud of crops through a depth camera, and then extracts key information such as main stem and canopy from the point cloud, thereby providing data support for subsequent positioning and decision-making.

[0003] In the extraction of key structures of crop point cloud, geometric model fitting methods (such as line and curve fitting based on RANSAC) are widely used for estimation of main stem shape and inference of rootstock position due to their robustness to noise and outliers. The core logic of such methods is to filter out an "effective point set" representing the trend of the main stem from the point cloud, fit a global geometric model (such as a straight line or a smooth curve), and then extrapolate the model to the ground to determine the position of the rootstock. In an ideal case, when the main stem of the crop is not blocked, the point cloud data is complete and dense, such methods can achieve high positioning accuracy.

[0004] However, the complexity of agricultural scenes poses a serious challenge to traditional geometric fitting methods, with the problem of severe occlusion being particularly prominent. In the field environment, the lower part of the crop stem (the area close to the ground) is easily blocked by the dense canopy, leaves or adjacent crops, resulting in the actual observed "main stem point set" being sparse and incomplete, containing only a small number of discrete points in the upper part of the main stem (far from the ground). The characteristics of this "high-sparse point set" directly break the core premise of traditional geometric fitting methods, i.e. there are enough observed points that can represent the overall trend of the main stem.

[0005] When relying only on the upper sparse points for model fitting, traditional methods will face insurmountable limitations:

[0006] The trend of the global model is completely determined by the distribution of the upper points, and cannot reflect the natural bending, deflection or growth direction changes that may exist in the blocked lower part (for example, the bending of the crop changes due to gravity); when the upper model is directly extrapolated to the ground, the error will increase sharply with the increase of the extrapolation distance, eventually leading to a serious deviation of the rootstock positioning result from the true position. This process is similar to "estimating the accurate position of the tree root underground only by observing the inclination of the tree top", which has very low reliability.

[0007] In addition, although there are attempts in the prior art to complete the missing lower segment structure by deep learning method, such methods rely on a large amount of labeled data for model training, and have poor generalization and high computational cost in complex and variable agricultural scenes, and the physical rationality of the completion result is difficult to guarantee, which is not suitable for resource-limited agricultural automation equipment.

[0008] Therefore, under the condition of serious occlusion in the agricultural scene, how to get rid of the dependence on the complete trunk point cloud, without the aid of deep learning data completion means, and only using the observable upper trunk information and the overall spatial distribution characteristics of the crop, to scientifically and stably infer the most possible growth path of the occluded lower segment, and finally realize the accurate positioning of the rootstock position, has become a key technical problem to be solved in the field. SUMMARY

[0009] To solve the problem of how to accurately infer the trunk information of the occluded area of the crop to realize accurate rootstock positioning, the present application proposes a fertilizing method for a fertilizing machine based on machine vision, which comprises the following steps:

[0010] Collecting an RGB-D image of a plant to be fertilized;

[0011] Segmenting the target plant pixels and ground pixels in the RGB-D image, clustering and analyzing the spatial points corresponding to the target plant pixels to obtain trunk point cloud clusters, and obtaining the trunk point cloud cluster closest to the ground as the reference point cloud cluster; if the reference point cloud cluster is a rootstock cluster, the fertilizing position is obtained;

[0012] If the reference point cloud cluster is not a rootstock cluster, the predicted extension direction of the rootstock is calculated, the centroid of the reference point cloud cluster is taken as the initial point, the predicted extension point of the rootstock is obtained from the initial point along the predicted extension direction, in response to the predicted extension point not reaching the point cloud plane corresponding to the ground pixel, the predicted extension point is taken as a new initial point, a new predicted extension direction of the rootstock is calculated, a new predicted extension point of the rootstock is obtained from the new initial point along the new predicted extension direction, in response to the new predicted extension point reaching the point cloud plane corresponding to the ground pixel, the new predicted extension point is taken as the rootstock point, and the fertilizing position is obtained; wherein the predicted extension direction is equal to the direction of the weighted cumulative vector of the self-predicted growth vector and the corrected growth vector, the self-predicted growth vector is equal to the weighted cumulative sum of the principal direction unit vector of the reference point cloud cluster and the gravity direction unit vector, and the corrected growth vector is equal to the weighted cumulative sum of the growth vectors of the plants in the neighborhood.

[0013] The application realizes accurate prediction of plant root and stem positions by accurately predicting the growth direction of the plant, thereby providing a basis for accurate fertilization positioning; further, when predicting the growth direction of the plant, the gravity factor affecting the growth direction of the plant and the original growth direction factor are introduced, so that the plant growth characteristics are more in line with the actual plant growth characteristics, and more accurate plant growth direction prediction is realized instead of simply fitting according to data; further, when predicting the growth direction of the plant, considering that the plants in the local area have similar growth direction characteristics, the growth direction of the surrounding plants is used to correct the self-predicted growth vector predicted by the self growth factor, thereby realizing more accurate plant growth direction prediction.

[0014] Preferably, the target plant pixels and the ground pixels in the RGB-D image are segmented, comprising:

[0015] Based on the RGB channel information in the RGB-D image, the target plant pixels and the ground pixels are segmented by using an image segmentation method.

[0016] Preferably, the trunk point cloud cluster is obtained by clustering analysis of the spatial points corresponding to the target plant pixels, comprising:

[0017] Based on the coordinate information and the depth channel data corresponding to the target plant pixels, the spatial points corresponding to the target plant pixels are obtained;

[0018] The perpendicularity of the spatial points corresponding to the target plant pixels to the ground is calculated;

[0019] Based on the perpendicularity, the RGB channel data and the spatial point coordinates, the spatial points corresponding to all target plant pixels are clustered to obtain a plurality of point cloud clusters;

[0020] Based on the perpendicularity, the trunk point cloud cluster is screened out from all point cloud clusters.

[0021] The application introduces a distinguishing feature that can better distinguish the trunk from the leaves for classification processing, thereby providing a basis for accurately extracting the trunk point cloud cluster.

[0022] Preferably, the perpendicularity of the spatial points corresponding to the target plant pixels to the ground is calculated, comprising:

[0023] Based on the coordinate information and the depth channel data corresponding to the ground pixels, the spatial points corresponding to the ground pixels are obtained; the spatial points corresponding to the ground pixels are used to fit a ground plane to obtain a normal vector of the ground plane;

[0024] The main direction vector at the spatial point corresponding to the target plant pixel is obtained, and the product of the normal vector and the main direction is taken as the perpendicularity of the spatial point corresponding to the target plant pixel to the ground.

[0025] The perpendicularity is obtained by vector multiplication, and the implementation is simple and efficient.

[0026] Preferably, the filtering of the trunk point cloud cluster from all point cloud clusters based on the perpendicularity comprises:

[0027] The average perpendicularity of all spatial points in each point cloud cluster is calculated.

[0028] All point cloud clusters are clustered into two categories by using the average perpendicularity, and all point cloud clusters in the category with a large average perpendicularity are taken as the trunk point cloud cluster.

[0029] The trunk point cloud cluster is accurately filtered by the feature that the trunk is generally perpendicular to the ground.

[0030] Preferably, if the reference point cloud cluster is a root cluster, the fertilization position is obtained, comprising:

[0031] If the reference point cloud cluster intersects with the spatial point corresponding to the ground pixel, the intersection position of the root cluster and the spatial point corresponding to the ground pixel is taken as the fertilization position.

[0032] Preferably, the method for obtaining the self-predicted growth vector comprises:

[0033]

[0034] wherein, represents the self-predicted growth vector, represents a unit vector in the direction of the main direction of the reference point cloud cluster, represents a unit vector in the direction of the gravity direction, represents the first weight, represents the second weight.

[0035] Preferably, the method for obtaining the first weight and the second weight comprises:

[0036] The distance between the initial point and the centroid of the reference point cloud cluster is recorded as the deviation distance, the opposite number of the cumulative sum of the preset parameter and the deviation distance is taken as the exponent, and the natural constant is taken as the base number to obtain the first weight;

[0037] The difference between 1 and the first weight is taken as the second weight.

[0038] The present application considers that the farther from the reference point cloud cluster, the lower the consistency of the growth direction at the reference point cloud cluster, and more accurately sets the weight, thereby providing a basis for accurate growth direction prediction.

[0039] Preferably, the method for obtaining the corrected growth vector comprises:

[0040] The target plant pixel is recorded as a target plant, the fertilized plant in the preset neighborhood of the target plant is recorded as a reference plant, the spatial point with the same height as the centroid of the reference point cloud cluster is obtained as a homologous point of the reference plant in all spatial points corresponding to the pixels of the reference plant, the unit vector with the main direction of the homologous point of the reference plant as the direction is taken as the growth vector of the reference plant at the homologous point, the inverse of the distance from the reference plant to the target plant is taken as the weight, the growth vectors of all homologous points of the reference plants are weighted and summed, and the obtained vector is recorded as a corrected growth vector.

[0041] Preferably, the method further comprises the following steps of:

[0042] The spatial point with a distance of a preset length from the initial point in the predicted extension direction is taken as the predicted extension point of the rhizome.

[0043] The present application has the following advantages:

[0044] The present application accurately predicts the growth direction of the plant, thereby realizing accurate prediction of the position of the rhizome of the plant and providing a basis for accurate fertilization positioning.

[0045] Further, when predicting the growth direction of the plant, the gravity factor affecting the growth direction of the plant and the original growth direction factor are introduced, so that the actual plant growth characteristics are more in line with the actual plant growth characteristics, and the more accurate prediction of the growth direction of the plant is realized.

[0046] Further, when predicting the growth direction of the plant, the growth direction of the surrounding plants is used to correct the self-predicted growth vector predicted by the growth factor of the plant itself, so that the more accurate prediction of the growth direction of the plant is realized. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 is a step flow chart of the fertilization machine positioning and fertilization method based on machine vision provided by the embodiment 1 of the present application;

[0048] Figure 2 is an RGB image of the plant to be fertilized provided by the embodiment 2 of the present application;

[0049] Figure 3 is a three-dimensional point cloud image of the plant to be fertilized provided by the embodiment 2 of the present application. DETAILED DESCRIPTION

[0050] Embodiment 1:

[0051] Please refer to Figure 1 which shows the step flow chart of the fertilization machine positioning and fertilization method based on machine vision provided by the embodiment 1 of the present application, and the method comprises the following steps:

[0052] S1: acquiring an RGB-D image of a plant to be fertilized.

[0053] Specifically, an RGB-D image of a plant to be fertilized is acquired by using an RGB-D camera installed on a fertilizer applicator.

[0054] S2: segmenting target plant pixels and ground pixels in the RGB-D image, clustering and analyzing spatial points corresponding to the target plant pixels to obtain a trunk point cloud cluster, and obtaining a trunk point cloud cluster closest to the ground as a reference point cloud cluster.

[0055] S20: segmenting target plant pixels and ground pixels in the RGB-D image.

[0056] It should be noted that under normal circumstances, fertilizers are generally applied to the roots of plants, so as to achieve precise fertilization and avoid waste of fertilizers, and therefore it is necessary to accurately locate the root and stem parts of the plant. Since the root and stem parts are generally located at the junction between the trunk of the plant and the ground, it is necessary to first segment the trunk information and the ground information in the RGB-D image.

[0057] Preferably, as an example, the target plant pixels and the ground pixels in the RGB-D image are segmented, comprising:

[0058] Based on the RGB channel information in the RGB-D image, a semantic segmentation network is used to segment the target plant region and the ground region, and the pixels in the target plant region are recorded as target plant pixels, and the pixels in the ground region are recorded as ground pixels.

[0059] S21: clustering and analyzing spatial points corresponding to the target plant pixels to obtain a trunk point cloud cluster, and obtaining a trunk point cloud cluster closest to the ground as a reference point cloud cluster.

[0060] It should be noted that since the root and stem parts are generally located at the junction between the trunk of the plant and the ground, the trunk closest to the ground is more likely to be the root and stem, and therefore the trunk closest to the ground needs to be selected first.

[0061] It should be further noted that the plant generally has leaves and a trunk, and the trunk information needs to be extracted from the plant first. Since the trunk of the plant generally has a large degree of perpendicularity to the ground, and the leaves have a small degree of perpendicularity to the ground, the trunk in the plant can be selected based on the degree of perpendicularity and the positional relationship of the spatial points.

[0062] Preferably, as an example, the target plant pixels and the ground pixels in the RGB-D image are segmented, comprising:

[0063] Based on the coordinate information and the depth channel data corresponding to the target plant pixels, the target plant pixels are converted into spatial points by using a camera perspective projection transformation method;

[0064] The perpendicularity of the spatial point corresponding to the target plant pixel to the ground is calculated.

[0065] Based on the perpendicularity, the RGB channel data and the spatial point coordinates, the spatial points corresponding to all target plant pixels are clustered to obtain a plurality of point cloud clusters.

[0066] The trunk point cloud cluster is screened out from all the point cloud clusters based on the perpendicularity.

[0067] The spatial point corresponding to the ground pixel is obtained, the distance from the centroid of each trunk point cloud cluster to the nearest ground spatial point is calculated, and the trunk point cloud cluster with the smallest distance is taken as the reference point cloud cluster.

[0068] It can be understood that, since the trunk generally has a large perpendicularity to the ground and the leaves have a small perpendicularity to the ground, and the colors of the trunk and the leaves are different, the trunk and the leaves will be classified into different clusters by the perpendicularity and the color. Meanwhile, there will be mutual occlusion between the trunk and the leaves, and some parts of the trunk cannot obtain spatial point information, so the trunk point cloud will not be continuously presented, and thus the trunk point cloud will present a plurality of point cloud clusters.

[0069] It should be added that the perpendicularity of the spatial point corresponding to the target plant pixel to the ground is calculated, including:

[0070] Based on the coordinate information and the depth channel data corresponding to the ground pixel, the ground pixel is converted into a spatial point by using a camera perspective projection transformation method; a ground plane is fitted by using the spatial point corresponding to the ground pixel, a normal vector of the ground plane is obtained, and the normal vector is a unit vector perpendicular to the ground plane;

[0071] The spatial points in a preset region are obtained with the spatial point corresponding to the target plant pixel as the center, the main direction of the spatial point corresponding to the target plant pixel is obtained by analyzing the spatial points in the preset region by using a PCA algorithm, a unit vector in the main direction is obtained and is denoted as a main direction vector, and the product of the normal vector and the main direction vector is taken as the perpendicularity of the spatial point corresponding to the target plant pixel to the ground.

[0072] It can be understood that, if the extension direction of the spatial point is perpendicular to the ground, the main direction of the spatial point is parallel to the normal vector of the ground plane, and thus the dot product of the vector obtained in the extension direction of the spatial point and the normal vector of the ground plane is large. Therefore, the perpendicularity of the spatial point to the ground can be more accurately evaluated based on this.

[0073] It should be further added that the trunk point cloud cluster is screened out from all the point cloud clusters based on the perpendicularity, including:

[0074] Calculate the average of the perpendicularity of all spatial points in each point cloud cluster;

[0075] All point cloud clusters are divided into two categories using the average of the perpendicularity, and all point cloud clusters in the category with a large average of the perpendicularity are taken as the trunk point cloud cluster.

[0076] It can be understood that the trunk of the plant has a large perpendicularity to the ground. Therefore, the point cloud cluster with a large perpendicularity is the trunk point cloud cluster.

[0077] S3: If the reference point cloud cluster is a rhizome cluster, a fertilization position is obtained; if the reference point cloud cluster is not a rhizome cluster, a predicted extension direction of the rhizome is calculated, a centroid of the reference point cloud cluster is taken as an initial point, a predicted extension point of the rhizome is obtained from the initial point along the predicted extension direction, in response to the predicted extension point not reaching a point cloud plane corresponding to a ground pixel, the predicted extension point is taken as a new initial point, a new predicted extension direction of the rhizome is calculated, a new predicted extension point of the rhizome is obtained from the new initial point along the new predicted extension direction, in response to the new predicted extension point reaching the point cloud plane corresponding to the ground pixel, the new predicted extension point is taken as a rhizome point, and a fertilization position is obtained; wherein the predicted extension direction is equal to a direction of a cumulative sum vector of a first force vector and a second force vector, the first force vector is equal to a weighted cumulative sum of a main direction unit vector of the reference point cloud cluster and a gravity direction unit vector, and the second force vector is equal to a vector formed by the initial point and a centroid of a point cloud corresponding to a target plant pixel.

[0078] S30: If the reference point cloud cluster is a rhizome cluster, a fertilization position is obtained.

[0079] Preferably, as an example, if the reference point cloud cluster is a rhizome cluster, a fertilization position is obtained, including:

[0080] If the reference point cloud cluster intersects with a space corresponding to a ground pixel, an intersection position of the rhizome cluster and the space corresponding to the ground pixel is taken as the fertilization position.

[0081] S31: If the reference point cloud cluster is not a rhizome cluster, a predicted extension direction of the rhizome is calculated, a centroid of the reference point cloud cluster is taken as an initial point, a predicted extension point of the rhizome is obtained from the initial point along the predicted extension direction, in response to the predicted extension point not reaching a point cloud plane corresponding to a ground pixel, the predicted extension point is taken as a new initial point, a new predicted extension direction of the rhizome is calculated, a new predicted extension point of the rhizome is obtained from the new initial point along the new predicted extension direction, in response to the new predicted extension point reaching the point cloud plane corresponding to the ground pixel, the new predicted extension point is taken as a rhizome point, and a fertilization position is obtained.

[0082] It should be noted that the reference point cloud cluster is not the rhizome, which means that the rhizome information is blocked by leaves and other objects, and the position of the rhizome needs to be predicted according to the information of the reference point cloud cluster closest to the rhizome.

[0083] Step one: calculate the predicted extension direction of the rhizome.

[0084] Preferably, as an example, the predicted extension direction of the rhizome is calculated, comprising:

[0085] First, calculate the self-predicted growth vector.

[0086] It should be noted that the plant growth generally does not appear mutation, so the plant generally grows to some extent along the original direction, and the plant growth is also affected by gravity, so the plant also grows to some extent along the direction of gravity.

[0087] The self-predicted growth vector satisfies the relationship:

[0088]

[0089] Among them, represents the self-predicted growth vector, represents a unit vector in the direction of the main direction of the reference point cloud cluster, represents a unit vector in the direction of the gravity direction, represents the first weight, represents the second weight.

[0090] Among them, the setting method of the first weight and the second weight comprises:

[0091] The distance between the initial point and the centroid of the reference point cloud cluster is recorded as the deviation distance, the opposite number of the cumulative sum of the preset parameter and the deviation distance is taken as the index, and the natural constant is taken as the base number. Calculate the first weight; the difference between 1 and the first weight is the second weight.

[0092] It can be understood that the unit vector in the direction of the main direction of the reference point cloud cluster reflects the original growth direction of the plant, and the unit vector in the direction of the gravity direction reflects the direction of gravity, and since the plant will be affected to some extent by the gravity and the original growth direction of the plant, the growth direction of the plant can be predicted to some extent through these two aspects.

[0093] Then, calculate the correction growth vector.

[0094] It should be noted that the growth direction of the plant is also affected by factors such as light and wind, but these two factors are difficult to measure. The plants in the nearby area are generally affected by similar external environment and resource factors, so the growth direction of the plants in the nearby area is also similar. For example, the plants in the nearby area are affected by similar factors such as light and wind, so the growth direction of the plants in the nearby area is similar. Therefore, the influence of factors such as light and wind can be reflected to some extent based on the growth direction of the plants in the nearby area.

[0095] The method for obtaining the corrected growth vector comprises:

[0096] The plant composed of target plant pixels is recorded as a target plant, the plants in the preset neighborhood of the target plant that have been fertilized are recorded as reference plants, the spatial points with the same height as the centroid of the reference point cloud are obtained from the spatial points corresponding to all pixels of the reference plants as the homologous points of the reference plants, the unit vector with the main direction of the homologous point of the reference plant as the direction is taken as the growth vector of the reference plant at the homologous point, the inverse of the distance from the reference plant to the target plant is taken as the weight, the growth vectors of all the homologous points of the reference plants are weighted and summed, and the obtained vector is recorded as the corrected growth vector.

[0097] It should be noted that the method for obtaining the main direction of the homologous point of the reference plant comprises:

[0098] The spatial points in the preset neighborhood are obtained as analysis spatial points with the homologous point of the reference plant as the center, and the PCA algorithm is used to process the analysis spatial points to obtain the main direction of the homologous point of the reference plant.

[0099] It can be understood that the closer the distance to the target plant, the more similar the external factors received by the target plant, and the higher the similarity of the growth direction of the target plant. By taking the inverse of the distance to the target plant as the weight, more reference is made to other plants close to the target plant, so that the corrected growth vector is more accurately obtained.

[0100] Then, the self-predicted growth vector is corrected by using the corrected growth vector to obtain.

[0101] The method for correcting the self-predicted growth vector by using the corrected growth vector comprises:

[0102] The angle between the growth vectors of all pairs of reference plants is calculated, the inverse of the average of the angles between the growth vectors of all pairs of reference plants is taken as the weight of the corrected growth vector, the difference between one and the weight of the corrected growth vector is taken as the weight of the self-predicted growth vector, the corrected growth vector and the self-predicted growth vector are weighted and summed to obtain a comprehensive predicted growth vector, and the direction of the comprehensive predicted growth vector is taken as the predicted extension direction of the rhizome.

[0103] It can be understood that the angle between the growth vectors of the reference plants reflects the difference in the growth direction of the reference plants, and the greater the difference in the growth direction of the reference plants, the greater the possibility that the growth direction of the target plant is different from that of the reference plant, and thus the growth direction of the target plant should not be predicted by excessively referring to the growth direction of the reference plant, and thus the weight is set based on the angle between the growth vectors of the reference plants to adjust the reference of the modified growth vector, thereby achieving accurate prediction of the growth direction of the target plant.

[0104] Step two: taking the centroid of the reference point cloud cluster as an initial point, and obtaining a predicted extension point of the rhizome along the predicted extension direction from the initial point.

[0105] Preferably, as an example, taking the centroid of the reference point cloud cluster as an initial point, and obtaining a predicted extension point of the rhizome along the predicted extension direction from the initial point, comprises:

[0106] Taking the centroid of the reference point cloud cluster as an initial point, and obtaining a spatial point with a preset length from the initial point in the predicted extension direction as the predicted extension point of the rhizome.

[0107] Step three: in response to the predicted extension point not reaching the point cloud plane corresponding to the ground pixel, taking the predicted extension point as a new initial point, calculating a new predicted extension direction of the rhizome, and obtaining a new predicted extension point of the rhizome along the new predicted extension direction from the new initial point.

[0108] Preferably, as an example, in response to the predicted extension point not reaching the point cloud plane corresponding to the ground pixel, taking the predicted extension point as a new initial point, calculating a new predicted extension direction of the rhizome, and obtaining a new predicted extension point of the rhizome along the new predicted extension direction from the new initial point, comprises:

[0109] In response to the predicted extension point not reaching the point cloud plane corresponding to the ground pixel, taking the predicted extension point as a new initial point, and calculating a new predicted extension direction of the rhizome according to the method in step one.

[0110] Obtaining a spatial point with a preset length from the new initial point in the predicted extension direction as the new predicted extension point of the rhizome.

[0111] It can be understood that through iterative prediction with small steps, the growth trend of the crops is more closely fitted, and the position of the rhizome of the plant is more accurately fitted, thereby providing a basis for accurate fertilization positioning.

[0112] Step four: in response to the new predicted extension point reaching the point cloud plane corresponding to the ground pixel, taking the new predicted extension point as a rhizome point to obtain a fertilization position.

[0113] Preferably, as an example, in response to the new predicted extension point reaching the point cloud plane corresponding to the ground pixel, the new predicted extension point is taken as the rhizome point, and the fertilization position is obtained, comprising:

[0114] In response to the new predicted extension point reaching the point cloud plane corresponding to the ground pixel, the new predicted extension point is taken as the rhizome point, and the rhizome point is taken as the fertilization position.

[0115] So far, the embodiment is completed.

[0116] Embodiment 2:

[0117] The embodiment is a specific embodiment of the fertilization machine positioning and fertilization method based on machine vision.

[0118] In order to illustrate the technical effect of the embodiment 1 of the present application, the technical effect of the technical solution will be illustrated below in combination with some experimental process images.

[0119] Experimental goal: verify the rhizome position positioning effect of the algorithm.

[0120] Experimental effect display:

[0121] The method in embodiment 1 is used to process the RGB-D image to obtain the positioning result, Figure 2 The RGB image is shown in FIG. 8, Figure 3 The three-dimensional point cloud is shown in FIG. 9, wherein the H point in the three-dimensional point cloud image is the predicted rhizome point.

[0122] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A machine vision-based method of positioning fertilization by a fertilizer applicator, characterized in that, The method comprises the following steps: Collecting an RGB-D image of a plant to be fertilized; Segmenting target plant pixels and ground pixels in the RGB-D image, clustering and analyzing the spatial points corresponding to the target plant pixels to obtain a trunk point cloud cluster, and obtaining a reference point cloud cluster closest to the ground; If the reference point cloud cluster is a rhizome cluster, a fertilization position is obtained; If the reference point cloud cluster is not a rhizome cluster, a predicted extension direction of the rhizome is calculated, the centroid of the reference point cloud cluster is taken as an initial point, the predicted extension point of the rhizome is obtained from the initial point along the predicted extension direction, in response to the predicted extension point not reaching the point cloud plane corresponding to the ground pixel, the predicted extension point is taken as a new initial point, a new predicted extension direction of the rhizome is calculated, the new predicted extension point of the rhizome is obtained from the new initial point along the new predicted extension direction, in response to the new predicted extension point reaching the point cloud plane corresponding to the ground pixel, the new predicted extension point is taken as a rhizome point, and a fertilization position is obtained; wherein the predicted extension direction is equal to the direction of the weighted cumulative vector of the self-predicted growth vector and the corrected growth vector, the self-predicted growth vector is equal to the weighted cumulative sum of the principal direction unit vector of the reference point cloud cluster and the gravity direction unit vector, and the corrected growth vector is equal to the weighted cumulative sum of the growth vectors of the plants in the neighborhood.

2. The machine vision-based positioning and fertilization method of claim 1, wherein, The method further comprises the following steps: Segmenting target plant pixels and ground pixels in the RGB-D image based on RGB channel information in the RGB-D image by using an image segmentation method.

3. The machine vision-based positioning and fertilization method of claim 1, wherein, The method further comprises the following steps: Based on the coordinate information and depth channel data of the target plant pixels, spatial points corresponding to the target plant pixels are obtained; Calculating the perpendicularity of the spatial points corresponding to the target plant pixels to the ground; Based on the perpendicularity, RGB channel data and spatial point coordinates, clustering of the spatial points corresponding to all target plant pixels is performed to obtain a plurality of point cloud clusters; Based on the perpendicularity, a trunk point cloud cluster is screened out from all point cloud clusters.

4. The machine vision-based positioning and fertilization method of claim 3, wherein, The method further comprises the following steps: Based on the coordinate information and depth channel data of the ground pixels, spatial points corresponding to the ground pixels are obtained; a ground plane is fitted using the spatial points corresponding to the ground pixels, and a normal vector of the ground plane is obtained; The product of the normal vector and the main direction is taken as the perpendicularity of the spatial points corresponding to the target plant pixels to the ground.

5. The machine vision-based positioning and fertilization method of claim 3, wherein, The method further comprises the following steps: Calculating the average perpendicularity of all spatial points in each point cloud cluster; All point cloud clusters are clustered into two categories using the average perpendicularity, and all point cloud clusters in the category with a larger average perpendicularity are taken as the trunk point cloud cluster.

6. The machine vision-based positioning and fertilization method of claim 1, wherein, If the reference point cloud cluster intersects with the space corresponding to the ground pixels, the intersection position of the rhizome cluster and the spatial point corresponding to the ground pixels is taken as the fertilization position. The method for obtaining the first weight and the second weight comprises the following steps:

7. The machine vision-based positioning and fertilization method of claim 1, wherein, ​ wherein, denotes a predicted growth vector, denotes a unit vector in the direction of the principal direction of the reference point cloud cluster, denotes a unit vector in the direction of the gravity direction, denotes a first weight, denotes a second weight.

8. The machine vision-based positioning and fertilization method of claim 7, wherein, ​ The distance between the initial point and the centroid of the reference point cloud cluster is recorded as a deviation distance, a first weight is calculated by taking the opposite of the sum of preset parameters as an index and taking a natural constant as a base number; A difference between 1 and the first weight is taken as a second weight.

9. The machine vision-based positioning and fertilization method of claim 1, wherein, The modified growth vector acquisition method comprises: A plant constituted by target plant pixels is recorded as a target plant, a fertilized plant in a preset neighborhood of the target plant is recorded as a reference plant, a spatial point with the same height as the centroid of the reference point cloud cluster is acquired from spatial points corresponding to all pixels of the reference plant as a homologous point of the reference plant, a unit vector with the main direction of the homologous point of the reference plant as the direction is taken as a growth vector of the reference plant at the homologous point, and the inverse of the distance from the reference plant to the target plant is taken as a weight, the growth vectors of all homologous points of the reference plant are weighted and summed, and the obtained vector is recorded as a modified growth vector.

10. The machine vision-based positioning and fertilization method of claim 1, wherein, The method for acquiring the predicted extension point of the rhizome from the initial point along the predicted extension direction comprises: A spatial point with a distance of a preset length from the initial point is acquired as the predicted extension point of the rhizome in the predicted extension direction.

Images