Automatic weeding method, device and storage medium

Abstract

The present application discloses an automatic weeding method, device and storage medium, belonging to the field of automated operation technology, the method comprising: acquiring crop images through a camera; identifying the position information of crops in the crop images through a position detection model; acquiring the target height of the camera from the ground; converting the position information into target position information in the camera coordinate system according to the target height; determining the knife-seedling distance and the seedling spacing according to the target position information; and performing weeding operations according to the determined knife-seedling distance and seedling spacing. The problem that crops may be damaged during automatic weeding in the prior art is solved, and the effect of obtaining the knife-seedling distance and the seedling spacing in real time and then accurately avoiding crops to avoid damaging crops is achieved.

Description

Technical Field

[0001] The invention relates to an automatic weeding method, device and storage medium, and belongs to the technical field of automated operations. Background Art

[0002] In traditional agricultural planting in my country, weeds directly affect agricultural production and reduce the quality of crops. At present, manual weeding is inefficient. Although large-scale chemical weeding has improved operating efficiency, it has caused environmental pollution and food safety problems. Therefore, automated intelligent mechanical weeding devices will become the research direction of green and sustainable agriculture in the future.

[0003] At present, domestic and foreign scholars have conducted a lot of research on intelligent weeding machines, among which the key technologies include: seedling identification and positioning, whole machine design and precise servo control, and weeding manipulator design for synchronous operation of weeding between rows and plants. In order to complete the weeding operation between rows, how to accurately locate crops in real time and effectively deal with weeds between rows without damaging crops is the primary technical difficulty in realizing the automation and intelligence of weeding machines. Summary of the invention

[0004] The object of the present invention is to provide an autonomous mobile chassis, a multi-span greenhouse chassis track changing method and a storage medium, so as to solve the problems existing in the prior art.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] According to a first aspect, an embodiment of the present invention provides an automatic weeding method, the method comprising:

[0007] Acquire crop images through cameras;

[0008] identifying position information of crops in the crop image by using a position detection model;

[0009] Get the target height of the camera from the ground;

[0010] Converting the position information into target position information in a camera coordinate system according to the target height;

[0011] Determine the distance between the blades and the distance between the seedlings according to the target position information;

[0012] Weeding operations are performed according to the determined distance between blades and seedlings and the distance between seedlings.

[0013] Optionally, obtaining a target height of the camera from the ground includes:

[0014] aligning the crop image with the acquired depth image;

[0015] Extracting a region of interest in the crop image;

[0016] Generating a binary image of the region of interest;

[0017] The target height is acquired according to the binarized image and the aligned depth image.

[0018] Optionally, generating a binary image of the region of interest includes:

[0019] A super green algorithm is used to separate green pixels in the region of interest, and the separated binary image is obtained.

[0020] Optionally, the acquiring the target height according to the binarized image and the aligned depth image includes:

[0021] Performing an AND operation on the binarized image and the aligned depth image;

[0022] An average value of depth values ​​in the processed binary image is calculated, and the average value is determined as the target height.

[0023] Optionally, determining the knife-seedling distance and the seedling-seedling distance according to the target position information includes:

[0024] Dividing the crops into left and right columns according to the coordinate value of the horizontal coordinate of each pixel in the target position information;

[0025] Calculate the distance between the two rows of crops according to the distance between the crops in the left and right rows;

[0026] The seedling spacing is calculated according to the spacing between two adjacent rows of crops in the left and right columns of crops.

[0027] Optionally, the calculating the distance between the cutters and seedlings according to the distance between the left and right rows of crops includes:

[0028] Get the lower boundary of the field of view;

[0029] For each column, obtaining the difference between the vertical coordinate of the crop adjacent to the lower boundary of the field of view and the lower boundary of the field of view;

[0030] The absolute value of the difference obtained is determined as the blade-to-sprout distance of each column.

[0031] Optionally, calculating the seedling spacing according to the spacing between two adjacent rows of crops in the left and right columns of crops includes:

[0032] For each column, calculate the difference between the vertical coordinates of two adjacent rows of crops;

[0033] The absolute value of the calculated difference was determined as the seedling spacing in each column.

[0034] Optionally, the image acquisition device is arranged at a preset height from the ground.

[0035] In a second aspect, an automatic weeding device is provided, the device comprising a memory and a processor, the memory storing at least one program instruction, the processor implementing the method as described in the first aspect by loading and executing the at least one program instruction.

[0036] According to a third aspect, a computer storage medium is provided, wherein at least one program instruction is stored in the computer storage medium, and the at least one program instruction is loaded and executed by a processor to implement the method according to the first aspect.

[0037] The crop image is acquired through a camera; the position information of the crop in the crop image is identified through a position detection model; the target height of the camera from the ground is acquired; the position information is converted into the target position information in the camera coordinate system according to the target height; the blade-to-seedling distance and the seedling-to-seedling distance are determined according to the target position information; and weeding is performed according to the determined blade-to-seedling distance and seedling-to-seedling distance. The problem that crops may be damaged during automatic weeding in the prior art is solved, and the effect of obtaining the blade-to-seedling distance and the seedling-to-seedling distance in real time and then accurately avoiding crops to avoid damaging them is achieved.

[0038] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention and implement it according to the contents of the specification, the following is a detailed description of the preferred embodiments of the present invention in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A method flow chart of an automatic weeding method provided by an embodiment of the present invention;

[0040] Figure 2 A possible schematic diagram of a weeding robot provided by an embodiment of the present invention operating in a field;

[0041] Figure 3 A possible schematic diagram of the position information of each crop detected by a position detection model provided in one embodiment of the present invention;

[0042] Figure 4 A schematic diagram of the relationship between a pixel coordinate system and an image coordinate system provided by an embodiment of the present invention;

[0043] Figure 5 A schematic diagram of a camera imaging principle provided by an embodiment of the present invention;

[0044] Figure 6A possible schematic diagram of the calculated knife-seedling distance and seedling-spacing distance provided for one embodiment of the present invention. DETAILED DESCRIPTION

[0045] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0047] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0048] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0049] Please refer to Figure 1 , which shows a method flow chart of an automatic weeding method provided by an embodiment of the present application, such as Figure 1 As shown, the method includes:

[0050] Step 101, acquiring crop images through a camera;

[0051] The camera described in this embodiment may be an RGBD camera. The RGBD camera may be set at a preset height from the ground, and the preset height may be set to different values ​​according to the height of the crop. For example, if the crop is a cabbage seedling, the preset height may be 70 cm.

[0052] Of course, in actual implementation, in order to ensure the image quality of the acquired crop images, the angle of the RGBD camera can be parallel to the ground.

[0053] In a possible implementation of this embodiment, the RGBD camera can be set in the weeding robot, such as Figure 2 As shown, the weeding robot consists of a mobile platform, an RGBD camera and a bracket. The RGBD camera is set 70 cm above the ground, and as the weeding robot moves, the RGBD camera can obtain crop images at its location in real time.

[0054] Optionally, since there may be weeds in the crop planting field, the acquired crop image may also include weeds, which is not limited in this embodiment.

[0055] Step 102, identifying position information of crops in the crop image by using a position detection model;

[0056] The location detection model is a pre-trained model. In actual implementation, the training steps of the location detection model include:

[0057] First, obtain the training set;

[0058] In this step, a set of crop images under different lighting conditions and different growth states can be acquired by using an RGBD camera, wherein each crop image can include crops and weeds growing around the crops.

[0059] In order to ensure the accuracy of training, the crop image set may include a large number of crop images, for example, 8,000 crop images. In addition, in actual implementation, in order to improve the generalization ability, after each crop image is acquired through the RGBD camera, the acquired crop image may be enhanced, which is not limited in this embodiment. Among them, the image enhancement method may include: random image reduction, splicing, cropping, etc.

[0060] In actual implementation, the acquired crop image set can be divided into a training set, a validation set, and a test set.

[0061] Second, an initial position detection model is trained according to the training set, and a trained position detection model is obtained.

[0062] In actual implementation, the Yolov5S detection model can be built on the Pytorch deep learning framework, and then the training set can be input into the Yolov5S detection model for training to obtain the trained location detection model.

[0063] Optionally, the trained location detection model can be verified through the validation set, and the test set can be input into the trained location detection model for testing to verify the training results and obtain the final location detection model. Figure 3 , which shows a possible schematic diagram of the position information of each crop detected by the trained position detection model.

[0064] In this step, the trained position detection model can be used to detect the position of crops in the crop image and the growth status of the crops.

[0065] Step 103, obtaining the target height of the camera from the ground;

[0066] This step includes:

[0067] First, aligning the crop image with the acquired depth image;

[0068] Optionally, the color image and the depth image acquired by the RGBD camera may be aligned, wherein the color image is the RGB three channels and the depth image is the D channel.

[0069] second, extracting a region of interest in the crop image;

[0070] In actual implementation, the central pixel ROI (region of interest) area in the crop image can be extracted, for example, the central 200*200 pixel ROI area is extracted.

[0071] Third, generating a binary image of the region of interest;

[0072] In the region of interest, the super green algorithm is used to separate green pixels, and the separated binary image is obtained. In actual implementation, the super green algorithm (2g-rb) and the Otsu threshold segmentation algorithm can be used to separate green pixels to obtain the ROI binary image.

[0073] Fourth, the target height is obtained according to the binarized image and the aligned depth image.

[0074] (1) performing an AND operation on the binary image and the aligned depth image;

[0075] (2) Calculate the average value of the depth values ​​in the processed binary image, and determine the average value as the target height.

[0076] Optionally, after performing the AND operation, outliers may be removed, and then an average value of the depth values ​​may be calculated, and the average value may be determined as the target height.

[0077] Step 104, converting the position information into target position information in a camera coordinate system according to the target height;

[0078] This step includes:

[0079] First, obtain the camera's internal parameter information;

[0080] The camera intrinsic parameter information includes the pixel size dx, dy in the camera, and the center point PPx, PPy of the pixel coordinate system.

[0081] Second, the acquired position information is converted into the target position information in the camera coordinate system according to the camera intrinsic parameter information and the target height.

[0082] In actual implementation, the acquired position information can be converted into target position information in the camera coordinate system according to the camera intrinsic parameter information, target height and camera imaging principle.

[0083] Optionally, refer to Figure 4 , which shows the relationship between the pixel coordinate system and the image coordinate system. In addition, please refer to Figure 5 , which shows a schematic diagram of a possible camera imaging principle of a camera.

[0084] Specifically, this step includes: let Xp, Yp be pixel coordinate points, Xc, Yc, Zc be camera coordinate points, Z be the acquired target height, and from the geometric relationship,

[0085]

[0086]

[0087] Assuming Xu and Yu are pixel coordinate points, the conversion relationship is as follows:

[0088] X p =(X u -pp x )dx

[0089] Y p =(Y u -pp y )dy

[0090] Substituting into the above formula, the transformation relationship between the pixel coordinate system and the camera coordinate system is:

[0091]

[0092]

[0093] Zc=z

[0094] Step 105, determining the distance between blades and seedlings and the distance between seedlings according to the target position information;

[0095] First, the crops are divided into two columns, left and right, according to the coordinate value of the horizontal coordinate of each pixel in the target position information;

[0096] After obtaining the target position information, determine whether the horizontal coordinate of the pixel coordinate is greater than the preset threshold. If it is greater, it is in the right column Right[]. If it is less than, it is in the left column Left[]. The above is only an example of using the target position information in the camera coordinate system for division. In actual implementation, the position information detected by the position detection model can also be used for judgment. At this time, if Figure 4 As shown, the two columns of coordinates are respectively sorted from large to small along the vertical axis v of the pixel coordinate system as the positive direction. The above-mentioned method for obtaining the knife-seedling distance and the seedling distance utilizes the coordinate conversion relationship in step 104 to convert the two columns of sorted coordinates from pixel coordinates to camera coordinates.

[0097] Second, the distance between the cutters and seedlings is calculated according to the distance between the crops in the left and right rows;

[0098] (1) Obtain the lower boundary of the field of view;

[0099] In actual implementation, Yu=800 is substituted into the expression in step 104 to obtain the lower boundary Yc of the field of view.

[0100] (2) For each column, obtain the difference between the vertical coordinate of the crop adjacent to the lower boundary of the field of view and the lower boundary of the field of view;

[0101] Mark the vertical coordinate of Right[0] in the camera coordinate system close to the boundary of the field of view as Yc_right[0] and the vertical coordinate of Left[0] as Yc_Left[0], take the absolute value of the difference with the lower boundary of the field of view Yc and add it to Dk2, which is the distance between the two rows of seedlings and the tool, that is, the tool-seedling distance, which are recorded as DKc_right[0] and DKc_left[0] respectively.

[0102] (3) The absolute value of the difference obtained is determined as the blade-to-sprout distance of each column.

[0103] Please refer to Figure 6 , the knife-seedling distance is the distance between the seedlings below the field of view and the cutting knife, which is:

[0104] Dk=Dk1+Dk2. Wherein, DK1 is the absolute value of the difference calculated in the above steps, and DK2 is a known value.

[0105] Third, the seedling spacing is calculated according to the spacing between two adjacent rows of crops in the left and right columns of crops.

[0106] (1) For each column, calculate the difference between the vertical coordinates of two adjacent rows of crops;

[0107] (2) The absolute value of the calculated difference is determined as the spacing between seedlings in each column.

[0108] Similarly, the difference between the camera coordinate values ​​of the sorted left and right rows Right[1] and Left[1] and the corresponding Right[0] and Left[0] is taken as the absolute value, which is the distance between the front and back seedlings in the left and right rows. Please refer to the seedling spacing. Figure 6 , the seedling spacing is DP.

[0109] Step 106, performing weeding operations according to the determined blade-seedling distance and seedling-spacing distance.

[0110] After determining the distance between the blade and the distance between the seedlings, you can start weeding.

[0111] In summary, the crop image is acquired through a camera; the position information of the crop in the crop image is identified through a position detection model; the target height of the camera from the ground is acquired; the position information is converted into the target position information in the camera coordinate system according to the target height; the blade-seedling distance and the seedling spacing are determined according to the target position information; and weeding is performed according to the determined blade-seedling distance and seedling spacing. The problem that crops may be damaged during automatic weeding in the prior art is solved, and the effect of obtaining the blade-seedling distance and the seedling spacing in real time and then accurately avoiding crops to avoid damaging crops is achieved.

[0112] The present application also provides an automatic weeding device, which includes a memory and a processor. The memory stores at least one program instruction, and the processor implements the above-mentioned method by loading and executing the at least one program instruction.

[0113] The present application also provides a computer storage medium, in which at least one program instruction is stored. The at least one program instruction is loaded and executed by a processor to implement the method described above.

[0114] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0115] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. An automatic weeding method, characterized in that: The method comprises: Acquire crop images through cameras; identifying position information of crops in the crop image by using a position detection model; Get the target height of the camera from the ground; Converting the position information into target position information in a camera coordinate system according to the target height; Determine the distance between the blades and the distance between the seedlings according to the target position information; Carry out weeding operation according to the determined distance between blades and seedlings and the distance between seedlings; Determining the knife-seedling distance and the seedling-seedling distance according to the target position information includes: According to the coordinate value of the horizontal coordinate of each pixel in the target position information, the crops are divided into two columns, left and right. After obtaining the target position information, it is determined whether the horizontal coordinate of the pixel coordinate is greater than a preset threshold. If it is greater, it is in the right column Right[], and if it is less than, it is in the left column Left[]. The two columns of coordinates are sorted from large to small along the vertical axis v of the pixel coordinate system as the positive direction; The knife-to-seedling distance is calculated according to the spacing between the left and right rows of crops, including: obtaining the lower boundary of the field of view; for each column, obtaining the difference between the vertical coordinate of the crop adjacent to the lower boundary of the field of view and the lower boundary of the field of view, marking the vertical coordinate of Right[0] in the camera coordinate system close to the field of view boundary as Yc_right[0] and the vertical coordinate of Left[0] as Yc_Left[0], taking the absolute value of the difference of the lower boundary of the field of view Yc and adding it to Dk2, that is, the distance between the seedlings and the cutter is the knife-to-seedling distance, which is recorded as DKc_right[0] and DKc_left[0] respectively; the absolute value of the difference obtained is determined as the knife-to-seedling distance of each column, and the knife-to-seedling distance is the distance between the seedlings and the cutting knife close to the bottom of the field of view, that is: Dk=Dk1+Dk2, wherein DK1 is the absolute value of the difference calculated in the above steps, and DK2 is a known value; The seedling spacing is calculated according to the spacing between two adjacent rows of crops in the left and right columns of crops, including: for each column, respectively calculating the difference between the vertical coordinates of two adjacent rows of crops; and determining the absolute value of the calculated difference as the seedling spacing of each column.

2. The method according to claim 1, characterized in that The obtaining of the target height of the camera from the ground comprises: aligning the crop image with the acquired depth image; Extracting a region of interest in the crop image; generating a binary image of the region of interest; The target height is acquired according to the binarized image and the aligned depth image.

3. The method according to claim 2, characterized in that The generating of the binary image of the region of interest comprises: A super green algorithm is used to separate green pixels in the region of interest, and the separated binary image is obtained.

4. The method according to claim 2, characterized in that: The step of obtaining the target height according to the binarized image and the aligned depth image includes: Performing an AND operation on the binarized image and the aligned depth image; An average value of depth values ​​in the processed binary image is calculated, and the average value is determined as the target height.

5. The method according to any one of claims 1 to 4, characterized in that: The image acquisition device is arranged at a preset height from the ground.

6. An automatic weeding device, characterized in that: The device includes a memory and a processor, wherein at least one program instruction is stored in the memory, and the processor implements the method according to any one of claims 1 to 5 by loading and executing the at least one program instruction.

7. A computer storage medium, characterized in that: The computer storage medium stores at least one program instruction, and the at least one program instruction is loaded and executed by a processor to implement the method according to any one of claims 1 to 5.

Images