Intelligent weeding device and weeding method based on visual navigation

Abstract

The application relates to an intelligent weeding device based on visual navigation and a weeding method, and relates to the technical field of intelligent weeding devices. In order to solve the problems in the prior art that the transverse movement of a weeding machine in a field and the increase and decrease of the number of weeding components need to be manually adjusted during use, the working process is easy to cause damage to the field ridge, the use convenience of the weeding machine is reduced, mechanical weeding is mostly inter-row weeding, the effect on inter-plant weeds is not ideal, weeds cannot be intelligently identified, the working path of the weeding machine cannot be planned for efficient weeding and the like, the application comprises a connecting frame, a translation mechanism, a turnover mechanism and a visual navigation mechanism; one side of the connecting frame is movably connected with a weeding mechanism, and the other side is connected with a supporting wheel; the weeding mechanism comprises a rack, weeding shovels and weeding fingers; the rack is movably connected to the connecting frame; the rear side of the rack is provided with a plurality of weeding shovels and a plurality of weeding fingers; and the upper end of the rack is provided with the visual navigation mechanism. The application is used for farmland weeding.

Description

Technical Field

[0001] This invention relates to the field of intelligent weeding equipment technology, specifically to an intelligent weeding device and weeding method based on visual navigation. Background Technology

[0002] Weed control is a crucial step in agricultural production. Existing weed control technologies in farmland mainly include manual weeding, mechanical weeding, and chemical weeding. Manual weeding is highly precise, but it is costly, inefficient, and time-consuming, and has been gradually replaced by manual methods with the advancement of agricultural machinery technology. Mechanical weeding eliminates weeds through physical actions such as cutting and tilling, and is an economical, safe, green, and sustainable weed control technology.

[0003] A lawnmower, also known as a lawn mower, lawn trimmer, or lawn trimmer, is a mechanical tool used for trimming lawns, vegetation, etc. It consists of a cutter head, engine, wheels, a walking mechanism, blades, a handle, and a control unit. The cutter head is mounted on the wheels, and the engine is mounted on the cutter head. The blades are mounted on the output shaft of the engine. The high-speed rotation of the blades by the engine greatly increases the speed, saving weeding workers' working time and reducing a lot of manpower. Weed mowers are needed in agricultural production.

[0004] Currently, in the use of agricultural weeding machines, the weeding components are usually connected to the frame. This means that the lateral movement of the weeding machine in the field and the increase or decrease of the number of weeding components need to be manually adjusted. During operation, the weeding machine is prone to damage to the field ridges, reducing its ease of use. At the same time, mechanical weeding is mostly done between rows, which is not ideal for weeds between plants. It also cannot intelligently identify weeds or plan the working path of the weeding machine for efficient weeding. Summary of the Invention

[0005] The purpose of this invention is to solve the problems in the existing technology of weeding machines, such as the need for manual adjustment of the lateral movement in the field and the increase or decrease of the number of weeding components, which can easily damage the field ridges during operation and reduce the ease of use of the weeding machine. At the same time, mechanical weeding is mostly between rows, which is not ideal for weeds between plants. It is also unable to intelligently identify weeds and plan the working path of the weeding machine for efficient weeding. Therefore, this invention provides an intelligent weeding device based on vision navigation.

[0006] The technical solution adopted by the present invention to solve the above problems is: an intelligent weeding device based on visual navigation, including a connecting frame, a translation mechanism, a flipping mechanism, a weeding mechanism, and a visual navigation system; the front end of the connecting frame is connected to a support wheel, and the rear end is movably connected to the weeding mechanism, which includes a frame, inter-row weeding units, and inter-plant weeding units; the frame is movably connected to the connecting frame, and multiple inter-row weeding units and multiple inter-plant weeding units are spaced apart on the rear side of the frame; the top of the frame is provided with a visual navigation mechanism; the translation mechanism is used to control the translational movement of the weeding mechanism along the connecting frame; the flipping mechanism is used to control the up-and-down flipping movement of the left and right sides of the weeding mechanism; and the visual navigation system is used to identify weeds and control the weeding operation of the weeding device.

[0007] Furthermore, the frame includes a main frame and two flip frames. The main frame is slidably connected to the connecting frame via a translation mechanism, and the left and right sides of the main frame are respectively rotatably connected to flip frames via flip mechanisms.

[0008] Furthermore, the translation mechanism includes a translation cylinder and a sliding frame. The front end of the sliding frame is slidably connected to the connecting frame, and the rear end is connected to the main frame. One end of the translation cylinder is mounted on the connecting frame, and the other end is mounted on the sliding frame.

[0009] Furthermore, the tilting mechanism includes a tilting cylinder and a connecting shaft. The main frame and the tilting frame are rotatably connected by the connecting shaft. One end of the tilting cylinder is connected to the main frame by a rotating shaft, and the other end is connected to the tilting frame by a rotating shaft.

[0010] Furthermore, the inter-row weeding unit includes an inter-row frame, a connecting plate, an inter-row hydraulic cylinder, a crossbeam, inter-row wheels, multiple inter-row weeding plates, and weeding shovels. The inter-row frame is connected to the main frame or a tilting frame. The front end of the connecting plate is hinged to the inter-row frame, and the rear end is hinged to the crossbeam. One end of the inter-row hydraulic cylinder is connected to the inter-row frame via a pivot, and the other end is connected to the crossbeam via a pivot. The front end of the crossbeam is provided with inter-row wheels and inter-row weeding plates, and the rear end is provided with weeding shovels. The inter-row weeding plates are arranged in two rows on the left and right sides of the crossbeam. The inter-row weeding plates and the weeding shovels are fixedly connected to the crossbeam via spiral brackets. The weeding shovels have a V-shaped structure, with a shovel tip at the front end and shovel wings on the left and right sides of the shovel tip.

[0011] Furthermore, the inter-plant weeding unit includes an inter-plant frame, a rotating plate, an inter-plant hydraulic cylinder, a connecting beam, inter-plant ground wheels, multiple inter-plant weeding plates, and multiple weeding fingers. The inter-plant frame is connected to the main frame or a tilting frame. The front end of the rotating plate is hinged to the inter-plant frame, and the rear end is hinged to the connecting beam. One end of the inter-plant hydraulic cylinder is connected to the inter-plant frame via a rotating shaft, and the other end is connected to the connecting beam via a rotating shaft. The front end of the connecting beam is provided with an inter-plant ground wheel, the middle part is provided with multiple inter-plant weeding plates, and the rear end is provided with multiple weeding fingers. The inter-plant weeding plates are arranged in two rows on the left and right sides of the connecting beam, and the weeding fingers are arranged in two rows on the left and right sides of the connecting beam. The inter-plant weeding plates and the weeding fingers are all fixedly connected to the connecting beam via a spiral truss.

[0012] Furthermore, the weeding boards between plants are arranged in two rows on the left and right sides of the connecting beam, and each row of weeding boards is divided into multiple soil-covering board groups. Each soil-covering board group consists of two weeding boards arranged in a crisscross pattern. The weeding fingers are arranged in two rows on the left and right sides of the connecting beam, and each row of weeding fingers consists of multiple weeding fingers. The angle α between the weeding fingers and the ground is 25°-40°.

[0013] Furthermore, the visual navigation system includes a camera, a range counter, an orientation sensor, a drive status sensor, and a controller. The output of the camera is connected to the input of the image processing module in the controller. The range counter, the orientation sensor, and the drive status sensor are all electrically connected to the controller. The controller outputs control signals to the driver, which drives a rotary motor. The rotary motor controls the rotation of the weeding finger. The camera is used to collect video information of weeds in the field. The range counter is used to calculate the range of the weeding equipment. The orientation sensor is used to monitor the speed and direction of the weeding equipment. The drive status sensor is used to collect the action parameters of the translation cylinder, the tilting cylinder, the inter-row cylinder, and the inter-plant cylinder.

[0014] Another technical solution adopted by the present invention to solve the above problems is: an intelligent weeding method, which includes identifying weeds in the actual total coverage area corresponding to the target weeding machine through a visual recognition algorithm;

[0015] The control unit controls the inter-row weeding unit and the inter-plant weeding unit to remove various weeds in the area, and generates a corresponding removal completion signal after the removal is completed;

[0016] The clearing completion signal is transmitted to the visual navigation system, so that the visual recognition system can perform secondary recognition on the sub-area to be cleared and generate a corresponding clearing completion confirmation signal;

[0017] In response to the confirmation signal that the weeding is completed, the target weeding device is controlled to move to the next sub-area to be weeded, and the process returns to the step of identifying the location of all weeds in one of the sub-areas to be weeded within the total area to be covered by the target weeding machine through a visual recognition algorithm, until the weeds in the total area to be weeded are cleared.

[0018] Furthermore, the steps for the visual recognition algorithm to identify weeds are as follows:

[0019] Multiple first videos are acquired, each of which is a video of weeds in a field and carries information about the type of weeds.

[0020] Multiple first image sets are obtained based on multiple first videos; the first image sets include multiple first images, and the first images carry weed species information of the first videos;

[0021] The optimal deep residual network structure is searched to obtain the first deep residual network structure;

[0022] The deep residual network is trained based on multiple sets of the first image and the weed species information to obtain a paddy field weed recognition model, wherein the deep residual network has the structure of the first deep residual network.

[0023] Get the second image;

[0024] The second image is input into the paddy field weed recognition model to obtain the first recognition result;

[0025] The deep residual network structure includes an input part, an intermediate part, and an output part;

[0026] The middle part consists of 4 residual blocks. Each residual block consists of multiple convolutional layers and bypass connections. Each convolutional layer is composed of BatchNorm batch normalization, ReLU activation function and Conv2D convolution connected in sequence. The output part is in the form of multiple outputs. Each output is for a type of weed, and the presence of multiple weeds can be judged at the same time.

[0027] The structure of the intermediate part is determined by finding the optimal deep residual network structure from the ENAS search space, the contents of which include:

[0028] The number of convolutional layers contained in each residual block, ranging from 1 to 5;

[0029] In each residual block, the stride of the first convolutional layer is 2, and the stride of the remaining convolutional layers is 1.

[0030] The number of channels in each residual block's convolutional layer is selected from 16, 32, 64, 128, 256, and 512 for the first layer. If the number of layers in the current residual block exceeds 2, the number of channels in the last convolutional layer is 4 times that of the first convolutional layer; the number of channels in the remaining layers is the same as that of the first layer.

[0031] The kernel size for each convolutional layer is selected from three sizes: 1x1, 3x3, and 5x5.

[0032] The size of the ENAS search space is (5*6*3)^4. The rules of the above search space are encoded into the search space of AutoKears. AutoKears automatically trains and compares deep residual network structures to select the one with the highest recognition rate.

[0033] During the search process, the loss function is calculated as follows: each output corresponds to a cross-entropy error, and the average of the cross-entropy errors of all outputs is used as the loss of the entire network.

[0034] The step of obtaining multiple first image sets based on multiple first videos includes:

[0035] A third image set is obtained for each of the first videos, wherein the third image set is an image set composed of all image frames of the first video in a time sequence.

[0036] Construct the first image set;

[0037] A first image is obtained from the third image set. The first image is then checked for blurriness and similarity to existing images in the first image set. If the first image is not blurry and is not similar to any existing image in the first image set, the first image is used as the first target image, and an image at a first preset interval of frames is obtained as the next first image. If the first image is blurry or similar to any existing image in the first image set, an image at a second preset interval of frames is obtained as the next first image.

[0038] Store the first target image in the first image set;

[0039] The step of detecting whether the first image is blurry includes: calculating the gradient matrix of the first image, calculating the variance of all elements of the gradient matrix, and if the variance is less than a predetermined variance threshold, then the first image is determined to be blurry; if the variance is greater than or equal to the predetermined variance threshold, then the first image is determined to be unblurry.

[0040] Detecting whether the first image is similar to existing images in the first image set includes:

[0041] Bilinear interpolation is used to compress the size of the first image and all images in the first image set to the first size; then, the super-green method is used to convert all the compressed images into grayscale images.

[0042] Calculate the mean value of all pixels in the grayscale image, binarize the grayscale image, record pixels greater than the mean value as 1 and pixels less than the mean value as 0, and obtain a matrix containing only 0 or 1; calculate the absolute value of the difference between the matrix corresponding to the first image and the matrix corresponding to each image in the first image set, and obtain a difference matrix containing only 0 and 1.

[0043] Calculate the sum of all elements of the difference matrix. If the sum of all elements of the difference matrix is ​​less than a preset difference threshold, then the first image is determined to be similar to the corresponding image in the first image set. If the sum of all elements of the difference matrix is ​​greater than or equal to the preset difference threshold, then the first image is determined to be dissimilar to the corresponding image in the first image set.

[0044] The present invention has the following beneficial technical effects:

[0045] This invention adjusts the lateral translation of the weeding mechanism through a translation mechanism, allowing the weeder to move freely laterally along the field terrain, thus improving the ease of use of the weeder. The flipping mechanism drives the flipping frame to flip upward, thereby causing the left and right side structures of the weeding mechanism to flip upward as a whole, realizing the adjustment of the working width of the weeder. This allows the weeder to better adapt to the field environment and improves its versatility.

[0046] The weeding shovel and weeding fingers used in this invention are bolted to the end of a spiral bracket or spiral truss. The spiral bracket or spiral truss is then connected to a crossbeam or connecting beam. The spiral bracket and spiral truss are generally S-shaped with an inverted U-shaped lower part. The structure has a shock absorption effect to prevent breakage and damage due to uncontrollable factors such as soil hardening exceeding the compressive strength of the weeding component material.

[0047] This invention employs a coordinated weeding operation with alternating row-to-row and plant-to-plant weeding units. Plant-to-plant weeding boards and weeding fingers remove weeds from the ridges, while row-to-row weeding boards remove weeds from the sides of the ridges. Weeding shovels remove weeds from the furrows, achieving efficient weed control both between rows and between plants simultaneously. The weeding shovels also trim the soil on the ridges broken by the weeding fingers. The two inter-plant weeding boards, arranged in a crisscross pattern, are highly efficient at removing weeds between plants with minimal damage to the ridges. The weeding fingers have outer and inner weeding teeth, with gaps between the teeth serving as seedling avoidance space. The alternating vertical weeding action efficiently removes weeds of varying heights.

[0048] This invention requires real-time monitoring of the field in front of the weeder during its movement. The camera in the visual navigation mechanism monitors the situation in front of the weeder in real time, so that the control unit can make real-time adjustments to the weeder while it is moving, thereby improving the operability of the weeder.

[0049] This invention uses a deep residual network-based artificial intelligence recognition model to identify field weeds by inputting a video of field weeds and information on weed species. This results in intelligent and efficient identification of field weeds. By comparing the distribution characteristics of the weeds, the invention automatically controls the weeding mechanism to perform field weeding. Attached Figure Description

[0050] Figure 1 This is a schematic diagram of the structure of the present invention;

[0051] Figure 2 yes Figure 1 The main view;

[0052] Figure 3 yes Figure 1 The left view;

[0053] Figure 4 yes Figure 1 Top view;

[0054] Figure 5 This is a schematic diagram of the flipping mechanism;

[0055] Figure 6 This is a schematic diagram of the translation mechanism;

[0056] Figure 7 This is a three-dimensional structural diagram of the inter-row weeding unit;

[0057] Figure 8 yes Figure 7 The main view;

[0058] Figure 9 yes Figure 7 Top view;

[0059] Figure 10 yes Figure 7 The left view;

[0060] Figure 11 This is a three-dimensional structural diagram of an inter-plant weeding unit;

[0061] Figure 12 yes Figure 11 The main view;

[0062] Figure 13 yes Figure 11 Top view;

[0063] Figure 14 yes Figure 11 The left view;

[0064] Figure 15 This is one of the structural diagrams of a weeding finger;

[0065] Figure 16 This is the second schematic diagram of the structure of the weeding finger;

[0066] Figure 17 This is the third schematic diagram of the structure of the weeding finger;

[0067] Figure 18 This is a schematic diagram of the interrow backfill board structure;

[0068] Figure 19 This is a schematic diagram of the structure of the interplant mound;

[0069] Figure 20 This is a schematic diagram of a weeding machine operating in the field;

[0070] Figure 21 This is a schematic diagram showing the distribution of weed control methods in the field;

[0071] Figure 22 This is the control principle diagram of the present invention;

[0072] Figure 23 This is a flowchart of a method for intelligent weed identification provided in an embodiment of the present invention;

[0073] Figure 24 This is a flowchart illustrating a method for intelligently identifying weeds and obtaining a first image set, provided in an embodiment of the present invention.

[0074] Figure 25 This is a schematic diagram of the structure of the deep residual network in an embodiment of the present invention;

[0075] In the diagram: 1. Connecting frame; 2. Translation mechanism; 21. Translation cylinder; 22. Sliding frame;

[0076] 3. Main frame; 4. Tilting frame; 5. Tilting mechanism; 51. Tilting cylinder; 52. Connecting shaft; 6. Support wheel;

[0077] 7. Inter-row weeding unit; 71. Inter-row frame; 72. Connecting plate; 73. Inter-row hydraulic cylinder; 74. Crossbeam; 75. Inter-row ground wheel; 76. Spiral support; 77. Inter-row weeding plate; 78. Weeding shovel;

[0078] 8. Inter-plant weeding unit; 81. Inter-plant frame; 82. Rotating plate; 83. Inter-plant hydraulic cylinder; 84. Connecting beam; 85. Inter-plant ground wheel; 86. Spiral truss; 87. Inter-plant weeding board;

[0079] 88. Weeding finger; 881. Rotary motor; 882. External weeding tooth; 883. Internal weeding tooth; 9. Visual navigation system. Detailed Implementation

[0080] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. The specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0081] Specific implementation method one: Combining Figures 1-20 This embodiment describes a vision-guided intelligent weeding device, comprising a connecting frame 1, a translation mechanism 2, a flipping mechanism 5, a weeding mechanism, and a vision navigation system 9. The connecting frame 1 has support wheels 10 connected to its front end and a weeding mechanism movably connected to its rear end. The weeding mechanism includes a frame, inter-row weeding units 7, and inter-plant weeding units 8. The frame is movably connected to the connecting frame 1. Multiple inter-row weeding units 7 and multiple inter-plant weeding units 8 are spaced apart on the rear side of the frame. The vision navigation mechanism 9 is located at the top of the frame. The translation mechanism 2 controls the translational movement of the weeding mechanism along the connecting frame 1. The flipping mechanism 5 controls the up-and-down flipping movement of the left and right sides of the weeding mechanism. The vision navigation system 9 identifies weeds and controls the weeding operation of the device. The frame includes a main frame 3 and two flipping frames 4. The main frame 3 is slidably connected to the connecting frame 1 via the translation mechanism 2. The left and right sides of the main frame 3 are rotatably connected to the flipping frames 4 via the flipping mechanisms 5.

[0082] Specific Implementation Method Two: Combining Figures 1-20 This embodiment describes a translation mechanism 2, which includes a translation cylinder 21 and a sliding frame 22. The front end of the sliding frame 22 is slidably connected to the connecting frame 1, and the rear end is connected to the main frame 3. One end of the translation cylinder 21 is mounted on the connecting frame 1, and the other end is mounted on the sliding frame 22.

[0083] In this embodiment, the translation cylinder 21 is arranged in a horizontal direction. The end of the cylinder rod of the translation cylinder 21 is connected to the connecting frame 1, and the cylinder barrel of the translation cylinder 21 is installed on the sliding frame 22. The extension and retraction of the cylinder rod pushes the sliding frame 22 to move horizontally left and right along the connecting frame 1.

[0084] The translation cylinder 21 used in this embodiment is a general standard part, and its structure and principle can be learned by those skilled in the art through technical manuals; the sliding frame 22 and the connecting frame 1 preferably adopt a structure such as a slide rail, dovetail groove and linear guide rail to achieve relative sliding, which can complete the translation movement.

[0085] Other components and connections are the same as in Implementation 1.

[0086] Specific implementation method three: Combining Figures 1-20 This embodiment describes a tilting mechanism 5, which includes a tilting cylinder 51 and a connecting shaft 52. The main frame 3 and the tilting frame 4 are rotatably connected by the connecting shaft 52. One end of the tilting cylinder 51 is connected to the main frame 3 via a rotating shaft, and the other end is connected to the tilting frame 4 via a rotating shaft. Two tilting cylinders 51 are used in total, each controlling one tilting frame 4.

[0087] In this embodiment, the cylinder rod of the tilting cylinder 51 is connected to the tilting frame 4 via a rotating shaft, and the cylinder barrel of the tilting cylinder 51 is connected to the main frame 3 via the rotating shaft. The extension and retraction of the cylinder rod pulls the tilting frame 4 to rotate up and down around the connecting shaft 52, controlling the raising and lowering of the inter-row weeding unit 7 and inter-plant weeding unit 8 installed on the tilting frame 4. The tilting cylinder 51 used in this embodiment is a general standard part, and its structure and principle can be learned by those skilled in the art through technical manuals.

[0088] Other components and connections are the same as in Implementation 1.

[0089] Specific implementation method four: Combination Figures 1-20 In this embodiment, the inter-row weeding unit 7 includes an inter-row frame 71, a connecting plate 72, an inter-row hydraulic cylinder 73, a crossbeam 74, an inter-row ground wheel 75, multiple inter-row weeding plates 77, and a weeding shovel 78.

[0090] The inter-row frame 71 is connected to the main frame 3 or the tilting frame 4. The front end of the connecting plate 72 is hinged to the inter-row frame 71, and the rear end is hinged to the crossbeam 74. One end of the inter-row cylinder 73 is connected to the inter-row frame 71 via a rotating shaft, and the other end is connected to the crossbeam 74 via a rotating shaft. The front end of the crossbeam 74 is provided with an inter-row ground wheel 75 and an inter-row weeding plate 77, and the rear end is provided with a weeding shovel 78. The inter-row weeding plates 77 are divided into two rows and are respectively arranged on the left and right sides of the crossbeam 74. The inter-row weeding plates 77 and the weeding shovel 78 are both fixedly connected to the crossbeam 74 via a spiral bracket 74. The weeding shovel 78 has a V-shaped structure, and the front end of the weeding shovel 78 is a shovel tip. The left and right sides of the shovel tip are provided with shovel wings.

[0091] In this embodiment, the weeding board 77 and weeding shovel 78 are both bolted to the end of the spiral bracket 74, which is then connected to the crossbeam 74. The spiral bracket 74 is generally S-shaped with an inverted U-shaped lower part. Its structure has a shock-absorbing effect, preventing breakage and damage due to uncontrollable factors such as soil hardening that exceed the compressive strength of the material of the weeding board 77 and weeding shovel 78.

[0092] In this embodiment, two connecting plates 72 are used, one upper and one lower. The front end of the connecting plate 72 is hinged to the inter-row frame 71, and the rear end is hinged to the crossbeam 74. The inter-row cylinder 73 is disposed between the two connecting plates 72. The end of the cylinder barrel of the inter-row cylinder 73 is hinged to the inter-row frame 71, and the end of the cylinder rod of the inter-row cylinder 73 is connected to the crossbeam 74 through a rotating shaft. The crossbeam 74 has a waist-shaped groove along the vertical direction, and the rotating shaft can slide in the waist-shaped groove. In this embodiment, the inter-row weeding plates 77 are divided into two rows and disposed on the left and right sides of the crossbeam 74 respectively. Multiple inter-row weeding plates 77 can be disposed in each row. In this embodiment, one inter-row weeding plate 77 is preferably disposed on each side.

[0093] Other components and connections are the same as in Implementation 1.

[0094] Specific Implementation Method Five: Combining Figures 1-20 This embodiment describes the inter-plant weeding unit 8, which includes an inter-plant frame 81, a rotating plate 82, an inter-plant hydraulic cylinder 83, a connecting beam 84, an inter-plant ground wheel 85, multiple inter-plant weeding plates 87, and multiple weeding fingers 88.

[0095] The inter-plant frame 81 is connected to the main frame 3 or the tilting frame 4. The front end of the rotating plate 82 is hinged to the inter-plant frame 81, and the rear end is hinged to the connecting beam 84. One end of the inter-plant cylinder 83 is connected to the inter-plant frame 81 through a rotating shaft, and the other end is connected to the connecting beam 84 through a rotating shaft. The front end of the connecting beam 84 is provided with an inter-plant ground wheel 85, the middle part is provided with multiple inter-plant weeding plates 87, and the rear end is provided with multiple weeding fingers 88. The inter-plant weeding plates 87 are arranged in two rows on the left and right sides of the connecting beam 84, and the weeding fingers 88 are arranged in two rows on the left and right sides of the connecting beam 84. The inter-plant weeding plates 87 and the weeding fingers 88 are both fixedly connected to the connecting beam 84 through a spiral truss 86.

[0096] In this embodiment, the weeding board 87 and weeding finger 88 are bolted to the end of the spiral truss 86, which is then connected to the connecting beam 84. The spiral truss 86 is generally S-shaped with an inverted U-shaped lower part. Its structure has a shock-absorbing effect, preventing the weeding board 87 and weeding finger 88 from breaking and being damaged due to uncontrollable factors such as soil hardening exceeding the compressive strength of the material.

[0097] In this embodiment, there are two rotating plates 82, one above the other. The front end of the rotating plate 82 is hinged to the inter-plant frame 81, and the rear end is hinged to the connecting beam 84. The inter-plant cylinder 83 is set between the two rotating plates 82. The end of the cylinder of the inter-plant cylinder 83 is hinged to the inter-plant frame 81. The end of the cylinder rod of the inter-plant cylinder 83 is connected to the connecting beam 84 through a rotating shaft. The connecting beam 84 has a waist-shaped groove along the vertical direction, and the rotating shaft can slide in the waist-shaped groove.

[0098] In this embodiment, the weeding boards 77 are arranged in two rows on the left and right sides of the crossbeam 74 respectively. Multiple weeding boards 77 can be installed in each row. In this embodiment, it is preferable to use one weeding board 77 on each side.

[0099] Other components and connections are the same as in Implementation 1.

[0100] Specific Implementation Method Six: Combination Figures 1-22 This embodiment describes a method where the inter-plant weeding boards 87 are arranged in two rows on the left and right sides of the connecting beam 84. Each row of inter-plant weeding boards 87 is divided into multiple soil-covering board groups, and each soil-covering board group consists of two inter-plant weeding boards 87 arranged in a crisscross pattern.

[0101] The weeding fingers 88 are arranged in two rows on the left and right sides of the connecting beam 84 respectively. Each row of weeding fingers 88 consists of multiple weeding fingers 88. The angle α between the weeding fingers 88 and the ground is 25°-40°. The weeding fingers 88 are driven by a rotary motor 881. The main body of the weeding fingers 88 is a disc structure. Multiple outer weeding teeth 882 are evenly arranged on the outer circumference of the disc. Multiple inner weeding teeth 883 are evenly arranged on the circumference of the middle part of the disc. The angle β between the inner weeding teeth 883 and the disc is 40°-60°.

[0102] In this embodiment, the weeding boards 87 are arranged in two rows on the left and right sides of the connecting beam 84. Each row of weeding boards 87 is divided into multiple soil-covering board groups. Each soil-covering board group consists of two weeding boards 87 arranged in a crisscross pattern. The number of weeding boards 87 and weeding fingers 88 can be freely adjusted according to the spacing between seedlings. The number of outer weeding teeth 882 and inner weeding teeth 883 is also adjustable and can be freely adjusted according to the spacing between seedlings. The gaps between the outer weeding teeth 882 serve as seedling avoidance space, and the distribution area of ​​the outer weeding teeth 882 and inner weeding teeth 883 serves as weeding space.

[0103] The preferred technical solution of this embodiment adopts seven inter-row weeding units 7 and six inter-plant weeding units 8. Three inter-row weeding units 7 and two inter-plant weeding units 8 are installed on the main frame 3, and the inter-row weeding units 7 and inter-plant weeding units 8 are arranged alternately. Two inter-row weeding units 7 and two inter-plant weeding units 8 are installed on the left and right side flipping frames 4, respectively, and the inter-row weeding units 7 and inter-plant weeding units 8 are arranged alternately. In this layout, the weeding operation of one row of crops is completed by two inter-row weeding units 7 and one inter-plant weeding unit 8.

[0104] This embodiment uses weeding boards 87 and weeding fingers 88 to remove weeds from the ridges, weeding boards 77 between rows to remove weeds from the two sides of the ridges, and weeding shovels 78 to remove weeds from the furrows. The weeding shovels 78 simultaneously and efficiently remove weeds from both between rows and between plants. The weeding shovels 78 can also trim the soil on the ridges broken by the weeding fingers 88. The two inter-row weeding boards 87, arranged in a crisscross pattern, are highly efficient at removing weeds between plants with minimal damage to the ridges. The weeding fingers 88 are equipped with outer weeding teeth 882 and inner weeding teeth 883, with the gaps between the teeth serving as a space to avoid seedlings. The alternating vertical weeding action effectively removes weeds of different heights.

[0105] In the preferred embodiment, the spacing of the weeding fingers 88 along the ridge direction is set according to the different sowing spacing of the crops. The spacing of the weeding fingers 88 is set at the corresponding distance for different sowing spacings. This arrangement creates a space between adjacent weeding fingers 88 to avoid damage to crop seedlings while weeding.

[0106] Other components and connections are the same as in Implementation 1.

[0107] Specific implementation method seven: Combination Figures 22-25 This embodiment describes a visual navigation system 9 comprising multiple cameras, a range counter, an orientation sensor, a drive status sensor, and a controller. The output of each camera is connected to the input of an image processing module in the controller. The range counter, orientation sensor, and drive status sensor are all electrically connected to the controller. The controller outputs control signals to a driver, which drives a rotary motor 881. The rotary motor 881 controls the rotation of the weeding finger 88. The cameras are used to collect video information of weeds in the field. The range counter is used to calculate the range of the weeding equipment. The orientation sensor is used to monitor the speed and direction of the weeding equipment. The drive status sensor is used to collect the action parameters of the translation cylinder 21, the tilting cylinder 51, the row-to-row cylinder 73, and the plant-to-plant cylinder 83.

[0108] Other components and connections are the same as in Implementation 1.

[0109] Specific implementation method eight: Combination Figures 22-25 This embodiment describes the present invention, highlighting that the algorithm used in the examples is the preferred method for intelligent weeding. Other algorithms can also achieve intelligent weeding functionality. The preferred intelligent weeding method in this embodiment includes identifying weeds within the actual total coverage area corresponding to the target weeding machine using a visual recognition algorithm.

[0110] The inter-row weeding unit 7 and inter-plant weeding unit 8 are controlled to remove various weeds in the area, and a corresponding removal completion signal is generated after the removal is completed;

[0111] The clearing completion signal is transmitted to the visual navigation system 9, so that the visual recognition system can perform secondary recognition on the sub-area to be cleared and generate a corresponding clearing completion confirmation signal;

[0112] In response to the confirmation signal that the weeding is completed, the target weeding device is controlled to move to the next sub-area to be weeded, and the process returns to the step of identifying the location of all weeds in one of the sub-areas to be weeded within the total area to be covered by the target weeding machine through a visual recognition algorithm, until the weeds in the total area to be weeded are cleared.

[0113] Intelligent weeding methods based on vision recognition and motion control can be implemented as computer programs tangibly contained in computer-readable storage media. Part or all of the computer program can be loaded into and / or installed on an electronic device via ROM and / or a communication unit. When the computer program is loaded into RAM and executed by a processor, one or more steps of the intelligent weeding methods based on vision recognition and motion control described above can be performed.

[0114] Alternatively, in other embodiments, the processor can be configured in any other suitable manner to perform a smart weeding method based on vision recognition and motion control.

[0115] This includes: using visual recognition algorithms to identify the coordinates of all weeds within a sub-area to be weeded within the total actual coverage area corresponding to the target weeder;

[0116] The sub-area to be weeded is the area covered by the target weeding machine in a single operation; the control robot removes all weeds corresponding to their coordinate positions and generates a corresponding removal completion signal after removal is completed;

[0117] The clearing completion signal is transmitted to the visual recognition system, so that the visual recognition system can perform secondary recognition of the sub-area to be cleared and generate a corresponding clearing completion confirmation signal;

[0118] In response to the confirmation signal that the weeding is completed, the target weeder is controlled to move to the next sub-area to be weeded, and the process returns to the step of identifying the coordinates of all weeds in one of the sub-areas to be weeded within the total area to be covered by the target weeder through a visual recognition algorithm, until the weeds in the total area to be weeded are cleared.

[0119] Specific Implementation Method Nine: Combining Figures 21-25 This embodiment describes the steps for the visual recognition algorithm to identify weeds as follows:

[0120] Multiple first videos are acquired, each of which is a video of weeds in a field and carries information about the type of weeds.

[0121] Multiple first image sets are obtained based on multiple first videos; the first image sets include multiple first images, and the first images carry weed species information of the first videos;

[0122] The optimal deep residual network structure is searched to obtain the first deep residual network structure;

[0123] The deep residual network is trained based on multiple sets of the first image and the weed species information to obtain a paddy field weed recognition model, wherein the deep residual network has the structure of the first deep residual network.

[0124] Get the second image;

[0125] The second image is input into the paddy field weed recognition model to obtain the first recognition result;

[0126] The deep residual network structure includes an input part, an intermediate part, and an output part;

[0127] The middle part consists of 4 residual blocks. Each residual block consists of multiple convolutional layers and bypass connections. Each convolutional layer is composed of BatchNorm batch normalization, ReLU activation function and Conv2D convolution connected in sequence. The output part is in the form of multiple outputs. Each output is for a type of weed, and the presence of multiple weeds can be judged at the same time.

[0128] The structure of the intermediate part is determined by finding the optimal deep residual network structure from the ENAS search space, the contents of which include:

[0129] The number of convolutional layers contained in each residual block, ranging from 1 to 5;

[0130] In each residual block, the stride of the first convolutional layer is 2, and the stride of the remaining convolutional layers is 1.

[0131] The number of channels in each residual block's convolutional layer is selected from 16, 32, 64, 128, 256, and 512 for the first layer. If the number of layers in the current residual block exceeds 2, the number of channels in the last convolutional layer is 4 times that of the first convolutional layer; the number of channels in the remaining layers is the same as that of the first layer.

[0132] The kernel size for each convolutional layer is selected from three sizes: 1x1, 3x3, and 5x5.

[0133] The size of the ENAS search space is (5*6*3)^4. The rules of the above search space are encoded into the search space of AutoKears. AutoKears automatically trains and compares deep residual network structures to select the one with the highest recognition rate.

[0134] During the search process, the loss function is calculated as follows: each output corresponds to a cross-entropy error, and the average of the cross-entropy errors of all outputs is used as the loss of the entire network.

[0135] The step of obtaining multiple first image sets based on multiple first videos includes:

[0136] A third image set is obtained for each of the first videos, wherein the third image set is an image set composed of all image frames of the first video in a time sequence.

[0137] Construct the first image set;

[0138] A first image is obtained from the third image set. The first image is then checked for blurriness and similarity to existing images in the first image set. If the first image is not blurry and is not similar to any existing image in the first image set, the first image is used as the first target image, and an image at a first preset interval of frames is obtained as the next first image. If the first image is blurry or similar to any existing image in the first image set, an image at a second preset interval of frames is obtained as the next first image.

[0139] Store the first target image in the first image set;

[0140] The step of detecting whether the first image is blurry includes: calculating the gradient matrix of the first image, calculating the variance of all elements of the gradient matrix, and if the variance is less than a predetermined variance threshold, then the first image is determined to be blurry; if the variance is greater than or equal to the predetermined variance threshold, then the first image is determined to be unblurry.

[0141] Detecting whether the first image is similar to existing images in the first image set includes:

[0142] Bilinear interpolation is used to compress the size of the first image and all images in the first image set to the first size; then, the super-green method is used to convert all the compressed images into grayscale images.

[0143] Calculate the mean value of all pixels in the grayscale image, binarize the grayscale image, record pixels greater than the mean value as 1 and pixels less than the mean value as 0, and obtain a matrix containing only 0 or 1; calculate the absolute value of the difference between the matrix corresponding to the first image and the matrix corresponding to each image in the first image set, and obtain a difference matrix containing only 0 and 1.

[0144] Calculate the sum of all elements of the difference matrix. If the sum of all elements of the difference matrix is ​​less than a preset difference threshold, then the first image is determined to be similar to the corresponding image in the first image set. If the sum of all elements of the difference matrix is ​​greater than or equal to the preset difference threshold, then the first image is determined to be dissimilar to the corresponding image in the first image set.

[0145] Working principle of the invention:

[0146] This invention adjusts the lateral translation of the weeding mechanism through the extension and translation mechanism 2, allowing the weeder to move freely laterally along the field terrain, effectively improving the ease of use of the weeder; through the flipping mechanism 5, the flipping frame 4 is flipped upward, thereby causing the left and right side structures of the weeding mechanism to flip upward as a whole, realizing the adjustment of the working width of the weeder, thus enabling the weeder to better adapt to the field environment and improving the versatility of the weeder.

[0147] During the movement of the weeder, real-time monitoring of the field in front of the weeder is required. The camera in the visual navigation mechanism 9 collects video of the weeds in front of the weeder, identifies the types of weeds in the field, and uses an artificial intelligence recognition model based on deep residual networks to identify the input images and obtain the recognition results. This intelligently and efficiently achieves the identification of weeds in the paddy field, so that the control unit can make real-time adjustments to the weeder while it is moving. The hydraulic system controls the row cylinders and plant cylinders to precisely control the raising and lowering of each row weeding unit and plant weeding unit, improving the operability of the weeder.

[0148] The embodiments described above are merely preferred and exemplary and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A vision navigation based intelligent weeding device, characterized in that: The utility model provides a kind of weeding device, including connecting frame (1), translation mechanism (2), turnover mechanism (5), weeding mechanism and visual navigation system (9);The front end of the connecting frame (1) is connected with support wheel (10), rear end movably connected with weeding mechanism, and the weeding mechanism includes rack, inter-row weeding unit (7) and inter-plant weeding unit (8); The rack is movably connected on connecting frame (1), and a plurality of inter-row weeding units (7) and a plurality of inter-plant weeding units (8) are arranged at the rear side of the rack in intervals, and a visual navigation system (9) is arranged on the top of the rack, the translation mechanism (2) is used to control the translation movement of the weeding mechanism along the connecting frame (1), the turnover mechanism (5) is used to control the up-down turnover movement of the left and right sides of the weeding mechanism, and the visual navigation system (9) is used to identify weeds and control the weeding operation of the weeding device; The rack includes a main rack (3) and two turnover racks (4), the main rack (3) is slidably connected on the connecting frame (1) by the translation mechanism (2), and the left and right sides of the main rack (3) are respectively rotatably connected with the turnover racks (4) by the turnover mechanisms (5); The inter-row weeding unit (7) includes an inter-row rack (71), a connecting plate (72), an inter-row oil cylinder (73), a cross beam (74), an inter-row ground wheel (75), a plurality of inter-row weeding plates (77) and a weeding shovel (78); The inter-row rack (71) is connected with the main rack (3) or the turnover rack (4), the front end of the connecting plate (72) is hingedly connected with the inter-row rack (71), and the rear end is hingedly connected with the cross beam (74), one end of the inter-row oil cylinder (73) is connected with the inter-row rack (71) through a rotating shaft, the other end is connected with the cross beam (74) through a rotating shaft, the front end of the cross beam (74) is provided with the inter-row ground wheel (75) and the inter-row weeding plate (77), and the rear end is provided with the weeding shovel (78), the inter-row weeding plates (77) are divided into two rows and arranged on the left and right sides of the cross beam (74), respectively, the inter-row weeding plates (77) and the weeding shovel (78) are fixedly connected with the cross beam (74) through a spiral support (76), the weeding shovel (78) is in V-shaped structure, the front end of the weeding shovel (78) is a shovel tip, and the left and right sides of the shovel tip are provided with shovel wings; The inter-plant weeding unit (8) includes an inter-plant rack (81), a rotating plate (82), an inter-plant oil cylinder (83), a connecting beam (84), an inter-plant ground wheel (85), a plurality of inter-plant weeding plates (87) and a plurality of weeding fingers (88); The inter-plant frame (81) is connected with the main frame (3) or the turnover frame (4), the front end of the rotating plate (82) is hinged with the inter-plant frame (81), the rear end is hinged with the connecting beam (84), one end of the inter-plant oil cylinder (83) is connected with the inter-plant frame (81) through a rotating shaft, the other end is connected with the connecting beam (84) through a rotating shaft, the front end of the connecting beam (84) is provided with an inter-plant ground wheel (85), the middle part is provided with a plurality of inter-plant weeding plates (87), and the rear end is provided with a plurality of weeding fingers (88), the inter-plant weeding plates (87) are divided into two rows and arranged on the left and right sides of the connecting beam (84), respectively, the weeding fingers (88) are divided into two rows and arranged on the left and right sides of the connecting beam (84), respectively, and the inter-plant weeding plates (87) and the weeding fingers (88) are fixedly connected with the connecting beam (84) through the spiral truss (86); The inter-plant weeding plates (87) are divided into two rows and arranged on the left and right sides of the connecting beam (84), respectively, each row of inter-plant weeding plates (87) is divided into a plurality of soil covering plate groups, and each soil covering plate group is composed of two inter-plant weeding plates (87) arranged in a left-right crossing mode; the weeding fingers (88) are divided into two rows and arranged on the left and right sides of the connecting beam (84), respectively, each row of weeding fingers (88) is composed of a plurality of weeding fingers (88), and the included angle α between the weeding fingers (88) and the ground is 25°-40°; The weeding fingers (88) are in a disc structure, a plurality of outer weeding teeth (882) are uniformly arranged on the circumferential outer side of the disc, a plurality of inner weeding teeth (883) are uniformly arranged on the middle part of the disc in a circumferential mode, and the included angle β between the inner weeding teeth (883) and the disc is 40°-60°.

2. The vision navigation based intelligent weeding device as claimed in claim 1, wherein: The translation mechanism (2) comprises a translation oil cylinder (21) and a sliding frame (22), the front end of the sliding frame (22) is slidingly connected with the connecting frame (1), and the rear end is connected with the main frame (3); one end of the translation oil cylinder (21) is arranged on the connecting frame (1), and the other end is arranged on the sliding frame (22).

3. The visual navigation based intelligent weeding device as claimed in claim 1, wherein: The turnover mechanism (5) comprises a turnover oil cylinder (51) and a connecting shaft (52), the main frame (3) and the turnover frame (4) are rotationally connected through the connecting shaft (52), one end of the turnover oil cylinder (51) is connected with the main frame (3) through a rotating shaft, and the other end is connected with the turnover frame (4) through a rotating shaft.

4. The visual navigation based weeding intelligent device according to claim 1, wherein: The visual navigation system (9) comprises a camera, a distance counter, an orientation sensor, a driving state sensor and a controller, an output end of the camera is connected with an input end of an image processing module in the controller, the distance counter, the orientation sensor and the driving state sensor are all electrically connected with the controller, the controller outputs a control signal to a driver, the driver drives a rotary motor (881), the rotary motor (881) controls rotation of a weeding finger (88), the camera is used for collecting field weed video information, the distance counter is used for calculating a weeding device distance, the orientation sensor is used for monitoring a weeding device speed and direction, and the driving state sensor is used for collecting motion parameters of a translation oil cylinder (21), a turnover oil cylinder (51), an inter-row oil cylinder (73) and an inter-plant oil cylinder (83).

Images