Weeding machine and weeding alignment method

Abstract

The invention provides a weeding machine and a weeding alignment method, and belongs to the field of agricultural machinery. The problem that alignment is inaccurate in the traditional weeding alignment process is solved. A weeding machine comprises a weeding machine main body used for driving the weeding machine to walk; the multi-mode vision assembly is arranged on the front side of the weeding machine body and used for collecting picture information and depth information of seedlings and weeds; the weeding translation mechanism is connected with the weeding machine body, and a weeding assembly is arranged on the near-ground side; the fixed end of the translation driving assembly is connected with the weeding machine main body, and the movable end is connected with the weeding translation mechanism for driving the weeding translation mechanism to act; and the controller is used for responding to a signal of the multi-mode vision assembly to control the translation driving assembly to drive the weeding translation mechanism and the weeding assembly to act. The weeding machine is mainly used for weeding.

Description

Technical Field

[0001] The invention belongs to the field of agricultural machinery, and in particular relates to a weeding machine and a weeding alignment method. Background Art

[0002] In the agricultural field, weeding is a crucial link that directly affects the growth and yield of crops. Traditional weeding methods, such as manual weeding, chemical weeding and mechanical weeding, all have certain limitations. Manual weeding is inefficient and labor-intensive; although chemical weeding is efficient, it may pollute the environment, and long-term use may cause weed resistance; and mechanical weeding may damage crops due to its low precision.

[0003] In order to solve these problems, the research and development of intelligent weeding robots has become an important direction for the development of agricultural science and technology. The key technologies of intelligent weeding robots include intelligent sensing technology, robot platform and mechanical weeding device. Intelligent sensing technology mainly distinguishes crops and weeds through crop row recognition technology and weed recognition technology. The design of the robot platform needs to consider the weeding efficiency in different crops and working environments. Common robot platforms include wheeled, crawler, foot and composite. As the end effector of the robot, the design of the mechanical weeding device directly affects the weeding effect.

[0004] With the advancement of technology, the future development trend of intelligent weeding robots will focus on four aspects: intelligent perception, precision weeding, efficient operation and intelligent management. Intelligent perception technology will provide more comprehensive farmland environment information through multi-sensor information fusion. Precision weeding will combine robot technology and automatic control technology to achieve high-precision weeding operations. Efficient operation will improve the quality and efficiency of weeding operations through artificial intelligence technology. Intelligent management will use technologies such as the Internet of Things, big data and cloud computing to achieve remote monitoring and intelligent decision-making of robot operations.

[0005] In addition, some advanced agricultural robotics companies, such as FarmWise, are developing AI-driven weeding robots that use image recognition technology to identify and remove weeds in the field, thereby reducing the use of chemical herbicides.

[0006] In image recognition technology, when using image information for comparison, there may be cases of misidentification, because when there are many weeds and sunlight is shining, the shadow of the seedlings will have a certain impact on the identification of the seedling strip, thus causing certain difficulties in the weeding alignment process. Summary of the invention

[0007] In view of this, the present invention aims to provide a weeder and a weeding alignment method to solve the problem of inaccurate alignment in the traditional weeding alignment process.

[0008] To achieve the above object, the present invention adopts the following technical solution. According to one aspect of the present invention, a weeder is provided, comprising:

[0009] The main body of the weeder is used to drive the weeder to move;

[0010] A multimodal vision component is provided on the front side of the main body of the weeder and is used to collect image information and depth information of seedlings and weeds;

[0011] A weeding translation mechanism is connected to the main body of the weeder, and a weeding assembly is arranged on the ground-close side;

[0012] A translation drive assembly, the fixed end of which is connected to the main body of the weeder, and the movable end of which is connected to the weeding translation mechanism to drive the weeding translation mechanism to move;

[0013] The controller is used to respond to the signal of the multimodal vision component to control the translation drive component to drive the weeding translation mechanism and the weeding component to move.

[0014] Furthermore, the multimodal vision components are provided in two and are distributed on both sides of the lawn mower body.

[0015] Furthermore, the multimodal vision component includes an RGB camera and a depth of field camera.

[0016] Furthermore, the multimodal vision component is connected to the lawn mower body via a camera bracket, wherein the camera bracket is a telescopic bracket.

[0017] Furthermore, the translation drive assembly is a hydraulic push rod.

[0018] Furthermore, the weeding translation mechanism is provided with a displacement sensor for collecting its position, and the displacement sensor is connected to the controller.

[0019] Furthermore, the lawn mower body is provided with a ground wheel, the ground wheel is provided with a Hall sensor, and the Hall sensor is connected to the controller.

[0020] Furthermore, a floor proximity switch is provided on the weeding component. The floor proximity switch has no signal when the weeding component is out of the weeding state and has a signal when the weeding component is in the weeding state. The floor proximity switch is connected to the controller.

[0021] According to another aspect of the present invention, a weeding alignment method is provided, using the above-mentioned weeder, comprising the following steps:

[0022] S1, the multimodal vision component starts working and transmits the video stream to the controller;

[0023] S2, the controller intercepts the current video stream frame, identifies the seedlings and weeds in the intercepted frame image respectively and compares them with the model library. If there is a high light ratio environment, the depth frame data is obtained to identify the seedlings and weeds;

[0024] S3, calibrating the positions of all seedlings in the frame image, and converting them into seedling belt information through Hough transformation according to the direction of the weeder, and calculating the angle and curvature of the seedling belt row;

[0025] S4, the controller calculates the center position of the seedling belt, and converts the camera matrix into a quaternion matrix through the rotation matrix to obtain the actual deviation distance information between the weeding component and the seedling belt;

[0026] S5, the controller calculates the driving speed and performs spatial model conversion between the modal vision component and the weeding component, and the controller calculates the moving time of the weeding component;

[0027] S6, when the weeding time point is reached, the controller controls the weeding translation mechanism to move so that the weeding component reaches the target position, meeting the weeding conditions.

[0028] Furthermore, the model library in step S2 is obtained by labeling and training the seedling strips and weeds in the pictures by the multimodal vision component in real time when the weeder moves forward at a low speed.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] 1. The present invention adopts the following technical solutions: a weeder body, used to drive the weeder to move; a multimodal vision component, arranged on the front side of the weeder body, used to collect image information and depth information of seedlings and weeds; a weeding translation mechanism, connected to the weeder body, with a weeding component arranged on the ground side; a translation drive component, a fixed end of which is connected to the weeder body, and a movable end of which is connected to the weeding translation mechanism, used to drive the weeding translation mechanism to move; a controller, used to respond to the signal of the multimodal vision component to control the translation drive component to drive the weeding translation mechanism and the weeding component to move. It can collect image information and depth information through multimodal vision components, and compare them with the database in the controller. After comprehensive detection of multiple information such as the outline, texture, and color of the seedlings, it can accurately identify the location of the seedlings in the current scene, and then drive the weeding translation mechanism to move through the translation drive component, thereby ultimately changing the position of the weeding component, so that the weeding component can be adjusted accordingly according to the position of the seedling belt, thereby improving the accuracy of weeding and preventing the seedlings from being accidentally injured. In scenes with a lot of weeds, since the weeds have formed row data, the template matching method of the prior art cannot find regular row data or there are multiple regular row data, resulting in deviations in seedling belt matching and recognition or failure to align rows, thereby causing the controlled tool position to be not in the middle of the two rows of seedling belts, resulting in seedling injury. The above solution can well solve the technical problem of inaccurate alignment;

[0031] 2. This weeder is equipped with a depth-of-field camera to avoid the traditional treatment method, which will cause the seedlings and shadows to match as a whole due to the lateral shadows produced by the seedlings when exposed to sunlight, resulting in the widening of the seedling belt range, thus causing the deviation of the weeding position. The depth-of-field camera can add seedling height information on the basis of identifying the outline, texture, and color of the seedlings. Because the shadow is projected onto the ground and there is no height difference information with the ground, the recognition defects of the original technology can be solved more accurately, and the accuracy of seedling recognition in complex scenes can be improved;

[0032] 3. Under high light ratio conditions, the existing technology using a single RGB camera cannot capture the status of the seedlings. To address this situation, a depth camera is used to directly identify the outline and position of the seedlings and perform corresponding processing, which improves the applicability of the weeder;

[0033] 4. This lawn mower is equipped with a floor proximity switch, which can control whether the multimodal vision component performs recognition according to the presence or absence of a signal, thereby preventing the multimodal vision component from always being in a working state and improving the operating economy of the lawn mower. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The accompanying drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the accompanying drawings:

[0035] Figure 1 A front view of a weeder according to the present invention;

[0036] Figure 2 A top view of a weeder according to the present invention;

[0037] Figure 3 A side view of a weeder according to the present invention;

[0038] Figure 4 It is an axonometric view of a weeder according to the present invention;

[0039] Figure 5 It is a structural schematic diagram of the weeding translation mechanism of the present invention;

[0040] Figure 6 It is a structural schematic diagram of the hydraulic push rod of the present invention;

[0041] Figure 7 The figure is a relative position arrangement diagram of the ground wheel and the Hall sensor according to the present invention;

[0042] Figure 8 is a schematic diagram of the structure of the multimodal visual component of the present invention;

[0043] Fig. 9 is a schematic diagram of the structure of the controller according to the present invention;

[0044] Fig.10 This is a schematic diagram of the structure of the HMI touch screen of the present invention;

[0045] Figure 11-13 The figure is a schematic diagram of the control principle of the weed control method of the present invention.

[0046] Multimodal vision component 1; controller 2; weeding translation mechanism 3; weeding component 4; ground wheel 5; displacement sensor 6; Hall sensor 7; floor proximity switch 8; HMI touch screen 9; camera bracket 10; hydraulic push rod 11; weeding machine body 12. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely explain the technical solutions in the embodiments of the present invention. It should be noted that the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict, and the embodiments described are only part of the embodiments of the present invention, not all of the embodiments.

[0048] It should be noted that the descriptions of the present invention regarding directions such as "left", "right", "left side", "right side", "upper", "lower", "top", "bottom", etc. are all defined based on the relationship between the orientations or positions shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the structure described must be constructed and operated in a specific orientation, and therefore, cannot be understood as a limitation on the present invention. In the description of the present invention, the meaning of "multiple" is more than two, unless otherwise clearly and specifically defined.

[0049] In the description of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a direct connection, or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0050] The present embodiment is described with reference to the accompanying drawings. According to one aspect of the present invention, there is provided a weeder, comprising:

[0051] The weeder body 12 is used to drive the weeder to move; the weeder body 12 can be connected to a mobile vehicle such as a tractor through a connecting component in the prior art, thereby helping the weeding process to be completed smoothly.

[0052] The multimodal visual component 1 is arranged on the front side of the lawnmower body 12, and is used to collect image information and depth information of seedlings and weeds; wherein, the image information and depth information include various information such as the outline, texture, color, etc. of the seedlings, and the depth information can help improve the accuracy. The length, width and height of the lawnmower body 12 are 7088mm, 2475mm and 1828mm, and the main beam of the lawnmower is made of Q345 low alloy steel, which has excellent mechanical properties, low temperature properties and processing properties. The weeding knife is made of 65Mn material, which has high toughness and wear resistance in the quenched state. The controller 2 is also electrically connected to the fifth generation mobile communication technology, and can upload the current real-time working status and position of the lawnmower to the cloud server for display and storage, so that managers can view the lawnmower status and historical records.

[0053] The weeding translation mechanism 3 is connected to the weeder body 12, and a weeding assembly 4 is arranged on the ground side; the weeding assembly 4 used in the present application is specifically a twelve-way weeding unit, which can also be increased or decreased according to actual needs. The twelve-way weeding unit uses a plunger cylinder and a four-link profiling technology to achieve a consistent pressure of multiple units on the ground, thereby ensuring a consistent weeding depth, and completing the treatment of inter-row weeds in the furrows respectively without damaging the twelve seedling strips. The position of the weeding assembly 4 can be adjusted by the weeding translation mechanism 3, and the position of the weeding assembly 4 is adjusted as a whole. Because the relative distance between the seedling strips is fixed, and the relative distance between each group of weeding knives of the weeding assembly 4 is fixed, each seedling strip can be accurately treated accordingly after the overall adjustment. The weeding translation mechanism 3 is specifically connected to the weeder body 12 by a three-point suspension method through a positioning and suppressing wheel mechanism, and other connection methods can also be used for connection according to actual needs. The specific structure of the weeding translation mechanism 3 is a fixed C-beam and a mobile seat structure. The mobile seat structure can slide relative to the weeder body 12 in the C-beam through three sets of pulleys. There will be no creeping and shaking during movement, ensuring the stability of the translation of the mechanism. When the translation drive assembly is driven by two hydraulic push rods 11, they are distributed on both sides of the mobile seat. The specific connection form is selected according to actual conditions. The offset result is calculated by the multimodal vision and machine learning algorithm controller device, that is, the controller 2, and the left hydraulic push rod or the right hydraulic push rod is controlled by the multimodal vision and machine learning algorithm controller to push the twelve-way weeding unit to achieve dynamic adjustment of the weeding knife at the center of the two seedling belts.

[0054] The translation drive assembly, the fixed end is connected to the weeding machine body 12, and the movable end is connected to the weeding translation mechanism 3 to drive the weeding translation mechanism 3 to move; the translation drive assembly is specifically set to a hydraulic push rod 11 for driving, and the number can be set according to actual conditions. In order to ensure good mobile stability, the number of translation drive assemblies can be set to two, which are respectively set on both sides of the movable end. When the two translation drive assemblies drive the weeding translation mechanism 3, they can be driven in a coordinated manner to achieve the purpose of stable and accurate driving. When two hydraulic cylinders are used, a one-way hydraulic cylinder can be used as the type of hydraulic cylinder. For the specific fixed position of the hydraulic cylinder, the cylinder body can be fixed to the weeding machine body 12 by bolts to complete the installation. The movable end of the hydraulic cylinder, that is, the hydraulic rod, is connected to the end face of the corresponding side of the weeding translation mechanism 3 by bolts, and the connection form can be adjusted accordingly according to actual conditions. For example, the use of a ball screw structure for driving is also within the inventive spirit of this application. For other drives, the specific connection method can be adaptively designed according to the actual structure.

[0055] Controller 2 is used to respond to the signal of multimodal vision component 1 to control the translation drive component to drive the weeding translation mechanism 3 and weeding component 4 to move. Controller 2 performs multimodal clustering analysis and positioning on the images collected by the camera, and then extracts the seedlings to perform Hough transform in the direction of vehicle travel to calculate the angle and curvature of the crop row. Controller 2 is specifically used as a multimodal vision and machine learning algorithm controller device. It uses the Ubuntu system of the arm hardware platform as a carrier, runs a multimodal vision and machine learning algorithm system on the ROS robot operating system based on the python language, and connects to the camera through the network port or USB port to collect images and complete the operation of the recognition program.

[0056] In this embodiment, two multimodal vision components 1 are provided and distributed on both sides of the lawn mower body 12. The use of two multimodal vision components 1 can simultaneously collect data on both sides of the lawn mower body 12, which can improve the accuracy of collection and prevent the inability to perform normal visual recognition when one side of the multimodal vision component 1 is damaged. Using two multimodal vision components 1 at the same time can improve the accuracy of visual recognition.

[0057] In this embodiment, the multimodal visual component 1 includes an RGB camera and a depth of field camera. The RGB camera and the depth of field camera are a group and are installed on the corresponding camera bracket 10 at the same time. For the RGB camera and the depth of field camera, it is necessary to meet the adjustable viewing angle and adjust it to the front of the lawn mower as the camera's visible field of view, and at least 2 seedling strips can be photographed, and each seedling strip has 8-10 seedlings as the ideal field of view. The RGB camera and the depth camera are DFRobot brand SEN0583 model TOF cameras. SEN0583 is divided into two parts, one is an ordinary RGB camera, and the other is a TOF module including a TOF array transmitter and a TOF infrared light receiving camera. By measuring the "flight time" of the light signal between the transmitter and the receiver to calculate the distance between the two, a depth camera for TOF ranging can be realized.

[0058] In actual application, the RGB camera collects image information of the field, and the intelligent weeder system recognizes the emergence of seedlings and forms seedling belts through multimodal vision and machine learning crop row recognition. In the case of high light ratio, the seedling height information collected by the depth camera needs to be integrated into the RGB camera to determine the seedling boundary and eliminate interference from shadows or dark light environments.

[0059] In this embodiment, the multimodal vision component 1 is connected to the main body 12 of the weeder through a camera bracket 10, wherein the camera bracket 10 is a telescopic bracket. The height of the support can be changed according to actual requirements. The RGB camera and the depth camera 1 are installed on the camera bracket 10 through a U-shaped card, and the height of the U-shaped card can be adjusted according to the row spacing and seedling spacing of the actual weeding environment to meet the field of view of the RGB camera and the depth camera 1. At least 2 rows of seedlings and 8-10 seedlings. The camera bracket 10 adopts a 40 square tube with a height of 800mm, and is installed on the crossbeam of the main body 12 of the weeder through a U-shaped card.

[0060] In this embodiment, the weeding translation mechanism 3 is provided with a displacement sensor 6 for collecting its position, and the displacement sensor 6 is connected to the controller 2. The displacement sensor 6 can detect the current position of the weeding translation mechanism 3 and feed it back to the controller 2, and can form a closed-loop control with the controller 2. The specific installation method of the displacement sensor 6 is that one end is installed on the frame of the weeding machine body 12, and the other end is installed on the weeding translation mechanism 3. When the weeding translation mechanism 3 translates left and right, it can drive the displacement sensor 6 to stretch or shrink. According to the different stretching strokes, the stroke simulation signal is fed back to the multimodal vision and machine learning algorithm controller, that is, the controller 2 for processing. After collecting the signal, the multimodal vision and machine learning algorithm system converts the analog quantity in real time to calculate the current position of the displacement sensor extension, thereby determining the position amount of the current left and right offset of the weeding mechanism, and assisting visual recognition to calculate the actual offset.

[0061] As for the power supply, the tractor power supply system provides a 12V power supply to meet the power requirements of the RGB camera and depth camera, multimodal vision and machine learning algorithm controller device, namely the controller 2, displacement sensor 6, Hall sensor 7, floor proximity switch 8, and HMI touch screen 9.

[0062] In this embodiment, the weeder body 12 is provided with a ground wheel 5, and the ground wheel 5 is provided with a Hall sensor 7, and the Hall sensor 7 is connected to the controller 2. The specific installation position of the ground wheel 5 is installed on the ground side of the weeder body 12, and can be fixed by bolts. During the movement of the weeder body 12, the wheel body of the ground wheel 5 will contact the ground and rotate relatively. During the rotation, the Hall sensor 7 will count the rotation of the wheel body of the ground wheel 5. Then, if the wheel diameter of the wheel body is known, the current travel speed of the vehicle body can be calculated by counting and the wheel diameter, and the longitudinal distance of the left and right deviation position of the seedling belt from the weeding component 4 can be identified according to the images collected by the current RGB camera and the depth camera, and it is determined when the weeding component 4 performs the left and right deviation action. The rotation counting method is that the Hall sensor is installed on the ground wheel with a metal grid structure. When the weeder is weeding, the ground wheel rotates, and when the grid metal sheet blocks the Hall sensor, a digital IO signal is given to the controller 2. Controller 2 collects IO signals and calculates the number of IO signals per unit time in real time through a multimodal vision and machine learning algorithm system. The current vehicle speed is converted based on the number, time and preset ground wheel diameter.

[0063] In this embodiment, the weeding component 4 is provided with a floor proximity switch 8, which has no signal when the weeding component 4 is out of the weeding state and has a signal when the weeding component 4 is in the weeding state, and the floor proximity switch 8 is connected to the controller 2. The above-mentioned weeding machine body 12 is also provided with an HMI touch screen 9 for displaying control and operating status, and the connection method with the controller can be a conventional connection method. The HMI touch screen 9 is connected to the multimodal vision and machine learning algorithm controller device via HDMI, and the device operation interface is made through pyqt, and the display of the device operation status, the input of abnormal status and related parameters can be completed through the interface. The multimodal vision and machine learning algorithm controller, that is, the controller 2, collects the floor proximity switch signal, and judges the current working state through the multimodal vision and machine learning algorithm system, thereby stopping or starting recognition.

[0064] According to another aspect of the present invention, a weeding alignment method is provided, using the above-mentioned weeder, comprising the following steps:

[0065] S1, the multimodal vision component 1 starts working, specifically the RGB camera and the depth camera start working, and transmit the video stream to the controller 2;

[0066] S2, controller 2 captures the current video stream frame, identifies the seedlings and weeds in the frame image captured by the RGB camera through a multimodal vision algorithm and compares them with the model library. If there is a high light ratio environment, the depth camera obtains the depth frame data to identify the seedlings and weeds, thereby enhancing the accuracy and robustness of crop detection;

[0067] S3, the controller 2 calibrates the positions of all seedlings in the frame image through the machine, and converts them into seedling belt information through Hough transformation according to the direction of the weeder, and calculates the angle and curvature of the seedling belt row;

[0068] S4, the controller 2 calculates the center position of the seedling belt through machine learning, and converts the camera matrix into a quaternion matrix through a rotation matrix to obtain the actual deviation distance information between the weeding component 4 and the seedling belt;

[0069] S5, the controller 2 calculates the driving speed through the Hall sensor and then performs a spatial model conversion between the modal vision component 1 and the weeding component 4, specifically, a spatial model conversion between the camera and the weeding knife. The controller 2, i.e., the multimodal vision and machine learning crop row recognition intelligent weeding machine system, calculates the time for the weeding component 4, i.e., the knife to move;

[0070] S6, after reaching the weeding time point, the controller 2 controls the weeding translation mechanism 3 to move so that the weeding component 4 reaches the target position and meets the weeding conditions. Specifically, when reaching the weeding time point, the position detected by the displacement sensor 6 on the weeding translation mechanism 3 and the calculated offset position are comprehensively processed to determine the actual reached position, and the multimodal vision and machine learning crop row recognition intelligent weeder system intelligently decides the current control of the left hydraulic push rod or the right hydraulic push rod action, so that the weeding knife accurately reaches the corresponding position.

[0071] In this embodiment, the model library in step S2 is obtained by real-time collection of pictures of seedlings and weeds by the multimodal vision component 1 when the weeder is moving at a low speed, and the seedling strips and weeds in the pictures are labeled and trained. After the optimal model library is trained, it is deployed in the intelligent weeder system for multimodal vision and machine learning crop row recognition.

[0072] See attached Fig.13 , the RGB camera and the depth camera use the visual sensor to obtain the image information of the current field in real time and transmit it to the multi-modal vision and machine learning algorithm controller device of Neousys NRU-230V through the network port, that is, controller 2. Controller 2 uses the visual algorithm of the intelligent weeder system of internal multi-modal vision and machine learning crop row recognition to identify the seedlings and detect the seedling belt through Hough transform, and then sends the offset data to the controller of the STM32F405RGT6 main control chip through RS485.

[0073] The controller of the STM32F405RGT6 main control chip collects multiple information such as the position information of the displacement sensor through the ADC interface, the speed measurement information of the encoder through the input capture detection, and the landing switch status through the IO input port, and integrates the offset data sent by the visual processor to calculate the position where the current translation mechanism should be offset, and controls the switch of the solenoid valve of the weeding translation mechanism 3 through the switch quantity to achieve the left translation or right translation of the weeding translation mechanism 3.

[0074] When in use, the weeder body 12 is mounted on the rear suspension of the tractor through a suspension mechanism and is towed by the tractor. The left and right deviations of the seedling belts are identified through the RGB camera and the depth camera on the camera bracket 10, and the hydraulic push rod 11 is powered by the tractor oil system. The hydraulic push rod 11 controls the twelve-way weeding assembly 4 to move left and right to ensure that the weeding knife is dynamically kept in the middle of the two seedling belts.

[0075] In the above description, the controllers, sensors, control programs, and model libraries that may be involved all adopt existing technologies and will not be described in detail here.

[0076] The embodiments of the present invention disclosed above are only used to help illustrate the present invention. The embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. According to the content of this specification, many modifications and changes can be made. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well.

Claims

1. A weeder, characterized in that: include: A weeder body (12) is used to drive the weeder to move; A multimodal vision component (1) is arranged on the front side of the weeder body (12) and is used to collect image information and depth information of seedlings and weeds; A weeding translation mechanism (3) is connected to the weeder body (12), and a weeding assembly (4) is provided on the ground side; A translation drive assembly, the fixed end of which is connected to the weeder body (12), and the movable end of which is connected to the weeding translation mechanism (3) for driving the weeding translation mechanism (3) to move; The controller (2) is used to respond to the signal of the multi-modal vision component (1) to control the translation drive component to drive the weeding translation mechanism (3) and the weeding component (4) to move.

2. A weeder according to claim 1, characterized in that: The multimodal visual components (1) are provided in two numbers and are distributed on both sides of the lawn mower body (12).

3. A weeder according to claim 1 or 2, characterized in that: The multimodal vision component (1) comprises an RGB camera and a depth-of-field camera.

4. A weeder according to claim 3, characterized in that: The multimodal vision component (1) is connected to a lawn mower body (12) via a camera bracket (10), wherein the camera bracket (10) is a telescopic bracket.

5. A weeder according to claim 1, 2 or 4, characterized in that: The translation driving component is a hydraulic push rod (11).

6. A weeder according to claim 1, characterized in that: The weeding translation mechanism (3) is provided with a displacement sensor (6) for collecting its position, and the displacement sensor (6) is connected to the controller (2).

7. A weeder according to claim 1 or 6, characterized in that: The weeder body (12) is provided with a ground wheel (5), the ground wheel (5) is provided with a Hall sensor (7), and the Hall sensor (7) is connected to the controller (2).

8. A weeder according to claim 7, characterized in that: The weeding component (4) is provided with a floor proximity switch (8), wherein the floor proximity switch (8) has no signal when the weeding component (4) is out of the weeding state, and has a signal when the weeding component (4) is in the weeding state, and the floor proximity switch (8) is connected to the controller (2).

9. A weeding method using a weeder according to claim 1, 2, 4, 6 or 8, characterized in that: The following steps are involved: S1, the multimodal vision component (1) starts working and transmits the video stream to the controller (2); S2, the controller (2) intercepts the current video stream frame, identifies the seedlings and weeds in the intercepted frame image respectively and compares them with the model library. If there is a high light ratio environment, the depth frame data is obtained to identify the seedlings and weeds; S3, calibrating the positions of all seedlings in the frame image, and converting them into seedling belt information through Hough transformation according to the direction of the weeder, and calculating the angle and curvature of the seedling belt row; S4, the controller (2) calculates the center position of the seedling belt, and converts the camera matrix into a quaternion matrix through a rotation matrix, thereby obtaining information on the actual deviation distance between the weeding component (4) and the seedling belt; S5, the controller (2) calculates the driving speed and performs a spatial model conversion between the modal vision component (1) and the weeding component (4), and the controller (2) calculates the moving time of the weeding component (4); S6, after reaching the weeding time point, the controller (2) controls the weeding translation mechanism (3) to move so that the weeding component (4) reaches the target position, thus meeting the weeding conditions.

10. A weed control method according to claim 9, characterized in that: The model library in step S2 is obtained by labeling and training the seedling strips and weeds in the pictures by the multimodal visual component (1) when the weeder is moving at a low speed and collecting pictures of the seedlings and weeds in real time.

Images