Intelligent laser weeding method and device

Abstract

The invention relates to an intelligent laser weeding method and device. The method comprises the steps that S1, a navigation path is formulated for a weeding device; s2, issuing the navigation path; s3, after weeding work is started, weed images are shot in the field walking process of the weeding device; identifying and judging the types of weeds and the sizes of plants in the identified images; s4, determining the laser irradiation area of each weed region, and correcting the laser irradiation power of each weed region in real time according to the type of weeds and the plant size; s5, determining position adjusting parameters of the laser and the size of an irradiation light spot; and S6, moving the laser to a position right above each weed area, controlling the laser to emit laser downwards, adjusting a laser spot to a corresponding size, and controlling the laser power to a corresponding correction value to irradiate the weed area. By means of the method, the laser irradiation power can be automatically adjusted in real time according to the types of weeds and the sizes of plants, the weeding effect can be improved, and meanwhile unnecessary energy waste is reduced.

Description

Technical Field

[0001] The invention relates to image processing technology, and in particular to a laser intelligent weeding method and device. Background Art

[0002] Weeds are one of the main constraints to sustainable agricultural production. The nutrients they absorb from the soil are basically the same as those of crops. The competition for nutrients, water and sunlight by weeds greatly harms the growth and development of crops. Weeds cause huge economic losses in agricultural production every year. In addition, weeds, as the transmission medium and intermediate host of many farmland pests and diseases, can cause outbreaks of farmland pests and diseases, leading to reduced crop yield and quality.

[0003] At present, in the process of agricultural production, field weeding operations are mainly carried out by spraying herbicides manually or by machine. This weeding method not only has an adverse effect on the growth and development of current crops, but also the residual herbicides pollute the soil and are difficult to remove, affecting the yield of subsequent crops, especially for intercropping and rotation farmland. It also poses a serious threat to the health of workers. Pesticide residues on crops after harvest will also affect the quality and safety of agricultural products.

[0004] With the development of technology, intelligent laser weeding robots are now available on the market. The robot can be controlled to walk automatically in the field through path planning and automatic navigation. During the robot's walking process, it can identify and locate weeds through image recognition, and control the laser to output laser to the weeds, so that the laser beam is accurately focused on the grass, and the grass is cut or burned to death through thermal effect. This technology can achieve accurate elimination of weeds and prevent crops from being exposed to herbicides.

[0005] The laser output power and laser irradiation duration for a single weed of existing laser weeding robots are set to specific values, without considering factors such as the type of weed and the size of the plant. For killing smaller and more fragile weeds, the set laser power and irradiation duration are often sufficient to ensure effective killing of weeds, which will cause unnecessary energy waste; while for killing larger and stronger weeds, the set laser power and irradiation duration may be insufficient to ensure effective killing of weeds, which will affect the weeding effect. Summary of the invention

[0006] Based on the above description, the present invention provides a laser intelligent weeding method and weeding device, which can automatically adjust the laser output power according to the type of weeds and the size of the plants, improve the weeding effect and reduce unnecessary energy waste.

[0007] The technical solution of the present invention to solve the above technical problems is as follows:

[0008] A laser intelligent weeding method comprises the following steps:

[0009] S1. Formulate a navigation path for the weeding device according to the position and direction of the field ridge;

[0010] S2, sending the navigation path to the weeding device;

[0011] S3. After starting the weeding work, the weeding device is controlled to move along the navigation path; the image below the weeding device is photographed and collected during the weeding device's movement in the field; the collected image is processed to identify the weed features in the image so as to realize the identification and judgment of the weed type and plant size, and the center coordinate position of each weed area is located according to the relative position of each weed area in the image, and the area of ​​each weed area is calculated;

[0012] S4, determining the laser irradiation area of ​​each weed region, and correcting the laser irradiation power of each weed region in real time according to the type and plant size of the weeds;

[0013] S5, determining the position adjustment parameters of the laser according to the central coordinate position of each weed area, and determining the size of the irradiated light spot according to the area of ​​the weed area;

[0014] S6. Move the laser to the top of each weed area in turn according to the position adjustment parameters, control the laser to emit laser downward and adjust the laser spot to a corresponding size, and at the same time control the laser power to a corresponding correction value to irradiate the weed area.

[0015] As a preferred solution: in step S3, a weed recognition model based on deep learning is constructed, a large number of image samples of different types of weeds are input into the weed recognition model, and the model is trained and tested to achieve the required accuracy; during the operation of the weed removal device, after the image data is obtained by shooting, the image is output to the weed recognition model, and the weed recognition model automatically identifies the type of weeds and selects the area where the weeds are located, and the area in the frame is the weed area; two diagonal lines are drawn from the weed area, and the intersection of the two diagonal lines is the center point of the rest of the weeds; the image is then processed by coordinate processing to obtain the coordinates of the center point of the weed area and the coordinates of the four corner points, the coordinates of the corner points in the image are converted into real-world coordinates, and the area of ​​each weed area is calculated accordingly.

[0016] As a preferred solution: in step S4, after the weed recognition model recognizes the weeds in the image and selects the area where the weeds are located, the image processing unit segments the image of the selected area to obtain a target image; the tip features of the weed leaves in the target image are recognized and located, and the tip point closest to the reference point is selected as the effective point based on the coordinates of the center point of the target image, and the distance between the effective point and the reference point is calculated according to the coordinates, and the distance is defined as the coverage radius of the weed plant. The larger the coverage radius, the larger the plant; the laser irradiation power required for each type of weed in different plant sizes is calibrated in advance; during the weeding process, when the type and plant size of the weed are recognized, the corresponding laser irradiation power is automatically called, that is, the corrected irradiation power is obtained.

[0017] As a preferred solution: when shooting an image in step S3, fill light is applied to the shooting area below to obtain an image after fill light is applied, and the part with water marks after fill light will show high reflectivity; after obtaining the target image in step S4, a copy of the target image is copied, and the copy of the target image is binarized to obtain a binary image, in which the outline of the weed plant and the outline of the water marks can be clearly reflected, and then the block covered by the outline is intercepted, and the block is evenly divided into a grid array, and a certain number of pixel points at the edge and center of each grid unit are scanned and the color values ​​of these pixel points are read, and the average color value of these pixel points is calculated and compared with the preset color value, and when the current value is greater than the latter, it is considered that there is a water mark in the current grid unit, otherwise it is considered that there is no water mark; then the number of grid units with water marks is counted and the ratio F1 thereof to the total number of grids is calculated; a number of groups of numerical intervals are preset for the F1 value, and a power correction coefficient k1 is calibrated for each group of numerical intervals, k1>1, and different numerical intervals correspond to different k1 values; specifically, the larger the numerical interval, the larger the k1 value;

[0018] In step S4, the weed plant contours in the binary image are extracted simultaneously, the target image original image and the copy are aligned and the contour is mapped to the target image original image, an image of a certain size is intercepted in the area within the contour on the target image original image as a color comparison image, the color values ​​of several pixels in the color comparison image are randomly scanned, the average color value is calculated, and the average setting is compared with the reference color value of the weed leaf and the color value deviation value F2 is calculated (the deviation value is equal to the color value difference obtained by subtracting the reference color value from the leaf color value, and then the color value difference is divided by the reference color value to obtain the deviation value). If the color is gray and light, it means that there is dust accumulation on the weed leaf; similarly, it is necessary to preset several groups of numerical intervals for the leaf color value deviation value F2, and calibrate the power correction coefficient k2 for each group of numerical intervals, k2>1, and different numerical intervals correspond to different k2 values; the larger the numerical interval, the larger the k2 value;

[0019] When correcting the laser irradiation power, according to the formula P1=P0×k1×k2, P0 is the calibrated laser irradiation power corresponding to the current type and size of the weed plant, and P1 is the corrected laser irradiation power.

[0020] As a preferred solution: in step S5, the required movement parameters of the laser, i.e., the position adjustment parameters, are calculated according to the coordinates of the current position of the laser and the center coordinates of the weed area to be treated; the size of the irradiated light spot is equal to the size of the inscribed circle of the rectangular boundary of the weed area.

[0021] A laser intelligent weeding device, used to execute the weeding method described in any one of claims 1 to 5, the device comprises a main control module, and also comprises an obstacle avoidance sensor, a storage module, a camera module, a flash module, a power module, a light spot adjustment mechanism, a motion control module, a laser drive module, a traction control module, a steering gear control module, a navigation positioning module and a communication module connected to the main control module.

[0022] Compared with the prior art, the technical solution of the present application has the following beneficial technical effects: the above method can realize real-time automatic adjustment of the laser irradiation power according to the type of weeds and the size of the plants, which can improve the weed control effect and reduce unnecessary energy waste. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is a flow chart of the method in Example 1;

[0024] Figure 2 This is a block diagram of the hardware principle of the device in Example 2. DETAILED DESCRIPTION

[0025] Reference Figure 1 , a laser intelligent weeding method, the method in this embodiment includes the following steps.

[0026] S1. A navigation path is established for the weeding device according to the position and direction of the field ridges.

[0027] S2. Send the navigation path to the weeding device.

[0028] S3. After starting the weeding work, the weeding device is controlled to move along the navigation path; the image below the weeding device is photographed and collected during the walking process in the field; the collected image is processed to identify the weed features in the image so as to realize the recognition and judgment of the weed type and plant size, and the center coordinate position of each weed area is located according to the relative position of each weed area in the image, and the area of ​​each weed area is calculated.

[0029] In this step, a weed recognition model based on deep learning is constructed, and a large number of image samples of different types of weeds are input into the weed recognition model. The model is trained and tested to achieve the required accuracy. During the operation of the weed removal device, after the image data is captured, the image is output to the weed recognition model. The weed recognition model automatically identifies the type of weeds and selects the area where the weeds are located. The area in the frame is the weed area. Two diagonal lines are drawn from the weed area, and the intersection of the two diagonal lines is the center point of the rest of the weeds; the image is then processed by coordinate processing to obtain the coordinates of the center point of the weed area and the coordinates of the four corner points. The coordinates of the corner points in the image are converted into real-world coordinates, and the area of ​​each weed area is calculated accordingly.

[0030] S4. Determine the laser irradiation area of ​​each weed region, and modify the laser irradiation power of each weed region in real time according to the type and plant size of the weeds.

[0031] In this step, after the weed recognition model identifies the weeds in the image and selects the area where the weeds are located, the image processing unit segments the image of the selected area to obtain the target image. The tip features of the weed leaves in the target image are identified and located, and the center point coordinates of the target image are used as a reference. The tip point closest to the reference point is selected as the effective point, and the distance between the effective point and the reference point is calculated based on the coordinates. The distance is defined as the coverage radius of the weed plant. The larger the coverage radius, the larger the plant. It is necessary to calibrate the laser irradiation power required for each type of weed in different plant sizes in advance. During the weeding process, when the type and plant size of the weed are identified, the corresponding laser irradiation power is automatically called, that is, the corrected irradiation power is obtained.

[0032] S5, determining the position adjustment parameters of the laser according to the central coordinate position of each weed area, and determining the size of the irradiated light spot according to the area of ​​the weed area;

[0033] In this step, the required movement parameters of the laser, i.e., the position adjustment parameters, are calculated based on the coordinates of the current position of the laser and the center coordinates of the weed area to be treated. The size of the irradiated light spot is equal to the size of the inscribed circle of the rectangular boundary of the weed area. The inscribed circle can be obtained by drawing a circle with the center point of the rectangular boundary as a circle and half of the short side of the rectangular boundary as a radius. The center of the inscribed circle is the axis of the laser after the position is adjusted.

[0034] S6. Move the laser to the top of each weed area in turn according to the position adjustment parameters, control the laser to emit laser downward and adjust the laser spot to a corresponding size, and at the same time control the laser power to a corresponding correction value to irradiate the weed area.

[0035] The above method can realize real-time automatic adjustment of laser irradiation power according to the type of weeds and the size of plants, which can improve the weed control effect and reduce unnecessary energy waste.

[0036] In actual application environments, due to the influence of rainfall and wind, some weeds will have water or dust on their surface. When the weed leaves are covered with water or dust, the reflection of water and the light-blocking effect of dust will weaken the laser irradiating the weeds, which will have an adverse effect on the weed control effect.

[0037] As an improved solution, based on the above, in this embodiment, when the laser irradiation power is corrected in real time, the surface water and dust accumulation of weeds are also taken into account. Specifically, when shooting an image in step S3, fill light is applied to the shooting area below to obtain a fill light image, and the part with water marks after fill light will show high reflectivity; after obtaining the target image in step S4, a copy of the target image is copied, and the copy of the target image is binarized to obtain a binarized image, in which the outline of the weed plant and the outline of the water marks can be clearly reflected, and then the block covered by the outline is intercepted, and the block is evenly divided into a grid array, and a certain number of pixel points at the edge and center of each grid unit are scanned and the color values ​​of these pixel points are read, and the average color value of these pixel points is calculated and compared with the preset color value. When the current value is greater than the latter, it is considered that there is a water mark in the current grid unit, otherwise it is considered that there is no water mark. Then the number of grid units with water marks is counted and the ratio F1 to the total number of grids is calculated.

[0038] In this embodiment, it is also necessary to preset several groups of numerical intervals for the F1 value, and calibrate the power correction coefficient k1 for each group of numerical intervals, k1>1, and different numerical intervals correspond to different k1 values. Specifically, the larger the numerical interval, the larger the k1 value.

[0039] In step S4, the outline of the weed plant in the binary image is extracted at the same time, the original target image and the copy are aligned and the outline is mapped to the original target image, an image of a certain size is intercepted in the area within the outline on the original target image as a color comparison image, the color values ​​of several pixels in the color comparison image are randomly scanned, the average color value is calculated, and the average setting is compared with the reference color value of the weed leaf and the color value deviation value F2 is calculated (the deviation value is equal to the leaf color value minus the reference color value to obtain the color value difference, and then the color value difference is divided by the reference color value to obtain the deviation value). If the color is gray and light, it means that there is dust accumulation on the weed leaf. Similarly, it is necessary to preset several groups of numerical intervals for the leaf color value deviation value F2, and calibrate the power correction coefficient k2 for each group of numerical intervals, k2>1, and different numerical intervals correspond to different k2 values. The larger the numerical interval, the larger the k2 value.

[0040] When correcting the laser irradiation power, according to the formula P1=P0×k1×k2, P0 is the calibrated laser irradiation power corresponding to the current type and size of the weed plant, and P1 is the corrected laser irradiation power.

[0041] Embodiment 2:

[0042] Reference Figure 2 , a laser intelligent weeding device, used to execute the weeding method in embodiment one.

[0043] The device in this embodiment includes a main control module, and also includes an obstacle avoidance sensor, a storage module, a camera module, a flash module, a power module, a light spot adjustment mechanism, a motion control module, a motion mechanism, a laser drive module, a traction control module, a servo control module, a navigation and positioning module and a communication module connected to the main control module.

[0044] The obstacle avoidance sensor is used to sense obstacles around the weeding device and feed back obstacle information to the main control module; the camera module is used to capture images below the weeding device; the flash module is used to fill in light to the object below the weeding device; the spot adjustment mechanism is used to adjust the size of the laser spot directed at the weeds; the motion control module is used to control the movement of the motion mechanism; the laser drive module is used to control the start and stop of the laser and adjust the output power of the laser; the traction control module is used to control the power motor of the weeding device to control the movement of the weeding device; the servo control module is used to control the steering motor of the weeding device to control the direction of travel of the weeding device; the navigation and positioning module is used to guide the weeding device to move along the navigation path; and the communication module is used for communication and data transmission between the weeding device and the cloud platform.

[0045] In this embodiment, the motion mechanism is a three-axis motion platform, which is connected to the main body of the weeding device. The laser is installed on the three-axis motion platform and the direction of the laser is downward. The position of the laser can be adjusted by controlling the movement of the three-axis motion platform.

[0046] In addition, the light spot adjustment mechanism in this embodiment is a zoom lens mechanism, which is arranged on the output optical path of the laser, and the zoom lens mechanism adjusts the size of the light spot by adjusting the position of the focusing lens. Specifically: In this embodiment, in order to simplify the calculation, the height of the weeds is directly ignored, and the height h of the laser from the ground is directly used as the standard when calculating the light spot size; after obtaining the radius of the aforementioned inscribed circle, the height position to which the focusing lens needs to be adjusted can be converted according to the radius and the height h, and then the up and down adjustment amount of the focusing lens can be calculated according to the current height position of the focusing lens; the adjustment amount can be converted into the control parameters of the zoom lens mechanism (for example, when the zoom lens mechanism adopts a linear motor, the support frame is used to fix the column focusing lens and then the support frame is connected to the shaft end of the linear motor, and the main body of the linear motor is connected to the main body of the weeding device. The up and down adjustment amount is the up and down stroke of the linear motor, and the above control can be achieved by controlling the rotation angle of the linear motor).

[0047] The weed control device in this embodiment is recommended for use in the early stage of weed growth. In this stage, the weeds are small in size and the height difference between the top of the weeds and the ground is small. Therefore, the spot size is calculated based on the height h of the laser from the ground, which basically does not affect the weed control effect.

[0048] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A laser intelligent weeding method, characterized in that: The following steps are involved: S1. Formulate a navigation path for the weeding device according to the position and direction of the field ridge; S2, sending the navigation path to the weeding device; S3. After starting the weeding work, the weeding device is controlled to move along the navigation path; the image below the weeding device is photographed and collected during the weeding device's movement in the field; the collected image is processed to identify the weed features in the image so as to realize the identification and judgment of the weed type and plant size, and the center coordinate position of each weed area is located according to the relative position of each weed area in the image, and the area of ​​each weed area is calculated; S4, determining the laser irradiation area of ​​each weed region, and correcting the laser irradiation power of each weed region in real time according to the type and plant size of the weeds; S5, determining the position adjustment parameters of the laser according to the central coordinate position of each weed area, and determining the size of the irradiated light spot according to the area of ​​the weed area; S6. Move the laser to the top of each weed area in turn according to the position adjustment parameters, control the laser to emit laser downward and adjust the laser spot to a corresponding size, and at the same time control the laser power to a corresponding correction value to irradiate the weed area.

2. The laser intelligent weeding method according to claim 1 is characterized in that: In step S3, a weed recognition model based on deep learning is constructed, a large number of image samples of different types of weeds are input into the weed recognition model, and the model is trained and tested to achieve the required accuracy. During the operation of the weed removal device, after the image data is captured, the image is output to the weed recognition model, and the weed recognition model automatically identifies the type of weeds and selects the area where the weeds are located. The area in the frame is the weed area. Two diagonal lines are drawn from the weed area, and the intersection of the two diagonal lines is the center point of the rest of the weeds. The image is then processed by coordinate processing to obtain the coordinates of the center point of the weed area and the coordinates of the four corner points. The coordinates of the corner points in the image are converted into real-world coordinates, and the area of ​​each weed area is calculated accordingly.

3. The laser intelligent weeding method according to claim 1 is characterized in that: In step S4, after the weed recognition model recognizes the weeds in the image and selects the area where the weeds are located, the image processing unit segments the image of the selected area to obtain a target image; The tip features of the weed leaves in the target image are identified and located. The tip point closest to the reference point is selected as the effective point based on the coordinates of the center point of the target image. The distance between the effective point and the reference point is calculated according to the coordinates. The distance is defined as the coverage radius of the weed plant. The larger the coverage radius, the larger the plant. The laser irradiation power required for each type of weed in different plant sizes is calibrated in advance. During the weeding process, when the type and plant size of the weed are identified, the corresponding laser irradiation power is automatically called to obtain the corrected irradiation power.

4. The laser intelligent weeding method according to claim 1 is characterized in that: When shooting an image in step S3, fill light is applied to the shooting area below to obtain a fill light image. After the fill light is applied, the part with water marks will show high reflectivity. After obtaining the target image in step S4, a copy of the target image is obtained by duplicating the copy of the target image. The copy of the target image is binarized to obtain a binarized image. The outline of the weed plant and the outline of the water marks can be clearly reflected in the binarized image. Then, the image block covered by the outline is intercepted, and the image block is evenly divided into a grid array. A certain number of pixel points at the edge and center of each grid unit are scanned and the color values ​​of these pixel points are read. The average color value of these pixel points is calculated and compared with the preset color value. When the current value is greater than the latter, it is considered that there is a water mark in the current grid unit, otherwise it is considered that there is no water mark. Then, the number of grid units with water marks is counted and the ratio F1 thereof to the total number of grids is calculated. Several groups of numerical intervals are preset for the F1 value, and the power correction coefficient k1 is calibrated for each group of numerical intervals, k1>1, and different numerical intervals correspond to different k1 values; specifically, the larger the numerical interval, the larger the k1 value; In step S4, the weed plant contours in the binary image are extracted simultaneously, the target image original image and the copy are aligned and the contour is mapped to the target image original image, an image of a certain size is intercepted in the area within the contour on the target image original image as a color comparison image, the color values ​​of several pixels in the color comparison image are randomly scanned, the average color value is calculated, and the average setting is compared with the reference color value of the weed leaf and the color value deviation value F2 is calculated (the deviation value is equal to the color value difference obtained by subtracting the reference color value from the leaf color value, and then the color value difference is divided by the reference color value to obtain the deviation value). If the color is gray and light, it means that there is dust accumulation on the weed leaf; similarly, it is necessary to preset several groups of numerical intervals for the leaf color value deviation value F2, and calibrate the power correction coefficient k2 for each group of numerical intervals, k2>1, and different numerical intervals correspond to different k2 values; the larger the numerical interval, the larger the k2 value; When correcting the laser irradiation power, according to the formula P1=P0×k1×k2, P0 is the calibrated laser irradiation power corresponding to the current type and size of the weed plant, and P1 is the corrected laser irradiation power.

5. The laser intelligent weeding method according to claim 1 is characterized in that: In step S5, the required movement parameters of the laser, namely the position adjustment parameters, are calculated according to the coordinates of the current position of the laser and the center coordinates of the weed area to be treated; the size of the irradiated light spot is equal to the size of the inscribed circle of the rectangular boundary of the weed area.

6. A laser intelligent weeding device, used to perform the weeding method according to any one of claims 1 to 5, characterized in that: The device includes a main control module, and also includes an obstacle avoidance sensor connected to the main control module, a storage module, a camera module, a flash module, a power module, a light spot adjustment mechanism, a motion control module, a laser drive module, a traction control module, a steering gear control module, a navigation and positioning module, and a communication module.

Images