A flame and laser coupled weeding robot with weed density detection function

Abstract

The application discloses a laser and flame combined weeding robot capable of separating and treating inter-plant and inter-row weeds, comprising a moving platform, a weed identification device, a flame weeding device, a laser weeding device, a height limiting device and an upper computer, wherein plants pass below the moving platform during the moving process, the weed identification device collects images of the weeds passing below, and the upper computer analyzes and processes the images to obtain inter-plant weed position information and inter-row weed density information respectively. The laser weeding device irradiates inter-plant meristem weeds, and can rotate freely to select a suitable irradiation position according to the position and growth of the weeds. The flame weeding device identifies the density of inter-row weeds, controls the intensity of the flame to weed, and the flame reflection plate improves the weeding efficiency and protects the crops on both sides of the inter-row weeds to a certain extent. In addition, the height limiting device is designed to control the distance between the flame spraying device and the ground, adapt to the change of the terrain, maintain the flame nozzle within a suitable range, and ensure the weeding effect.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of agricultural engineering and agricultural robot technology, and particularly relates to a flame and laser coupled weeding robot with weed density detection function. BACKGROUND

[0002] Grass damage is one of the important reasons for the decrease of crop yield and quality. Weeds not only compete with crops for nutrients, water, sunlight and growing space, but also have an important influence on the occurrence and spread of field diseases and pests. Therefore, it is crucial to remove weeds in farmland. At present, the main methods of weed control in domestic fields are chemical herbicide weeding and manual weeding. Manual weeding has low efficiency and high labor intensity, and cannot meet the current weeding needs of field agriculture. In addition, the cost of chemical weeding is increasing, the weed resistance is strengthening, and the types of weeding are limited by herbicides. It has a great threat to the agricultural ecological environment, agricultural and food safety, and human health. Long-term use of chemical herbicides can cause soil acidification, nutrient reduction, soil void reduction, soil compaction and soil toxicity, which is not conducive to crop growth and leads to hidden yield reduction of crops.

[0003] Many new weeding methods have been researched by domestic and foreign experts, such as electric weeding, steam weeding, infrared radiation weeding, flame weeding, laser weeding and microwave radiation weeding. Due to the high cost, high technical requirements, large energy consumption and strict use environment of electric weeding, microwave radiation weeding and steam weeding, and the easy damage of equipment during weeding, the research progress is slow, and it cannot be used for large-scale promotion. The two methods of laser weeding and flame weeding are widely used abroad. Laser weeding can effectively kill weeds by laser irradiation of weed meristem, and flame weeding can kill weeds by spraying high-temperature flame on weeds, which can make the water of weeds evaporate rapidly and die in a short time, and also kill soil surface insect eggs and grass seeds, avoiding chemical pollution of farmland. It has important research value for the protection of farmland and its surrounding ecological environment. However, laser weeding has poor effect in the face of large area of weeds, and flame weeding method will cause waste of resources when there are only a small amount of weeds in the area. Therefore, a flame and laser coupled weeding robot with weed density detection function is designed, which can identify weeds in different areas during the process of walking according to the characteristics of more weeds in the row and less weeds in the plant, and use different weeding methods. Flame weeding is used in the row, and the flame intensity is adjusted according to the size of the identified weed density; laser point shooting method is used in the plant, which will not harm the crops. It can improve the efficiency of weed removal, reduce resource waste, increase soil organic carbon storage, improve soil fertility, and help to obtain high-yield and high-quality agricultural products. SUMMARY

[0004] In view of the deficiencies of the existing automatic flame weeding technology and laser weeding technology and equipment, the present application aims to provide a coupling type weeding robot system and an implementation method thereof, which adopts flame and laser weeding methods to respectively target inter-row and inter-plant, and efficiently removes weeds in the field by using the flame and laser weeding technology.

[0005] The specific technical solutions adopted by the present application are as follows:

[0006] A flame and laser coupling weeding robot with weed density detection function, characterized by a mobile platform, a weed and crop recognition device, a flame weeding device, a laser weeding device, a height limiting device and a host computer.

[0007] The weed recognition device is installed at the front end of the mobile platform and faces the ground, and identifies weeds and crops below during the movement of the mobile platform. The inter-plant weed recognition device at the left side is connected to the laser weeding device to identify weeds on the crop flight line, and the inter-row weed recognition device at the right side is connected to the flame weeding device to adjust the intensity of the flame according to the size of the weed density in the recognition area.

[0008] The flame weeding device comprises a gas delivery hose, a connecting rod, a flame spout, a flame reflection plate, a gas flow meter and a combustible gas container. The combustible gas container is fixed above the mobile platform. The gas outlet of the combustible gas container is connected to the connecting rod and the flame spout of the flame spraying device through the hose, and a gas flow meter is arranged at the tail of the hose. The flame spouts are arranged below the mobile platform, and the flame spouts extend into the flame reflection plate. When the flame spouts are above the weed area, the flame spraying operation is performed.

[0009] The laser weeding device comprises a laser execution device, which can rotate left and right to facilitate the treatment of weeds on the left and right sides without moving the laser execution device on the guide rail. The laser execution device can facilitate the focusing of the laser on the weed meristem to improve the weed killing efficiency. The laser weeding device is composed of five parts, including a host computer, a CO2 laser, a controller, a galvanometer system and an F-Theta lens.

[0010] The height limiting device comprises a sink wheel, an elastic telescopic rod and a sink wheel support. The height limiting device is used to control the distance between the flame reflection plate, the flame spout and the field ground. Since the field ground is uneven, the height limiting device is used to adjust the distance to prevent the flame spout from colliding with the ground and to maintain the heat receiving area of the crops, thereby maximizing the damage of the flame to the weeds.

[0011] The host computer is used to control the operation of the laser weeding device and the flame weeding device. After obtaining the position of the inter-plant weeds, the laser weeding device is controlled to operate. After obtaining the size of the inter-row weed density, the intensity of the flame is controlled to perform weeding, thereby reducing the waste of resources in the weed-free area.

[0012] In the above technical solution, the weeds pass under the flame weeding device and the flame reflection plate during the movement of the mobile platform, and the crops pass through the middle position of the flame reflection plate.

[0013] In the above technical solution, a new method for detecting weed density is designed. The new method calculates the size of the weed density in advance, divides the density value into grades, and uses the density grade of the weeds in each picture as a label to train Deeplabv3+. The model can quickly identify the density grade in the region. The following steps can be used to achieve this:

[0014] Step 1: Collection and preprocessing of weed data. Collect image data sets containing weeds, perform preprocessing steps, and adjust all pictures to a uniform format of 512x512 pixels. Use data augmentation methods to expand the sample capacity of the training set. Use random rotation, random scaling, random translation, and image flipping to expand the sample capacity while ensuring information integrity, ultimately improving the accuracy and generalization ability of the model.

[0015] Step 2: Use the open-source labelme annotation tool to annotate the corn image. Train the Deeplabv3+ model using the annotated data set to achieve crop segmentation. Use the Color Index of Vegetation Extraction (CIVE) and Otsu threshold segmentation algorithm to achieve vegetation segmentation. According to the crop segmentation result, remove the crop area from the obtained vegetation segmentation result, and consider the remaining green vegetation as weeds. Calculate the ratio of the number of weed pixels in the image to the total number of pixels in the image to obtain the weed density. According to actual requirements, define three flame intensity levels, such as low, medium, and high. According to the grading rules, label the density grade of the original image as a whole image.

[0016] Step 3: Split the data set with weed density grade labels into training set, validation set, and test set. The training set will be used to train the model, the validation set will be used to adjust the model's hyperparameters, and the test set will be used to evaluate the model's performance.

[0017] Step 4: Train the Deeplabv3+ model. Train the Deeplabv3+ model on the training data with weed density grade labels. Deeplabv3+ is a convolutional neural network architecture commonly used for image segmentation tasks. During training, the model learns to predict the weed density grade of each pixel in the input image using annotated training data. After training, use the test set to evaluate the performance of the trained model. Adjust the model's hyperparameters and architecture as needed to improve its performance.

[0018] Step 5: Use the trained model to predict the weed density level in the new image. To do this, input the new image into the model, which will predict the overall weed density level in the image.

[0019] The present application has the beneficial effects of:

[0020] The present application can efficiently and accurately guide the mobile platform to walk along the crop row line under the navigation of the depth camera and the radar, the inter-plant weed recognition camera identifies and locates the inter-plant weeds below the progress, and the inter-row weed recognition camera detects the density of the inter-row weeds below the progress. The flame weeding device is fixed below the mobile platform, and the flame is sprayed to rapidly evaporate the water in the weed cells to kill the weeds, and at the same time, the high-temperature disinfection and sterilization of the soil and the killing of the weed seeds on the surface of the soil are realized. The laser weeding device uses a laser beam to focus on the weed meristem to realize accurate weed killing, which has the advantages of high efficiency, accuracy, energy saving, etc., and will not cause harm to crops. Both the laser weeding and the flame weeding reduce the use of pesticides, protect the environment, and avoid resource waste in the case of no weeds or few weeds by adjusting the intensity of the flame under different densities of inter-row weeds and using laser point-to-point weeding. Moreover, the present application also designs a flame reflection plate and a height limiting device to protect the corn plants and prevent the mechanism from being damaged, which improves the efficiency of weeding while achieving good weeding effect. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 is a three-dimensional structure schematic diagram of a flame and laser coupled weeding robot with weed density detection function.

[0022] Figure 2 is a structure schematic diagram of the flame spraying device in the present application.

[0023] Figure 3 is a structure schematic diagram of the laser device in the present application.

[0024] Figure 4 is a structure schematic diagram of the height limiting device in the present application.

[0025] Figure 5 is a structure schematic diagram of the inter-plant and inter-row weed recognition camera and the navigation camera in the present application.

[0026] For those skilled in the art, other related drawings can be obtained from the above drawings without creative labor. DETAILED DESCRIPTION

[0027] In order for those skilled in the art to better understand the present application, the technical solutions of the present application will be further described below in combination with specific embodiments.

[0028] Embodiment one

[0029] Combined with appendix Figures 1-5 As shown, the flame and laser coupled weeding robot system and its implementation method with weed density detection function of the present invention consists of a mobile platform 1, a weed identification device 2, a flame weeding device 3, a height limiting device 4, a laser weeding device 5, and a host computer 6.

[0030] The mobile platform 1 includes a frame 1.1, a battery pack 1.2, steering wheels 1.3, and drive wheels 1.4. The battery pack 1.2 is fixedly mounted on the frame 1.1, providing power for the mobile platform 1 to travel along its path and supplying power to the weed identification device 2, the flame weeding device 3, the height limiting device 4, the laser weeding device 5, and the host computer 6. There are two steering wheels 1.3, which are installed below the front end of the frame 1.1. There are two drive wheels 1.4, which are installed below the rear end of the frame 1.1. The drive wheels 1.4 have hub motors inside, and the hub motors and drive wheels are integrated into one structure. During movement, the mobile platform 1 can be turned by the differential rotation of the drive wheels.

[0031] The weed identification device 2 is located at the bottom of the front end of the mobile platform 1. Specifically, the inter-row weed identification depth camera 2.1 identifies weeds within the crop rows and obtains their location information as the mobile platform 1 moves forward, thereby determining the location of the weed's meristematic tissue; the row-to-row weed identification depth camera 2.2 acquires the density information of weeds between rows as the mobile platform 1 moves forward. These information are then transmitted to the host computer system for corresponding weeding operations.

[0032] The flame weeding device 3 is installed below the mobile platform 1 and used to spray flame to kill weeds. Specifically, the flame spraying device includes a gas supply hose 3.1, a connecting rod 3.2, a flame nozzle 3.3, a flame reflection plate 3.4, a gas flow meter 3.5 and a combustible gas container 3.6. The flame nozzle 3.3 is connected with the combustible gas container 3.6 through the gas supply hose 3.1 to ensure the supply of energy, the connecting rod 3.2 is installed below the mobile platform 1, and the two flame nozzles 3.3 are arranged side by side on the connecting rod 3.2 in the same direction. The gas flow meter 3.5 mainly controls the size of the combustible gas flow, thereby controlling the intensity of the flame. The host computer platform 6.2 controls the size of the flow of the gas flow meter 3.5 according to the weed density information identified by the weed identification device 2. The flame reflection plate 3.4 is installed below the mobile platform 1, and the flame nozzle 3.3 is extended below the flame reflection plate 3.4 by cooperating with the flame reflection plate 3.4, so as to isolate the flame nozzle 3.3 in the flame reflection plate 3.4, increase the contact area of the flame and the weeds, improve the temperature in the area, and be beneficial to kill the weeds and protect the corn plants. The flame reflection plate 3.4 has two, corresponding to the row position on both sides of the corn plants. Moreover, the flame reflection plate 3.4 has a downward lateral protection side on both sides, which encloses the flame in the flame reflection plate 3.4, gathers the heat, and improves the weeding efficiency.

[0033] The laser weeding device 5 is installed at the bottom of the mobile platform 1 and used to emit laser to kill weeds. Specifically, the laser weeding device includes a CO2 laser 4.1, a controller, a galvanometer system and an F-Theta lens, which can better find the meristem of the target weed given by the visual system during the movement, and kill the weeds by using laser beam to irradiate the meristem. After the weed identification device 2 detects the weed information, the information such as the position of the weed is transmitted to the host computer platform 6.1, and the laser weeding device 5 is controlled to perform weeding operation.

[0034] The height limiting device 4 includes a sink wheel 5.1, an elastic telescopic rod 5.2 and a sink wheel support 5.3. Specifically, the sink wheel 5.1 adopts a wheel with a large area to reduce the shaking of the flame weeding device 3 and keep the flame weeding device stable. The sink wheel 5.1 is connected with the mobile platform 1 through the sink wheel support 5.2, and the sink wheel 5.1 is connected with the bottom end of the sink wheel support 5.2. The sink wheel support 5.3 has the elastic telescopic rod 5.2 connected with the mobile platform 1, and the up and down movement of the sink wheel 5.1 is ensured through the elastic telescopic rod 5.2, so that the sink wheel 5.1 can always be in contact with the ground under the condition of uneven field, and the stable weeding operation is maintained. The distance between the flame reflection plate 3.4 and the ground is controlled by the height limiting device 4, so as to ensure that the flame reflection plate 3.4 is located within a reasonable range above the ground, and the plants are protected from being damaged by the flame without harming the plants.

[0035] The upper computer 6 has two, the upper computer 6 includes the upper computer one 6.1 and the upper computer two 6.2. The upper computer one 6.1 controls the execution of the laser weeding task. In detail, the upper computer one 6.1 transmits the information such as the position of the weeds into the upper computer one 6.1 by receiving the inter-plant weed identification information of the inter-plant weed identification depth camera 2.1 at the front end of the mobile platform 1, finds the meristem of the weeds, and controls the operation of the laser weeding device 5. The upper computer two 6.2 controls the execution of the flame weeding task. In detail, the upper computer two 6.2 controls the strength of the flame by receiving the inter-row weed identification information of the inter-row weed identification depth camera 2.2 at the front end of the mobile platform 1 and identifying the weed density information in the area through the upper computer two 6.2.

[0036] Embodiment two

[0037] Further, the flame and laser coupled weeding robot with weed density detection function of the present application adopts autonomous navigation technology, in which the radar 7 and the depth camera 8 are the core components of the navigation system. The radar 7 is installed at the center of the front end of the mobile platform 1 and is fixed at a high position using a vertical rod to ensure that the received signals are not disturbed. Its main function is to obtain the position information of the mobile platform 1 in real time. Based on these information, the driving wheels are controlled to move the mobile platform 1; the depth camera 8 is used to collect the image information of the crops in the field in the forward direction of the mobile platform 1. By processing these images, the system can identify the crop route and extract the navigation route. The navigation recognition algorithm will correct the deviation according to the extracted route to ensure that the mobile platform 1 moves along the correct route. The advantage is to realize the precise control of the mobile platform 1 and ensure its smooth operation in complex field environment. The combination of radar 7 and depth camera 8 enables the system to work reliably under different conditions, providing strong support for autonomous navigation.

[0038] Embodiment three

[0039] Further, the weed density level data set obtained by the semi-supervised data labeling method is used to train the Deeplabv3+ model to predict the weed density level of the target image. This can reduce the amount of calculation, reduce the data labeling time, improve the real-time performance of the operation, and greatly improve the speed of the weeding operation.

[0040] In detail, the steps and methods for predicting the weed density level in corn fields using the above-mentioned Deeplabv3+ are as follows:

[0041] Step 1: Obtain video information during the forward movement of the mobile platform in the field, and obtain high-definition pictures of weeds and corn plants using video key frame extraction technology.

[0042] Specifically, data preprocessing is performed to adjust the size of all images to 512x512 pixels. The sample capacity of the training set is expanded through data augmentation, using methods such as random rotation, random scaling, random translation, and image flipping. This ensures the integrity of the information while ultimately improving the accuracy and generalization ability of the model.

[0043] Step 2: Implement the weed density level dataset annotation. First, use the annotated crop dataset to train the Deeplabv3+ model to achieve crop segmentation. Second, use Otsu threshold segmentation and vegetation extraction color index (CIVE) to achieve unsupervised vegetation segmentation. The CIVE formula is:

[0044] CIVE = 0.441R - 0.811G + 0.385B + 18.78745 (1)

[0045] In the formula, R, G, and B represent the pixel values of the red, green, and blue channels of a pixel in the image.

[0046] Then, according to the crop segmentation result, remove the crop area from the obtained vegetation segmentation result, and consider the remaining green vegetation as weeds. After that, perform morphological operations such as dilation and erosion on the obtained weed mask image. Calculate the ratio of the number of weed pixels to the total number of pixels in the image to obtain the weed density, and use threshold to divide the weed density level of each picture. Use the labelme data labeling software to label the weed density level of the whole image.

[0047] Define the flame intensity level: According to actual needs, define three flame intensity levels, such as low, medium, and high.

[0048] Determine the threshold: According to the weed density value obtained by the deep learning model, determine three thresholds to divide the weed density value into three levels. For example, if the weed density value is between 0 and 0.25, it is classified as a low flame intensity level; if the weed density value is between 0.25 and 0.6, it is classified as a medium flame intensity level; if the weed density value is between 0.6 and 1, it is classified as a high flame intensity level.

[0049] Control the flame intensity: According to the level of the weed density value, control the flame intensity to achieve the purpose of killing weeds. For example, if the weed density value belongs to the low level, a weaker flame intensity can be used for control; if the weed density value belongs to the high level, a stronger flame intensity needs to be used for control.

[0050] Step 3: Divide the weed density level dataset obtained in step 2 into training set, validation set, and test set. Use the training set to train Deeplabv3+.

[0051] Step 4: After the training is completed, the model is evaluated and optimized. The model is tested using the validation dataset, and the accuracy, precision, recall, and other indicators of the model are calculated. The cross-entropy loss function is used to optimize the model. The trained Deeplabv3+ model is evaluated using the test set. Performance indicators such as IoU (Intersection over Union) or Dice coefficient can be used to measure the segmentation quality of the model.

[0052] Step 5: The model that meets the requirements is applied to actual field operations. The test image is input into the model, and the model will predict the weed density level of the entire image to guide the variable spray fire weeding operation.

[0053] In order to more clearly describe the spatial position relationship between the mobile platform and various elements on the mobile platform, the article uses terms related to spatial direction and position, such as "bottom", "below" and "above". The purpose of these terms is to more accurately describe the working space and logical operation sequence of various devices. Using these orientation terms can help readers better understand the spatial position relationship between various elements on the mobile platform, so as to better understand the structure and operation principle of the robot. At the same time, these terms can also make the article more rigorous and accurate, so that readers can better understand the information described in the article.

[0054] Therefore, the use of these orientation terms can make the article more clear, accurate and easy to understand, and also help readers better understand the structure and operation principle of the mobile platform. Through the above description of the design of the laser and flame combined weeding platform, those skilled in the art can clearly understand the weeding method under different conditions in the real field, and those skilled in the art can complete the weeding operation through computer software instructions.

[0055] The above invention is described by way of example, and it should be noted that any simple modification, modification or other equivalent replacement that does not deviate from the core of the invention and does not require creative labor can fall within the protection scope of the invention.

Claims

1. A flame-and-laser coupled weeding robot with weed density detection function, employing a weed density-based grading strategy and an adaptive flame-and-laser coupling method for weeding, characterized in that: By calculating the density of weeds and classifying them into three levels—low, medium, and high—the density level of weeds in each image is used as a label to train Deeplabv3+, resulting in a model that can quickly identify the density level within a region. The robot includes: a mobile platform, a weed identification device, a flame weeding device, a laser weeding device, a height limiting device, and a host computer. The weed identification device is installed at the bottom of the front end of the mobile platform. The weed identification device includes a weed identification depth camera between plants and a weed identification depth camera between rows. The weed identification depth camera between plants identifies weeds in the crop rows and obtains the location information of the weeds as the mobile platform moves forward. The weed identification depth camera between rows obtains the density information of the weeds between rows as the mobile platform moves forward. The flame weeding device includes a gas supply hose, a connecting rod, a flame nozzle, a flame reflector, a gas flow meter, and a combustible gas container. The combustible gas container is fixed above the mobile platform. The gas outlet of the combustible gas container is connected to the flame nozzle of the flame-spraying device through the hose, and the flow rate is controlled by the gas flow meter. The flame nozzle is inserted into the flame reflector. When the flame nozzle is above the weed area, the flame-spraying operation is performed. The laser weeding device includes a laser actuator that can rotate left and right to easily treat weeds at different locations between plants without requiring the laser actuator to move on a guide rail. The laser weeding device consists of five parts, including a computer, a CO2 laser, a controller, a galvanometer system, and an F-Theta lens, which facilitates laser focusing on the weed meristematic tissue and improves weed control efficiency. The height limiting device is used to control the distance between the flame reflector and the field ground. Because the field ground is uneven, the height limiting device is used to adjust the distance to prevent the flame nozzle from colliding with the ground, maximize the damage of the flame to weeds, and reduce energy waste. The host computer is used to control the operation of the laser weeding device and the flame weeding device. After obtaining the position of the weeds between the plants, it controls the laser weeding method to carry out the operation; after obtaining the density of the weeds between the rows, it controls the intensity of the flame to weed, thereby improving the real-time performance and efficiency of the operation.

2. The flame-and-laser coupled weeding robot with weed density detection function according to claim 1, characterized in that: As the mobile platform moves, weeds and crops pass beneath it. Two weed detection depth cameras at the front of the mobile platform identify weeds between rows and between plants, respectively, and transmit the density information of weeds between rows and the meristematic information of weeds between plants to the host computer for weeding operations.

3. The flame-and-laser coupled weeding robot with weed density detection function according to claim 1, characterized in that: The combustible gas container is connected to the flame nozzle via a hose. The flame nozzle is installed below the mobile platform via a connecting rod. The gas flow rate is controlled by a flow meter, which in turn controls the flame intensity. A flame deflector is installed below the flame nozzle. The flame nozzle cooperates with the flame deflector, extending the flame nozzle into the flame deflector to isolate the emitted flame within the flame deflector. This increases the contact area between the flame and the weeds, raises the temperature in the area, which is beneficial for killing the weeds and also protects the corn plants. Furthermore, the flame deflector has side-down protective sides on both sides, which envelop the flame within the flame deflector, concentrating the heat and improving weeding efficiency.

4. A flame-and-laser coupled weeding robot with weed density detection function according to claim 1, characterized in that, Height limiting devices are used on both sides of the flame nozzle. The height limiting devices include a sinker wheel, an elastic telescopic rod, and a sinker wheel bracket. One end of the elastic telescopic rod is connected to the moving platform, and the other end is connected to the sinker wheel bracket. A sinker wheel is installed at the bottom of the sinker wheel bracket. The elasticity of the elastic telescopic rod provides a downward elastic force to the sinker wheel bracket. The height limiting devices ensure the distance between the flame nozzle and the ground.

5. A flame-and-laser coupled weeding robot with weed density detection function according to claim 1, characterized in that: There are two sets of different wheels on both sides of the mobile platform. The front wheels have a large cross-sectional area and a lower coefficient of friction to adapt to loose soil sections and furrows in the field and prevent sinking. The rear wheels are the drive wheels and have a high coefficient of friction.

Images