Weed feature recognition and weeding tool switching system and switching method

Abstract

The invention discloses a weed feature recognition and weeding cutter switching system and method, and the system comprises a multi-source sensing fusion sensing layer, a deep learning algorithm decision-making layer and a cutter replacement execution layer, and a sensing end comprises a multispectral camera, a high-definition camera, a depth camera and an environment sensor. Features of weeds can be acquired in multiple dimensions and positioning can be realized, and a deep learning algorithm realizes input of a multi-source sensing fusion spliced image and output of a cutter correspondingly used by the weeds and a specific position of the cutter. According to the system, multi-dimensional characteristics of weeds are collected through a sensing layer multi-source sensor, a decision-making layer adopts a YOLOv8s algorithm to perform data fusion and weight distribution, a tool interaction model is dynamically constructed in combination with a DDPG algorithm, finally, an execution layer intelligently switches tools according to a decision-making instruction and generates an optimal weeding path, full-process automatic precise operation is achieved, and the efficiency is improved. The efficiency is improved, the adaptability is enhanced, and the practical value is great.

Description

Technical Field

[0001] The present invention relates to the fields of electronic technology, sensor technology and computer technology, and in particular to a weed feature recognition and weeding tool switching system and switching method. Background Art

[0002] With the continuous development of the times, intelligent equipment has been widely used in various industries. However, weeding operations in agriculture, forestry, and grasslands often cause accidents. The most typical example is the collision between an aircraft and a weeder during weeding operations on airport grasslands due to line of sight problems and improper weeding timing. This accident can directly cause deaths and huge economic losses. At the same time, manual weeding often faces problems such as driver fatigue and manpower limitations, resulting in low weeding efficiency. Therefore, more and more people are focusing on intelligent weeding equipment.

[0003] Existing intelligent weeding systems typically use a single tool to complete weeding tasks along a predetermined path. The inventors have discovered that existing intelligent weeding systems mostly use a single tool to complete weeding tasks. This design and operating mode offers certain economic advantages for simple pastures with a single weed type, but it often proves ineffective in complex conditions. A single tool cannot handle all weeding tasks, or a specific tool may be less efficient than a more suitable tool in an unsuitable weeding task. Summary of the Invention

[0004] The purpose of the present invention is to provide a weed feature recognition and weeding tool switching system and switching method to solve the problems of existing intelligent equipment weeding systems having a small scope of application and low efficiency in complex working conditions due to the use of a single tool.

[0005] To achieve the above objectives, the present invention provides the following technical solutions: a weed feature recognition and weeding tool switching system, comprising a perception layer for multi-source sensor fusion, a decision-making layer for deep learning algorithms, and an execution layer for tool replacement and path planning. The input of the decision-making layer is connected to the output of the perception layer, and the execution layer is connected to the decision-making layer, thereby realizing the connection between the modules and the transmission of data.

[0006] The perception layer includes a multispectral camera, a high-definition camera, a depth camera, and an environmental sensor. The multi-source sensing instrument of the perception layer is external to the mower itself and is used to obtain the multispectral characteristics, morphological characteristics, three-dimensional structure and environmental positioning data of vegetation in multiple dimensions, obtain the characteristics of weeds and locate them, and upload them to the computer;

[0007] The decision layer includes a feature fusion module based on the YOLOv8s algorithm and a tool decision module based on the DDPG algorithm, which are used to optimize, improve, and stitch multi-source sensor fusion images. The YOLOv8s algorithm uses multiple feature inputs and makes weight adjustments to provide a data set for subsequent tool control optimization. The decision layer is trained and matched using the data set, and uses the DDPG algorithm to build an interactive physical model of the tool and weeds to match the corresponding tool.

[0008] The execution layer includes multiple sets of switchable tools and a control module based on a path planning algorithm. The execution layer matches and replaces the tools according to the selected terrain and weed positions, and adjusts and plans the optimal weeding path.

[0009] Preferably, the multispectral camera is equipped with red edge band and near infrared band sensors to capture the red edge and near infrared signals of weeds to distinguish the types and corresponding states of weeds; the high-definition camera is used to capture the morphological characteristics of weeds, including the thickness, width and narrowness of leaves; the depth camera adopts three-dimensional imaging technology to obtain the overall three-dimensional structure of weeds; the environmental sensor includes a GPS positioning module and a light sensor for positioning and light compensation of collected information.

[0010] Preferably, the decision layer includes a YOLOv8s algorithm to achieve weight distribution and integration of multiple features. The decision layer specifically provides quadruple data through multi-source sensing instruments; and adjusts the input layer convolution so that the model supports four-channel input, wherein the features of some collected data are hidden deeper, and a frequency division scattering scheme is adopted to improve it, extracting signal features of different frequency bands and changing them into input, and finally completing the output through weight distribution and fitting of the main model.

[0011] Preferably, the feature processing optimization of multi-source perception data includes signal input, weight allocation, and target signal segmentation optimization. The weight allocation is to add a channel attention mechanism, which is added to the shallow layer and Neck part of the Backbone to increase the weight of the key signal band and achieve adaptive signal enhancement. The specific steps include:

[0012] First, the input data is converted into three sets of learnable weight matrices, divided into Q, K, and V. The similarity between Q and K is then calculated and recorded as the attention score. The attention score is then normalized using the Softmax function and finally multiplied by V and the weighted sum is obtained as the output. The target signal segmentation optimization is to collect the signal frequency analysis to extract the characteristics of the signal at different frequencies, thereby improving the small target detection capability.

[0013] In the perception layer and decision layer systems, the implementation of the YOLOv8s algorithm function depends on the convolution decomposition, attention mechanism weight calculation and loss function, including:

[0014] Convolution calculation amount: O(K 2 ·C in ·C out ·H·W)

[0015] Channel weight calculation: wc=σ(W2·ReLU(W1·GAP(F c ))

[0016] Loss function: FL(p t )=-α t (1-p t ) γ log(p t ))

[0017] Among them, K is the convolution kernel size, C in / C out is the number of input / output channels, H and W constitute the size of the feature map, GAP represents the global average pooling, W1 and W2 are the intervals of compression ratio limits, and for the loss function, there is p t is the predicted probability of the model, α t The category weight is set to 0.5, and γ is the focusing parameter, which is set to 1.5.

[0018] Preferably, the core tool of the decision layer adopts the DDPG algorithm to construct an interactive physical model of the tool and weeds. The core mechanism of the DDPG algorithm is the reward function; the characteristic information of the weeds, the current speed and type of the tool are used as input to interact with the actual weeding effect to complete the construction of the physical model, and finally realize the solution of the output action, including the speed and the type of tool used. The specific implementation of the reward function is to use weeding efficiency as a reward, and energy consumption and tool jamming during weeding operations as penalties. The final value is calculated using the function to determine whether the conditions such as the use of the tool and the speed are suitable.

[0019] Preferably, the base value of the reward function mechanism is set to 10, effective weeding is set as a positive reward, and the value is increased, and ineffectiveness or a situation similar to a tool stuck is set as a negative reward, and the value is reduced accordingly, including the reward function:

[0020]

[0021] in, Indicates the amount of weeds removed in a certain period of time, I t represents the actual current of the motor at time t, Represents the rated current of the motor. The formula is divided into three parts, representing the weeding effect, energy consumption penalty, and jamming penalty.

[0022] Preferably, the execution layer tool set includes a serrated knife, a shovel knife and a double-action blade, which are respectively adapted to the physical characteristics of different types of weeds.

[0023] Preferably, the execution layer path optimization adopts the A* path planning optimization algorithm, which divides the collected grassland data into grids, constructs a cost map, and then completes the planning of the weeding priority path through the designed cost function. The cost function includes moving distance, weed type priority and soil parameters. The weeding work is generally from near to far, from important to less important, and from suitable planting land to less suitable planting land. The short weeding distance, priority of weeding type and fertility of soil are set to high weights to complete the planning of the weeding path. The cost function includes:

[0024]

[0025] Where D(n) is the distance from the node to the target point, is the weed density, H(n) is the land hardness data, w d Represents the weight coefficient of distance, w p Represents the weight coefficient of weed density, w h The weight coefficient representing soil hardness.

[0026] The present invention also provides a weed feature recognition and weeding tool switching method, comprising the following steps:

[0027] S1. First, the multi-source sensing instruments in the perception layer capture the characteristic signals of the four weeds and upload them to the computer decision layer;

[0028] After receiving this information, the decision layer analyzes and processes it, using the YOLOv8s algorithm to divide and scatter the quadruple data and assign weights, ultimately enhancing the data. Furthermore, the DDPG algorithm is used at the decision layer to build a physical model and perform tool selection, training, and analysis.

[0029] S3: Submit the information about the weed location and the selected tool to the execution layer. After receiving the information, the execution layer completes the final tool replacement and plans a matching path through the A* algorithm.

[0030] Compared with the prior art, the present invention has the following beneficial effects:

[0031] 1. The present invention uses electronic, sensor, and computer technologies to extract and analyze weed features, match tools, and plan paths. This solves the complex weeding tasks of multiple weed species, thereby resolving existing problems of low efficiency and poor adaptability, and possesses great practical value.

[0032] 2. In the present invention, through multi-source sensor fusion technology, the system can obtain weed characteristics from multiple dimensions such as spectrum, morphology, three-dimensional structure and environment, and combine with deep learning algorithms to achieve high-precision identification and positioning, effectively distinguishing weed types and states; at the same time, the dynamic decision-making mechanism based on reinforcement learning can autonomously optimize tool selection and operating parameters according to the real-time environment and tool status, significantly enhancing the system's adaptability to different weeds and complex farmland scenarios.

[0033] 3. This invention utilizes an intelligent switching mechanism for multiple tool types to flexibly match the physical characteristics of different weeds. Combined with a path planning algorithm based on environmental data, it dynamically generates an optimal weeding path. This collaborative mechanism not only reduces repetitive work and energy consumption, but also avoids equipment damage caused by tool misoperation, ultimately improving overall weeding efficiency and resource utilization. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0035] In the attached figure:

[0036] Figure 1 is a schematic diagram of the weed feature recognition and weeding tool switching system of the present invention;

[0037] Figure 2 It is a flow chart of the weed feature recognition and weeding tool switching method of the present invention. DETAILED DESCRIPTION

[0038] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0039] Example: Figure 1 As shown, an embodiment of the present invention provides a weed feature recognition and weeding tool switching system for calculating suitable tools and reasonable weeding paths according to the weed features and distribution decisions in agricultural, forestry and grassland fields.

[0040] In this embodiment, the weed species identification and weeding tool switching system includes a multi-source sensor fusion perception layer, a deep learning algorithm decision-making layer, and an execution layer for tool replacement and path planning. The multi-source sensor fusion layer is composed of a multispectral camera, a high-definition camera, a depth camera, and environmental sensors. The perception layer captures information across four dimensions and analyzes and fits this multi-dimensional information to improve the reliability of the dataset. The multi-source sensor instrument is external to the mower body. The input of the decision-making layer is connected to the output of the perception layer via a Wi-Fi module, and the decision-making layer is connected to the execution layer.

[0041] The three types of tools on the execution layer include a serrated knife, a shovel knife and a double-action blade, which are used for grass weeds, broadleaf weeds and vine weeds respectively. Improper use of the tools may make weed cleaning difficult or even cause the tools to get stuck.

[0042] Optionally, the specific content of the preset conditions is not restricted. The perception layer captures the status, morphological features, three-dimensional structure and other features of the vegetation through a multispectral camera, a high-definition camera, a depth camera and an environmental sensor and uploads them to a computer. The optimization, improvement and splicing of the multi-source sensor fusion image are input through the YOLOv8s algorithm and weight adjustment is made to provide a more reliable data set for subsequent tool control optimization. The decision layer matches the most suitable tool through training on the data set and uses the DDPG algorithm to build an interactive physical model of the tool and weeds to match the most suitable tool. The execution layer plans the most suitable weeding path according to the selected terrain and the above-mentioned weed position and tool matching information.

[0043] The system's perception layer consists of a multispectral camera, a high-definition camera, a depth camera, and environmental sensors. The multispectral camera captures the red edge and near-infrared signals of weeds to distinguish their type and state. The high-definition camera directly captures weed morphological features, such as blade thickness, width, and length. The depth camera captures the overall three-dimensional structure of the weeds. The environmental sensors include GPS and light sensing for positioning and light compensation of the collected information. The perception layer provides four layers of feature information to the decision layer, which incorporates the YOLOv8s algorithm to weight and integrate multiple features. The specific implementation involves preparing a dataset, using a multi-source sensor fusion instrument to provide four layers of data; data processing, adjusting the input layer convolution to support four-channel input. Some of the collected data has relatively hidden features, so a frequency scattering scheme is employed to improve this, extracting signal features from different frequency bands and converting them into input. Finally, the output is achieved through weight allocation and fitting of the main model.

[0044] In the weed feature recognition and weeding tool switching system of the present invention, the integrated optimization algorithm mainly consists of signal input, weight assignment, and target signal segmentation optimization. The weight assignment is achieved by adding a channel attention mechanism to the shallow layer and neck of the Backbone, increasing the weights of key signal bands and achieving adaptive signal enhancement. The specific implementation steps are to first convert the input data into three sets of learnable weight matrices, divided into Q, K, and V, then calculate the similarity between Q and K as the attention score, and then normalize the attention score through the Softmax function, and finally multiply it with V and take the weighted sum to obtain the output. The target signal segmentation optimization is to collect the signal through frequency division analysis to extract the features exhibited by the signal at different frequencies, improving the detection ability of small targets.

[0045] In the above-mentioned perception layer and decision layer systems, the implementation of the YOLOv8s algorithm function depends on the convolution decomposition, attention mechanism weight calculation and loss function.

[0046] Convolution calculation amount: O(K 2 ·C in ·C out ·H·W)

[0047] Channel weight calculation: wc=σ(W2·ReLU(W1·GAP(F c ))

[0048] Loss function: FL(p t )=-α t (1-p t ) γ log(p t )

[0049] Where K is the convolution kernel size, C in / C out is the number of input / output channels, H and W constitute the size of the feature map, GAP represents the global average pooling, W1 and W2 are the intervals of compression ratio limits, and for the loss function, there is p t is the predicted probability of the model, α t The category weight is set to 0.5, and γ is the focusing parameter and is set to 1.5.

[0050] Through the above design, the present invention achieves: analyzing and integrating the signals collected from multiple dimensions of the perception layer through the YOLOv8s algorithm, highlighting the key signals through the idea of ​​weight, weakening the irrelevant signals, and outputting the final multi-dimensional data fitting as support for the tool physical platform model cooperation experiment.

[0051] In the weed feature recognition and weeding tool switching system of the present invention, the decision-making layer's core tool adaptation utilizes the DDPG algorithm, constructing a physical model of the interaction between tools and weeds. The core mechanism of the DDPG algorithm is a reward function. Specifically, the physical model is constructed by using the weed feature information, current tool speed, and type as inputs, interacting with the actual weeding effect. This process ultimately results in the output action, including the speed and tool type used. The reward function is implemented by using weeding efficiency as a reward and energy consumption and tool jamming during weeding operations as penalties. The function calculates the final value, which is then used to determine whether the tool usage, speed, and other conditions are suitable.

[0052] In this embodiment of the present invention, the reward function mechanism has a base value of 10. Effective weeding is rewarded positively, with the value increasing. Ineffective weeding or a tool-jamming condition is rewarded negatively, with the value decreasing accordingly. DDPG combines deterministic policy gradients and deep Q-networks, with its core design being a reward mechanism.

[0053] Reward function:

[0054] in, Indicates the amount of weeds removed in a certain period of time, I t represents the actual current of the motor at time t, Represents the rated current of the motor. The formula is divided into three parts, representing the weeding effect, energy consumption penalty, and jamming penalty.

[0055] Through the above design, the present invention achieves: building a physical interaction model between weeds and cutting tools through a dataset of weed features, training and calculating the types of cutting tools corresponding to different features, and locating the specific positions of weeds. Finally, uploading the results to the execution layer shows the weed positions and the corresponding cutting tools.

[0056] In the weed feature recognition and weeding tool switching system of the present invention, the execution layer path optimization adopts the A* path planning optimization algorithm. The core of the algorithm implementation is the cost function enhancement. Optionally, the priority of weed removal in weeding work can be preset, and the path of reasonable planning task can be enhanced by calculating the condition weights. The specific implementation steps include rasterizing the collected grassland data, constructing a cost map, and then completing the planning of the weeding priority path through the designed cost function. The cost function contains three items: distance, weed type, and soil hardness. Weeding work is generally from near to far, from important to less important, and from suitable planting land to less suitable planting land. Therefore, the close weeding distance, priority of weeding type and fertility of soil are set to high weights to complete the planning of the weeding path. The core mechanism of this algorithm is the cost function.

[0057] Cost function:

[0058] Where D(n) is the distance from the node to the target point, is the weed density, H(n) is the land hardness data, w d Represents the weight coefficient of distance, w p Represents the weight coefficient of weed density, w h The weight coefficient representing soil hardness.

[0059] like Figure 2As shown, the present invention also provides a weed feature recognition and weeding tool switching method. The overall execution steps of the system include: first, the perception layer uses multi-source sensing instruments to capture four weed feature signals and upload them to the computer decision layer; after receiving this information, the decision layer analyzes and processes it, and uses the YOLOv8s algorithm to divide and scatter the four-fold data and complete weight distribution, ultimately achieving data enhancement; at the same time, the decision layer also uses the DDPG algorithm to implement the construction of a physical model and tool selection training and analysis; finally, the weed location and selected tool information are submitted to the execution layer. After receiving the information, the execution layer completes the final tool replacement work and plans a matching path using the A* algorithm.

[0060] In summary, the present invention provides a weed feature recognition and weeding tool switching system and switching method. By adopting electronic technology, sensor technology, and computer technology, weed feature extraction and analysis, tool matching, and path planning are completed. This can cope with the complex weeding task of multiple weed species, thereby solving the existing problems of low efficiency and low adaptability, and has great practical value.

[0061] Finally, it should be noted that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art will be able to modify the technical solutions described in the aforementioned embodiments or substitute equivalents for some of the technical features. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A weed feature recognition and weeding tool switching system, characterized by: It includes a perception layer for multi-source sensor fusion, a decision-making layer for deep learning algorithms, and an execution layer for tool replacement and path planning. The input of the decision-making layer is connected to the output of the perception layer, and the execution layer is connected to the decision-making layer, thereby realizing the connection between each module and the transmission of data; The perception layer includes a multispectral camera, a high-definition camera, a depth camera, and an environmental sensor. The multi-source sensing instrument of the perception layer is external to the mower itself and is used to obtain the multispectral characteristics, morphological characteristics, three-dimensional structure and environmental positioning data of vegetation in multiple dimensions, obtain the characteristics of weeds and locate them, and upload them to the computer; The decision layer includes a feature fusion module based on the YOLOv8s algorithm and a tool decision module based on the DDPG algorithm, which are used to optimize, improve, and stitch multi-source sensor fusion images. The YOLOv8s algorithm uses multiple feature inputs and makes weight adjustments to provide a data set for subsequent tool control optimization. The decision layer is trained and matched using the data set, and uses the DDPG algorithm to build an interactive physical model of the tool and weeds to match the corresponding tool. The execution layer includes multiple sets of switchable tools and a control module based on a path planning algorithm. The execution layer matches and replaces the tools according to the selected terrain and weed positions, and adjusts and plans the optimal weeding path.

2. The weed feature recognition and weeding tool switching system according to claim 1, characterized in that: The multispectral camera is equipped with red-edge and near-infrared band sensors to capture the red-edge and near-infrared signals of weeds to distinguish the type and corresponding status of the weeds; the high-definition camera is used to capture the morphological characteristics of the weeds, including the thickness, width and narrowness of the leaves; the depth camera uses three-dimensional imaging technology to obtain the overall three-dimensional structure of the weeds; the environmental sensor includes a GPS positioning module and a light sensor for positioning and light compensation of the collected information.

3. The weed feature recognition and weeding tool switching system according to claim 2, characterized in that: The decision layer includes the YOLOv8s algorithm to achieve weight distribution and integration of multiple features. The decision layer specifically provides quadruple data through multi-source sensor instruments; and adjusts the input layer convolution so that the model supports four-channel input, and finally completes the output through weight distribution and fitting of the main model.

4. The weed feature recognition and weeding tool switching system according to claim 3, characterized in that: The feature processing optimization of multi-source perception data includes signal input, weight allocation, and target signal segmentation optimization. The weight allocation is to add a channel attention mechanism, which is added to the shallow layer and Neck part of Backbone. The specific steps include: First, the input data is converted into three sets of weight matrices, divided into Q, K, and V. The similarity between Q and K is then calculated and recorded as the attention score. The attention score is then normalized using the Softmax function and finally multiplied by V and the weighted sum is obtained as the output. The target signal segmentation optimization is to collect the signal frequency analysis to extract the characteristics of the signal at different frequencies. In the perception layer and decision layer systems, the implementation of the YOLOv8s algorithm function depends on the convolution decomposition, attention mechanism weight calculation and loss function, including: Convolution calculation amount: O(K 2 ·C in ·C out ·H·W); Channel weight calculation: wc=σ(W2·ReLU(W1·GAP(F c )); Loss function: FL(p t )=-α t (1-p t ) γ log(p t )); Among them, K is the convolution kernel size, C in / C out is the number of input / output channels, H and W constitute the size of the feature map, GAP represents the global average pooling, W1 and W2 are the intervals of compression ratio limits, and for the loss function, there is p t is the predicted probability of the model, α t The category weight is set to 0.5, and γ is the focusing parameter, which is set to 1.

5.

5. The weed feature recognition and weeding tool switching system according to claim 3, characterized in that: The core tool of the decision-making layer uses the DDPG algorithm to construct a physical model of the interaction between the tool and weeds. The core mechanism of the DDPG algorithm is the reward function. The characteristic information of the weeds, the current tool speed, and type are used as input to interact with the actual weeding effect to complete the construction of the physical model and ultimately solve the output action, including the speed and tool type used. The specific implementation of the reward function is to use weeding efficiency as a reward and energy consumption and tool jamming during weeding operations as penalties. The final value is calculated using the function to determine whether the conditions such as tool use and speed are suitable.

6. The weed feature recognition and weeding tool switching system according to claim 5, characterized in that: The base value of the reward function mechanism is set to 10. Effective weeding is set as a positive reward, and the value is increased. Ineffective weeding or a situation such as a tool stuck occurs is set as a negative reward, and the value is reduced accordingly. The reward function includes: ； in, Indicates the amount of weeds removed in a certain period of time, I t represents the actual current of the motor at time t, Represents the rated current of the motor. The formula is divided into three parts, representing the weeding effect, energy consumption penalty, and jamming penalty.

7. The weed feature recognition and weeding tool switching system according to claim 1, characterized in that: The execution layer tool set includes a serrated knife, a shovel knife and a double-action blade, which are respectively adapted to the physical characteristics of different types of weeds.

8. The weed feature recognition and weeding tool switching system according to claim 7, characterized in that: The execution layer path optimization adopts the A* path planning optimization algorithm. By rasterizing the collected grassland data, a cost map is constructed. Then, the weeding priority path is planned using a designed cost function. The cost function includes moving distance, weed type priority, and soil parameters. Short weeding distance, priority of weeding type, and fertile soil are set as high weights to complete the weeding path planning. The cost function includes: ； Where D(n) is the distance from the node to the target point, is the weed density, H(n) is the land hardness data, w d Represents the weight coefficient of distance, w p Represents the weight coefficient of weed density, w h The weight coefficient representing soil hardness.

9. A method for weed feature recognition and weeding tool switching, characterized in that: The following steps are involved: S1. First, the multi-source sensing instruments in the perception layer capture the characteristic signals of the four weeds and upload them to the computer decision layer; S2, the decision layer, analyzes and processes the received information. It uses the YOLOv8s algorithm to divide and scatter the quadruple data and assign weights, ultimately enhancing the data. Simultaneously, the DDPG algorithm is used at the decision layer to build a physical model and perform tool selection, training, and analysis. S3: Submit the information about the weed location and the selected tool to the execution layer. After receiving the information, the execution layer completes the final tool replacement and plans a matching path through the A* algorithm.

Images