Sensing system and method for identifying invasive plants

The sensing system efficiently detects invasive plants by using a CCD camera and microcontroller for on-board processing in the HSV model, addressing high processing capacity and signal transmission issues, resulting in a cost-effective and efficient solution for invasive plant identification and control.

WO2024234068A1PCT designated stage expired Publication Date: 2024-11-21ZASSO BRASIL IND E COMERCIO DE MAQUINAS LTDA +2

Patent Information

Application Number
PCT/BR2024/050019
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-12
Filing Date
2024-01-23
Publication Date
2024-11-21

AI Technical Summary

Technical Problem

Existing sensing systems for invasive plants require high processing capacity and are prone to signal transmission failures, making them expensive and inefficient.

Method used

A sensing system utilizing a CCD RGB camera connected to a microcontroller via DCMI for on-board processing in the HSV color model, performing image acquisition and processing between 10-15 FPS, and sending binary signals via RS485 to activate machinery, with image processing involving conversion from RGB565 to RGB24, convolution, and thresholding to efficiently detect green areas.

Benefits of technology

The system achieves efficient and cost-effective invasive plant detection with reduced processing complexity and minimized signal interference, enabling precise and timely action on invasive plants.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure IMGF000007_0001
    Figure IMGF000007_0001
  • Figure IMGF000008_0001
    Figure IMGF000008_0001
  • Figure IMGF000008_0002
    Figure IMGF000008_0002
Patent Text Reader

Abstract

The sensing system for identifying invasive plants comprises an RGB CCD camera for image acquisition, connected to a microcontroller via a DCMI interface for on-board processing of the image data and detection of areas corresponding to the colour green in the HSV colour model. The device acquires and processes the image, and when it detects a green-coloured area, it sends a binary signal via an RS485 interface to the machine to which it is coupled for it to perform the programmed action for the predetermined duration.
Need to check novelty before this filing date? Find Prior Art

Description

SENSING SYSTEM AND METHOD FOR IDENTIFYING INVASIVE PLANTS Technical sector

[0001] This invention pertains to the agricultural technology sector, specifically relating to a sensing system and method for identifying invasive plants. This system processes images captured by a camera, detecting areas that correspond to the color green and, consequently, represent invasive plants. When the presence of an invasive plant is identified in land and plantations, the system sends a signal to the machinery to which it is connected, identifying the plant's presence and triggering it to perform the programmed action. For example, in the case of electric weeding, it delivers an electric discharge to the invasive plant. State of the art

[0002] Sensing invasive plants in crops is a technique that has become increasingly important in modern agriculture. This is because invasive plants can harm crop development by competing for nutrients, water, and sunlight, as well as harboring pests and diseases.

[0003] Using advanced technologies such as satellite imagery, drones, sensors, and data processing algorithms, it is possible to detect the presence of invasive plants quickly and efficiently, enabling the adoption of integrated management measures to minimize their negative effects on agricultural production. Sensing also helps reduce the use of weed control techniques, as it allows for the precise identification of infested areas, avoiding unnecessary use of these techniques and reducing environmental impacts.

[0004] Patent document CN 103348964 describes a smart weeding device and its method of use. The system describes a weeding system that captures terrain images and performs image definition processing to define the plant via a cloud, using radio frequency for data transmission. A limitation of this type of device is the possibility of signal transmission failure between the system's devices. Furthermore, the image processing of this invention performs convolution directly after RGB image capture. requiring high processor capacity and more processing time.

[0005] Patent document CN 105469020 describes a system equipped with weed sensing and a spraying device. The image obtained by the system is processed using the Super-Green Model, which uses an enhancement equation that assigns a weight of two to the green band of the pixel, subtracting the values ​​of the other two remaining bands, each of which have a weight of one. In other words, the grayscale pixel is given by 2G-RB. The grayscale image is passed through the binarization filter using the threshold established by the OTSU Method for defining the exclusion threshold. This invention has the same limitation as the previous one—the need for high processing capacity—and does not perform convolution in its image processing, yielding different results.

[0006] US patent document 2018 168141 describes an autonomous device with an imaging system that determines which plants should be eliminated. The camera on the device identifies the types of plants identified and whether they should be eliminated or not. This device has the limitation of requiring very high processing and storage capacity from the system, making the equipment expensive.

[0007] Patent document CN113498639 describes weeding equipment with image processing. The imaging system uses a camera to determine the location and type of intentionally planted commercial plants, with a camera located between the rows identifying the plants in that location and their number. Although this equipment features sensors for plant identification, its purpose and processing method are to identify plant types and count them. The equipment requires high processing capacity, making it very expensive. Novelties and purpose of the invention

[0008] The present invention aims to provide a sensing system and method for identifying invasive plants that effectively resolves the limitations of the prior art mentioned above.

[0009] The sensing system for identifying invasive plants that comprises a CCD camera “charge-coupled device” RGB “Red / Green / Blue Color Space” for image acquisition, connected to a microcontroller, via DCMI (Digital Camera Memory Interface) interface, for onboard processing of image data and detection of areas corresponding to the color green in the HSV (Hue / Saturation / Value color space) color model. The device performs image acquisition and processing between 10 and 15 FPS (Frames per Second). When a green area is detected, the device sends a binary signal via RS485 interface to the machine to which it is coupled, which performs the programmed action for the specified time.

[0010] Optionally, the machine that receives the signal from the system is an electric weeding machine. Advantages and technical effects of the invention

[0011] The sensing system for identifying invasive plants, which is the object of the present invention, results in the following advantages and achieves the following technical effects over sensing systems for identifying invasive plants in the state of the art: - Prior-art sensing systems for identifying invasive plants feature inefficient or overly complex image processing methods, requiring high processor capacity and consequently making the system very costly. This problem is solved by using a microprocessor that receives RGB565 images, transforms them to RGB24, converts them to HSV, applies convolution, and obtains the desired result through a threshold. - State-of-the-art sensing systems for identifying invasive plants use radiofrequency signal transmission between the camera and the equipment's drive, increasing the risk of failure and allowing signal interference. This problem is solved by the present invention through a camera system with a microprocessor directly connected to the equipment's power module; - The system of the present invention presents image processing with low complexity, and a microprocessor can be used, making the system efficient and low cost. List of attached drawings

[0012] In order that the present invention may be fully understood and carried out to be put into practice by any technician in this technological sector, it must be described in a clear, precise and sufficient manner, based on the attached drawings, listed below, which illustrate preferred ways of implementing the sensing system: Figure 1 - diagram illustrating the system of the invention; Figure 2 - block diagram of interconnection of the system equipment; and Figure 2 - flowchart of the image processing method of the invention. Detailed description of the invention

[0013] Figure 1 illustrates the sensing system for detecting invasive plants, which comprises an RGB CCD camera (1) for image acquisition, connected to a microcontroller (2) via DCMI interface for onboard processing of image data and detection of areas corresponding to the color green in the HSV color model. The system performs image acquisition and processing between 10 and 15 FPS. When a green area is detected, the microcontroller (2) processes the image and sends a binary signal with a duration according to the pre-established time, via RS485 interface to the power module (3) of the machine (4) that activates the tool. (5) keeping it active for the desired time (according to the application and decision in the machine module).

[0014] Power module (3) is powered via a power unit (6), usually a battery or similar object.

[0015] Preferably the binary signal being sent contains a duration of 300 ms.

[0016] Alternatively, the machine (4) is an electric weeding machine, with module (3) being a power module, which activates the tool (5), which is an electric weeder, delivering an electric discharge to the invasive plants, more specifically, weeds.

[0017] Figure 2 illustrates the interconnection blocks of the system's electronic equipment, where the energy derived from the machine's battery (4) sends an input voltage to the circuit, passing through the power supply protection assembly, protecting the circuit against surges, which is then forwarded to the regulators. When forwarding the electric current to the regulators, they adapt it according to the subsequent device, microcontroller (2) or camera (1), so that the device functions correctly, without receiving a voltage that is too high or too low. The image acquisition by microcontroller (2) is performed through its digital camera interface (DCMI), receiving the frame and adjusting the camera configuration (1) through the communication channel (I2C). After processing the image, the microcontroller (2) emits a signal to the module (3) through a transceiver that performs the RS-485 bus of the emitted signal, activating the tool (5) of the machine (4). In parallel, the microcontroller (2) activates the LED lights of the machine (4) according to the average luminance of the image (YAVG) acquired in the camera (1).

[0018] Figure 2 illustrates the image processing method captured by the camera (1) performed by the microcontroller (2). This process consists of four main steps: Image Capture; Preprocessing; Processing; and Result. The camera (1) captures the image in RGB565, which then undergoes preprocessing, which consists of sending the image frame via RGB565, where the image's color bands are allocated in two bytes. Five bits are reserved for the red band, six bits for the green band, and five bits for the blue band. The image is transported to the microcontroller (2) through the DCMI data channel. Upon receiving the image, the microcontroller (2) begins processing, using a DMA "Direct Memory Access" to convert RGB565 to RGB24, which requires 24 bits for pixel allocation, with one byte per band, facilitating processing.When the pixel allocated in memory is in RGB565 format, it is blended due to occupying two bytes per pixel; when converted to RGB24, it has three bytes per pixel. Next, to limit processing to shades of green, the RGB cube is converted to the HSV color system. In this step, a green matrix range and a minimum value level are defined, and from this range defined for shades of green, the saturation output variable is obtained. After conversion to the HSV color system, the image is convoluted, passing a two-dimensional filter (mask) through the saturation result to eliminate noise and highlight regions with a greater number of interconnected green-toned pixels. By achieving saturation concentration, it is possible, through a Threshold, to eliminate acquisition noise that enters the green spectrum.

[0019] Convolution accumulates the saturation value identified in the HSV color model by convolving an increase in intensity across a population of pixels neighbors, thus allowing the exclusion of isolated green points. After binarization, if the count of pixels detected as green resulting from all processing exceeds the threshold stipulated as the minimum for signal activation, the signal is sent that enables the machinery to act.

[0020] Threshold is the last Threshold Filter applied after saturation convolution that quantifies the green pixels resulting from all previous processing to identify the presence of the green set. HSV Conversion

[0021] The conversion from RGB24 to HSV occurs from the RGB Cube, being executed pixel by pixel using the following equations, starting with the difference equation:

[0022] 4 = Cmax - Cmin

[0023] Where:

[0024] Cmax: indicates the band with the highest value within the pixel;

[0025] Cmin: indicates the band with the lowest value within the same pixel;

[0026] A: indicates difference.

[0027] Hue (H) is calculated for each of the pixel's band dominance situations, indicating the angle of the color band that the pixel represents according to its most prominent value. For cases where the difference between the majority band and the minority band is equal, the angle of the color band will be zero. The Hue calculation for each of its situations can be visualized in the equation below: Equation 1 - Hue (H) calculation

[0028] Pixel saturation (5) is calculated by dividing the difference (A) between the majority band (Cmax) and the minority band (Cmin) by the majority band. For cases where the majority band is zero, then the pixel saturation is also zero, as shown in the following equation: Equation 2 - calculation of pixel saturation (S)

[0029] And finally, the Pixel Value (7) is simply the value of your most significant color band and can be visualized in the following equation:

[0030] V = Cmax (4)

[0031] Each of these equations represents a property of the analyzed pixel, thus facilitating segmentation by color tones and even a specific color parameter. Convolution

[0032] Because the objective of the invention does not depend on a specific format, it was decided to use a Gaussian mask with a unitary standard deviation ( <J = 1 ) que simula um filtro morfológico circular de ordem três, tendo como centro cartesiano o pixel central, os valores da máscara (G) são gerados através da seguinte equação: Equation 3 - Calculation of the Gaussian mask (G)

[0033] Choosing this type of mask results in a higher value for pixels adjacent to the central one than for diagonal pixels. This allows scanning the image and aggregating the central pixel based on its neighboring pixels in a heterogeneous manner, maintaining spatial symmetry. Image processing

[0034] In this way, the processing steps of the image captured by the camera (1) consist of: A) Image capture by the camera; B) Sending the image frame via RGB565; C) Conversion of the image from RGB565 to RGB24; D) Conversion from RGB24 to HSV, and from the range defined for shades of green, the saturation output variable is obtained; E) A convolution is performed on the image, passing a two-dimensional filter through the saturation result, to eliminate noise and highlight regions with a greater number of interconnected green tone pixels; and F) The saturation concentration is determined and through a Threshold, noise from the acquisition that enters the green spectrum is eliminated and the final image is obtained. If the count value of pixels detected as green resulting from all processing exceeds the limit stipulated as the minimum for signal activation, the signal that enables the action of the tool (4) of the machine (5) is sent.

[0035] After the final image is obtained, the microcontroller (2) sends a signal to the module (3) of the machine (4) which will activate its tool (5).

[0036] The processing steps are optimized and employ the double-buffering technique, which consists of capturing a new image while analyzing the previous one, to facilitate implementation. This creates a pipeline between the acquisition and processing processes. When the acquired frame is copied by DMA and converted to RGB24, it switches between two memory addresses that will contain the frame data. After copying the frame, a new acquisition is performed, and then processing of the frame begins. When the new capture is complete, this new frame is copied and converted to the other memory address, and the new capture restarts while processing of the previous image begins.

Claims

CLAIMS 1 - “SENSING SYSTEM FOR PLANT IDENTIFICATION INVASORAS”, characterized by comprising a CCD RGB camera (1) for image acquisition, connected to a microcontroller (2), via DCMI interface, for embedded processing of image data and detection of areas corresponding to the color green in the HSV color model; the system performs image acquisition and processing between 10 and 15 FPS, when a green area is detected, the microcontroller (2) processes the image and sends a binary signal with a duration according to the pre-established time, via RS485 interface to the power module (3) of the machine (4) that activates the tool (5) keeping it active for the desired time, according to the application and decision in the machine module (4). 2 - “SENSING SYSTEM FOR PLANT IDENTIFICATION INVASORAS”, according to claim 1, characterized in that the interconnection of the electronic equipment of the system occurs in the following sequence: the energy derived from the machine's battery (4) sends an input voltage to the circuit, passing through the power supply protection set, protecting the circuit against surges, and forwarded to the regulators; when forwarding the electric current to the regulators, they adapt it according to the subsequent device, microcontroller (2) or the camera (1), so that the device works correctly, without receiving a voltage that is too high or too low; the acquisition of the image by the microcontroller (2) is carried out through its DCMI digital camera interface, receiving the frame and adjusting the camera configuration (1) through the I2C communication channel;after processing the image, the microcontroller (2) emits a signal to the module (3) through a transceiver that performs the RS-485 bus of the emitted signal, activating the tool (5) of the machine (4).; 3 - “SENSING SYSTEM FOR PLANT IDENTIFICATION INVADERS”, according to any of the previous claims, characterized preferably, the machine (4) is an electric weeding machine, with the module (3) being a power module, which drives the tool (5) it is an electric weeder that carries out an electric discharge on invasive plants, more specifically, weeds. 4 - “SENSING METHOD FOR PLANT IDENTIFICATION INVADERS”, implemented by the system defined in claim 1, characterized by comprising the following steps: A) Image capture by the camera (1); B) Sending the image frame via RGB565; C) Conversion of the image from RGB565 to RGB24; D) Conversion from RGB24 to HSV, and from the range defined for shades of green, the saturation output variable is obtained; E) A convolution is performed on the image, passing a two-dimensional filter through the saturation result, to eliminate noise and highlight regions with a greater number of interconnected green tone pixels; and F) The saturation concentration is determined and through a Threshold, noise from the acquisition that enters the green spectrum is eliminated and the final image is obtained. If the count value of pixels detected as green resulting from all processing exceeds the limit stipulated as minimum for signal activation, the signal that enables the action of the tool (4) of the machine (5) is sent. 5 - “SENSING METHOD FOR IDENTIFYING INVASIVE PLANTS”, according to claim 4, characterized in that step B) consists of pre-processing, which consists of sending the image frame via RGB565, where the color bands of the image are allocated in two bytes; where five bits are reserved for the red band, six bits for the green band and five bits for the blue color band. 6 - “SENSING METHOD FOR IDENTIFYING INVADING PLANTS”, according to claim 4, characterized in that step C) consists of the following process: upon receiving the image, the microcontroller (2) uses a DMA to convert RGB565 to RGB24, which requires 24 bits for pixel allocation, with one byte per band, facilitating processing; when the pixel allocated in memory is in RGB565 format, it is mixed due to occupying two bytes per pixel, when converted to RGB24, it has three bytes per pixel. 7 - “SENSING METHOD FOR IDENTIFYING INVASIVE PLANTS”, according to claim 4, characterized in that step D) consists of converting the RGB cube to the HSV color system; in this step a green matrix band and a minimum value level are delimited and, from from this range defined for green tones the saturation output variable is obtained.

Citation Information

Patent Citations

  • Intelligent weeding device and use method thereof

    CN103348964A

  • Weed positioning spraying-weeding system and positioning spraying-weeding method thereof

    CN105469020A

  • Weeding device and weeding method

    CN113498639A

  • Robot vehicle and method using a robot for an automatic treatment of vegetable organisms

    US20180168141A1

  • METHOD FOR RECOGNIZING A TYPE OF WEED IN A NATURAL ENVIRONMENT, SYSTEM FOR RECOGNIZING A TYPE OF WEED IN A NATURAL ENVIRONMENT, WEED CONTROL METHOD AND COMPUTER PROGRAM PRODUCT

    BR112018073098A2

Cited By

  • Method and system for electrical weeding between plant rows of a plantation field

    WO2026170271A1