Continuous profiling variable spraying method and system based on RGBD visual inspection

Abstract

The invention relates to the technical field of intelligent precise spraying, in particular to a continuous profiling variable spraying method and system based on RGBD visual inspection, and the method comprises the steps: designing a profiling sprayer based on RGBD visual inspection, collecting a tree crown image, and obtaining an RGB image and a depth image; preprocessing the RGB image and the depth image, and correspondingly determining an RGB image matrix and a depth image matrix; determining an ideal spraying coverage area, dividing the crown contour to obtain spraying segments, and determining a spraying positioning point and a positioning angle; the RGB image matrix and the depth image matrix are integrated, the volume and the leaf area of each spraying section are obtained by combining a profiling spraying machine, and the spraying amount and the wind speed of the spraying sections are correspondingly determined; and matching a nozzle in the profiling sprayer, and controlling the profiling sprayer to move towards the crown to a corresponding position to perform profiling variable spraying on the crown. The method for dynamically matching the spray head in the profiling spraying machine with the crown contour is achieved, and profiling variable spraying with the high pesticide effective utilization rate is achieved.

Description

Technical Field

[0001] This invention relates to the field of intelligent precision spraying technology, specifically to a continuous shape-mimicking variable spraying method and system based on RGBD visual detection. Background Technology

[0002] In practical applications, contour spraying for fruit trees and other trees is a precision pesticide spraying technology and method. Based on the detected characteristics such as the shape of the fruit tree outline and the area density of the canopy leaves, the ideal spray flow rate, air delivery speed, and spatial position of the spray point are calculated. Under the processing of microprocessor instructions, the nozzle group is automatically controlled to reach the ideal position and spray pesticides with appropriate spray parameters to improve the uniformity of droplet distribution in the fruit tree.

[0003] Currently, contour sprayers mainly include variable boom contour sprayers, double rocker arm contour sprayers, robotic arm contour sprayers, gantry contour sprayers, and three-in-one contour sprayers. Existing structural devices suffer from problems such as bulky and cumbersome contouring mechanisms, low automation, poor contouring accuracy, high operation difficulty, and low efficiency. Furthermore, in terms of contour spray point positioning, manual control is generally used, resulting in poor positioning accuracy and high operation difficulty.

[0004] To address the technical shortcomings of existing contour sprayers that rely on manual control, current technologies typically divide trees into static spray zones at fixed heights. While these zones use fixed nozzles and ideal spray distances, they fail to adequately consider the actual spray coverage and angle during operation. Furthermore, the number of nozzles is not rationally adjusted and optimized based on actual spraying needs, potentially leading to ineffective spraying in localized areas or unreasonable overlap between spray coverage zones, thus reducing the overall efficiency and accuracy of the spraying operation. Summary of the Invention

[0005] To address the technical problem that existing spraying methods cannot adjust spraying according to tree dynamics, leading to localized dry spraying and unreasonable overlap, thus reducing spraying efficiency and accuracy, the present invention aims to provide a continuous shape-mimicking variable spraying method based on RGBD visual detection. The specific technical solution adopted is as follows:

[0006] Design a contour sprayer based on RGBD visual detection, control the contour sprayer to acquire tree canopy images, and obtain RGB images and depth images respectively;

[0007] Preprocess the RGB and depth images, and determine the corresponding RGB image matrix and depth image matrix;

[0008] The ideal spray coverage area is determined by using a contour sprayer, the depth image matrix is ​​analyzed, the tree canopy outline is divided into spray segments, and the spray positioning point and positioning angle of each spray segment are determined.

[0009] By combining the RGB image matrix and the depth image matrix, and using a contour sprayer to obtain the volume and leaf area of ​​each spray segment, the spray volume and wind speed of the spray segment are determined accordingly.

[0010] Match the nozzles in the contour sprayer to the spray volume and wind speed, and control the contour sprayer to move toward the tree canopy to the corresponding position to perform contour-contact variable spraying on the tree canopy.

[0011] Preferably, the RGBD vision detection-based contour sprayer includes a spray vehicle and contour arms disposed opposite to each other on the spray vehicle, as well as an RGBD camera module and a spray supply system disposed between the contour arms. Multiple robotic arms are connected to the contour arms, and a spray mechanism is provided at the end of the robotic arms away from the contour arms.

[0012] Preferably, the spraying mechanism includes a fan, a connector, and a nozzle arranged sequentially on the robotic arm toward the end away from the contour arm, wherein the fan and the nozzle are fixed together by the connector.

[0013] Preferably, the contour sprayer is controlled to acquire tree canopy images, obtaining RGB images and depth images respectively, including:

[0014] Control the contour sprayer to move toward the tree canopy and control the spray supply system to output a frequency signal, and update the position of the contour sprayer based on the frequency signal;

[0015] Based on the position of the contour sprayer, perpendicular to the planting row corresponding to the tree, an RGBD camera is used to capture images of the tree canopy, resulting in RGB images and depth images respectively.

[0016] Preferably, preprocessing the RGB image and depth image to correspondingly determine the RGB image matrix and depth image matrix includes:

[0017] The RGB image and depth image are filtered and cropped sequentially to obtain the corresponding target region in the RGB image and depth image;

[0018] Detection is performed based on the target region. If a tree canopy exists in the target region, the corresponding RGB image matrix and depth image matrix are determined.

[0019] Preferably, the ideal spray coverage area is determined using a contour sprayer, the depth image matrix is ​​analyzed, the canopy outline is divided into spray segments, and the spray positioning points and positioning angles for each spray segment are determined, including:

[0020] The ideal spray coverage area is obtained by combining the spray angle and the ideal spray distance from the nozzle to the tree canopy, based on the nozzle itself.

[0021] The tree canopy outline arc length is output based on the depth image matrix, and the tree canopy outline is divided by the ideal spray coverage area to obtain spray segments;

[0022] Analyze each spray segment to determine the corresponding spray positioning point and positioning angle.

[0023] Preferably, the canopy outline arc length is output based on the depth image matrix, and the canopy outline is divided in combination with the ideal spray coverage area to obtain spray segments, including:

[0024] The coordinate point matrix corresponding to the tree canopy outline is determined based on the depth image matrix, and the outline curve equation corresponding to the tree canopy is obtained. The outline curve equation is fitted to the coordinate point matrix using a polynomial method to obtain the arc length of the tree canopy outline.

[0025] By comparing the ideal spray coverage area with the arc length of the tree canopy outline, the tree canopy outline is divided, the number of nozzles to be used is determined, and the corresponding spray segments are obtained based on the number of nozzles used.

[0026] Preferably, each spray segment is analyzed to determine the corresponding spray positioning point and positioning angle, including:

[0027] Based on the segmented spraying, the tree canopy outline arc data and corresponding arc length are obtained, and the slope and actual spraying distance are determined accordingly.

[0028] The positioning angle and position of the nozzle for spraying are determined based on the slope and the actual spraying distance.

[0029] Preferably, by combining the RGB image matrix and the depth image matrix, and using a contour sprayer to obtain the volume and leaf area of ​​each spray segment, the spray volume and wind speed of the corresponding spray segment are determined, including:

[0030] The distance from the RGBD camera module to the contour arm is measured, and the travel distance within the interval between adjacent rising edges of the frequency signal is determined in conjunction with the spray supply system. The volume of the spray segment is determined in conjunction with the depth image matrix.

[0031] The relative leaf area index of the corresponding spray segment is obtained from the RGB image matrix;

[0032] The spray volume and wind speed of the spray segments are obtained by combining the overall volume and relative leaf area index with the fan speed.

[0033] To address the aforementioned problems, this invention also provides a continuous shape-mimicking variable spraying system based on RGBD visual detection. The system includes a processor, a communication interface, a memory, and a communication bus. The processor, communication interface, and memory communicate with each other via the communication bus. The processor calls logical instructions from the memory to execute any of the aforementioned continuous shape-mimicking variable spraying methods based on RGBD visual detection.

[0034] The present invention has the following beneficial effects:

[0035] 1. First, a contour-following sprayer based on RGBD (Red, Green, Blue Depth) visual detection is designed. After acquiring tree canopy images, preprocessing is performed to effectively improve the quality and consistency of image data. The RGB image matrix and depth image matrix corresponding to the tree canopy are obtained respectively, realizing accurate perception and analysis of the fruit tree canopy morphology. Next, the relevant data involved in the spraying operation are specifically analyzed, and the spraying segments are dynamically divided. Finally, specific data acquisition is carried out for the spraying segments to determine the spraying position of the contour-following sprayer, so as to realize the method of dynamic matching of the nozzle to the tree canopy contour in the contour-following sprayer, and realize contour-following variable spraying with high pesticide utilization efficiency.

[0036] 2. The continuous shape-following variable spray system based on RGBD visual detection provided by this invention has the same beneficial effects as the continuous shape-following variable spray method based on RGBD visual detection provided by this invention, and will not be described in detail here. Attached Figure Description

[0037] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 The flowchart illustrates the steps of a continuous shape-mimicking variable spraying method based on RGBD visual detection, as provided in one embodiment of the present invention.

[0039] Figure 2 This is a schematic diagram of the structure of a contour sprayer based on RGBD visual detection, provided in one embodiment of the present invention.

[0040] Figure 3 This is a schematic diagram of a tree canopy photographed using a continuous shape-mimicking variable spraying method based on RGBD visual detection, as provided in an embodiment of the present invention.

[0041] Figure 4This is a schematic diagram of the canopy outline curve corresponding to the spray segment in the continuous shape-mimicking variable spraying method based on RGBD visual detection provided in an embodiment of the present invention;

[0042] Figure 5 This is a comparative diagram of the dynamic division of spray segments and the division of spray zones according to a fixed height of the tree crown in a continuous shape-mimicking variable spraying method based on RGBD visual detection provided in an embodiment of the present invention.

[0043] Figure 6 This is a schematic diagram of a continuous contour-following variable spraying method based on RGBD visual detection provided in an embodiment of the present invention.

[0044] In the picture:

[0045] 1. Spray truck; 2. Contouring arm; 3. RGBD camera module; 4. Robotic arm; 5. Spraying mechanism;

[0046] 10. Brushless wheel; 11. Carriage; 30. Support rod; 31. RGBD camera; 40. Servo motor; 41. Connecting rod; 50. Fan; 52. Connector; 53. Nozzle. Detailed Implementation

[0047] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a continuous contour-following variable spraying method and system based on RGBD visual detection proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0048] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0049] The following description, in conjunction with the accompanying drawings, details the specific scheme of the continuous contour-following variable spraying method and system based on RGBD visual detection provided by this invention.

[0050] To better illustrate, RGBD visual inspection is an advanced visual perception technology that combines RGB color images with depth information. It uses a camera to simultaneously collect color and distance data of a target, forming an image containing three-dimensional spatial information to achieve accurate recognition of the shape, position, and contour of an object. In agricultural and forestry applications, this visual inspection technology can efficiently detect the three-dimensional structure, canopy morphology, and growth status of trees.

[0051] Contouring spraying is a technology that uses the detected three-dimensional shape of trees for precise pesticide application. It involves collecting data using RGBD equipment and automatically adjusting the direction, angle, and volume of the spray to ensure the spray area closely matches the actual shape of the tree. Contouring spraying offers several advantages: first, it significantly reduces pesticide and fertilizer use, preventing spraying into non-target areas and minimizing environmental pollution; second, it improves control effectiveness by precisely covering areas prone to pests and diseases in the tree canopy, thus enhancing control efficiency; and third, it adapts to the different growth stages and morphological characteristics of trees, making it particularly suitable for orchards, forests, and other environments with complex, three-dimensional planting structures.

[0052] Please see Figure 1 The diagram illustrates a flowchart of a continuous shape-mimicking variable spraying method based on RGBD visual detection provided in the first embodiment of the present invention. The method includes:

[0053] Step S1: Design a contour sprayer based on RGBD visual detection, control the contour sprayer to acquire tree canopy images, and obtain RGB images and depth images respectively;

[0054] Step S2: Preprocess the RGB image and depth image, and determine the corresponding RGB image matrix and depth image matrix;

[0055] Step S3: Determine the ideal spray coverage area using a contour sprayer, analyze the depth image matrix, divide the tree canopy outline into spray segments, and determine the spray positioning point and positioning angle for each spray segment;

[0056] Step S4: Combine the RGB image matrix and the depth image matrix, and use the contour sprayer to obtain the volume and leaf area of ​​each spray segment, and determine the spray volume and wind speed of the corresponding spray segment;

[0057] Step S5: Match the nozzles in the contour sprayer according to the spray volume and wind speed, and control the contour sprayer to move towards the tree canopy to the corresponding position to perform contour-contact variable spraying on the tree canopy.

[0058] Optionally, the proposed continuous shape-mimicking variable spraying method based on RGBD visual detection is applicable to various trees to dynamically achieve more efficient pesticide utilization. In this embodiment, fruit trees are used as a specific example.

[0059] This paper explains that, based on existing technologies, static spraying zones divided by fixed heights lead to localized drowsiness and unreasonable overlap of sprays. Therefore, a continuous shape-mimicking variable spraying method based on RGBD visual detection is proposed to achieve dynamic matching of the nozzle with the tree canopy outline, thereby realizing shape-mimicking variable spraying with high pesticide utilization efficiency.

[0060] Please see Figure 2Furthermore, the contour sprayer based on RGBD visual inspection includes a spray vehicle 1 and contour arms 2 arranged opposite to each other on the spray vehicle 1, as well as an RGBD camera module 3 and a spray liquid supply system arranged between the contour arms 2. Multiple robotic arms 4 are connected to the contour arms 2, and a spray mechanism 5 is provided at the end of the robotic arms 4 away from the contour arms 2.

[0061] Preferably, in this embodiment, the robotic arm 4 is provided with at least three sections, which are fixedly connected to the contour arm 2 by bolts through servo motors 40, and each section of the robotic arm 4 is connected to each other by bolts through servo motors 40 and connecting rods 41 to form an articulated arm.

[0062] As explained, in the specific implementation environment, the spray vehicle 1 includes brushless wheels 10 and a carriage 11. The RGBD camera module 3 is mounted on the carriage 11, and the spray supply system is located inside the carriage 11. To facilitate the normal operation of the spray supply system, an electrical control box is also configured inside the carriage 11. This box is an electronic control module, including a Linux development board, an STM32 development board, a multi-channel MOSFET (Metal Oxide Semiconductor) switching module, and a brushless motor driver for the wheels. The Linux development board is responsible for running the control algorithm for the spray vehicle 1 to perform contour spraying. The STM32 development board is mainly responsible for real-time data acquisition and fast response. The multi-channel MOSFET switching module controls the on / off state of different circuit branches according to control requirements to meet the task requirements in different scenarios. The brushless motor driver for the wheels drives the spray vehicle 1 to spray. The multiple components work together to achieve high-precision drive and control of the spray supply system, ensuring that the spray supply system can operate stably and reliably in various working environments.

[0063] It can be explained that the RGBD camera module 3 includes a support rod 30 and multiple RGBD cameras 31 disposed on the end of the support rod 30 away from the brushless wheel 10, and all RGBD cameras 31 are fixedly connected to the support rod 30 by bolts; in this embodiment, at least two RGBD cameras 31 are set to photograph the tree crown in order to accurately capture the three-dimensional structural information corresponding to the tree crown; at least one RGBD camera 31 is used to control the automatic driving of the spray truck 1, that is, to perceive the working environment of the spray truck 1 in real time and ensure the autonomous navigation and safe driving of the spray truck 1 in any terrain environment; preferably, in this embodiment, the RGBD camera that is relatively closer to the brushless wheel 10 is used to control the automatic driving of the spray truck 1.

[0064] The spray liquid supply system includes multiple components such as a water tank, pressure gauge, water pump, overflow valve, and water pipes. The water tank provides the core liquid storage, i.e., it stores the corresponding spraying agent. The pressure gauge is used to monitor the working pressure of the entire spray vehicle 1 in real time to ensure that it operates within a safe range. The water pump is the core power component, and centrifugal water pumps or plunger water pumps can be selected according to the different liquids being transported to meet the task requirements. The overflow valve is a safety protection device that automatically opens when the pressure exceeds a pre-set value, guiding excess liquid back to the water tank to prevent damage to pipelines or equipment due to overpressure. The water pipes are responsible for connecting the various components and transporting the liquid, and are usually made of corrosion-resistant PVC (Polyvinyl Chloride Pipe) or stainless steel pipes.

[0065] As an optional implementation, both the contour arm 2 and the RGBD camera module 3 are connected to the spray truck 1 via aluminum profile corner brackets. The aluminum profile corner brackets refer to L-shaped connectors 52 made of aluminum alloy material, which are commonly used in the assembly process of industrial equipment or mechanical structures. They can be fixed at the intersection of different components by bolts or screws, thereby enhancing structural stability and connection strength.

[0066] Furthermore, the spraying mechanism 5 includes a fan 50, a connector 52, and a nozzle 53 arranged sequentially on the robotic arm 4 toward the end away from the contour arm 2. The fan 50 and the nozzle 53 are fixed by the connector 52. The fan 50 refers to a brushless fan 50, which, along with the nozzle 53, is fixed to the connector 52 by bolts. The other end of the connector 52 is fixedly connected to the rotating arm of the servo motor 40 by bolts, so as to control the rotation and spraying of the nozzle 53 by the rotating arm, thereby expanding the applicable range.

[0067] Further, in step S1, the contour sprayer is controlled to acquire tree canopy images, obtaining RGB images and depth images respectively, including:

[0068] Step S11: Control the contour sprayer to move towards the tree canopy and control the spray supply system to output a frequency signal, and update the position of the contour sprayer based on the frequency signal.

[0069] Specifically, the Linux development board sends commands to control the spray truck to move at a certain speed, adapted to the current environment. The brushless motor driver of the wheels outputs a frequency signal FG, where FG is a PWM (Pulse Width Modulation) signal. By adjusting the duty cycle of the pulse, the motor speed and torque are controlled to ensure the spray truck can more stably update the position of the overall contour sprayer, improving the reliability of autonomous driving. When the Linux development board's I / O port detects the rising edge of the FG signal, it triggers an interrupt, indicating that the spray truck has reached the preset camera shooting position. At this time, the RGBD camera position number in the spray truck is displayed. The value increases by 1, which corresponds to the specific shooting location of the spray truck, for example, increasing sequentially from the initial position 0 to 1, 2, 3, etc.

[0070] Step S12: Based on the position of the contour sprayer, perpendicular to the planting row corresponding to the tree, execute the RGBD camera to capture tree canopy images, and obtain RGB images and depth images respectively.

[0071] Specifically, after determining the position of the contour sprayer according to the aforementioned step S11, and ensuring that the RGBD camera in the sprayer is perpendicular to the planting row corresponding to the tree, the Linux system program executes an RGBD camera scan to capture a tree canopy image, obtaining an RGB image, denoted as . Depth image, denoted as ,in, Indicates the height of the image; Indicates the width of the image; These correspond to the R, G, and B channels in an RGB image, and their sizes are all [missing information]. .

[0072] Further, step S2 includes:

[0073] Step S21: Filter and crop the RGB image and depth image in sequence to obtain the corresponding target region in the RGB image and depth image.

[0074] Specifically, a preset depth image distance threshold is defined as . It represents the distance between the RGBD camera and the center of the corresponding planting row of trees, indexed depth image. All exceeding the threshold The pixel position coordinates, i.e. In RGB images Lieutenant General pixel position coordinates The corresponding R, G, and B channel pixel values ​​are set to zero, meaning pixels exceeding the distance threshold are removed from the RGB image. The background fruit tree canopies are filtered out to prevent them from affecting the detection results, resulting in a filtered RGB image, denoted as . .

[0075] Next, based on RGB images The image is cropped to obtain the center region. The corresponding cropping logic is as follows: ,in, This represents the center region of the RGB image, i.e., the image width is 100 pixels; by cropping the depth image of the corresponding region, we obtain the center region image corresponding to the depth image. The target region is the central region of the RGB image and the depth image, respectively.

[0076] Please see Figure 3 Step S22: Detection is performed based on the target region. If a tree canopy exists in the target region, the corresponding RGB image matrix and depth image matrix are determined.

[0077] As an optional implementation, in this embodiment, a deep learning target algorithm is used to detect whether there is a tree crown in the target region. That is, by constructing a deep neural network model such as a convolutional neural network (CNN), YOLO (You Only Look Once), or SSD (Single Shot MultiBox Detector), feature extraction and target localization are performed on the target region.

[0078] Specifically, the presence of tree canopies is determined from the target region. If they exist, a detection box corresponding to the tree canopy is drawn, with the corresponding upper-left and lower-right pixel coordinates as follows: and Based on the pixel coordinates of the detection box, all pixels outside the detection box in the target region of the RGB image are cleared to zero, while the pixels within the detection box are retained, resulting in an RGB image matrix, denoted as . Similarly, based on the RGB image, the target region corresponding to the depth image is processed by combining the pixel coordinates of the detection box. Simultaneously, any target regions in the depth image exceeding a threshold are excluded. All pixels are also cleared to zero, resulting in a depth image matrix, denoted as . .

[0079] It should be noted that if no tree canopy is detected in the target area, the control arrays for the robotic arm, nozzle flow rate, and fan speed corresponding to each nozzle should be set to... ,in, This represents an array of parameters to be executed by the spray truck. Indicates the location number of the spray truck; This indicates that all control variables are empty. In this case, the parameter array is directly sent to the STM32 development board via serial communication. The STM32 development board executes the instructions to drive the nozzle to spray the trees normally, and then jumps to the subsequent step S5 to correspond to the data obtained after steps S1-S5. Perform the appropriate spraying operation.

[0080] Furthermore, step S3 includes:

[0081] Step S31: Determine the spray angle and the ideal spray distance from the nozzle to the tree canopy based on the nozzle. Combine the spray angle and the ideal spray distance to obtain the ideal spray coverage area.

[0082] Specifically, the spray angle is determined based on the nozzle, denoted as . This indicates the angular range covered by the mist sprayed from the nozzle; the ideal spray distance from the nozzle to the tree canopy is determined and denoted as... Based on the nozzle model, the corresponding instruction manual is consulted to obtain the recommended spray distance. For example, for the commonly used TP80015 nozzle, the instruction manual recommends a spray distance of 50cm for a spray angle of 110° and 75cm for a spray angle of 80°. The ideal spray coverage area is then calculated by combining the spray angle and the ideal spray distance using the following formula:

[0083]

[0084] in, Indicates the ideal spray coverage area; Indicates the ideal spray distance from the nozzle to the tree canopy; Indicates the spray angle.

[0085] Step S32: Output the canopy outline arc length based on the depth image matrix, and divide the canopy outline by combining the ideal spray coverage area to obtain spray segments.

[0086] It is explained that the growth of trees is affected by a variety of natural factors such as sunlight, water, soil nutrients, and wind, resulting in a high degree of complexity and individual differences in the shape of their crowns. This makes it difficult to fix the crown outline into any shape, and it is irregular. Therefore, a specific analysis is conducted in conjunction with the collected image data.

[0087] Further, step S32 includes:

[0088] Step S321: Determine the coordinate point matrix corresponding to the tree crown outline based on the depth image matrix, and obtain the outline curve equation corresponding to the tree crown. Fit the outline curve equation to the coordinate point matrix using a polynomial method to obtain the arc length of the tree crown outline.

[0089] Specifically, depth image matrix The mean value for the depth distance of the tree canopy outline is obtained by averaging the values ​​row by row. Depth data outside the detection box is removed, and the height direction of the RGB image and depth image is opposite to the height direction of the contour sprayer. Therefore, the coordinate point matrix corresponding to the tree crown outline is obtained by rewriting according to the height direction of the contour sprayer. The corresponding calculation formula is:

[0090]

[0091] in, A matrix representing the coordinate points corresponding to the tree canopy outline; Represents the depth image matrix Calculate the average value by row; Indicates the height of the depth image; , Each represents the pixel coordinate position of the detection box corresponding to the tree crown; This represents the actual height distance value corresponding to 1 pixel in the height direction of the depth image.

[0092] Next, obtain the equation of the outline curve corresponding to the tree crown, denoted as . This involves collecting three-dimensional coordinate data of the tree canopy surface and using a fitting method to transform discrete data points into continuous mathematical expressions in order to accurately describe the shape and characteristics of the tree canopy outline.

[0093] Then, the coordinate point matrix of the tree canopy outline The polynomial method is used to fit the contour curve equation. The formula for calculating the arc length of the tree crown outline is as follows:

[0094]

[0095]

[0096] in, Indicates the arc length of the tree crown outline; , These represent the upper and lower limits of the number of pixels corresponding to the tree crown; Representing the equation of the profile curve The derivative of .

[0097] Please see Figure 4 Step S322: Compare the ideal spray coverage area with the arc length of the tree canopy outline, divide the tree canopy outline, determine the number of nozzles to be used, and obtain the corresponding spray segments based on the number of nozzles used.

[0098] Specifically, based on the ideal spray coverage area and the arc length of the tree crown outline The ratio between them determines the number of nozzles required on a single-sided contour arm. The corresponding calculation formula is:

[0099]

[0100] in, Indicates the number of nozzles used; Indicates the ideal spray coverage area; It indicates the arc length of the tree crown outline.

[0101] It can be explained that, based on the ratio Divide the tree crown outline into piecewise curves, that is, when... At that time, using one nozzle, the contour curve equation is... Divide into 1 segment, denoted as ;when At that time, two nozzles were used to apply the contour curve equation. Divide into two equal segments, denoted as and ;when At that time, three nozzles were used to apply the contour curve equation. Divide into 3 equal segments, denoted as , and Among them, based on the contour curve equation The segmented divisions are called spray segments.

[0102] It should be noted that in this embodiment, only 3 nozzles are used for illustration. The number of nozzles can be increased or decreased according to the actual usage situation, based on the ideal spray coverage area and the arc length of the tree canopy outline. For example, when the trees are relatively taller than in this embodiment, the number can be increased to 4 or 5.

[0103] Step S33: Analyze each spray segment and determine the corresponding spray positioning point and positioning angle.

[0104] It is explained that when confirming the spray positioning point and positioning angle, the principle of full coverage of the spray surface is followed, that is, to ensure that the spray droplets can evenly and completely cover the entire target surface, leaving no uncovered areas or dead corners.

[0105] Further, step S33 includes:

[0106] Step S331: Based on the spray segmentation, obtain the tree canopy outline arc segment data and the corresponding arc length, and determine the slope and actual spray distance in sequence.

[0107] Specifically, the canopy outline arc segment data for each spraying segment is obtained according to step S32, and denoted as... That is, based on the contour curve equation The divided arc segment data, And obtain the corresponding canopy arc length for each spray segment, denoted as . Based on tree canopy outline arc data Get the pixel coordinates at the midpoint, i.e. The slope of the tangent at that point is Then, based on the arc length Combined with the spray angle of the nozzle The actual spray distance is calculated using the following formula:

[0108]

[0109] in, Indicates the actual spray distance; This indicates the canopy arc length corresponding to the spray segment; This indicates the spray angle of the nozzle.

[0110] Step S332: Obtain the positioning angle and position of the nozzle for spraying based on the slope and the actual spraying distance.

[0111] Specifically, the corresponding calculation formula is:

[0112]

[0113]

[0114]

[0115] in, This indicates the nozzle positioning angle corresponding to the spray segment; Indicates the slope; Indicates the nozzle positioning position corresponding to the spray segment; This represents the coordinates of the center point of the canopy outline arc segment corresponding to the spray segment.

[0116] In particular, during actual spraying from the nozzle, since the spray distance cannot be infinitely small, when season And determine the actual spraying distance according to step S33. Nozzle positioning angle and nozzle positioning position .

[0117] Further, step S4 includes:

[0118] Step S41: Measure the distance from the RGBD camera module to the contour arm, determine the travel distance within the interval between adjacent rising edges of the frequency signal by combining the spray supply system, and determine the volume of the spray segment by combining the depth image matrix.

[0119] Specifically, based on the spray truck, relevant parameters were statistically analyzed, namely, the horizontal distance from the RGBD camera module to the contouring arm was measured and denoted as... The distance traveled by the spray truck within the interval between adjacent rising edges of the FG signal output by the brushless motor driver is denoted as . Meanwhile, the diameter of the brushless motor wheel is measured based on the brushless motor driver and denoted as... The number of poles of the motor is denoted as... The formula for calculating the volume of any spray segment is as follows:

[0120]

[0121]

[0122] in, Indicates the volume of the spray segment; Represents the pixels in the spray segment; , These represent the upper and lower limits of the tree canopy outline in the spray segment, respectively; Indicates the distance threshold of the depth image; Represents the depth image matrix Calculate the average value by row; This represents the actual height distance value corresponding to 1 pixel in the height direction of the depth image.

[0123] Step S42: Obtain the relative leaf area index of the corresponding spray segment based on the RGB image matrix.

[0124] Specifically, the corresponding calculation formula is:

[0125]

[0126] in, The relative leaf area index corresponding to the canopy outline of the spray segment; Represents the pixels in the spray segment; This indicates the pixel's index within the spray segment; Represents an RGB image matrix The G channel component.

[0127] Step S43: Combine the volume and relative leaf area index with the fan to obtain the spray volume and wind speed of the spray segment.

[0128] Specifically, the corresponding calculation formula is:

[0129]

[0130]

[0131] in, This indicates the spray volume corresponding to the canopy outline of the spray segment; Indicates the volume of the spray segment. This indicates the amount of liquid medicine required per unit cubic meter; The relative leaf area index corresponding to the canopy outline of the spray segment; The wind speed corresponding to the canopy outline of the spray segment; Indicates the wind loss coefficient; This indicates the diameter of the air outlet of the fan in the contour sprayer.

[0132] It can be noted that, in this embodiment, the amount of liquid required per cubic meter is... 0.1L / m 3 .

[0133] Please combine Figures 4-6 To better illustrate this, a specific analysis is made of the three spray segments divided in step S3. When the number of arc segments in the canopy outline is 1, the upper and lower limits of the divided canopy outline are... The spray volume of the spray segment is obtained according to step S4. and wind speed .

[0134] When the tree crown outline is divided into 2 arc segments, the ordinate of the segment division point is: The upper and lower limits of the tree canopy outline are divided from top to bottom. and The spray volume of the spray segment is obtained according to step S4. and wind speed .

[0135] When the tree crown outline is divided into 3 arc segments, the ordinate of the 2 segment division points is: and The upper and lower limits of the tree canopy outline are divided from top to bottom. , and The spray volume of the spray segment is obtained according to step S4. and wind speed .

[0136] It can be explained that in step S5, the nozzle numbers are assigned to each canopy outline arc segment based on the principle of combining the shortest spatial distance with spatial distribution. Specifically, taking the three nozzles proposed in this embodiment as an example, the three nozzles are numbered from bottom to top as follows: And correspondingly determine the coordinates of the previous pixel position of each nozzle from bottom to top, and record them as follows: , and .

[0137] When the number of arc segments in the tree canopy outline is 1, using one nozzle, calculate the actual spray positioning points of the three nozzles to the current tree canopy outline arc segment. distance Take the nozzle number corresponding to the minimum distance, i.e. ,in, This indicates the nozzle matched from the contour sprayer when the spray segment is 1. ; This indicates the minimum value operation.

[0138] When the canopy outline is divided into 2 arc segments, the actual spray positioning points of the nozzles corresponding to the 2 canopy outline arc segments are as follows: and The two selected nozzles correspond from bottom to top to the two arc segments of the tree canopy outline. At this point, there are three possible distance combinations from the previous position of the three nozzles to the actual spray positioning point of the two nozzles. The determined distance combinations are as follows: Take the nozzle number combination corresponding to the smallest distance combination, that is... ,in, This indicates the nozzle combination matched from the contour sprayer when the spray is divided into two segments. .

[0139] When the tree canopy outline is divided into 3 arc segments, all 3 nozzles are used, and the 3 nozzles correspond to the 3 arc segments of the tree canopy outline from bottom to top.

[0140] Next, based on the nozzle matching status, the corresponding control parameters for the articulated robotic arm, nozzle flow rate, and fan speed are determined. If the nozzle is not used, the control parameters for the articulated robotic arm, nozzle flow rate, and fan speed are all empty. Then, if nozzles are used, based on the matching results of each nozzle, the corresponding control arrays for the articulated robotic arm, nozzle flow rate, and fan speed are stored. That is, to store the relevant parameter array to be executed by the spray truck, denoted as .

[0141] Then, the position that the current spray truck's contour arm should travel is calculated, and the distance traveled by the spray truck within the interval between adjacent rising edges of the FG signal output by the brushless motor driver is obtained based on step S4. The corresponding calculation formula is:

[0142]

[0143] in, Indicates the current position of the spray truck as it is about to proceed; Indicates the position of the spray truck before it moved; This indicates the floor function.

[0144] Specifically, regarding the current location where the spray truck is about to proceed... Provide specific details, when When the index is greater than or equal to 0, the relevant parameter array to be executed by the spray truck is obtained. Conversely, if When the value is less than 0, backtrack to step S1, that is, re-acquire tree canopy images and perform the analysis of steps S1-S5 until... And it equals 0.

[0145] Finally, the Linux development board will use the parameter array determined in step S5. The command is sent to the STM32 development board via serial communication, and the instruction is executed to drive the nozzle to dynamically match the canopy contour, thereby realizing shape-mimicking variable spraying of the tree canopy.

[0146] Understandably, firstly, a contour-following sprayer based on RGBD visual detection is designed. After acquiring tree canopy images, preprocessing effectively improves the quality and consistency of the image data. The RGB image matrix and depth image matrix corresponding to the tree canopy are obtained, enabling precise perception and analysis of the fruit tree canopy morphology. Next, relevant data involved in the spraying operation are specifically analyzed, and spraying segments are dynamically divided. Finally, specific data acquisition is carried out for each spraying segment to determine the spraying position of the contour-following sprayer, thus realizing a method for dynamically matching the nozzle to the tree canopy contour in the contour-following sprayer, achieving high-efficiency pesticide utilization through contour-following variable spraying.

[0147] The second embodiment of the present invention provides a continuous shape-following variable spraying system based on RGBD visual detection. The system includes a processor, a communication interface, a memory, and a communication bus. The processor, the communication interface, and the memory communicate with each other through the communication bus. The processor calls logical instructions in the memory to execute a continuous shape-following variable spraying method based on RGBD visual detection as described in any of the foregoing embodiments.

[0148] Understandably, when a continuous shape-following variable spraying system based on RGBD visual detection is in operation, multiple modules within the system work together to achieve the continuous shape-following variable spraying method based on RGBD visual detection provided in any of the foregoing embodiments of the present invention. Therefore, whether the system and program data are integrated or different hardware is configured to produce functions with similar effects to those achieved by the present invention, they all fall within the protection scope of the present invention. This system has the same beneficial effects as the aforementioned continuous shape-following variable spraying method based on RGBD visual detection, and will not be elaborated here.

[0149] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0150] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A continuous shape-mimicking spraying method based on RGBD visual detection, characterized in that, The method includes: Design a contour-following sprayer based on RGBD visual detection, control the contour-following sprayer to acquire tree canopy images, and obtain RGB images and depth images respectively; Preprocess the RGB and depth images, and determine the corresponding RGB image matrix and depth image matrix; The ideal spray coverage area is determined by using a contour sprayer, the depth image matrix is ​​analyzed, the tree canopy outline is divided into spray segments, and the spray positioning point and positioning angle of each spray segment are determined. By combining the RGB image matrix and the depth image matrix, and using a contour sprayer to obtain the volume and leaf area of ​​each spray segment, the spray volume and wind speed of the spray segment are determined accordingly. Match the nozzles in the contour sprayer to the spray volume and wind speed, and control the contour sprayer to move toward the tree canopy to the corresponding position to perform contour-contact variable spraying on the tree canopy.

2. The continuous shape-following variable spraying method based on RGBD visual detection according to claim 1, characterized in that, The RGBD vision detection-based contour sprayer includes a spray vehicle and contour arms mounted opposite each other on the spray vehicle, as well as an RGBD camera module and a spray supply system disposed between the contour arms. Multiple robotic arms are connected to the contour arms, and a spray mechanism is provided at the end of each robotic arm away from the contour arms.

3. The continuous shape-following variable spraying method based on RGBD visual detection according to claim 2, characterized in that, The spraying mechanism includes a fan, a connector, and a nozzle arranged sequentially on the robotic arm toward the end away from the contouring arm, with the fan and the nozzle fixed together by the connector.

4. The continuous shape-following variable spraying method based on RGBD visual detection according to claim 3, characterized in that, The contour sprayer is controlled to acquire images of the tree canopy, resulting in RGB and depth images, including: Control the contour sprayer to move toward the tree canopy and control the spray supply system to output a frequency signal, and update the position of the contour sprayer based on the frequency signal; Based on the position of the contour sprayer, perpendicular to the planting row corresponding to the tree, an RGBD camera is used to capture images of the tree canopy, resulting in RGB images and depth images respectively.

5. The continuous shape-following variable spraying method based on RGBD visual detection according to claim 1, characterized in that, Preprocessing the RGB and depth images, correspondingly determining the RGB image matrix and depth image matrix, includes: The RGB image and depth image are filtered and cropped sequentially to obtain the corresponding target region in the RGB image and depth image; Detection is performed based on the target region. If a tree canopy exists in the target region, the corresponding RGB image matrix and depth image matrix are determined.

6. The continuous shape-following variable spraying method based on RGBD visual detection according to claim 4, characterized in that, The ideal spray coverage area is determined using a contour-following sprayer. The depth image matrix is ​​analyzed, and the tree canopy outline is divided into spray segments. The spray positioning points and angles for each segment are then determined, including: The ideal spray coverage area is obtained by combining the spray angle and the ideal spray distance from the nozzle to the tree canopy, based on the nozzle itself. The tree canopy outline arc length is output based on the depth image matrix, and the tree canopy outline is divided by the ideal spray coverage area to obtain spray segments; Analyze each spray segment to determine the corresponding spray positioning point and positioning angle.

7. The continuous shape-following variable spraying method based on RGBD visual detection according to claim 6, characterized in that, The tree canopy outline arc length is output based on the depth image matrix. Combined with the ideal spray coverage area, the tree canopy outline is divided to obtain spray segments, including: The coordinate point matrix corresponding to the tree canopy outline is determined based on the depth image matrix, and the outline curve equation corresponding to the tree canopy is obtained. The outline curve equation is fitted to the coordinate point matrix using a polynomial method to obtain the arc length of the tree canopy outline. By comparing the ideal spray coverage area with the arc length of the tree canopy outline, the tree canopy outline is divided, the number of nozzles to be used is determined, and the corresponding spray segments are obtained based on the number of nozzles used.

8. The continuous shape-following variable spraying method based on RGBD visual detection according to claim 6, characterized in that, Analyze each spray segment to determine the corresponding spray positioning point and positioning angle, including: Based on the segmented spraying, the tree canopy outline arc data and corresponding arc length are obtained, and the slope and actual spraying distance are determined accordingly. The positioning angle and position of the nozzle for spraying are determined based on the slope and the actual spraying distance.

9. A continuous shape-following variable spraying method based on RGBD visual detection according to claim 6, characterized in that, By combining the RGB image matrix and the depth image matrix, and using a contour-following sprayer to obtain the volume and leaf area of ​​each spray segment, the spray volume and wind speed of each spray segment are determined accordingly, including: The distance from the RGBD camera module to the contour arm is measured, and the travel distance within the interval between adjacent rising edges of the frequency signal is determined in conjunction with the spray supply system. The volume of the spray segment is determined in conjunction with the depth image matrix. The relative leaf area index of the corresponding spray segment is obtained from the RGB image matrix; The spray volume and wind speed of the spray segments are obtained by combining the overall volume and relative leaf area index with the fan speed.

10. A continuous shape-following variable spray system based on RGBD visual detection, characterized in that, The system includes a processor, a communication interface, a memory, and a communication bus. The processor, the communication interface, and the memory communicate with each other through the communication bus. The processor calls logical instructions in the memory to execute the continuous shape-following variable spraying method based on RGBD visual detection as described in any one of claims 1 to 9.

Images