Expressway green belt maintenance system based on unmanned aerial vehicle and inspection robot

Abstract

The invention provides an expressway green belt maintenance system based on an unmanned aerial vehicle and an inspection robot, and the system comprises the steps: enabling the unmanned aerial vehicle to fly according to a preset green belt air track, collecting a camera image and a thermal imaging image of an expressway green belt, and preliminarily judging whether the expressway green belt has a pest and disease damage region or not; if the pest and disease damage area exists, after the unmanned aerial vehicle moves to the position above the pest and disease damage area, worm eggs, worm holes and disease spots on plant leaves of the pest and disease damage area are recognized, the pest and disease damage grade is obtained, and if the pest and disease damage grade is recognized to be low, a pest and disease damage control instruction and the position of the pest and disease damage area are sent to the inspection robot; if the inspection robot receives the pest control instruction, the inspection robot moves to the position of the pest control area and then stops moving, the spraying device is started, a biological control agent is released through the spraying device, and plant leaves in the pest control area are controlled. Through cooperative work of the inspection robot and the unmanned aerial vehicle, full-intelligent maintenance and pruning of the highway green belt are achieved.

Description

Technical Field

[0001] The present invention relates to the cross - field of agricultural intelligent irrigation and drone technology, and particularly to a highway green belt maintenance system based on drones and inspection robots. Background Art

[0002] At present, with the popularization of highways, the number of families with vehicles is increasing. Driving on highways is an important way of inter - city transportation, and the green belts along the way are an essential scenic line. The green belts not only have the functions of separating traffic and ensuring safety, but also can visually relieve drivers' fatigue. However, since highways are far from towns and cities, the green belts may wither if not maintained in time.

[0003] At present, the maintenance method of highways mainly relies on highway maintenance personnel to regularly inspect and repair the highway green belt area. On the one hand, the vehicle speed on highways is relatively fast, and highway maintenance personnel are prone to traffic accidents; on the other hand, highway maintenance personnel use roadblocks for isolation during inspection and repair, which will occupy the communication lane and cause traffic jams, and is likely to lead to traffic accidents due to emergency lane changes by following vehicles. Summary of the Invention

[0004] The present invention provides a highway green belt maintenance system based on drones and inspection robots. Its main purpose is to achieve full - intelligent maintenance and pruning of highway green belts through the collaborative work of inspection robots and drones, and solve the problem of traffic accidents prone to occur in manual maintenance.

[0005] In a first aspect, an embodiment of the present invention provides a highway green belt maintenance system based on drones and inspection robots. The drone is equipped with a visible - light camera, a macro - camera, and a thermal imager. During the operation of the highway green belt maintenance system, the inspection robot is located on both sides of the highway green belt, and the drone is located above the highway green belt. The straight - line distance between the drone and the inspection robot is within the wireless communication range, where:

[0006] The drone flies along a preset aerial trajectory of the green belt. During the flight at a first height, it collects the video images of the highway green belt through the camera and collects the thermal - imaging images of the highway green belt through the thermal imager. By comparing the video images and the thermal - imaging images with the standard images and the standard thermal - imaging images respectively, it preliminarily judges whether there are pest - and - disease - damaged areas in the highway green belt;

[0007] If there is such a pest and disease area, after the drone descends from the first height to the second height and moves to the position of the pest and disease area, it uses the macro camera to identify the eggs, wormholes, and disease spots on the plant leaves in the pest and disease area to obtain the pest and disease level. If the identified pest and disease level is relatively low, it sends a pest control instruction and the position of the pest and disease area to the inspection robot;

[0008] The inspection robot is equipped with a spraying device and a biological control agent. If the inspection robot does not receive the pest control instruction, it moves uniformly along the preset green belt guardrail trajectory and trims the high-speed green belt during the movement;

[0009] After the inspection robot receives the pest control instruction, it moves to the position of the pest and disease area and then stops moving, and turns on the spraying device to release the biological control agent through the spraying device to treat the plant leaves in the pest and disease area.

[0010] Furthermore, the steps of preliminarily judging whether there is a pest and disease area in the high-speed green belt by comparing the captured image and the thermal imaging image with the standard image and the standard thermal imaging image respectively include:

[0011] Extract the image feature vectors of the captured image. The image feature vectors include morphological features, color features, and texture features. The morphological features include contour area, perimeter, and shape complexity. The color features include hue, saturation, and lightness. The texture features include energy, contrast, correlation, and entropy;

[0012] Match the image feature vectors of the captured image with the image feature vectors of each standard image to obtain the similarity score between the captured image and each standard image;

[0013] Compare the average value of the similarity scores with the similarity threshold. If the average value is less than the similarity threshold, obtain the first judgment result, and the first judgment result is that there is a pest and disease area in the high-speed green belt.

[0014] Furthermore, the further steps of preliminarily judging whether there is a pest and disease area in the high-speed green belt by comparing the captured image and the thermal imaging image with the standard image and the standard thermal imaging image respectively include:

[0015] Compare the temperature values of the thermal imaging image and the standard thermal imaging image pixel by pixel and calculate the temperature difference;

[0016] Adopt an image segmentation algorithm to segment the thermal imaging image into different temperature regions, analyze the shape, size, and position of each region, and extract the thermal distribution region features of the thermal imaging image;

[0017] Compare the thermal distribution area features of the thermal imaging image with those of the standard thermal imaging image, and calculate the thermal distribution difference between the thermal imaging image and each standard thermal imaging image;

[0018] Set a temperature difference threshold and a thermal distribution pattern difference threshold. When the temperature difference exceeds the temperature difference threshold and the area ratio of the thermal distribution difference exceeding the thermal distribution pattern difference threshold exceeds 5%, a second judgment result is obtained, and the second judgment result is that there is a pest area in the high-speed green belt;

[0019] If at least one of the first judgment result and the second judgment result is that there is a pest area in the high-speed green belt, it is preliminarily determined that there is a pest area in the high-speed green belt.

[0020] Further, the inspection robot includes a fuselage body and an inspection controller. The fuselage body includes a first fixing bar, a second fixing bar, and a third fixing bar connected in an "H" shape. The top end of the first fixing bar is slidably and perpendicularly connected to one end of the second fixing bar, and the other end of the second fixing bar is slidably connected to the top end of the third fixing bar. The bottom ends of the first fixing bar and the third fixing bar are both connected to roller skating devices;

[0021] A slidable track is installed on the second fixing bar, the upper region of the first fixing bar close to the second fixing bar, and the upper region of the third fixing bar close to the second fixing bar. The slidable track is connected to one end of an extension arm, and the other end of the extension arm is connected to a pruning tool.

[0022] Further, during the operation of the inspection robot and when the inspection robot does not receive the pest control instruction, the inspection robot divides the high-speed green belt into several sections. For each section of the high-speed green belt, the inspection robot trims the high-speed green belt section by section according to a "Z" - shaped route.

[0023] Further, the inspection robot is also equipped with a motor, and both sides of the extension arm are respectively connected to the slidable track through buckles;

[0024] The first fixing bar is located on one side of the high-speed green belt, the second fixing bar is located above the high-speed green belt, and the third fixing bar is located on the other side of the high-speed green belt;

[0025] During the operation of the inspection robot and when the inspection robot has not received the pest control instruction, for each section of the highway green belt, after the inspection controller drives the extension arm to move to the preset position of the first fixing bar through the motor, the extension arm is fixed by the buckle. Then, through the trimming tool, the inspection robot moves from the starting point to the end point of this section of the highway green belt and trims one side of this section of the highway green belt.

[0026] The inspection controller controls the buckle to loosen, drives the extension arm to move to the preset position of the second fixing bar through the motor, fixes the extension arm by the buckle, the inspection robot moves from the end point to the starting point of this section of the highway green belt, and trims the upper side of this section of the highway green belt through the trimming tool.

[0027] The inspection controller controls the buckle to loosen, drives the extension arm to move to the preset position of the third fixing bar through the motor, fixes the extension arm by the buckle, the inspection robot moves from the starting point to the end point of this section of the highway green belt, and trims the other side of this section of the highway green belt through the trimming tool.

[0028] Further, during the operation of the inspection robot and after the inspection robot receives the pest control instruction, when the inspection robot moves to the position of the pest and disease area, the inspection controller controls the roller skating device to stop moving and the trimming tools to stop working.

[0029] Further, the steps of using the macro camera to identify the eggs, wormholes, and disease spots on the plant leaves in the pest and disease area to obtain the pest and disease level include:

[0030] The drone controls the macro camera to take close-up pictures of the pest and disease area of the highway green belt.

[0031] According to the close-up pictures, determine whether white, yellow, or black appears in the close-up pictures, set a color threshold, mark the pixel points of white, yellow, and black, and determine the number of eggs per unit area.

[0032] By analyzing the color gradient change, locate the percentage of the leaf area occupied by the wormholes and the percentage of the leaf area occupied by the disease spots in the close-up pictures.

[0033] According to the number of eggs per unit area, the percentage of the leaf area occupied by the wormholes, and the percentage of the leaf area occupied by the disease spots, determine the pest and disease level.

[0034] A highway green belt maintenance system based on a drone and an inspection robot proposed by an embodiment of the present invention has the following advantages:

[0035] (1) Through the cooperation of drones and inspection robots, the drones can fly in the air without being restricted by ground obstacles and can quickly cover large areas of highway green belts. The inspection robots are limited to moving on the guardrails, and their speed and coverage are relatively limited. Therefore, using the inspection robots as the operation terminals for spraying pesticides and pruning the highway green belts can improve the collaborative work efficiency. A single flight of a drone can cover green belts for several kilometers, enabling a preliminary scan of the overall area in a short time, quickly discovering potential problem areas, and then guiding the inspection robots to go for a detailed inspection, thus improving the maintenance work efficiency.

[0036] The camera is installed on the inspection robot and can only obtain images near its traveling route and in the horizontal direction. It is difficult to capture the situations in the high, far, and some blocked areas of the green belt. Therefore, compared with the solution of installing the camera on the inspection robot, the recognition accuracy of the present invention is higher, and thus the maintenance effect is better.

[0037] (2) The moving speed of the inspection robot is relatively slow. If it relies on the camera carried by it to monitor large areas of green belts, it will take a lot of time. The flight speed of the drone is fast. Through the cooperation of the drone and the inspection robot, the present invention can complete the image acquisition of large areas in a short time, greatly improving the monitoring efficiency.

[0038] (3) The drone flies over the green belt using its high-definition camera and, through thermal imaging technology, detects subtle changes such as the temperature and color of plant leaves to discover potential pest and disease areas; uses the macro camera carried by it to conduct a more detailed inspection of the plants in the pest and disease areas to further determine the type and severity of the pests and diseases, improving the recognition accuracy.

[0039] (4) In the embodiment of the present invention, the inspection robot is of the H type, does not occupy the driving lanes on the highway, and can avoid traffic congestion problems caused by maintenance work; and can achieve fully automatic maintenance without the need for manual participation, avoiding traffic accidents caused by manual participation.

[0040] (5) The embodiment of the present invention utilizes the fixed trajectory position information of the highway green belt. Without the need to carry a high-precision positioning device, the inspection robot can surely move to the position of the pest and disease area along the established trajectory of the highway green belt, achieving the function at the lowest cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a schematic structural diagram of a highway green belt maintenance system based on a drone and an inspection robot provided by an embodiment of the present invention;

[0042] Figure 2Schematic diagram of the structure of a highway green belt maintenance system based on an unmanned aerial vehicle and an inspection robot provided by a preferred embodiment of the present invention;

[0043] Figure 3 Schematic diagram of the internal electrical structure of an inspection robot provided by an embodiment of the present invention. Description of the drawings:

[0045] 10, unmanned aerial vehicle; 11, visible light camera;

[0046] 12, macro camera; 13, thermal imager;

[0047] 20, inspection robot; 21, first fixing strip;

[0048] 22, second fixing strip; 23, third fixing strip;

[0049] 24, sliding track; 25, extension arm;

[0050] 26, spraying device; 261, liquid storage barrel;

[0051] 262, infusion tube; 263, nozzle;

[0052] 27, trimming tool.

[0053] The realization, functional features and advantages of the object of the present invention will be further described with reference to the embodiments and the accompanying drawings. Detailed implementation manners

[0054] The following describes in detail the implementation manners of the present application. The examples of the implementation manners are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The implementation manners described below by referring to the accompanying drawings are exemplary only for explaining the present application and should not be construed as limiting the present application.

[0055] In the description of the present application, it should be understood that the orientation or positional relationship indicated by the terms "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "circumferential", "radial", etc. are based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as limiting the present application.

[0056] To enable those skilled in the art to better understand the solution of this application, the following will clearly and completely describe the technical solutions in the embodiments of this application with reference to the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative efforts fall within the scope of protection of this application.

[0057] In the embodiments of this application, "at least one" means one or more; "a plurality" means two or more. In the description of this application, terms such as "first", "second", "third", etc. are only used for the purpose of distinguishing descriptions, and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying an order. In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of this application, "a plurality" means two or more unless otherwise clearly and specifically defined.

[0058] The reference to "an embodiment" or "some embodiments" etc. described in this specification means that a specific feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of this application. Thus, the terms "include", "comprise", "have" and their variants in this specification all mean "including but not limited to" unless otherwise specifically emphasized in other ways.

[0059] It should be noted that "connection" in the embodiments of this application can be understood as electrical connection, and the connection of two electrical components can be a direct or indirect connection between the two electrical components. For example, when A is connected to B, it can be either a direct connection between A and B or an indirect connection between A and B through one or more other electrical components.

[0060] As Figures 1 to 3 shown, the drone 10 is equipped with a visible light camera 11, a macro camera 12, and a thermal imager 13. During the operation of the highway green belt maintenance system, the inspection robot 20 is located on both sides of the highway green belt, the drone 10 is located above the highway green belt, and the straight-line distance between the drone 10 and the inspection robot 20 is within the wireless communication range, where:

[0061] The drone 10 flies along a preset aerial trajectory of the green belt. During the flight at the first altitude, it collects the video images of the highway green belt through the camera, and collects the thermal imaging images of the highway green belt through the thermal imager 13. By comparing the video images and the thermal imaging images with the standard images and the standard thermal imaging images respectively, it preliminarily judges whether there are pest and disease areas in the highway green belt;

[0062] If there is such a pest and disease area, the drone 10 descends from the first altitude to the second altitude. After moving to the position of the pest and disease area, it uses the macro camera 12 to identify the eggs, wormholes, and disease spots on the plant leaves in the pest and disease area to obtain the pest and disease level. If the identified pest and disease level is relatively low, it sends a pest control instruction and the position of the pest and disease area to the inspection robot 20;

[0063] The inspection robot 20 is equipped with a spraying device 26 and a biological control agent. If the inspection robot 20 does not receive the pest control instruction, the inspection robot 20 moves uniformly along the preset green belt guardrail trajectory and trims the highway green belt during the movement;

[0064] If the inspection robot 20 receives the pest control instruction, it moves to the position of the pest and disease area and then stops moving, and turns on the spraying device 26 to release the biological control agent through the spraying device 26 to treat the plant leaves in the pest and disease area.

[0065] In this embodiment, the drone 10 and the inspection robot 20 cooperate with each other. The drone 10 is equipped with a visible light camera 11, a macro camera 12, and a thermal imager 13. The visible light camera 11, the macro camera 12, and the thermal imager 13 are all installed below the drone 10, and can clearly capture the highway green belt below.

[0066] During the operation of the highway green belt maintenance system, the drone 10 flies along a preset aerial trajectory of the green belt. Since the position of the highway green belt is fixed, the preset aerial trajectory of the drone 10 is also fixed. This aerial trajectory can be input into the drone 10 in advance, thereby reducing the equipment loaded by the drone 10 and improving the endurance of the drone 10.

[0067] In addition, at the start of work, since the flight speed of the drone 10 is faster than that of the inspection robot 20, the drone 10 flies at a high altitude, and the inspection robot 20 trims and maintains the high-speed green belt behind the drone 10. Specifically, the drone 10 first flies at a first altitude, takes pictures of the high-speed green belt, collects video images of the high-speed green belt through the camera, and collects thermal imaging images of the high-speed green belt through the thermal imager 13. The video images and the thermal imaging images are respectively compared with the standard images and the standard thermal imaging images to preliminarily determine whether there are pest and disease areas in the high-speed green belt.

[0068] If there is a pest and disease area, the drone 10 descends from the first altitude to the second altitude, and after moving to the position above the pest and disease area, at a position relatively close to the top of the pest and disease area by the drone 10, the specific value of this relatively close position can be determined according to the actual situation, and this embodiment does not make specific limitations on this; the macro camera 12 is used to identify the eggs, wormholes, and disease spots on the plant leaves in the pest and disease area to obtain the pest and disease level. If the identified pest and disease level is relatively light, a pest control instruction and the location of the pest and disease area are sent to the inspection robot 20.

[0069] The inspection robot 20 slowly moves along the high-speed green belt at a certain speed. When it does not receive a pest control instruction, the inspection robot 20 trims the three sides of the high-speed green belt during the movement; after the inspection robot 20 receives the pest control instruction, it stops moving after moving to the position of the pest and disease area, and turns on the spraying device 26 carried on the inspection robot 20 to release a biological control agent through the spraying device 26 to treat the plant leaves in the pest and disease area.

[0070] Among them, the steps of respectively comparing the video image and the thermal imaging image with the standard image and the standard thermal imaging image to preliminarily determine whether there are pest and disease areas in the high-speed green belt include:

[0071] Extract the image feature vectors of the video image. The image feature vectors include morphological features, color features, and texture features. The morphological features include contour area, perimeter, and shape complexity. The color features include hue, saturation, and lightness. The texture features include energy, contrast, correlation, and entropy;

[0072] Match the image feature vectors of the video image with the image feature vectors of each standard image to obtain the similarity scores of the video image and each standard image;

[0073] Compare the average value of the similarity scores with the similarity threshold. If the average value is less than the similarity threshold, a first judgment result is obtained, and the first judgment result is that there are pest and disease areas in the high-speed green belt.

[0074] Among them, comparing the captured image and the thermal imaging image with a standard image and a standard thermal imaging image respectively to preliminarily determine whether there is a pest and disease area in the highway green belt further includes:

[0075] Compare the temperature values of the thermal imaging image and the standard thermal imaging image pixel by pixel, and calculate the temperature difference;

[0076] Adopt an image segmentation algorithm to segment the thermal imaging image into different temperature regions, analyze the shape, size, and position of each region, and extract the thermal distribution region features of the thermal imaging image;

[0077] Compare the thermal distribution region features of the thermal imaging image with those of the standard thermal imaging image, and calculate the thermal distribution difference between the thermal imaging image and each standard thermal imaging image;

[0078] Set a temperature difference threshold and a thermal distribution pattern difference threshold. When the temperature difference exceeds the temperature difference threshold and the area ratio of the thermal distribution difference exceeding the thermal distribution pattern difference threshold exceeds 5%, a second judgment result is obtained, and the second judgment result is that there is a pest and disease area in the highway green belt;

[0079] If at least one of the first judgment result and the second judgment result is that there is a pest and disease area in the highway green belt, it is preliminarily determined that there is a pest and disease area in the highway green belt.

[0080] In the embodiment of the present invention, the acquisition process of the captured image and the thermal imaging image is as follows:

[0081] Captured image acquisition: The unmanned aerial vehicle 10 equipped with the visible light camera 11 flies over the highway green belt according to the planned route, and takes images at a certain time interval (such as every 3 - 5 seconds) or distance interval (such as every 50 - 100 meters). Ensure that the captured image is clear and can capture details such as the leaves and branches of plants, covering the characteristics of different types of plants and different growth stages.

[0082] Thermal imaging image acquisition: Synchronously use a thermal imaging device to obtain a thermal imaging image. The thermal imager 13 generates an image according to the thermal radiation situation on the plant surface, and different temperature regions are presented in different colors on the image. During the acquisition process, ensure that the parameter settings of the thermal imager 13 are correct, such as temperature resolution, frame rate, etc., to obtain accurate thermal imaging data and clearly reflect the temperature distribution of the plants.

[0083] Standard camera images: Under the condition of healthy growth of plants in the highway green belt, typical samples are selected in different seasons and at different time periods, and a large number of camera images are taken. These images should cover various plant species, growth forms, and situations under different environmental conditions (such as light, humidity, etc.). Screen and process these images, remove blurred and incomplete images, and establish a standard camera image library.

[0084] Standard thermal imaging images: Similarly, under the condition of plant health, use the thermal imager 13 to obtain thermal imaging images of different plants during normal physiological activities. Since the temperature of plants is affected by environmental factors (such as air temperature, light intensity), data should be collected under various environmental conditions. Analyze these thermal imaging images, determine the normal temperature range and thermal distribution characteristics of healthy plants under different environments, and establish a standard thermal imaging image library.

[0085] Combine the comparison results of camera images and thermal imaging images for comprehensive analysis. If obvious abnormal forms or symptoms of plants are found in the camera images, and at the same time the thermal imaging images show abnormal temperatures in the corresponding areas, then the possibility of pests and diseases in that area is very high.

[0086] Extract morphological features from the preprocessed camera images: Use contour detection algorithms (such as Canny edge detection combined with contour search algorithms) to extract the contour information of plants, and calculate morphological features such as contour area, perimeter, and shape complexity.

[0087] Extract color features from the preprocessed camera images: Convert the grayscale image to the HSV color space, extract the hue (H), saturation (S), and value (V) features of each pixel point, and statistically analyze the color histogram as the color feature vector.

[0088] Extract texture features from the preprocessed camera images: Use the gray-level co-occurrence matrix (GLCM) algorithm to calculate the co-occurrence probability of pixel pairs with different gray levels in the image, and extract texture features such as energy, contrast, correlation, and entropy.

[0089] Adopt the nearest neighbor algorithm (such as the K-nearest neighbor algorithm, KNN), and find the K nearest neighbors that are most similar to the feature vector of the real-time image in the set of feature vectors of the standard image library, and determine the similarity by calculating metrics such as Euclidean distance.

[0090] Set a similarity threshold (such as 0.8). When the similarity scores of the real-time image with all standard images are lower than this threshold, it is judged that there are pests and diseases in that area; otherwise, it is considered that the plants in that area grow normally.

[0091] Specifically, the step of using the macro camera 12 to identify the eggs, wormholes, and disease spots on the plant leaves in the pest and disease area to obtain the pest and disease level includes:

[0092] The UAV 10 controls the macro camera 12 to take close-up pictures of the pest and disease area of the high-speed green belt;

[0093] Based on the close-up pictures, determine whether white, yellow, or black appears in the close-up pictures, set a color threshold, mark the pixel points of white, yellow, and black, and determine the number of eggs per unit area;

[0094] Locate the percentage of the leaf area occupied by the wormholes and the percentage of the leaf area occupied by the lesions in the close-up pictures by analyzing the color gradient changes;

[0095] Determine the pest and disease level based on the number of eggs per unit area, the percentage of the leaf area occupied by the wormholes, and the percentage of the leaf area occupied by the lesions.

[0096] First, the steps of image processing and color pixel marking are as follows:

[0097] Color space conversion: Convert the picture from the RGB color space to the HSV (hue, saturation, value) color space. Because in the HSV space, the description of colors is more in line with human visual perception, and the judgment and threshold setting of colors are more intuitive. For example, for white in the HSV space, the hue (H) can be any value in the range of 0 - 360°, the saturation (S) is close to 0, and the value (V) is close to 1; for yellow, the hue (H) is approximately between 30° - 60°, and the saturation (S) and value (V) are relatively high; for black, the hue (H) is arbitrary, the saturation (S) is close to 0, and the value (V) is close to 0.

[0098] Set the color threshold: According to the characteristics of the above colors in the HSV space, set the corresponding threshold range. For example, the HSV threshold range for white can be set as H: 0 - 360, S: 0 - 0.1, V: 0.9 - 1; the HSV threshold range for yellow is set as H: 30 - 60, S: 0.5 - 1, V: 0.5 - 1; the HSV threshold range for black is set as H: 0 - 360, S: 0 - 0.1, V: 0 - 0.1.

[0099] Pixel point marking: Traverse each pixel point in the picture and determine whether its HSV value is within the set color threshold range. If it is within the range, mark the pixel point as the corresponding color (white, yellow, or black) and record its coordinates.

[0100] The steps for calculating the number of eggs are as follows:

[0101] Egg pixel point screening: According to the color characteristics of the eggs (such as white or yellow), screen out the pixel points that may belong to the eggs from the marked pixel points. For example, for white eggs, screen out the set of pixel points marked as white.

[0102] Cluster analysis: Use a clustering algorithm (such as the DBSCAN density clustering algorithm) to cluster the selected pixel points. Since the eggs are usually distributed in clusters, the pixel points belonging to the same egg can be divided into a cluster through clustering. Each cluster represents an egg.

[0103] Egg count statistics: Count the number of clusters obtained by clustering, which is the number of eggs. Then, calculate the number of eggs per unit area based on the actual area of the photographed area.

[0104] Area percentage analysis:

[0105] For wormholes and lesions, use an image segmentation algorithm (such as threshold segmentation, edge detection combined with region growing algorithm, etc.) to segment them from the leaf background. For example, for a wormhole, since its color may be darker (close to black), it can be initially segmented according to the color threshold of black, and then morphological operations (such as erosion and dilation) are used to remove noise and small interfering regions to obtain the accurate wormhole region. For lesions, they are segmented according to their color and shape characteristics. Lesions usually have irregular shapes and specific color ranges (such as yellow, brown, etc.).

[0106] Area calculation: Calculate the number of pixels in the segmented wormhole region and lesion region. Assume the number of pixels in the wormhole region is n hole , and the number of pixels in the lesion region is n spot , and the number of pixels corresponding to the total leaf area is n leaf . According to the proportional relationship between the number of pixels and the actual area (which can be obtained through camera calibration), calculate the actual areas of the wormhole and the lesion. Thus, calculate the percentage of the leaf area occupied by the wormhole as (n hole / n leaf )*100%, and the percentage of the leaf area occupied by the lesion as (n spot / n leaf )*100%.

[0107] Set the grading criteria: According to experience and relevant research, set the numerical ranges corresponding to different pest and disease levels. For example, the criteria for mild pest and disease damage can be that the number of eggs per unit area is less than 5 eggs per square centimeter, the percentage of the leaf area occupied by the wormhole is less than 5%, and the percentage of the leaf area occupied by the lesion is less than 10%; the criteria for moderate pest and disease damage are that the number of eggs per unit area is between 5 - 15 eggs per square centimeter, the percentage of the leaf area occupied by the wormhole is between 5% - 15%, and the percentage of the leaf area occupied by the lesion is between 10% - 30%; for severe pest and disease damage, the number of eggs per unit area is greater than 15 eggs per square centimeter, the percentage of the leaf area occupied by the wormhole is greater than 15%, and the percentage of the leaf area occupied by the lesion is greater than 30%.

[0108] Level judgment: Compare the calculated number of insect eggs per unit area, the percentage of the leaf area occupied by insect holes, and the percentage of the leaf area occupied by disease spots with the set level standards to determine the current pest and disease level. For example, if the number of insect eggs per unit area is 8 per square centimeter, the percentage of the leaf area occupied by insect holes is 8%, and the percentage of the leaf area occupied by disease spots is 15%, it is judged as moderate pest and disease according to the standard.

[0109] As Figure 1 and Figure 2 shown, the inspection robot 20 includes a fuselage body and an inspection controller. The fuselage body includes a first fixing bar 21, a second fixing bar 22, and a third fixing bar 23 connected in an "H" shape. The top end of the first fixing bar 21 is slidably and vertically connected to one end of the second fixing bar 22, and the other end of the second fixing bar 22 is slidably connected to the top end of the third fixing bar 23. The bottom ends of the first fixing bar 21 and the third fixing bar 23 are both connected to a roller skating device;

[0110] A slidable track 24 is installed on the second fixing bar 22, the upper region of the first fixing bar 21 close to the second fixing bar 22, and the upper region of the third fixing bar 23 close to the second fixing bar 22. The slidable track 24 is connected to one end of an extension arm 25, and the other end of the extension arm 25 is connected to a trimming tool 27.

[0111] During the operation of the inspection robot 20 and when the inspection robot 20 does not receive the pest control instruction, the inspection robot 20 divides the high-speed green belt into several sections. For each section of the high-speed green belt, the inspection robot 20 trims the high-speed green belt section by section along a "Z" - shaped route.

[0112] The high-speed green belt maintenance system provided by the embodiment of the present invention includes a drone 10 and an inspection robot 20. As Figure 2 shown, a visible light camera 11, a macro camera 12, and a thermal imager 13 are installed below the drone 10; the inspection robot 20 includes a first fixing bar 21, a second fixing bar 22, and a third fixing bar 23. As Figure 1 shown, a slidable track 24 is installed in the upper region where the first fixing bar 21 is connected to the second fixing bar 22, the second fixing bar 22, the upper region where the third fixing bar 23 is connected to the second fixing bar 22. An extension arm 25 is installed on this slidable track 24, and the extension arm 25 can slide up and down on the slidable track 24; a trimming tool 27 is installed at the end of the extension arm 25, and the trimming tool 27 is used to trim the high-speed green belt.

[0113] The bottom ends of the first fixing bar 21 and the second fixing bar 22 are installed with roller skating devices, which are used to drive the inspection robot 20 to move. Moreover, a spraying device 26 is carried on the inspection robot 20. The spraying device 26 includes a liquid storage bucket 261, a liquid delivery pipe 262 and a nozzle 263. After the biological control agent is mixed with water, it is placed on the liquid storage bucket 261 and flows down along the liquid delivery pipe 262 and is sprayed out from below the nozzle 263, so as to achieve biological control of the high-speed green belt.

[0114] Among them, Figure 3 As shown, an inspection controller, a roller skating device drive motor, a spraying device 26 drive motor, an extension arm 25 drive motor, a trimming tool 27 drive motor and a wireless communication module are also arranged inside the inspection robot 20. The drive motors are arranged beside the corresponding components and are used to drive the corresponding components. The roller skating drive motor is used to drive the wheels at the bottom end of the first fixing bar 21 to rotate. The spraying device 26 drive motor is used to open the connection between the liquid storage bucket 261 and the liquid delivery pipe 262. The extension arm 25 drive motor is used to drive the extension arm 25 to move to the corresponding position along the sliding track 24. The trimming tool 27 drive motor is used to drive the trimming tool 27 to rotate, so as to trim the high-speed green belt. The wireless communication module is used to realize communication with the drone 10.

[0115] During the operation of the highway maintenance system, the first fixing bar 21, the second fixing bar 22 and the third fixing bar 23 are erected on both sides of the highway green belt. And when the inspection robot 20 does not receive the biological control instruction sent by the drone 10, the inspection robot 20 trims only one section of the high-speed green belt each time. For example, it trims only 2m of the green belt each time, and trims both sides of the high-speed green belt in a "Z" - shaped route when trimming. Specifically, the inspection controller first controls the extension arm 25 to slide along the sliding track 24 to the first fixing bar 21 and fixes the extension arm 25 through the buckles on both sides of the extension arm 25; trims the left side of the high-speed green belt. At this time, the inspection robot 20 has moved from one end of this section of the high-speed green belt to the other end; then the buckles on both sides of the extension arm 25 are released, the extension arm 25 moves from the first fixing bar 21 to the second fixing bar 22, and the inspection robot 20 trims the upper side of this section of the high-speed green belt. The inspection robot 20 moves from the other end of this section of the high-speed green belt to one end; finally, the buckles on both sides of the extension arm 25 are released, the extension arm 25 moves from the second fixing bar 22 to the third fixing bar 23, and the extension arm 25 is fixed through the buckle, and the inspection robot 20 moves from one end of the high-speed green belt to the other end to achieve trimming of the right side of the high-speed green belt.

[0116] After the inspection robot 20 receives the biological control instruction and moves to the location of the pest and disease area, the pruning tool 27 stops working first, and the control spray device 26 starts working. The biological control agent is sprayed onto the highway green belt through the spray device 26, thereby realizing the pest control of the highway green belt.

[0117] Finally, it should be noted that a wireless communication module is also set inside the drone 10, and wireless communication between the inspection robot 20 and the drone 10 is realized through this wireless communication module.

[0118] A highway green belt maintenance system based on the drone 10 and the inspection robot 20 proposed in the embodiment of the present invention has the following advantages:

[0119] (1) Through the cooperation of the drone 10 and the inspection robot 20, the drone 10 can fly in the air without being restricted by ground obstacles and can quickly cover a large area of the highway green belt. The inspection robot 20 is limited to moving on the guardrail, and its speed and coverage are relatively limited. Therefore, the inspection robot 20 is used as an operation terminal to spray medicine and trim the highway green belt, improving the collaborative work efficiency. The drone 10 can cover a green belt of several kilometers in one flight, can conduct a preliminary scan of the overall area in a short time, quickly discover potential problem areas, and then guide the inspection robot 20 to go for a detailed inspection, improving the maintenance work efficiency.

[0120] The camera is installed on the inspection robot 20 and can only obtain images near its traveling route and in the horizontal direction. It is difficult to capture the situations of the high, far, and some blocked areas of the green belt. Therefore, compared with the solution where the camera is installed on the inspection robot 20, the recognition accuracy of the present invention is higher, and thus the maintenance effect is better.

[0121] (2) The moving speed of the inspection robot 20 is relatively slow. If it relies on it to carry a camera to monitor a large area of the green belt, it will take a lot of time. The drone 10 has a fast flight speed. Through the cooperation of the drone 10 and the inspection robot 20, the image acquisition of a large area can be completed in a short time, greatly improving the monitoring efficiency.

[0122] (3) The drone 10 flies over the green belt using its high-definition camera, and through thermal imaging technology, it detects subtle changes in the temperature, color, etc. of plant leaves to discover potential pest and disease areas; it uses the macro camera 12 carried by it to conduct a more detailed inspection of the plants in the pest and disease areas to further determine the type and severity of the pests and diseases, improving the recognition accuracy.

[0123] (4) In the embodiment of the present invention, the inspection robot 20 is of the H type and does not occupy the driving lane on the highway, which can avoid traffic congestion problems caused by maintenance work; and it can achieve fully automatic maintenance without the need for manual participation, avoiding traffic accidents caused by manual participation.

[0124] (5) The embodiment of the present invention utilizes the fixed trajectory position information of the highway green belt. Without the need to carry a high-precision positioning device, the inspection robot 20 can surely move to the position of the pest area along the established trajectory of the highway green belt, achieving the function at the lowest cost.

[0125] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is used as an example. In actual applications, the above functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above.

[0126] The above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A highway green belt maintenance system based on drones and inspection robots, characterized in that: The drone is equipped with a visible light camera, a macro camera and a thermal imager. During the operation of the highway green belt maintenance system, the inspection robot is located on both sides of the highway green belt, and the drone is located above the highway green belt. The straight-line distance between the drone and the inspection robot is within the wireless communication range, wherein: The UAV flies according to a preset green belt aerial trajectory, collects a camera image of the high-speed green belt through the camera during the flight at the first altitude, and collects a thermal imaging image of the high-speed green belt through the thermal imager, and compares the camera image and the thermal imaging image with the standard image and the standard thermal imaging image, respectively, to preliminarily determine whether there is a pest and disease area in the high-speed green belt; If the pest area exists, the drone descends from the first height to the second height and moves to a position above the pest area, and uses the macro camera to identify insect eggs, insect holes, and disease spots on plant leaves in the pest area to obtain the pest level. If the identified pest level is relatively light, the drone sends a pest control instruction and the location of the pest area to the inspection robot; The inspection robot is equipped with a spray device and a biological control agent. If the inspection robot does not receive the pest control instruction, the inspection robot moves at a constant speed along a preset green belt guardrail trajectory and trims the high-speed green belt during the movement; If the inspection robot receives the pest control instruction, it moves to the location of the pest area and then stops moving, and turns on the spray device to release the biological control agent through the spray device to control the plant leaves in the pest area.

2. The highway green belt maintenance system based on drones and inspection robots according to claim 1 is characterized in that: The camera image and the thermal imaging image are compared with the standard image and the standard thermal imaging image respectively to preliminarily determine whether there is a pest and disease area in the highway green belt, the steps include: Extracting an image feature vector of the camera image, wherein the image feature vector includes morphological features, color features, and texture features, wherein the morphological features include contour area, perimeter, and shape complexity, the color features include hue, saturation, and brightness, and the texture features include energy, contrast, correlation, and entropy; Matching the image feature vector of the camera image with the image feature vector of each standard image to obtain a similarity score between the camera image and each standard image; The average of the similarity scores is compared with a similarity threshold. If the average is smaller than the similarity threshold, a first judgment result is obtained, and the first judgment result is that there is a pest and disease area in the highway green belt.

3. The highway green belt maintenance system based on drones and inspection robots according to claim 2 is characterized in that: The method further comprises comparing the camera image and the thermal imaging image with the standard image and the standard thermal imaging image respectively to preliminarily determine whether there is a pest and disease area in the highway green belt. Comparing the temperature values ​​of the thermal imaging image and the standard thermal imaging image pixel by pixel, and calculating the temperature difference; Using an image segmentation algorithm, the thermal imaging image is segmented into different temperature regions, the shape, size, and position of each region are analyzed, and the heat distribution region characteristics of the thermal imaging image are extracted; Comparing the heat distribution area characteristics of the thermal imaging image with the heat distribution area characteristics of standard thermal imaging images, and calculating the heat distribution difference between the thermal imaging image and each standard thermal imaging image; A temperature difference threshold and a heat distribution mode difference threshold are set. When the temperature difference exceeds the temperature difference threshold, and the area where the heat distribution difference exceeds the heat distribution mode difference threshold accounts for more than 5%, a second judgment result is obtained, and the second judgment result is that there is a pest area in the highway green belt; If at least one of the first judgment result and the second judgment result is that there is a pest and disease area in the highway green belt, it is preliminarily judged that there is a pest and disease area in the highway green belt.

4. The highway green belt maintenance system based on drones and inspection robots according to claim 1 is characterized in that: The inspection robot comprises a body and an inspection controller, wherein the body comprises a first fixing bar, a second fixing bar and a third fixing bar connected in an "H" shape, wherein the top of the first fixing bar is slidably connected to one end of the second fixing bar, and the other end of the second fixing bar is slidably connected to the top of the third fixing bar, and the bottom ends of the first fixing bar and the third fixing bar are both connected to a roller sliding device; The second fixing bar, the upper end area of ​​the first fixing bar close to the second fixing bar, and the upper end area of ​​the third fixing bar close to the second fixing bar are provided with a slidable track, the slidable track is connected to one end of the extension arm, and the other end of the extension arm is connected to the pruning tool.

5. The highway green belt maintenance system based on drones and inspection robots according to claim 4 is characterized in that: During the operation of the inspection robot, and before the inspection robot receives the pest control instruction, the inspection robot divides the highway green belt into several sections. For each section of the highway green belt, the inspection robot trims the highway green belt section by section along a "Z"-shaped route.

6. The highway green belt maintenance system based on drones and inspection robots according to claim 5 is characterized in that: The inspection robot is also equipped with a motor, and both sides of the extension arm are connected to the slidable track through buckles; The first fixing strip is located on one side of the high-speed green belt, the second fixing strip is located above the high-speed green belt, and the third fixing strip is located on the other side of the high-speed green belt; During the operation of the inspection robot, and the inspection robot has not received the pest control instruction, for each section of the high-speed green belt, the inspection controller drives the extension arm to move to the preset position of the first fixing bar through the motor, fixes the extension arm through the buckle, and uses the trimming tool to move the inspection robot from the starting point to the end point of the section of the high-speed green belt and trim one side of the section of the high-speed green belt; The inspection controller controls the buckle to release, drives the extension arm to move to the preset position of the second fixing strip through the motor, fixes the extension arm through the buckle, and the inspection robot moves from the end point of the section of the high-speed green belt to the starting point, and trims the upper side of the section of the high-speed green belt through the trimming tool; The inspection controller controls the release of the buckle, drives the extension arm to move to the preset position of the third fixed bar through the motor, fixes the extension arm through the buckle, and the inspection robot moves from the starting point to the end point of the section of the high-speed green belt, and trims the other side of the section of the high-speed green belt through the trimming tool.

7. The highway green belt maintenance system based on drones and inspection robots according to claim 4 is characterized in that: During the operation of the inspection robot, and after the inspection robot receives the pest control instruction, when the inspection robot moves to the location of the pest area, the inspection controller controls the wheel skating device to stop moving and the pruning tools to stop working.

8. The highway green belt maintenance system based on drones and inspection robots according to claim 1 is characterized in that: The steps of using the macro camera to identify insect eggs, insect holes, and disease spots on the plant leaves in the pest area to obtain the pest level include: The drone controls the macro camera to take close-up pictures of the pest and disease area of ​​the highway green belt; According to the close-up picture, determine whether white, yellow or black appears in the close-up picture, set a color threshold, mark the white, yellow and black pixel points, and determine the number of eggs per unit area; Locating the percentage of the leaf area occupied by the wormholes and the percentage of the leaf area occupied by the diseased spots in the close-up image by analyzing the color gradient change; The pest level is determined based on the number of insect eggs per unit area, the percentage of the leaf area occupied by the insect holes, and the percentage of the leaf area occupied by the disease spots.

Images