A method and system for intelligent water spraying control of sprinkler trucks

Abstract

This invention discloses an intelligent water spraying control method and system for sprinkler trucks. The control system includes a vision recognition system, a data processing unit, and a controller. The vision recognition system is used to collect images of plants in green belts during the operation of the intelligent sprinkler truck. During the operation of the sprinkler truck, the system combines the information collected from the images to control the pitch angle of the water cannon and the dwell time at a certain height, so as to achieve effective spraying of plants at different heights. This can effectively reduce the labor intensity of workers and improve work efficiency.

Description

Technical Field

[0001] This invention belongs to the field of sprinkler truck technology, and particularly relates to an intelligent sprinkler truck sprinkler control method and system. Background Technology

[0002] Water sprinkler trucks, also known as spray trucks, are commonly used for urban road washing, dust suppression, landscaping, and emergency firefighting. They play a vital role in improving urban environmental management, road cleanliness, and the city's image. However, some problems exist in actual operation. The operation is relatively cumbersome. During operation, the sprinkler truck also needs to spray plants in roadside green belts. Because the plants in these belts vary in height and distribution, the spraying angle needs frequent adjustments, making the spraying process tedious and laborious. Manual operation often results in uneven spraying.

[0003] There are two main methods in the field of machine vision-based plant detection:

[0004] One approach is the traditional image processing detection method. This method relies on human observation to determine the feature differences between different height levels of the green belt, such as color and texture, and then designs segmentation thresholds or uses regional contrast methods to achieve segmentation and distinguish plants of different heights. However, this method is greatly affected by adverse environmental conditions such as lighting, weather, background, and occlusion, and lacks robustness. In the actual operation of sprinkler trucks, due to the large number of pedestrians and complex background environments, this method performs poorly and is not suitable for plant detection by sprinkler trucks.

[0005] Secondly, there's the object detection method, which utilizes a DCNN (Deep Convolutional Neural Network) model to detect, classify, and select plant bounding boxes. However, since green belt plants are not isolated entities but rather composed of many interconnected plants, the bounding box method in images captured by sprinkler trucks often includes a lot of useless background information due to the viewing angle. Furthermore, some areas are even composed of plants of varying heights, meaning object detection methods can only provide the center location of the object, lacking recognition of the overall plant features. Moreover, this method mostly relies on bounding box recognition, which easily generates multiple overlapping bounding boxes in areas with plants of different heights, increasing the complexity of the recognition process. Therefore, this method is also unsuitable for plant detection in sprinkler trucks.

[0006] The semantic segmentation method used in this invention can classify each plant pixel in the image, separate the plant area from the background area along the edge of the plant area, and achieve better segmentation results. Finally, different spraying strategies are formulated based on the segmentation results. Summary of the Invention

[0007] The purpose of this invention is to provide an intelligent water spraying control method and system for water trucks, so as to solve the problem of complex and cumbersome water spraying operations.

[0008] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0009] A method for intelligent water spraying control of a water truck, applied to the water spraying control system of an intelligent water truck, the method comprising:

[0010] S101. Collect images of green belt plants while the intelligent sprinkler truck is in motion, label the plant areas according to the height of the plants into three categories: high, medium and low, establish a green belt plant dataset, and divide it into training set and test set;

[0011] S102. A Unet semantic segmentation model with added SeNet attention mechanism is adopted. Since the plant region exhibits a large feature size in the image, a ResNet50 network with a deeper number of layers is selected as the backbone network to extract feature maps. The error is calculated using the cross-entropy loss function, and the backpropagation training of the network model is completed. The model is tested on the test dataset, and the trained semantic segmentation model is output after the prediction error reaches an acceptable range.

[0012] The trained semantic segmentation model is deployed on the vehicle-mounted intelligent terminal hardware. During the actual spraying operation of the sprinkler truck, the vehicle-mounted camera is used to take pictures of the plants in the green belt. The obtained images are input into the segmentation model, which classifies each pixel in the image and segments the plant image regions of different heights.

[0013] S103. Develop corresponding spraying strategies based on the area of ​​plant regions at different height levels;

[0014] S104. During the actual spraying of plants by the sprinkler truck, images of plants in the green belt are captured by the on-board camera and input into the plant image semantic segmentation model to obtain the height distribution of each plant. According to the spraying strategy, a signal is sent to the control system to control the pitch angle and spraying time at a certain height when the water bubbles are sprayed.

[0015] Furthermore, the annotation of plant regions includes: calling a pre-built semantic segmentation model for plant images to identify and segment plant regions at different heights.

[0016] Furthermore, the labeling of the plant areas includes dividing the collected green belt plant images into two parts along the horizontal direction, labeled as Region 1 and Region 2 from left to right. The plant areas corresponding to the segmentation results obtained in Region 1 are far away from the sprinkler truck, making their height features difficult to distinguish visually, which has a significant impact on the recognition accuracy. Therefore, this part is discarded.

[0017] For the segmentation results in Region 2, there are certain differences in the area and number of plant species in different height levels. An exhaustive method was used to formulate corresponding spraying strategies.

[0018] Furthermore, in order to ensure that the processing speed of the detection algorithm can keep up with the speed at which the sprinkler truck is moving and taking pictures, and to avoid the situation where the sprinkler truck has already left the current spraying area before the detection algorithm has obtained a result, it is necessary to set the camera shooting interval.

[0019] An intelligent control system for a sprinkler truck includes a vision recognition system, a data processing unit, and a controller to realize the above-mentioned intelligent sprinkler truck control method.

[0020] The visual recognition system is used to collect images of plants in green belts during the operation of the intelligent sprinkler truck;

[0021] The data processing unit is used to identify and segment the collected plant images; and to determine the spraying angle of the intelligent sprinkler truck based on the height of the segmented plants and the area of ​​the region.

[0022] The controller is used to adjust the spray angle of the water cannon.

[0023] The beneficial effects of this invention are as follows: This invention utilizes a semantic segmentation network model to achieve intelligent spraying functionality for sprinkler trucks. A deep learning semantic segmentation model is used to obtain information about the plant areas in roadside green belts, segmenting them into plant areas of different heights. The segmentation results are input into a spraying strategy calculation model to obtain the sprinkler truck's spraying strategy for the current target area. Finally, the water cannon control and adjustment system in this invention can automatically adjust the vertical angle of the water cannon and control the spray volume according to a pre-set communication protocol and the established spraying strategy. This effectively improves the automation capabilities of sprinkler trucks, achieving intelligent upgrades and enabling sprinkler trucks to perform spraying operations based on plant height, plant type, and the area of ​​different plant regions, thus effectively reducing water waste during operation. Attached Figure Description

[0024] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a flowchart illustrating an intelligent water spraying control method for a sprinkler truck disclosed in an embodiment of the present invention.

[0026] Figure 2This is a plant segmentation network structure diagram disclosed in an embodiment of the present invention.

[0027] Figure 3 This is a diagram illustrating a water truck spraying strategy disclosed in an embodiment of the present invention. Specific implementation methods

[0028] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present invention.

[0029] This invention discloses an intelligent water spraying control method for sprinkler trucks, applied to the control system of the intelligent water spraying control method for sprinkler trucks. The control system includes a vision recognition system, a data processing unit, and a controller. The vision recognition system is used to collect images of plants in green belts during the operation of the intelligent sprinkler truck. During the operation of the sprinkler truck, the system combines the information collected from the images to control the pitch angle of the water cannon spray and the dwell time at a certain height, so as to achieve the purpose of effectively spraying different plants. This can effectively reduce the labor intensity of workers and improve work efficiency.

[0030] Specifically, please refer to Figure 1 The intelligent water spraying control method for a sprinkler truck disclosed in this embodiment includes the following steps:

[0031] S101. Collect images of green belt plants while the intelligent sprinkler truck is in motion, label the plant areas according to the plant height into three categories: high, medium, and low, establish a green belt plant dataset, and divide it into training and test sets.

[0032] Specifically, a visual recognition system is used to collect images of the area in front of the sprinkler truck.

[0033] During the actual operation of the sprinkler truck, images of roadside green belt plants were captured. A green belt plant dataset was created using the labelme annotation tool, and the green belt plants were divided into three categories: tall, medium, and short. The Unet semantic segmentation model, which has less data requirements, was selected, and the pre-trained ResNet50 was used as the backbone network for model training.

[0034] S102. A semantic segmentation model with an added attention mechanism is trained and tested on a test dataset. Once the prediction error is within an acceptable range, the trained semantic segmentation model is obtained. The trained semantic segmentation model is then deployed to the vehicle-mounted intelligent terminal hardware. During the actual spraying operation of the sprinkler truck, the vehicle-mounted camera is used to take pictures of the plants in the green belt. The resulting images are input into the semantic segmentation model, which classifies each pixel in the image and segments plant image regions of different heights.

[0035] Optionally, a pre-built semantic segmentation model for plant images can be invoked to identify and segment plant regions at different heights.

[0036] The image was divided into two parts horizontally, with the segmentation ratio determined based on the camera's installation angle. These parts were labeled Region 1 and Region 2 from left to right. The plants detected in Region 1 were too far from the sprinkler truck, making height features difficult to distinguish, so this region was discarded. The plants detected in Region 2 were the targets to be sprayed by the sprinkler truck, and the spraying strategy for this region was determined based on its segmentation results.

[0037] Please see Figure 2 The plant segmentation network structure diagram uses the Unet semantic segmentation network. Semantic segmentation requires determining the category of each pixel in the image for accurate segmentation. Unet is a commonly used semantic segmentation model; it is simple, efficient, easy to understand, and easy to build. However, in small datasets like green belts, state-of-the-art models do not have a significant advantage over the lightweight Unet. Therefore, the Unet semantic segmentation model is suitable for real-time detection of roadside green belt plants by sprinkler trucks. Its U-shaped structure can be called an "encoder-decoder" structure. The encoder consists of two 3×3 convolutional layers plus a 2×2 max-pooling layer, forming a downsampling module repeated four times. The decoder consists of an upsampling convolutional layer, feature concatenation, and two 3×3 convolutional layers repeatedly. It can fuse the deep semantic information of green belt plant images after multiple convolutions with shallow features such as canopy texture, enabling better differentiation of plants of different heights through features such as tree height and canopy texture, and separation from complex backgrounds. This allows for real-time segmentation of green belt plant images captured by sprinkler truck cameras.

[0038] The plant segmentation in this example can be viewed as a multi-class classification task, and cross-entropy is used to calculate the loss value: in For positive sample labels, For network output.

[0039] Deep learning can learn autonomously on a large amount of sample data, while the Unet network used can achieve sufficient segmentation accuracy with a small amount of sample data.

[0040] S103. Develop corresponding spraying strategies based on the area of ​​plant regions at different height levels;

[0041] Optionally, for the segmentation results in Region 2, there are certain differences in the area and number of plant species in different height levels. An exhaustive method can be used to formulate corresponding spraying strategies.

[0042] The specific spraying strategy is as follows:

[0043] Since the water truck will drive along the lane line closest to the green belt when spraying plants, and the width of the green belt is roughly the same, only the elevation angle of the water cannon needs to be controlled and adjusted.

[0044] This invention, based on the obtained segmented image, first determines the height levels within the target region: the high, medium, and low height levels correspond to water cannon spraying elevation angles of θ1, θ2, and θ3, respectively, where θ1 > θ2 > θ3. The number of plant height levels in the current region is represented by n.

[0045] If n=1, then choose the corresponding angle θ. i That's all;

[0046] If n=2, then the range of spray pitch angles [θ] needs to be determined. i θ j ], where i > j;

[0047] If n=3, then the spray pitch angle range is [θ3, θ1].

[0048] After determining the spray pitch angle, a delay time t needs to be inserted at equal intervals during the spraying process. This delay represents the dwell time at the corresponding spray angle, with a corresponding delay weight of w. i .

[0049] The delay weight w is calculated based on the area of ​​different plant height levels. i Let the areas of tall, medium, and short plants be denoted as S1, S2, and S3 respectively, and the total area of ​​the planted area be denoted as S. The spraying delay weight can be calculated using the following formula: Where T is the total time it takes for a sprinkler truck to spray the same area, and T is a fixed value.

[0050] During the operation of the intelligent sprinkler truck, the coverage area of ​​its video and image capture region two is as follows: The driving speed is v, and the computation time of the plant segmentation network model is [time value missing]. Every t seconds, a frame of image is taken and input into the segmentation network model. The water cannon spray coverage area is... To ensure that the water cannon can adjust its spray angle in real time according to changes in the image displayed while the water truck is moving, the following conditions are required: .

[0051] Please see Figure 3 This embodiment discloses a water truck spraying strategy, which specifically includes the following steps:

[0052] S201. Check the number of plant height categories within the photo area;

[0053] S202. Determine how many plant height categories there are;

[0054] If the plant height category is 1, then execute S203: determine the elevation angle of the water cannon spray according to the plant height category; S204: determine the spraying time according to S203;

[0055] If the plant height category is not 1, then execute S205: determine the pitch angle range of the water cannon spraying according to the plant height category; S206: allocate the delay weight of water cannon spraying according to the plant area of ​​different height categories.

[0056] After completing the above procedures, spraying can begin.

[0057] S104. During the actual spraying of plants by the sprinkler truck, images of plants in the green belt are captured by the on-board camera and input into the plant image semantic segmentation model to obtain the height distribution of each plant. According to the spraying strategy, a signal is sent to the control system to control the pitch angle and spraying time at a certain height when the water bubbles are sprayed.

[0058] In this embodiment, the RK3288 vehicle-mounted chip is selected as the central processing chip, and the plant segmentation network model is deployed on it. Video images of the sprinkler truck operation are acquired through the vehicle-mounted camera, and the corresponding spraying strategy is calculated. Based on the spraying strategy, a corresponding serial port protocol is established to send the pitch angle range and delay weight to the STM32 microcontroller, thereby achieving water cannon pitch angle control.

[0059] This invention involves a water truck using an onboard visual recognition system to capture images of plants in the green belt at fixed intervals during its operation. A data processing unit then processes these images, segmenting them using a pre-defined plant segmentation network model, labeling them from left to right as Region 1 and Region 2. Plants detected in Region 1 are too far from the truck to be easily distinguishable due to height differences, and are therefore discarded. Plants detected in Region 2 are the targets for spraying, and the segmentation results for this region determine the spraying strategy. The spraying strategy determines the water cannon's elevation angle and the spraying time at a specific height based on the plant's height, species, and area. This ensures that plants at different heights are sprayed at different heights, avoiding the drawbacks of previous methods relying on worker experience, reducing worker workload, and effectively saving water while improving spraying efficiency.

[0060] An embodiment of an intelligent control system for a sprinkler truck:

[0061] The intelligent control system of the sprinkler truck in this embodiment includes a vision recognition system, a data processing unit, and a controller to realize the intelligent sprinkler truck control method described above.

[0062] The visual recognition system is used to collect images of plants in green belts during the operation of the intelligent sprinkler truck;

[0063] The data processing unit is used to identify and segment the collected plant images; and to determine the spraying angle of the intelligent sprinkler truck based on the height of the segmented plants and the area of ​​the region.

[0064] The controller is used to adjust the spray angle of the water pump.

[0065] Since the intelligent water spraying control method for sprinkler trucks has already been described in the embodiments of the intelligent water spraying control method for sprinkler trucks, this embodiment will not repeat the description of the intelligent water spraying control method for sprinkler trucks.

[0066] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0067] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0068] The above embodiments can be combined arbitrarily. The descriptions of the disclosed embodiments and the features recorded in the embodiments of this specification can be substituted or combined with each other, so that those skilled in the art can implement or use this application.

[0069] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for intelligent water spraying control of a sprinkler truck, characterized in that: The method of applying machine vision technology to the sprinkler control system of an intelligent sprinkler truck includes: S101. Collect images of green belt plants while the intelligent sprinkler truck is in motion, label the plant areas according to the height of the plants into three categories: high, medium and low, establish a green belt plant dataset, and divide it into training set and test set; S102. A semantic segmentation model with an added attention mechanism is trained and tested on a test dataset. Once the prediction error is within an acceptable range, the trained semantic segmentation model is obtained. The trained semantic segmentation model is then deployed to the vehicle-mounted intelligent terminal hardware. During the actual spraying operation of the sprinkler truck, the vehicle-mounted camera is used to take pictures of the plants in the green belt. The resulting images are input into the semantic segmentation model, which classifies each pixel in the image and segments plant image regions of different heights. S103. Develop corresponding spraying strategies based on the area of ​​plant regions at different height levels; S104. During the actual spraying of plants by the sprinkler truck, images of the plants in the green belt are captured by the vehicle-mounted camera and input into the plant image semantic segmentation model to obtain the height distribution of each plant. According to the spraying strategy, a signal is transmitted to the control system to control the pitch angle of the water cannon spray and the spraying time at a certain height; the spraying strategy includes adjusting the water cannon pitch angle to meet the following conditions: The coverage area of ​​Region 2 is as follows: The driving speed is v, and the computation time of the plant segmentation network model is [time value missing]. A frame of image is taken every t seconds and input into the plant image segmentation network model. The water cannon spray coverage area is... .

2. The intelligent water spraying control method for a sprinkler truck according to claim 1, characterized in that: The annotation of plant regions includes: calling a pre-built semantic segmentation model for plant images to identify and segment plant regions at different heights.

3. The intelligent water spraying control method for a sprinkler truck according to claim 1, characterized in that: The labeling of the plant areas includes dividing the collected green belt plant images into two parts along the horizontal direction, labeled as Region 1 and Region 2 from left to right. The plant areas corresponding to the segmentation results obtained in Region 1 are far away from the sprinkler truck, making their height features difficult to distinguish visually and significantly affecting the recognition accuracy. Therefore, this part is discarded.

4. The intelligent water spraying control method for a sprinkler truck according to claim 3, characterized in that: For the segmentation results in Region 2, there are certain differences in the area and number of plant species in different height levels. An exhaustive method was used to formulate corresponding spraying strategies.

5. The intelligent water spraying control method for a sprinkler truck according to claim 1, characterized in that: The spraying strategy includes a formula for calculating the weighting of spraying delay: Among them, the area calculation delay weight w for different plant height levels is used. i The areas of tall, medium, and short plants are denoted as S1, S2, and S3, respectively. The total area of ​​the plant area is denoted as S. T is the total time spent by the sprinkler truck spraying the same area, and T is a fixed value.

6. The intelligent water spraying control method for a sprinkler truck according to claim 5, characterized in that: The spraying strategy includes adjusting the elevation angle of the water cannon spray. Within region two, there are three height levels: high, medium, and low, corresponding to water cannon spray elevation angles of θ1, θ2, and θ3 respectively, where θ1 > θ2 > θ3. The number of plant height levels in the current region is represented by n. If n=1, then choose the corresponding angle θ. i That's all; If n=2, then the range of spray pitch angles [θ] needs to be determined. i θ j ], where i > j; If n=3, then the spray pitch angle range is [θ3, θ1].

7. An intelligent control system for a sprinkler truck, characterized in that: It includes a visual recognition system, a data processing unit, and a controller to implement the intelligent water spraying control method for a sprinkler truck as described in any one of claims 1-6; The visual recognition system is used to collect images of plants in green belts during the operation of the intelligent sprinkler truck; The data processing unit is used to identify and segment the collected plant images; and to determine the spraying angle of the intelligent sprinkler truck based on the height of the segmented plants and the area of ​​the region. The controller is used to adjust the spray angle of the water pump.

Images