Intelligent spraying control strategy for watering cart

Abstract

The invention relates to an intelligent spraying control strategy for a watering cart, and belongs to the technical field of intelligent environment operation equipment, and the intelligent spraying control strategy for the watering cart comprises the steps: extracting a multi-scale feature map of a to-be-sprayed region image, processing the multi-scale feature map through employing a plurality of parallel decoupling heads, and obtaining a to-be-sprayed region image; obtaining a sprayable road area, a green belt area and a non-motorized participant area in the to-be-sprayed area image; determining depth information of a sprayable road area, a green belt area and a non-motorized participant area in the to-be-sprayed area image by adopting a preset space sensing module, and determining spatial positions of the sprayable road area, the green belt area and the non-motorized participant area based on the depth information; and performing spraying control based on the spatial positions of the sprayable road area, the green belt area and the non-motorized participant area. Intelligent spraying of all spraying areas can be achieved, and the spraying precision and efficiency are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent environmental operation equipment, and in particular to an intelligent spraying control strategy for a sprinkler truck. Background Art

[0002] With the continuous development of artificial intelligence, it has been widely used in various fields, including urban road cleaning and green belt irrigation scenarios.

[0003] In multi-task intelligent spraying tasks, at the perception level, although multi-sensor fusion solutions (such as lidar, millimeter-wave radar, etc.) can improve accuracy, they have problems such as high hardware cost and strong data heterogeneity. Although the vision-based single sensor solution has cost advantages, it is difficult to take into account the real-time and accuracy of multi-task goals (such as pedestrian detection, green belt segmentation and sprayable area detection), resulting in inaccurate recognition of spraying targets for multi-task intelligent spraying and the inability to achieve accurate intelligent spraying.

[0004] It can be seen that the existing multi-task intelligent spraying tasks cannot achieve precise intelligent spraying. Summary of the Invention

[0005] In view of this, it is necessary to provide an intelligent spraying control strategy for a sprinkler truck to solve the problem that the existing multi-task intelligent spraying tasks cannot achieve precise intelligent spraying.

[0006] In order to solve the above problems, in the first aspect, the present invention provides an intelligent spraying control strategy for a sprinkler truck, comprising: Extract multi-scale feature maps from the image of the area to be sprayed, and process them using multiple parallel decoupling heads to obtain sprayable road areas, green belt areas, and non-motorized participant areas in the image of the area to be sprayed. A preset spatial perception module is used to determine the depth information of the sprayable road area, green belt area, and non-motorized participant area in the image of the area to be sprayed, and the spatial position of the sprayable road area, green belt area, and non-motorized participant area is determined based on the depth information; Spraying control is performed based on the spatial location of sprayable road areas, green belt areas, and non-motorized participant areas.

[0007] In a possible embodiment of the present invention, extracting a multi-scale feature map of an image of an area to be sprayed includes: A feature extraction module combined with an attention mechanism is used to divide the image of the area to be sprayed into multiple non-overlapping windows. The feature extraction module is a pyramid feature extraction structure combined with a sliding window mechanism. A pyramid feature extraction structure combined with a sliding window mechanism is used to extract features from multiple non-overlapping windows to obtain multi-scale feature maps.

[0008] In a possible embodiment of the present invention, a plurality of parallel decoupling heads are used, including a non-motorized participant area identification decoupling head, a sprayable road area identification decoupling head, and a green belt area identification decoupling head, wherein: The loss function of the non-motorized participant region identification disaggregation head is:

[0009]

[0010]

[0011]

[0012] in, The loss function for the disentangled head to identify non-motorized actor regions, 、 as well as is the weight coefficient, is the binary cross loss function, is a smooth loss function, is the first intersection-over-union loss function, C is the number of detection categories, is the true label vector, is the predicted probability of category c, N is the number of samples during training, are the coordinates of the true bounding box during training, are the coordinates of the predicted bounding box during training, smooth is a smooth loss function, is the ground-truth bounding box area of ​​the non-motorized participant area, The predicted bounding box area for the non-motorized participant area; The loss function of the decoupling head for green belt area identification is:

[0013]

[0014] in, The loss function for identifying the decoupling head for the green belt area, is the second intersection-over-union loss function, P is the predicted segmentation area of ​​the green belt, and G is the actual segmentation area of ​​the green belt; The decoupling heads for sprayable road area identification are:

[0015] in, Loss function for identifying the decoupling head for sprayable road areas.

[0016] In one possible embodiment of the present invention, a preset spatial perception module is used to determine depth information of the sprayable road area, green belt area, and non-motorized participant area in the image of the area to be sprayed, including: The spatial perception module is used to determine the depth value of each pixel in the image of the area to be sprayed; The depth values ​​of the pixel points are fused into the pixel points corresponding to the sprayable road area, green belt area and non-motorized participant area to obtain the depth information of the sprayable road area, green belt area and non-motorized participant area.

[0017] In one possible embodiment of the present invention, spraying control is performed based on the spatial positions of the green belt area, the sprayable road area, and the non-motorized participant area, including: Calculate the intersection-and-union ratio of the current spraying range and the green belt area, and construct a spraying operation reward function based on the intersection-and-union ratio; Adjusting the spraying parameters based on the reward function so that the intersection-over-union ratio of the spraying range controlled by the spraying parameters and the green belt area is greater than a preset threshold; determining the movement trend of non-motorized participants based on the spatial positions of the non-motorized participant areas at adjacent time points; The spraying range is adjusted based on the movement trend so that the spraying range is within the sprayable road area and avoids the non-motorized participant area.

[0018] The beneficial effects of the present invention are as follows: the intelligent spraying control strategy for the sprinkler truck provided by the present invention can obtain multi-scale features of the image of the area to be sprayed by performing multi-scale feature extraction on the image of the area to be sprayed, which is convenient for accurate identification of each spraying area; the multi-scale feature map is processed by multiple parallel decoupling heads, so as to realize multi-task parallel processing of the image of the area to be sprayed, and parallel identification of the sprayable road area, green belt area and non-motorized participant area in the image of the area to be sprayed, thereby reducing hardware cost and improving recognition accuracy; the depth information of each spraying area is identified by the spatial perception module, and the depth information is integrated with each spraying area to obtain the spatial position of each spraying information, which can accurately identify the positional relationship between each spraying area and the sprinkler truck, and perform spraying control based on the spatial position of each spraying area, thereby realizing intelligent spraying of each spraying area, improving spraying accuracy and efficiency, and reducing resource waste. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 A schematic diagram of a flow chart of an intelligent spraying control strategy for a sprinkler truck provided in an embodiment of the present invention; Figure 2 A schematic diagram of a multi-scale feature extraction method provided by an embodiment of the present invention; Figure 3 A schematic diagram of the structure of a machine vision model provided by an embodiment of the present invention; Figure 4 A schematic diagram of feature extraction provided by an embodiment of the present invention; Figure 5 A schematic diagram of a flow chart of a depth information acquisition method provided by an embodiment of the present invention; Figure 6 A schematic diagram of deep fusion provided by an embodiment of the present invention; Figure 7 A schematic diagram of distance perception provided by an embodiment of the present invention; Figure 8 A schematic diagram of determining a region of interest provided by an embodiment of the present invention; Figure 9 A schematic diagram of a spraying system provided by an embodiment of the present invention; Figure 10 A schematic flow chart of a spraying control method provided by an embodiment of the present invention; Figure 11 A schematic diagram of a water cannon spraying provided by an embodiment of the present invention; Figure 12 A schematic diagram of calculating the actual area provided by an embodiment of the present invention; Figure 13 A schematic diagram showing the intersection-to-combination ratio of spray coverage areas provided by an embodiment of the present invention; Figure 14 A schematic diagram of the interaction process between a model and an environment provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0021] The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein the accompanying drawings constitute a part of the present invention and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not used to limit the scope of the present invention.

[0022] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present invention. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute a separate or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0023] A specific embodiment of the present invention, as Figure 1 As shown, a sprinkler truck intelligent spraying control strategy is disclosed, including: S101, extracting a multi-scale feature map of the image of the area to be sprayed, and processing the multi-scale feature map using multiple parallel decoupling heads to obtain sprayable road areas, green belt areas, and non-motorized participant areas in the image of the area to be sprayed.

[0024] In an embodiment of the present invention, an image of the area to be sprayed is acquired by an image acquisition device on a sprinkler truck. While the sprinkler truck is in motion, the image acquisition device continuously captures images within a preset range in front of the sprinkler truck. The images should include the sprayable road area, the green belt area, and the non-motorized participant area. Specifically, video images within the preset range in front of the sprinkler truck can be captured in the form of a video stream, and features are extracted from each frame of the video image. A multi-scale feature map includes features at multiple scales of the image of the area to be sprayed. Because key information such as the sprayable road area, the green belt area, and the non-motorized participant area are represented significantly differently in the image, non-motorized participants appear randomly and occupy a smaller image area, while the visual information of the green belt and the sprayable road area changes continuously with road conditions and occupies a larger area in the image, a deep learning model incorporating a window attention mechanism and a moving window attention mechanism can be used to extract features. Features of different areas are divided into non-overlapping windows. At the same time, a feature pyramid structure is used in the feature extraction process to achieve more efficient multi-scale fusion and reduce information loss, thereby improving the model's ability to capture multi-scale features. Furthermore, for different target objects, using parallel decoupling heads to process multi-scale feature maps can effectively identify multiple target objects and achieve efficient collaborative processing of multiple tasks.

[0025] S102: Using a preset spatial perception module to determine the depth information of the sprayable road area, green belt area, and non-motorized participant area in the image of the area to be sprayed, and determining the spatial positions of the sprayable road area, green belt area, and non-motorized participant area based on the depth information.

[0026] In an embodiment of the present invention, the sprayable road area, green belt area, and non-motorized participant area in the image of the area to be sprayed extracted in S101 are all two-dimensional data, which differs from the actual three-dimensional physical world. Therefore, in order to achieve precise operation of the intelligent spraying system, a three-dimensional spatial position perception module is also required to obtain spatial position information in the real physical world. The preset spatial perception module can be a monocular depth estimation model, which estimates scene depth information in real time through a monocular camera to extract depth information of the sprayable road area, green belt area, and non-motorized participant area in the image of the area to be sprayed, and uses the extracted depth information to determine the spatial position of the sprayable road area, green belt area, and non-motorized participant area.

[0027] S103, performing spraying control based on the spatial positions of the sprayable road area, green belt area, and non-motorized participant area.

[0028] In an embodiment of the present invention, after determining the spatial positions of the sprayable road area, green belt area and non-motorized participant area, intelligent spraying control is performed based on the spatial positions of each area to ensure the accuracy of the sprinkler truck's spraying control.

[0029] The intelligent spraying control strategy for sprinkler trucks provided by the present invention can obtain multi-scale features of the images of the areas to be sprayed by performing multi-scale feature extraction on the images of the areas to be sprayed, thereby facilitating the accurate identification of each spraying area; by processing the multi-system feature maps through multiple parallel decoupling heads, it can realize multi-task parallel processing of the images of the areas to be sprayed, and parallelly identify the sprayable road areas, green belt areas and non-motorized participant areas in the images of the areas to be sprayed, thereby reducing hardware costs and improving recognition accuracy; by identifying the depth information of each spraying area through a spatial perception module, and fusing the depth information with each spraying area, the spatial position of each spraying information is obtained, which can accurately identify the positional relationship between each spraying area and the sprinkler truck, and perform spraying control based on the spatial position of each spraying area, thereby realizing intelligent spraying of each spraying area, improving spraying accuracy and efficiency, and reducing resource waste.

[0030] In some possible embodiments of the present invention, Figure 2 As shown, the multi-scale feature map of the image of the area to be sprayed is extracted, including: S201, using a feature extraction module combined with an attention mechanism to divide the image of the area to be sprayed into multiple non-overlapping windows; the feature extraction module is a pyramid feature extraction structure combined with a sliding window mechanism; S202 : Perform feature extraction on the multiple non-overlapping windows using the pyramid feature extraction structure combined with the sliding window mechanism to obtain a multi-scale feature map.

[0031] In the embodiment of the present invention, Figure 3 As shown, in the intelligent spraying control strategy for sprinkler trucks adopted in the embodiment of the present invention, when extracting features from the image of the spraying area, a pre-trained machine vision model can be used for extraction. The machine vision model can be a Swin-MPM model, which uses the Swin-Transformer feature extraction block as the core unit of feature extraction to give full play to its advantage in processing global information. In each basic building block, the received feature map is divided into non-overlapping windows. At the same time, a feature pyramid structure is used in the feature extraction process to achieve more efficient multi-scale fusion and reduce information loss, so as to enhance the model's ability to capture multi-scale features, wherein, represents the feature set obtained through multi-scale processing, It means that the features extracted at different scales are targeted at different task requirements and perceived object characteristics.

[0032] The embodiment of the present invention provides a machine vision model that can realize multi-scale feature extraction of images of the area to be sprayed, thereby realizing feature extraction of non-motorized traffic participants, green belts, and sprayable road areas, thereby facilitating the subsequent accurate identification of different areas.

[0033] In some possible embodiments of the present invention, a plurality of parallel decoupling heads are used, including a non-motorized participant area identification decoupling head, a sprayable road area identification decoupling head, and a green belt area identification decoupling head, wherein: The loss function of the non-motorized participant region identification disaggregation head is:

[0034]

[0035]

[0036]

[0037] in, The loss function for the disentangled head to identify non-motorized actor regions, 、 as well as is the weight coefficient is the binary cross loss function, is a smooth loss function, is the first intersection-over-union loss function, C is the number of detection categories, is the true label vector, is the predicted probability of category c, N is the number of samples during training, are the coordinates of the true bounding box during training, are the coordinates of the predicted bounding box during training, smooth is a smooth loss function, is the ground-truth bounding box area of ​​the non-motorized participant area, The predicted bounding box area for the non-motorized participant area; The loss function of the decoupling head for green belt area identification is:

[0038]

[0039] in, The loss function for identifying the decoupling head for the green belt area, is the second intersection-over-union loss function, P is the predicted segmentation area of ​​the green belt, and G is the actual segmentation area of ​​the green belt; The decoupling heads for sprayable road area identification are:

[0040] in, Loss function for identifying the decoupling head for sprayable road areas.

[0041] In this embodiment of the present invention, a parallel decoupling head is used for feature recognition of non-motorized participants, green belts, and sprayable roads, effectively achieving efficient collaborative processing of multiple tasks. For the identification of non-motorized participant areas, the location and target category of non-motorized traffic participants are analyzed from the feature map, and the location and target category information of non-motorized traffic participants are obtained through different branches. This can effectively reduce the number of parameters and computational complexity of the algorithm model while improving its generalization ability. The decoupling process includes classification and regression tasks. The classification task refers to detecting whether non-motorized participants exist in the environment, using the binary cross entropy loss function:

[0042] The regression task predicts the location information of non-motorized participants by minimizing the difference between the bounding box and the true location, using a smooth loss function:

[0043] in, ; The intersection-over-union function is further used to optimize the overlap of the bounding boxes:

[0044] In the segmentation and decoupling process of the green belt area, the green belt plants are divided into three levels: high, medium and low. According to the area of ​​the plant areas at different height levels, the corresponding spraying strategy is formulated. According to the characteristics of the green belt extracted , the probability distribution of segmentation prediction can be expressed as:

[0045] Among them, S represents semantic category information, , that is, the classification result of each pixel, E represents the edge detail information, that is, the fine features such as segmentation edge and shape, and I is the feature of each pixel.

[0046] To reduce computing power requirements, the Swin-Unet structure is used to decouple the semantic information of the green belt area. At the same time, the cross entropy loss and the IoU loss function are combined. The cross entropy loss function is used for pixel-level multi-classification problems, while the IoU loss function is used to measure the overlap between the predicted and the true segmented area. It is defined as follows:

[0047] like Figure 4 As shown in the figure, the feature map is downsampled to 640×480, and then the key information is extracted by taking the channel extreme value method. Finally, color filling is performed to distinguish plants at different height levels.

[0048] The sprayable road area decoupling head is responsible for completing the road area segmentation task, providing accurate area information for the spraying system, and assisting in the decision-making of valve opening adjustment. Its decoupling part is designed with a simple convolution block, using the same cross entropy loss function and intersection-over-union loss function as the green belt segmentation decoupling process as the loss. In the multi-task learning framework, the reasonable allocation of loss weights for each task has a decisive impact on the performance of Swin-MPM. For the three tasks of non-motorized participant detection, green belt segmentation, and sprayable road area detection, the loss weights correspond to 、 and , the total loss function of Swin-MPM can be expressed as:

[0049] To avoid the time and training cost of manually adjusting weights, a multi-task loss balancing method based on homoscedastic uncertainty is introduced. The loss weights of multiple tasks are balanced according to the homoscedastic uncertainty of different tasks. Homoscedastic uncertainty is independent of the specific input data and remains unchanged for all input data, and only varies between different tasks. For example, in the pedestrian detection task, bounding box regression requires higher accuracy to achieve accurate positioning of the target, while the result of classification is a discrete label with lower uncertainty. As a sufficient statistic, the total loss of multi-task learning can be expressed by considering the likelihood function of each output as the product of independent distributions:

[0050] in, 、 as well as Represent the detection results of non-motorized participants, green belts and sprayable roads respectively. For vector , the process from feature extraction to decoupling head output can be expressed as:

[0051] in, 、 as well as is the loss weight.

[0052] The embodiment of the present invention provides three parallel decoupling head loss functions and dynamically adjusts the loss function weights to ensure the accuracy of the model in detecting non-motorized participants, segmenting green belts, and detecting sprayable road areas.

[0053] In some possible embodiments of the present invention, Figure 5 As shown, a preset spatial perception module is used to determine the depth information of the sprayable road area, green belt area, and non-motorized participant area in the image of the area to be sprayed, including: S501, using a spatial perception module to determine the depth value of each pixel in the image of the area to be sprayed; S502: The depth values ​​of the pixel points are integrated into the pixel points corresponding to the sprayable road area, the green belt area, and the non-motorized participant area to obtain the depth information of the sprayable road area, the green belt area, and the non-motorized participant area.

[0054] In the embodiment of the present invention, the spatial perception module is a pre-trained monocular depth estimation model that estimates the scene depth information in real time through a monocular camera. Figure 6 As shown, the input of the neural network is a 2D image , by fitting the depth estimation function, the corresponding depth is obtained , and perform scene depth information fusion to provide effective and reliable spatial position information for subsequent decision-making processes. Detection of non-motorized participants in the image, extracting spatial position information such as Figure 7 As shown in , the non-motorized participant decoupling head outputs the object detection frame information, maps the detection frame position information x, y, w, h to the depth map, calculates the depth mean of the corresponding area in the depth map, and obtains the distance information between the non-motorized participant and the camera. For green belt perception, as Figure 8As shown in the figure, based on the operating sequence of the sprinkler system (spraying the green belt from near to far), the rightmost area of ​​the green belt closest to the sprinkler in the input image is selected as the area of ​​interest to extract the green belt height level distribution and its corresponding depth distance information. In order to determine the scope of the sprayable road area, the area detection results need to be further processed to extract its right boundary and calculate the road width information. In actual scenes, due to the limited field of view of the camera, the depth change of the right boundary of the sprayable road area captured at the same time usually shows a smooth nonlinear trend. The cubic polynomial fitting method is used to model the depth change of the right boundary, and an analytical expression describing the depth distribution is obtained. The right boundary point of the area range and its corresponding depth value are extracted through the perception results, and the depth value is associated with its vertical coordinate in the image to establish a polynomial fitting model. After the fitting is completed, the fitting function can clearly describe the law of the transformation of the right boundary depth value with the image coordinate. By analyzing the fitting function, the position of its minimum point is determined, and the depth value D corresponding to the point is extracted, which is the closest distance between the camera and the right boundary, as shown in Figure 9 Based on the calculated closest distance D to the right boundary and the camera installation height H, the road width L required to be covered by the forward valve in the spraying system is calculated, thereby obtaining the true value of the sprayable road area under the current road conditions.

[0055] The embodiment of the present invention realizes the perception of the real spatial position of each area by integrating the spatial perception module, and provides an accurate spraying range for subsequent intelligent spraying control.

[0056] In some possible embodiments of the present invention, Figure 10 As shown, spraying control is performed based on the spatial locations of green belt areas, sprayable road areas, and non-motorized participant areas, including: S1001, calculating the intersection-over-union ratio of the current spraying range and the green belt area, and constructing a spraying operation reward function based on the intersection-over-union ratio; S1002, adjusting the spraying parameters based on the reward function so that the intersection-over-intersection ratio of the spraying range controlled by the spraying parameters and the green belt area is greater than a preset threshold S1003, determining a movement trend of the non-motorized participant based on the spatial positions of the non-motorized participant area at adjacent time points; S1004: Adjust the spraying range based on the movement trend so that the spraying range is within the sprayable road area and avoids the non-motorized participant area.

[0057] In an embodiment of the present invention, in order to prevent non-motorized participants from being affected during the spraying process, it is necessary to implement a target tracking function for non-motorized participants, and calculate the delay time for stopping or resuming spraying by using the moving speed of the object in the video image. Considering that the camera frame rate is 30 FPS, the time interval between each detection frame is 0.033 s. In general road scenarios, the average moving speed of non-motorized participants is 5~25 km / h, while the speed of the sprinkler truck does not exceed 25km / h. Through the analysis of frame interval and moving speed, it can be obtained that the pixel offset range of the same non-motorized participant is approximately 10~50 pixels. The relative movement speed between the non-motorized participant and the sprinkler truck is calculated by the pixel offset of the center position of the detection frame. When a non-motorized participant appears in the perception field of view, the distance between the non-motorized participant and the camera is first extracted. Then, the bounding box area is transformed to determine whether the non-motorized participant is far away from or close to the spraying range. The specific calculation process is as follows: Detect the distance between the pedestrian and the camera, and judge its movement trend by the change of the bounding box area. Suppose the depth distance of the pedestrian target at time t is , the depth distance at time t+Δt is , the movement trend can be expressed as:

[0058] Where T is the depth distance difference between adjacent moments. If T < 0, the pedestrian is close to the spraying range; if T > 0, the pedestrian is far away from the spraying range.

[0059] Record the relative distance when the target breaks into the spray coverage area and combine it with the depth change offset T Calculate its relative motion speed with the frame interval Δt v :

[0060] If the target is close to the spraying range, the spraying width will be dynamically adjusted or spraying will be stopped to ensure safe operation of the system.

[0061] When executing the spraying control strategy, first initialize the spraying distance and set the initial spraying distance =0, time =0 and set the maximum spraying distance and minimum spraying distance Extract the boundary distance L of the sprayable area and detect non-motorized traffic participants. Use the camera sensor to detect whether non-motorized traffic participants have entered the spraying range. If a non-motorized traffic participant is detected continuously and approaches the spraying range, calculate the object distance for the current frame. Use depth information and pixel distance changes to calculate the relative distance between the non-motorized traffic participant and the camera in the current frame, and calculate the target distance for the next frame. Predict the position of the non-motorized traffic participant in the next frame, calculate its relative distance to the camera, record its intrusion speed, and calculate the spraying resumption delay t. Record the relative speed of the non-motorized traffic participant entering the spraying range. Based on the speed and position of the non-motorized traffic participant, calculate the spraying resumption delay t. Adjust the spraying distance L based on the current frame object distance and the next frame target distance to ensure that the spraying area covers all areas of the road to the right while avoiding the object. Resume spraying after a delay of t. After the calculated delay t, resume spraying to ensure that the spraying area covers all areas of the road to the right.

[0062] The embodiment of the present invention can prevent the sprinkler truck from affecting non-motorized participants when spraying by adding avoidance control for non-motorized participants.

[0063] In an embodiment of the present invention, for the control of the spraying range, it is necessary to combine the intersection and union ratio of the spraying range and the green belt area to perform spraying control. First, it is necessary to establish a mathematical relationship model between the water cannon parameters and the water flow coverage area. When the water cannon action changes, it guides the update of the environmental state. It is also used to evaluate the environmental feedback after the water cannon action is executed. At the same time, with the spraying water cannon as the intelligent agent, a Markov decision process is established through factors such as state, action, transition probability and reward to construct a complete decision-making framework. In the green belt spraying decision-making process, the actual spraying water flow coverage area is difficult to measure directly, but it can be obtained from the spraying angle, water flow angle, water flow initial velocity and relative distance to the green belt. The initial water flow velocity can be expressed as:

[0064] in, is the initial velocity of the water flow at the outlet, A is the cross-sectional area of ​​the outlet, d is the radius of the outlet, Q is the water flow rate of the outlet per unit time, P is the water pressure, is the density of water.

[0065] like Figure 11 As shown, the water flow sprayed by the water monitor is usually fan-shaped, and the size and shape of the water flow are affected by the water flow angle. , initial water velocity , nozzle angle The spray coverage is calculated as follows:

[0066]

[0067] in, and They are the maximum height and minimum height of the water flow when the spraying distance is D, and the cross-sectional height of the water flow in the vertical direction Used to calculate the intersection-over-union ratio of the spray coverage area and the green belt area, where .

[0068] Slice the green belt perception results and calculate the spray coverage intersection ratio in the direction of water spraying, such as Figure 12 As shown, the number of plant pixels at each height level of the green belt is extracted within the area of ​​interest, and the ratio of the number of pixels in different height categories to the area of ​​each height interval covered by the water flow is calculated. Finally, the number of pixels in each category and the height interval are output, which are used to calculate the spraying delay ratio and the height range to be sprayed. After obtaining the boundary range of the green belt plant distribution in the image coordinate system, it is also necessary to convert it to the real height. According to the principle of geometric imaging, the pixel height can be converted to the actual height through the camera installation height and its focal length. The actual height and its pixel height on the image plane have the following proportional relationship:

[0069] in, is the actual height of the green belt, is the pixel height of the green belt in the image of the area to be sprayed, is the depth distance between the camera and the subject, f is the focal length of the camera.

[0070] By constructing a spraying operation reward function based on the intersection-union ratio, the spraying parameters are adjusted based on the reward function so that the intersection-union ratio of the spraying range controlled by the spraying parameters and the green belt area is greater than a preset threshold.

[0071] In an embodiment of the present invention, when calculating the intersection-and-union ratio of the spraying range and the actual area of ​​the green belt area, the height range of the green belt in the real world can be calculated based on the height boundary range of the green belt in the image of the area to be sprayed, and the intersection-and-union ratio of the spray coverage area is calculated. The intersection-and-union ratio of the spray coverage area is obtained by accumulating the three different height levels. The intersection-and-union ratio of the spray coverage area between each height level of the green belt plants is calculated separately. The calculation formula is as follows:

[0072] Among them, such as Figure 13 As shown, Indicates different height level categories of green belts, Indicates the proportion of area perception results of plants at different height levels, Indicates the height of the actual green belt area at different height levels, Indicates the spraying height. The specific meaning of the subscript is Figure 13 As shown in , based on the above formula, the intersection and union ratio of the spray coverage areas between the height levels of green belt plants can be calculated.

[0073] Furthermore, the spraying parameters are adjusted based on the reward function, including: The spraying angle, water flow angle and spraying speed of the water cannon of the sprinkler truck are adjusted based on the reward function.

[0074] In the embodiment of the present invention, the water cannon is used as an intelligent agent, and its action space includes: spraying angle, water flow angle and spraying speed. According to the specification parameters of the water cannon, the value ranges of the spraying angle, water flow angle and spraying speed are 0~90°, 0~120° and 0~24.26 m / s respectively. By discretizing the values, a complete action space is constructed. The state space needs to accurately reflect the information of the current spraying environment, including the green belt perception result information and the water cannon spraying state information. The interaction process between the model and the environment is as follows Figure 14 As shown. Reward function The intersection-over-union ratio of the water flow coverage area Aspray and the perceived green belt area Atarget is used as the measurement standard. The reward obtained after each action is executed is calculated as follows:

[0075] Among them, k is an adjustable parameter, and 5 can be selected to control the exponential growth rate. By nonlinearly and dynamically amplifying the contribution of high intersection-union ratios, the spraying agent is encouraged to get closer to the ideal state of high coverage and high precision during training. At the same time, when the spraying intersection-union ratio is greater than 0.8, higher rewards are given to help the spraying agent remember excellent behaviors and improve learning efficiency. Through continuous interactive learning with the environment, the model gradually optimizes the spraying strategy to achieve efficient green belt spraying operations. Based on reward feedback, the agent will dynamically adjust the spraying angle, water flow angle and initial spraying velocity to optimize the spray coverage intersection-union ratio. When the intersection-union ratio approaches 1, it means that the spray coverage area is close to overlapping with the target green belt area, which can be expressed as:

[0076] in, 、 as well as The optimal values ​​of spray angle, water flow angle and initial spray velocity are obtained through continuous adjustment and optimization of the system through reinforcement learning strategy. During the interaction, the sum of all reward values ​​is calculated as follows:

[0077] in, is the reward factor, is the reward calculation function.

[0078] The present invention constructs a reward function through the intersection and union ratio of the green belt area and the spraying range, and optimizes the spraying angle, water flow angle and initial spraying velocity to ensure that the green belt area is fully sprayed and save water resources.

[0079] Those skilled in the art will appreciate that all or part of the process steps of the above-described embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, such as a magnetic disk, an optical disk, a read-only memory, or a random access memory.

[0080] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by any technician familiar with this technical field within the technical scope disclosed by the present invention should be covered by the scope of protection of the present invention.

Claims

1. A sprinkler truck intelligent spraying control strategy, characterized in that: include: Extracting a multi-scale feature map of the image of the area to be sprayed, and processing the multi-scale feature map using multiple parallel decoupling heads to obtain a sprayable road area, a green belt area, and a non-motorized participant area in the image of the area to be sprayed; Determine depth information of the sprayable road area, green belt area, and non-motorized participant area in the image of the area to be sprayed using a preset spatial perception module, and determine the spatial positions of the sprayable road area, green belt area, and non-motorized participant area based on the depth information; Spraying control is performed based on the spatial positions of the sprayable road area, green belt area, and non-motorized participant area.

2. The intelligent spraying control strategy of a sprinkler truck according to claim 1 is characterized in that: The step of extracting a multi-scale feature map of the image of the area to be sprayed comprises: A feature extraction module combined with an attention mechanism is used to divide the image of the area to be sprayed into multiple non-overlapping windows; the feature extraction module is a pyramid feature extraction structure combined with a sliding window mechanism; The pyramid feature extraction structure combined with the sliding window mechanism is used to perform feature extraction on the multiple non-overlapping windows to obtain a multi-scale feature map.

3. The intelligent spraying control strategy of a sprinkler truck according to claim 1 is characterized in that: The plurality of parallel decoupling heads include a non-motorized participant area identification decoupling head, a sprayable road area identification decoupling head, and a green belt area identification decoupling head, wherein: The loss function of the non-motorized participant region identification decoupling head is: in, The loss function for the disentangled head to identify non-motorized actor regions, 、 as well as is the weight coefficient, is the binary cross loss function, is a smooth loss function, is the first intersection-over-union loss function, C is the number of detection categories, is the true label vector, is the predicted probability of category c, N is the number of samples during training, are the coordinates of the true bounding box during training, are the coordinates of the predicted bounding box during training, smooth is a smooth loss function, is the ground-truth bounding box area of ​​the non-motorized participant area, The predicted bounding box area for the non-motorized participant area; The loss function of the green belt area identification decoupling head is: in, The loss function for identifying the decoupling head for the green belt area, is the second intersection-over-union loss function, P is the predicted segmentation area of ​​the green belt, and G is the actual segmentation area of ​​the green belt; The sprayable road area identification decoupling head is: in, Loss function for identifying the decoupling head for sprayable road areas.

4. The intelligent spraying control strategy of a sprinkler truck according to claim 1 is characterized in that: The method of using a preset spatial perception module to determine depth information of the sprayable road area, green belt area, and non-motorized participant area in the image of the area to be sprayed includes: The spatial perception module is used to determine the depth value of each pixel in the image of the area to be sprayed; The depth values ​​of the pixel points are fused into the pixel points corresponding to the sprayable road area, green belt area and non-motorized participant area to obtain the depth information of the sprayable road area, green belt area and non-motorized participant area.

5. The intelligent spraying control strategy of a sprinkler truck according to claim 1 is characterized in that: The spraying control based on the spatial positions of the green belt area, the sprayable road area and the non-motorized participant area includes: Calculate the intersection-and-union ratio of the current spraying range and the green belt area, and construct a spraying operation reward function based on the intersection-and-union ratio; Adjusting the spraying parameters based on the reward function so that the intersection-over-union ratio of the spraying range controlled by the spraying parameters and the green belt area is greater than a preset threshold; determining a movement trend of the non-motorized participant based on the spatial positions of the non-motorized participant area at adjacent time points; The spraying range is adjusted based on the movement trend so that the spraying range is within the sprayable road area and avoids the non-motorized participant area.

Images