Intelligent control system for water spraying of watering cart to avoid pedestrians

Abstract

The invention provides an intelligent control system for water spraying of a watering cart to avoid pedestrians. The intelligent control system comprises a sensing module, an edge calculation module, a spraying control module, a control strategy generation module and a communication bus module. The sensing module integrates an RGB camera, an infrared thermal imager and a millimeter wave radar and is used for collecting images, thermal imaging and distance data. The edge calculation module realizes pedestrian detection, tracking and path prediction based on a multi-modal fusion recognition algorithm; the control strategy generation module generates a nozzle avoidance control instruction according to the prediction result; the spraying control module comprises a plurality of angle-adjustable nozzles, and each nozzle is provided with an independent control unit and can be started and stopped in different areas; the communication bus module is used for realizing real-time synchronization of system data and a control instruction; according to the invention, accurate avoidance control of the pedestrian target in the sprinkling operation path can be realized, and the safety and the operation efficiency of the system in a complex environment are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of urban operation vehicle control, and particularly to an intelligent control system for a sprinkler truck to avoid spraying water on pedestrians. Background Art

[0002] As one of the equipment for municipal environmental sanitation operations, sprinkler trucks are widely used in regular sprinkler operations in areas such as urban main roads, commercial blocks, and around green belts. However, there are still technical limitations: First, the existing sprinkler truck control systems still mainly rely on manual operation or simple timed switch control, and cannot maintain the operation efficiency while avoiding pedestrians, resulting in problems such as unnecessary pauses and frequent operation interruptions in spraying control; Second, the existing sprinkler trucks lack the ability to accurately and real-time identify and predict the behavior of dynamic pedestrians, resulting in the inability to make timely and effective avoidance responses when pedestrians suddenly enter the sprinkling path, and there are problems such as mis-spraying or safety hazards; Therefore, an intelligent control system for a sprinkler truck to avoid spraying water on pedestrians is proposed. Summary of the Invention

[0003] In view of this, the present invention provides an intelligent control system for a sprinkler truck to avoid spraying water on pedestrians, so as to solve or alleviate the technical problems existing in the prior art and at least provide a beneficial option.

[0004] The technical solution of the present invention is realized as follows: An intelligent control system for a sprinkler truck to avoid spraying water on pedestrians, comprising a sensing module, including three sensor devices: an RGB camera, an infrared thermal imager, and a millimeter-wave radar, for collecting image information, thermal imaging data, and target distance information in front of and on the side of the sprinkler truck; an edge computing module, connected to the sensing module through a physical communication line, receiving the data output by the sensing module and detecting, tracking, and predicting the future path of pedestrian targets based on a preset multi-modal fusion recognition algorithm; a spraying control module, connected to the edge computing module through a wired control channel, the spraying control module includes adjustable-angle nozzles distributed in multiple sectors, and each nozzle is equipped with an independent electromagnetic control unit for adjusting the spraying state according to a control instruction; a control strategy generation module, communicatively connected to the edge computing module and the spraying control module respectively through a logic processing interface, for generating an avoidance control strategy according to the pedestrian path prediction result and sending corresponding instructions to the spraying control module; a communication bus module, connecting the above-mentioned sensing module, edge computing module, control strategy generation module, and spraying control module through a CAN or EtherCAT industrial bus for realizing data synchronization and instruction transmission between modules.

[0005] Further preferably, the edge computing module is an integrated embedded AI processing unit. The AI processing unit selects an embedded platform of the Jetson Orin or Jetson Xavier model, has a computing power of 20 TOPS or more, and can complete the fusion processing of the data collected by the sensing module, image target recognition, and path prediction operation based on continuous frames within 100 milliseconds. The operation results are used as the input of the control strategy generation module.

[0006] Further preferably, the multi-modal fusion recognition algorithm is constructed by the YOLOv8 network model and incorporates a Transformer structure to enhance the spatial feature expression ability for identifying occluded or partially visible pedestrian targets in images. The path prediction operation algorithm selects the long short-term memory neural network (LSTM) structure to build a time series prediction model, model the continuous frame trajectory of the target, and combine a Kalman filter to correct the prediction deviation in real time.

[0007] Further preferably, the spraying control module is provided with adjustable-angle nozzles in three independent zones. Each nozzle is equipped with a servo motor and a solenoid valve control unit, can adjust the spraying angle within the range of 30 to 120 degrees, and execute opening and closing actions according to the instructions of the control strategy generation module. The instructions are uniformly transmitted by the communication bus module.

[0008] Further preferably, after receiving the path prediction result provided by the edge computing module, the control strategy generation module analyzes the predicted path of each target pedestrian based on the set time window logic, and determines whether it will enter the spraying sector covered by any nozzle within time T through calculation. If the judgment result is "yes", a control instruction to close the nozzle is sent to the spraying control module at the moment of T - Δt milliseconds. If the judgment result is "no", the spraying state remains unchanged. If the predicted path deviates from the threshold range or the target exits the spraying influence area, the spraying action is immediately restored.

[0009] Further preferably, the infrared thermal imager in the sensing module is installed in the central area in front of the sprinkler truck for collecting human infrared radiation images under low light conditions. The millimeter-wave radar is installed in the areas on both sides of the sprinkler truck for detecting the relative distance and speed information of lateral dynamic obstacles. The RGB camera is installed in the front part of the vehicle for obtaining forward visible light images under daytime conditions. The three types of data are processed by the edge computing module after time series alignment and spatial fusion.

[0010] Further preferably, the edge computing module integrates an adaptive parameter adjustment module, which is used to self-learn and adjust the recognition threshold, time window parameters, and nozzle closing delay value based on the pedestrian recognition rate, path prediction error, and spraying control execution lag information accumulated during the sprinkler operation process, and the updated parameters are fed back to the edge computing and control strategy generation module in real time.

[0011] Further preferably, the communication bus module adopts the CAN or EtherCAT industrial bus protocol to stably and rapidly transmit the sensing data, control strategy, and execution instructions among the functional modules, and the communication protocol supports error code verification and master-slave control modes.

[0012] Further preferably, the system reserves a docking interface with the urban management Internet of Things platform to upload the operation data, recognition results, and system status information through the standardized API protocol.

[0013] Further preferably, the sensing module, edge computing module, control strategy generation module, spraying control module, and communication bus module are each encapsulated in an independent functional unit and interconnected and communicate through standard industrial control interfaces.

[0014] Due to the above technical solutions adopted in the embodiments of the present invention, it has the following advantages: First, the present invention adopts an adjustable-angle nozzle structure with sector distribution, combined with the control strategy output by the edge computing module, to achieve independent control of the spraying area. Each nozzle is equipped with an independent electromagnetic control unit and an angle adjustment mechanism, which can achieve precise start-stop and spraying angle adjustment according to the real-time avoidance strategy, so as to only close the local nozzle area where there is an intersection of the predicted pedestrian paths, and the other areas maintain normal spraying, effectively improving the continuity and efficiency of the sprinkler operation and avoiding the interruption of the vehicle-level spraying.

[0015] Second, the present invention constructs a sensing module based on an RGB camera, an infrared thermal imager, and a millimeter-wave radar, combines a deep recognition network model of YOLOv8 + Transformer, and substitutes the pedestrian path prediction of the LSTM + Kalman filter combination to achieve high-precision recognition and future path prediction of dynamic pedestrians, enabling the system to make early judgments and responses within the 100 - 300 ms time window before the sprinkler nozzle executes the action, improving the safety of pedestrian avoidance and the real-time performance of the system response.

[0016] The above summary is only for the purpose of the specification and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features of the present invention will become apparent by reference to the drawings and the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0018] Figure 1 It is a system module relationship diagram of the present invention. Detailed implementation manners

[0019] In the following text, only some exemplary embodiments are briefly described. As those skilled in the art can recognize, the described embodiments can be modified in various different ways without departing from the spirit or scope of the present invention. Therefore, the drawings and the description are considered to be exemplary in nature rather than restrictive.

[0020] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0021] As Figure 1 shown, the embodiment of the present invention provides an intelligent control system for a sprinkler truck to avoid pedestrians when spraying water, including: A perception module, which includes three sensor devices, namely an RGB camera, an infrared thermal imager, and a millimeter-wave radar, and is used to collect image information, thermal imaging data, and target distance information in front of and on the side of the sprinkler truck; An edge computing module, which is connected to the perception module through a physical communication line, receives the data output by the perception module, and detects, tracks, and predicts the future path of the pedestrian target based on a preset multi-modal fusion recognition algorithm; A spraying control module, which is connected to the edge computing module through a wired control channel. The spraying control module includes adjustable-angle nozzles distributed in multiple sectors, and each nozzle is equipped with an independent electromagnetic control unit, which is used to adjust the spraying state according to the control instruction; A control strategy generation module, which is communicatively connected to the edge computing module and the spraying control module respectively through a logic processing interface, and is used to generate an avoidance control strategy according to the pedestrian path prediction result and send the corresponding instruction to the spraying control module; A communication bus module, which connects the above-mentioned perception module, edge computing module, control strategy generation module, and spraying control module through a CAN or EtherCAT industrial bus, and is used to realize data synchronization and instruction transmission between modules.

[0022] Embodiment 1: The intelligent control system of this embodiment is installed on the basis of a conventional sprinkler truck chassis and is composed of a perception module, an edge computing module, a control strategy generation module, a spraying control module, and a communication bus module. Each sub-module is encapsulated with a standard industrial connection interface; The perception module consists of the following components: Two RGB cameras are installed above the central axis of the sprinkler truck, with a horizontal viewing angle of 90°, used to collect visible light images during the day; One infrared thermal imager is installed in the upper-middle part of the vehicle head, with a vertical elevation angle of 15°, which can be used for detecting human heat sources in nighttime or low visibility environments; Two millimeter-wave radars are symmetrically arranged on both sides of the vehicle body below the rearview mirrors, to detect the distance and speed information of dynamic objects in real time.

[0023] The signal lines of the above sensors are integrally connected to the edge computing module through a waterproof wire harness. The edge computing module uses the Jetson Orin NX platform, runs the Ubuntu L4T system, and integrates the TensorRT acceleration library, which is used to deploy the YOLOv8-T+Transformer fusion detection network and the LSTM path prediction network; The spraying control module is an electronic control unit, equipped with 5 sector nozzles. Each sector covers a 36° fan-shaped area, and the overall front covers a 180° sprinkling range. The nozzle is equipped with a closed-loop angle servo system to achieve real-time adjustment of the spraying angle. The response time of each solenoid valve is 25ms, meeting the requirements of millisecond-level fast spraying start and stop; The 5 sector nozzles form a multi-segment adjustable nozzle array through the angle adjustment of the electronic control unit.

[0024] The communication bus module is a high anti-interference CAN-FD industrial bus, with a backbone bandwidth of 2Mbps. It uses a star topology to connect each functional module. This communication bus module communicates with the upper vehicle-mounted PLC controller, supporting master-slave communication, redundancy detection, CRC check, and disconnection protection to ensure the stable operation of the control system.

[0025] After the system is started, the perception module continuously obtains image frames, thermal imaging frames, and radar point cloud data at a frequency of 20Hz, and transmits them to the edge computing module for preprocessing. The image data is first sent to the YOLOv8-T network for candidate target detection, and the output format is class, x_center, y_center, width, height, confidence. After the detection layer, the Transformer is used to fuse the spatial context in multiple frames of images to enhance the recognition ability of targets in the boundary area; Based on the detection results of consecutive frame images, the edge computing module calls the fusion prediction composed of the LSTM neural network and the Kalman filter, for each detected pedestrian target to generate a trajectory sequence within a future time period , with a prediction duration of 1 second and a prediction time step of every 100ms; The control strategy generation module sets a fixed prediction time window T, for example, T = 1000 ms, and sets an early closing threshold Δt = 300 ms. The front spraying area of the sprinkler is divided into several fixed spatial sectors Z1, Z2,..., Zn. The coverage range of each sector is defined by a set of angular ranges and forward distance ranges in a two-dimensional coordinate system; The edge computing module performs the following calculation and judgment process: For each target pedestrian , traverse the position of each time point in its trajectory sequence in turn , map each position point to the current vehicle body coordinate system of the sprinkler, and judge whether there is a trajectory point , where ; If there is an intersection, record the entry time point and the sector number j; According to the above judgment results, the control strategy generation module makes the following decisions: If , and , then at the system time point Δt, generate a control instruction to close the nozzle ; If no point is found to fall into the spraying area within the trajectory prediction, the nozzle maintains its existing open state unchanged; If the system detects that the target deviates from the historical stable trend in the current prediction sequence, the deviation value exceeds the set threshold ε, or all consecutive prediction points are outside the sector, it is determined that the target has left the spraying influence area, and the control strategy generation module will immediately send a resume spraying instruction to the spraying control module to activate the nozzle .

[0026] The above judgment logic is continuously executed at a fixed period (such as 100 ms), and the latest perception input and trajectory prediction results are used for each update.

[0027] When the detected target enters the LSTM prediction module, trajectory modeling is performed with a time window t = 10 frames to predict the x and y coordinate trajectory points (a total of 40) within the next 2 seconds. Each target trajectory will be marked with its motion trend angle θ and average speed v, and then input to the control strategy generation module; The control strategy generation module is based on the following avoidance decision function F(p, θ, v, S) internally: F(p, θ, v, S) = 1 indicates triggering nozzle S avoidance; F = 0 indicates maintaining normal spraying; where p is the current coordinate point of the target, θ is the motion direction angle, v is the current speed, and S is the nozzle sector number.

[0028] When the system calculates that a certain pedestrian will enter the spraying area S within the next Δt, the control strategy module sends the instruction "close nozzle S" to the spraying control module, and the delay compensation is corrected by weighting the historical response delay. After receiving the signal, the spraying control module immediately performs the nozzle state switch. If the target stays away from the path or remains stationary for more than the threshold T_i, the system resumes spraying to ensure that cleaning is not missed and intelligent avoidance can be achieved.

[0029] Embodiment 2: Each nozzle unit in this embodiment is provided with the following components: Electromagnetic drive valve; spray angle servo motor; status feedback Hall sensor; single-chip microcomputer local controller (STM32H7).

[0030] Nozzle response process: Receive the control signal CAN_msg from the spraying control module; Judge according to the instruction to execute "OPEN", "CLOSE" or "ADJUST_ANGLE(θ)"; Locally execute the action, record the completion timestamp, and return the ACK signal and the status words of the current angle, valve position, and whether there is an abnormality; The feedback data is updated once per second and uploaded to the edge computing module, the upper computer, and the city platform via the communication bus module to achieve full-process traceability.

[0031] Embodiment 3: This embodiment is linked with the urban management platform, and the sprinkler system accesses the urban Internet of Things platform through the dual-channel communication of MQTT + RESTful API, supporting the following interface capabilities: Upload the operation status; obtain the parameter update package; submit the self-check log.

[0032] Each sprinkler is configured with a unique ID and establishes a key handshake connection with the management platform. The platform can view the path trajectory, avoidance trigger frequency, recognition success rate, etc. of each vehicle in real time. The administrator can set the "whitelist for avoiding high pedestrian flow areas" or enable the full avoidance mode during specific time periods in the platform background.

[0033] Embodiment 4: This embodiment is based on the application process of the intelligent control system for the sprinkler to avoid pedestrians of the present invention, and is applicable to the intelligent sprinkler operation on urban main roads in the municipal environmental sanitation system; In the intelligent environmental sanitation pilot area of a certain city, a modified sprinkler of model XJ2100 is selected as the carrier. An infrared thermal imager is installed at the mid-axis position in the front of the vehicle body, a set of 77GHz millimeter-wave radars are installed on both sides, and two wide-angle RGB cameras are set above the front bumper, facing the left front and right front views respectively, forming a multi-modal perception matrix for comprehensively perceiving the dynamic pedestrian status within 6 meters in front; All of the above sensing devices are connected to the Jetson Orin NX edge AI module located in the main control area of the cab through USB and SPI serial ports, providing local real-time computing capabilities. This edge AI module is installed with the Ubuntu 20.04 embedded system and runs the YOLOv8+Transformer object detection model optimized by TensorRT, combined with the LSTM+Kalman trajectory prediction algorithm combination; After the sprinkler operation is started, the system continuously receives RGB images, thermal imaging matrices, and millimeter-wave speed / distance data at a rate of 30 frames per second, and performs spatial alignment, time synchronization, and image fusion processing on the three types of data through the SensorFusion SDK in the Jetson module; The system divides the spraying area in the front of the sprinkler into three independent controllable sectors A, B, and C (each sector is 60°, and the total spraying angle is 180°), and each sector is connected to a servo-angle adjustable nozzle and its solenoid valve control unit; The control strategy generation module receives the predicted trajectories of each pedestrian and determines whether there is a predicted path that will cross the spraying coverage area of any sector within the next 1.0 second; If there is an intersection, calculate the crossing time point T, and send a control signal to the spraying control module to set the nozzle to enter the closed state Δt (set to 300 ms) in advance; when the pedestrian completely leaves the predicted area, the control module sends an opening instruction to resume spraying, and multiple nozzles can be independently controlled according to different targets.

[0034] The instructions are transmitted through the CAN bus protocol, and all nozzle actions are synchronously driven, with a control response delay of less than 80 ms. If the system detects that the target has left the monitoring range or the prediction accuracy has dropped below the threshold, it will preferentially enter the default state of resuming sprinkling.

[0035] The field test scenario was from 8:30 to 9:00 on Friday morning. The main road is a six-lane two-way road with a high pedestrian density. The sprinkler truck travels along the green belt at a speed of 10 kilometers per hour. The system can identify 8-12 pedestrians per minute on average, and about 3 of them enter the spraying prediction area.

[0036] Connect to the urban intelligent environmental sanitation platform through the EtherCAT communication module, and periodically upload operation status data, avoidance times, recognition confidence information, and target recognition frame screenshots through the MQTT protocol, which is convenient for the background supervision center to conduct operation review and scheduling.

[0037] As described above, it is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of various changes or substitutions thereof, and these should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims described above.

Claims

1. An intelligent control system for a sprinkler truck to spray water to avoid pedestrians, characterized in that: include The perception module includes three sensor devices: RGB camera, infrared thermal imager and millimeter wave radar, which are used to collect image information, thermal imaging data and target distance information in front and on the sides of the sprinkler truck; An edge computing module is connected to the perception module via a physical communication line, receives data output by the perception module, and detects, tracks, and predicts future paths of pedestrian targets based on a preset multimodal fusion recognition algorithm; A spraying control module is connected to the edge computing module through a wired control channel, and the spraying control module includes a plurality of adjustable angle nozzles distributed in sectors, each nozzle is equipped with an independent electromagnetic control unit for performing spraying state adjustment according to control instructions; A control strategy generation module, which is respectively connected to the edge computing module and the spraying control module through a logic processing interface, and is used to generate an avoidance control strategy according to the pedestrian path prediction result, and send corresponding instructions to the spraying control module; The communication bus module connects the above-mentioned perception module, edge computing module, control strategy generation module and spraying control module through the CAN or EtherCAT industrial bus to achieve data synchronization and command transmission between modules.

2. According to claim 1, an intelligent control system for a sprinkler truck to spray water to avoid pedestrians is characterized by: The edge computing module is an integrated embedded AI processing unit. The AI ​​processing unit uses a Jetson Orin or Jetson Xavier model embedded platform with a computing power of 20 TOPS or above. It can complete the fusion processing of the data collected by the perception module, image target recognition, and path prediction operations based on continuous frames within 100 milliseconds. The calculation results are used as input to the control strategy generation module.

3. According to claim 1, an intelligent control system for a sprinkler truck to spray water to avoid pedestrians is characterized by: The multimodal fusion recognition algorithm is constructed by the YOLOv8 network model and substituted into the Transformer structure to enhance the spatial feature expression capability, and is used to identify pedestrian targets that are occluded or partially visible in the image; the path prediction algorithm uses the long short-term memory neural network LSTM structure to construct a time series prediction model, models the target continuous frame trajectory, and combines the Kalman filter to correct the prediction deviation in real time.

4. According to claim 1, an intelligent control system for a sprinkler truck to spray water to avoid pedestrians is characterized by: The spray control module is provided with three independently partitioned adjustable angle nozzles, each nozzle is equipped with a servo motor and a solenoid valve control unit, which can adjust the spray angle within the range of 30 to 120 degrees, and execute opening and closing actions according to the instructions of the control strategy generation module, and the instructions are uniformly transmitted by the communication bus module.

5. According to claim 1, an intelligent control system for a sprinkler truck to spray water to avoid pedestrians is characterized by: After receiving the path prediction results provided by the edge computing module, the control strategy generation module analyzes the predicted path of each target pedestrian based on the set time window logic, and determines by calculation whether it will enter the spraying sector covered by any nozzle within time T; If the judgment result is "yes", a control instruction to close the nozzle is sent to the spray control module at T-Δt milliseconds. If the judgment result is "no", the spraying state is maintained unchanged. If the predicted path deviates from the threshold range or the target leaves the spraying influence area, the spraying action is resumed immediately.

6. According to claim 1, an intelligent control system for a sprinkler truck to spray water to avoid pedestrians is characterized by: The infrared thermal imager in the sensing module is installed in the front central area of ​​the sprinkler truck to collect infrared radiation images of the human body under low light conditions; The millimeter wave radar is installed on both sides of the sprinkler truck to detect the relative distance and speed information of lateral dynamic obstacles; The RGB camera is installed at the front of the vehicle to obtain forward visible light images under daytime conditions. The three types of data output by the infrared thermal imager, millimeter wave radar and RGB camera are processed by the edge computing module after time alignment and spatial fusion.

7. According to claim 1, an intelligent control system for a sprinkler truck to spray water to avoid pedestrians is characterized by: The edge computing module integrates an adaptive parameter adjustment module, which is used to self-learn and adjust the recognition threshold, time window parameters and nozzle closing delay value based on the pedestrian recognition rate, path prediction error and spraying control execution lag information accumulated during the watering operation, using a gradient optimization algorithm. The updated parameters are fed back to the edge computing and control strategy generation module in real time.

8. According to claim 1, an intelligent control system for a sprinkler truck to spray water to avoid pedestrians is characterized by: The communication bus module adopts the CAN or EtherCAT industrial bus protocol to stably and quickly transmit the perception data, control strategies and execution instructions between the functional modules. The CAN or EtherCAT industrial bus protocol supports error checking and master-slave control mode.

9. The intelligent control system for a water sprinkler truck to avoid pedestrians by spraying water according to claim 1, characterized in that: The system reserves an interface for docking with the city management Internet of Things platform, and uploads operation data, recognition results and system status information through a standardized API protocol.

10. The intelligent control system for a water sprinkler truck to spray water to avoid pedestrians according to claim 1, characterized in that: The perception module, edge computing module, control strategy generation module, spray control module and communication bus module are each encapsulated in an independent functional unit, and interconnection and communication are achieved through a standard industrial control interface.

Images